Add dgggen tool to generate production-shaped call graphs#166
Merged
Conversation
The DGG sample corpus tops out at depth 4 and 11 spans per trace, well short of the depths of 15+ and hundreds of spans the Du et al. paper reports for Alibaba production traces. dgggen generates DGG-format JSON graphs with those production shape distributions: heavy-tailed sizes up to 450 nodes, depths to 20, hub fan-out, and repeated-call multiplicities into the hundreds, stratified to cover the tail in a small corpus. Output is deterministic per seed and feeds the existing dgg2motel and test_corpus.sh pipeline; test_corpus.sh gains CHECK_ARGS for raising motel check limits.
There was a problem hiding this comment.
Pull request overview
Adds a new Go tool (dgggen) to generate deterministic, production-shaped DGG JSON call graphs (heavy-tailed depth/size/fan-out), plus tests and documentation to enable stress-testing motel with more realistic topology distributions.
Changes:
- Add
tools/dgggengenerator that builds stratified, tail-covering tree-shaped DGG graphs and writes them to disk. - Add a test suite validating determinism, structural validity, and span/fan-out bounds.
- Update demo docs and
tools/dgg2motel/test_corpus.shto support raisedmotel checklimits viaCHECK_ARGS.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tools/dgggen/main.go | New DGG corpus generator with production-shaped distributions and bounding logic |
| tools/dgggen/main_test.go | Tests for determinism, graph validity, and bound enforcement |
| tools/dgg2motel/test_corpus.sh | Adds CHECK_ARGS passthrough to motel check for larger graphs |
| docs/demos/demo-dgg.md | Documents how to generate and validate a production-shaped corpus |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+330
to
+337
| walk = func(n *genNode) { | ||
| if len(n.children) == 0 && (largest == nil || n.mult > largest.mult) { | ||
| largest = n | ||
| } | ||
| for _, c := range n.children { | ||
| walk(c) | ||
| } | ||
| } |
Comment on lines
14
to
16
| RATE="${RATE:-}" # override traffic rate if set | ||
| CHECK_ARGS="${CHECK_ARGS:-}" # extra flags for motel check, e.g. "--max-depth 25" | ||
|
|
Comment on lines
+46
to
+50
| # shellcheck disable=SC2086 | ||
| if ! "$MOTEL" check $CHECK_ARGS "$f" > /dev/null 2>&1; then | ||
| echo "CHECK FAIL: $rel" | ||
| "$MOTEL" check "$f" 2>&1 | sed 's/^/ /' | ||
| # shellcheck disable=SC2086 | ||
| "$MOTEL" check $CHECK_ARGS "$f" 2>&1 | sed 's/^/ /' |
clampSpans only targeted leaf nodes, but assignMultiplicities can set mult=2 on interior nodes. An interior mult multiplies the entire subtree, so a deep spine with an interior mult could keep staticSpans above maxStaticSpans even after all leaf mults reached 1. Fix: walk all nodes with mult > 1, not just leaves. Replace unquoted $CHECK_ARGS expansion in test_corpus.sh with a read -ra array to avoid word-splitting and glob expansion at the call sites.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
dgggen, a new tool that generates synthetic DGG-format call graphs with shape distributions matching production measurements from the Du et al. paper on Alibaba microservice traces. This enables stress testing with realistic graph topologies that reach the tail of the distribution (depths 15+, hundreds of nodes, high fan-out).Key Changes
New tool:
tools/dgggen/main.go— Generates deterministic, stratified call graphs with:Test suite:
tools/dgggen/main_test.go— Validates:Documentation:
docs/demos/demo-dgg.md— Added section showing:Test infrastructure:
tools/dgg2motel/test_corpus.sh— Enhanced to support:CHECK_ARGSenvironment variable for passing custom motel check flagsImplementation Details
https://claude.ai/code/session_014HnMuCVwwmEnBBfPRsERfn