Graph Visualization: Added hierarchy based on frames, faster function to get the 'pbtxt' and 'dot' formats, and implicit edges are added between commands and subcommands.#2527
Conversation
6e34820 to
702dd46
Compare
| currentGraph, err := createGraphFromDependencyGraph(dependencyGraph) | ||
| currentGraph.removeNodesWithZeroDegree() | ||
| output := "" | ||
| currentGraph, err := createGraphFromDependencyGraph(ctx, dependencyGraph) |
There was a problem hiding this comment.
It would be more canonical go with:
currentGraph, err := createGraphFromDependencyGraph(ctx, dependencyGraph)
if err != nil {
return []byte{}, err
}
currentGraph.joinNodesByFrame()
currentGraph.joinNodesWithZeroDegree()
output := []byte{}
if format == service.GraphFormat_PBTXT {
output = currentGraph.getGraphInPbtxtFormat()
} else if format == service.GraphFormat_DOT {
output = currentGraph.getGraphInDotFormat()
}
return output, nilThere was a problem hiding this comment.
You are right, I added the error checking.
| for _, currentEdge := range g.edgeIdToEdge { | ||
| lines := strconv.Itoa(currentEdge.source.id) + " -> " + strconv.Itoa(currentEdge.sink.id) + ";\n" | ||
| output += lines | ||
| func (g *graph) dfs(currentNode *node, visited []bool, visitedNodes *[]*node) { |
There was a problem hiding this comment.
This recursive implementation makes me a bit nervous that a long chain of dependencies could cause a stack overflow.
There was a problem hiding this comment.
Golang has an infinite stack. So unless we expect this to end up being a memory consumption issue, we should be ok there.
There was a problem hiding this comment.
Thank you, I was not expecting an infinite stack. However, I implemented an iterative version (using a breadth first search). I think in some point it could end up a unnecessary memory consumption issue for huge graphs (millions of nodes and hundred of millions of edges).
gapis/resolve/dependencygraph2/graph_visualization/graph_structure.go
Outdated
Show resolved
Hide resolved
gapis/resolve/dependencygraph2/graph_visualization/graph_structure.go
Outdated
Show resolved
Hide resolved
gapis/resolve/dependencygraph2/graph_visualization/graph_visualization.go
Outdated
Show resolved
Hide resolved
| newNode := getNewNode(int(nodeId), label) | ||
| newNode.name = name | ||
| newNode.attributes = attributes | ||
| newNode.isEndOfFrame = command.CmdFlags(ctx, api.CmdID(cmdNodeId), &api.GlobalState{}).IsEndOfFrame() |
There was a problem hiding this comment.
Using an empty GlobalState actually works?
@AWoloszyn, is this safe in general? And if so, why does CmdFlags have a GlobalState argument?
There was a problem hiding this comment.
This is not safe in general. I expect we would be ok in Vulkan, but it will likely segfault in GLES.
The way you would normally do this is get all of the EndOfFrame events from the trace (using a path.Events resolvable), and use those.
Alternatively you could insert those parameters into the dependency graph when creating it, which will have a valid global state.
gapis/resolve/events.go
There was a problem hiding this comment.
I will store the CmdFlags in the dependency graph command nodes, and switch this to get the CmdFlags from the dependency graph in a separate PR.
| frameNumber := 1 | ||
| nodes := g.getSortedNodes() | ||
| for _, currentNode := range nodes { | ||
| if !visited[currentNode.id] && currentNode.isEndOfFrame { |
There was a problem hiding this comment.
Just an observation: Forward dependencies break this strategy. For example, if an image is created in frame 1 and destroyed in frame 2, the creation has a dependency to the destruction. Following that edge in the DFS would cause some of the frame 2 commands to be incorrectly placed in frame 1.
However, I think the issues from this should be pretty small, since nodes with forward dependencies (like resource destruction) probably don't have a lot of other dependencies. I also don't see an obviously more correct way to handle forward dependencies, since just ignoring them in the DFS will leave lots of orphan nodes not associated with any frame.
702dd46 to
1428ee9
Compare
| lines := strconv.Itoa(currentEdge.source.id) + " -> " + strconv.Itoa(currentEdge.sink.id) + ";\n" | ||
| output += lines | ||
| func (g *graph) bfs(sourceNode *node, visited []bool, visitedNodes *[]*node) { | ||
| head, tail := 0, 0 |
There was a problem hiding this comment.
Do you need tail? It looks like tail should always be the end of visitedNodes.
Also, it looks like things will behave very strangely if visitedNodes were somehow not empty to start. I know this should not happen, but if you set head := len(*visitedNodes), it would be a non-issue.
There was a problem hiding this comment.
No, I do not . I was using tail for better understanding of the algorithm, but I think it is okay remove this.
I am assuming that always visitedNodes is empty at the beginning, but could be useful for some cases set head := len(*visitedNodes) such as get nodes from two frames, so I changed it. Thank you.
| output.WriteString("\"\n") | ||
| output.WriteString("op: \"") | ||
| output.WriteString(currentNode.label) | ||
| output.WriteString("\"\n") |
There was a problem hiding this comment.
I don't doubt that individual WriteString calls are faster, but I think a single WriteString or fmt.Fprintf call per output line would be easier to read, and I suspect the performance difference is extremely small.
There was a problem hiding this comment.
Yes, the performance is almost the same considering small lengths of strings. Based on local becnhmarking and https://gist.github.com/dtjm/c6ebc86abe7515c988ec , using concatenation is a little faster than fmt.Fprintf. So I added concatenation by line.
1428ee9 to
cfce6e9
Compare
to get the 'pbtxt' and 'dot' formats, and implicit edges are added between commands and subcommands. Hierarchy based on frames: Using a depth first search starting in every VK_QUEUE_PRESENT command. Implicit edges added between commands and subcommands to correctly defined the frames in the depth first search.
cfce6e9 to
bed1338
Compare
No description provided.