Repository navigation
TF-TRT C++ conversion - #52012
Merged
copybara-service[bot] merged 1 commit intoOct 11, 2021
Merged
TF-TRT C++ conversion#52012
Conversation
tfeher
force-pushed
the
trt_cpp_conversion_example
branch
from
September 16, 2021 07:48
e6d0699 to
45680b7
Compare
This was referenced Sep 16, 2021
bixia1
self-requested a review
September 16, 2021 16:40
Contributor
|
We can avoid the change to the utilities for freezing the graph by updating out SavedModelBundle with the converted GraphDef. Something like this: |
sanjoy
removed their request for review
September 20, 2021 20:30
tfeher
force-pushed
the
trt_cpp_conversion_example
branch
2 times, most recently
from
September 26, 2021 22:49
afaa5af to
d876eae
Compare
tfeher
marked this pull request as ready for review
September 26, 2021 22:51
Contributor
Author
|
@bixia1 ready for review. |
tfeher
commented
Sep 26, 2021
bixia1
reviewed
Sep 30, 2021
bixia1
left a comment
bixia1
left a comment
Contributor
There was a problem hiding this comment.
I am still working on reviewing trt_convert.cc .
bixia1
reviewed
Oct 1, 2021
bixia1
left a comment
bixia1
left a comment
Contributor
There was a problem hiding this comment.
Still working on trt_convert.cc.
bixia1
reviewed
Oct 1, 2021
Comment on lines
+62
to
+66
| // We could set item_conifg.feed_nodes and item_config.fetch_nodes to the | ||
| // nodes in the signature def. Alternatively, we could also set collection | ||
| // 'train_op'. Grappler can use these to determine the input and outputs. | ||
| // If none of these are set, then it will use the SignatereDef from | ||
| // the MetaGraphDef. See grappler_item_builder.cc for details. |
Contributor
There was a problem hiding this comment.
If we will go with the API where users provide input/output names, we should setup ItemConfig fetch_nodes/feed_nodes.
If we will go with the API where users provide SignatureDef, we need to clear the SignatureDef in the input meta_graph_def and keep only the one we need.
Also, if we will need to run grappler a few times, we probably want to avoid constructing GrapplerItem each time by somehow reusing the GrapplerItem, see TF code here.
Contributor
Author
There was a problem hiding this comment.
- I have set up feed_nodes and fetch_nodes.
- If a previous conversion had
RunMetaOptimizer(item, ..., out_graph_def), can we just replace the item's graphitem.graph = out_graph_defbefore the nextRunMetaOptimizercall? For now we have only a single grappler pass, but it might matter in the follow up PR where we need to inline the graph before freezing it.
tfeher
commented
Oct 4, 2021
Comment on lines
+62
to
+66
| // We could set item_conifg.feed_nodes and item_config.fetch_nodes to the | ||
| // nodes in the signature def. Alternatively, we could also set collection | ||
| // 'train_op'. Grappler can use these to determine the input and outputs. | ||
| // If none of these are set, then it will use the SignatereDef from | ||
| // the MetaGraphDef. See grappler_item_builder.cc for details. |
Contributor
Author
There was a problem hiding this comment.
- I have set up feed_nodes and fetch_nodes.
- If a previous conversion had
RunMetaOptimizer(item, ..., out_graph_def), can we just replace the item's graphitem.graph = out_graph_defbefore the nextRunMetaOptimizercall? For now we have only a single grappler pass, but it might matter in the follow up PR where we need to inline the graph before freezing it.
tfeher
force-pushed
the
trt_cpp_conversion_example
branch
2 times, most recently
from
October 7, 2021 17:30
09cd906 to
b02e321
Compare
Contributor
Author
|
@bixia1 I have removed the circular dependency, and added trt_convert_api as a dependency of trt_op_libs. Let me know if there are other issues that we want to address in this PR. |
Contributor
|
Can you fix the PR description, such as to remove the information that is obsolete? The PR description will become part of the commit message. |
bixia1
reviewed
Oct 7, 2021
sherhut
removed their request for review
October 8, 2021 12:23
tfeher
force-pushed
the
trt_cpp_conversion_example
branch
from
October 8, 2021 17:21
b02e321 to
5cbde12
Compare
tfeher
commented
Oct 8, 2021
bixia1
approved these changes
Oct 11, 2021
google-ml-butler
Bot
added
kokoro:force-run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TF-TRT C++ interface to convert models.
Differences compared to the Python API:
convert_to_static_engineconversion param allows to convert dynamic engines to static engines.The steps for model conversion by
ConvertAndBuild(On the Python side, steps 4-5 are done by a separate
buildfunction.)Related PRs: