…rch#145381)

Summary:
Pull Request resolved: pytorch#145381

In this diff we are trying to introduce some stateful API to enable "fullgraph_package" mode which will force inductor to use AOTI as a backend. Different from PR pytorch#141700, we didn't try to populate the package file into caching system, instead we bypass caching to simplify the implementation in the current form.

Similar to PR pytorch#141700, I did a quick benchmark to the loading time and it looks like the following:
- Precompile
```
buck run mode/opt scripts/zhxchen17:precompile
```
- Load using cache:
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader cache
```
Output:
```
real    0m24.593s
user    0m59.342s
sys     0m17.201s
```
- Load using load_fullgraph_package
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader precompile
```
Output:
```
real    0m10.907s
user    0m9.210s
sys     0m1.173s
```

Test Plan:
buck run mode/opt caffe2/test:test_export -- -r test_fullgraph_package_basic
_function

Differential Revision: D68459341

…rch#145381)

Summary:

In this diff we are trying to introduce some stateful API to enable "fullgraph_package" mode which will force inductor to use AOTI as a backend. Different from PR pytorch#141700, we didn't try to populate the package file into caching system, instead we bypass caching to simplify the implementation in the current form.

Similar to PR pytorch#141700, I did a quick benchmark to the loading time and it looks like the following:
- Precompile
```
buck run mode/opt scripts/zhxchen17:precompile
```
- Load using cache:
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader cache
```
Output:
```
real    0m24.593s
user    0m59.342s
sys     0m17.201s
```
- Load using load_fullgraph_package
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader precompile
```
Output:
```
real    0m10.907s
user    0m9.210s
sys     0m1.173s
```

Test Plan:
buck run mode/opt caffe2/test:test_export -- -r test_fullgraph_package_basic
_function

Differential Revision: D68459341

…rch#145381)

Summary:

In this diff we are trying to introduce some stateful API to enable "fullgraph_package" mode which will force inductor to use AOTI as a backend. Different from PR pytorch#141700, we didn't try to populate the package file into caching system, instead we bypass caching to simplify the implementation in the current form.

Similar to PR pytorch#141700, I did a quick benchmark to the loading time and it looks like the following:
- Precompile
```
buck run mode/opt scripts/zhxchen17:precompile
```
- Load using cache:
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader cache
```
Output:
```
real    0m24.593s
user    0m59.342s
sys     0m17.201s
```
- Load using load_fullgraph_package
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader precompile
```
Output:
```
real    0m10.907s
user    0m9.210s
sys     0m1.173s
```

Test Plan:
buck run mode/opt caffe2/test:test_export -- -r test_fullgraph_package_basic
_function

Differential Revision: D68459341

Summary:

Design doc: https://docs.google.com/document/d/1Z15cBBPjoZ7gH00TSgCdgaYko7a7Br-ERd3_hA-g2IU/edit?usp=sharing

In this diff we are trying to introduce a new API pre torch.compile() object which will force inductor to use AOTI as a backend. Different from PR #141700.

Similar to PR #141700, I did a quick benchmark to the loading time and it looks like the following:
- Precompile
```
buck run mode/opt scripts/zhxchen17:precompile
```
- Load using cache:
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader cache
```
Output:
```
real    0m24.593s
user    0m59.342s
sys     0m17.201s
```
- Load using load_fullgraph_package
```
time buck run mode/opt scripts/zhxchen17:precompile -- --loader precompile
```
Output:
```
real    0m10.907s
user    0m9.210s
sys     0m1.173s
```

Test Plan:
buck run mode/opt caffe2/test:test_export -- -r test_fullgraph_package_basic
_function

Differential Revision: D68459341

Following up PR #145381, we implement
a new API for compiling fullgraph models using the cpp wrapper, and save/load
compiled artifacts to disk.

Sticky cache is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in sticky cache first, and match by the guards on inputs
only.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, fullgraph=True, sticky_cache="my_dir/my_model")

compiled_f(*args)

compiled_f.save_sticky_cache(prefix="/dir1")

...

compiled_f.load_sticky_cache(prefix="/dir2")
```

Since this is touching many layers of the torch.compile system, we start from the
simple case of forward only graph, static shape and flat tensor inputs/outputs.
Once the overall API converges, we can gradually remove the sticky_cache.unimplemented()
calls from the code.

Following up PR #145381, we implement
a new API for compiling fullgraph models using the cpp wrapper, and save/load
compiled artifacts to disk.

Sticky cache is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in sticky cache first, and match by the guards on inputs
only.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, fullgraph=True, sticky_cache="my_dir/my_model")

compiled_f(*args)

compiled_f.save_sticky_cache(prefix="/dir1")

...

compiled_f.load_sticky_cache(prefix="/dir2")
```

Since this is touching many layers of the torch.compile system, we start from the
simple case of forward only graph, static shape and flat tensor inputs/outputs.
Once the overall API converges, we can gradually remove the sticky_cache.unimplemented()
calls from the code.

Following up PR #145381, we implement
a new API for compiling fullgraph models using the cpp wrapper, and save/load
compiled artifacts to disk.

Sticky cache is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in sticky cache first, and match by the guards on inputs
only.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, fullgraph=True, sticky_cache="my_dir/my_model")

compiled_f(*args)

compiled_f.save_sticky_cache(prefix="/dir1")

...

compiled_f.load_sticky_cache(prefix="/dir2")
```

Since this is touching many layers of the torch.compile system, we start from the
simple case of forward only graph, static shape and flat tensor inputs/outputs.
Once the overall API converges, we can gradually remove the sticky_cache.unimplemented()
calls from the code.

Following up PR #145381, we implement
a new API for compiling fullgraph models using the cpp wrapper, and save/load
compiled artifacts to disk.

Sticky cache is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in sticky cache first, and match by the guards on inputs
only.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, fullgraph=True, sticky_cache="my_dir/my_model")

compiled_f(*args)

compiled_f.save_sticky_cache(prefix="/dir1")

...

compiled_f.load_sticky_cache(prefix="/dir2")
```

Since this is touching many layers of the torch.compile system, we start from the
simple case of forward only graph, static shape and flat tensor inputs/outputs.
Once the overall API converges, we can gradually remove the sticky_cache.unimplemented()
calls from the code.

Following up PR #145381, we implement
a new API for compiling fullgraph models using the cpp wrapper, and save/load
compiled artifacts to disk.

Sticky cache is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in sticky cache first, and match by the guards on inputs
only.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, fullgraph=True, sticky_cache="my_dir/my_model")

compiled_f(*args)

compiled_f.save_sticky_cache(prefix="/dir1")

...

compiled_f.load_sticky_cache(prefix="/dir2")
```

Since this is touching many layers of the torch.compile system, we start from the
simple case of forward only graph, static shape and flat tensor inputs/outputs.
Once the overall API converges, we can gradually remove the sticky_cache.unimplemented()
calls from the code.

Following up PR #145381, we implement
a new API for compiling models using the cpp wrapper, and save/load
compiled artifacts to disk.

Package is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in compile package first, and match by the serialized guards.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, package="my_dir/my_model")

compiled_f(*args)

compiled_f.save_package(prefix="/dir1")

...

compiled_f.load_package(prefix="/dir2")
```

Following up PR #145381, we implement
a new API for compiling models using the cpp wrapper, and save/load
compiled artifacts to disk.

Package is now designed to be a per-torch.compile() object living with
the compilation context. Each time a recompilation happens, it will collect
the compiled artifacts into a lookup table. When a new set of inputs is
passed to the compiled callable, before we enter the dynamo cache, we will
perform a lookup in compile package first, and match by the serialized guards.

API names are tentative but the workflow roughly looks like the following:

```
def f(...): ...

compiled_f = torch.compile(f, package="my_dir/my_model")

compiled_f(*args)

compiled_f.save_package(prefix="/dir1")

...

compiled_f.load_package(prefix="/dir2")
```