Repository navigation
feat: Implement V2 FBCaching and Optimize Existing FBCache - #621
Conversation
|
Could you provide the reference cache implementation in diffusers? |
I looked into implementing caching in diffusers. FBCache (in ParaAttn): link DeepCache: link Should I align my current implementation with the style and structure of these existing caches? |
Yeah, I think maybe we need to align the current implementation if we would like to scale up the codebase and support more cache mechanisms in the future. Better check how diffusers maintain the codebase. |
I got it. i will make those caches to the format that diffuser supports |
| import torch | ||
| from diffusers.models.modeling_outputs import Transformer2DModelOutput | ||
|
|
||
| from nunchaku.caching.FBCache import check_and_apply_cache |
There was a problem hiding this comment.
For Python file, we'd better call it fbcache instead of FBCache.
There was a problem hiding this comment.
rename the file to fbcache.py.
Description
This PR is a new V2 version of FB caching, complete with test cases and examples. Additionally, it includes optimizations for the existing FBCache implementation.
Key Changes
Implemented V2 FB Caching:
Introduced a new FBCacheV2 with a revised caching logic.
Added test cases.
Included an example script of FBCacheV2.
Optimized Existing FBCache:
V2 Performance comparsion
Inference Time Comparison
Without Cache (Baseline)
With Cache (multi_threshold=0.12)
DoubleFBCache Performance comparsion
Key Findings
Cache Effectiveness
FBCache refactoring and optimzation
FBCache.py (Update) delivers marginal but consistent performance improvements over the original FBCache.py by eliminating GPU-CPU synchronization overhead. The optimized version is 1.2% faster in realistic workflows while maintaining identical functionality.