Skip to content

feat: Implement V2 FBCaching and Optimize Existing FBCache - #621

Merged
lmxyy merged 7 commits into
nunchux-ai:mainfrom
Bluear7878:v2_cache
Aug 27, 2025
Merged

lmxyy merged 7 commits into
nunchux-ai:mainfrom
Bluear7878:v2_cache

Conversation

@Bluear7878

Copy link
Copy Markdown
Contributor

Description

This PR is a new V2 version of FB caching, complete with test cases and examples. Additionally, it includes optimizations for the existing FBCache implementation.

Key Changes
Implemented V2 FB Caching:

Introduced a new FBCacheV2 with a revised caching logic.

Added test cases.

Included an example script of FBCacheV2.

Optimized Existing FBCache:

V2 Performance comparsion

Inference Time Comparison

Without Cache (Baseline)

  • V1 Baseline: ~27.6s (50 steps)
  • V2 Baseline: ~29.0s (50 steps)

With Cache (multi_threshold=0.12)

V1 FBCache Performance:
├── Run 1: 9.70s
├── Run 2: 9.83s
├── Run 3: 9.92s
└── Average: 9.82s (196.3ms/step)

V2 FBCache Performance:
├── Run 1: 10.74s
├── Run 2: 10.81s
├── Run 3: 10.85s
└── Average: 10.80s (216.0ms/step)

DoubleFBCache Performance comparsion

Key Findings

Single_Threshold V1 Time (s) V2 Time (s) Difference
0.08 22.02 23.37 +1.35s (+6.1%)
0.10 18.18 18.80 +0.61s (+3.4%)
0.12 13.98 14.53 +0.55s (+4.0%)
0.15 13.27 13.96 +0.69s (+5.2%)
0.20 13.28 13.95 +0.68s (+5.1%)

Cache Effectiveness

Method Without Cache With Cache Speedup Effectiveness
V1 27.6s 9.82s 17.78s 64.4%
V2 29.0s 10.80s 18.20s 62.8%

FBCache refactoring and optimzation
FBCache.py (Update) delivers marginal but consistent performance improvements over the original FBCache.py by eliminating GPU-CPU synchronization overhead. The optimized version is 1.2% faster in realistic workflows while maintaining identical functionality.

Metric Original Optimized Improvement
Similarity Check (avg) 3.98ms 3.98ms ~0%
Cache Operations 0.0023ms 0.0022ms 4.3%
Workflow Simulation 10.34ms 10.22ms 1.2%
GPU-CPU Sync Overhead 0.10ms 0ms 100%

@lmxyy

lmxyy commented Aug 19, 2025

Copy link
Copy Markdown
Collaborator

Could you provide the reference cache implementation in diffusers?

@Bluear7878

Copy link
Copy Markdown
Contributor Author

Could you provide the reference cache implementation in diffusers?

I looked into implementing caching in diffusers.
FasterCache: link

FBCache (in ParaAttn): link

DeepCache: link

Should I align my current implementation with the style and structure of these existing caches?
Any specific one I should prioritize as a reference?

@lmxyy

lmxyy commented Aug 19, 2025

Copy link
Copy Markdown
Collaborator

Could you provide the reference cache implementation in diffusers?

I looked into implementing caching in diffusers. FasterCache: link

FBCache (in ParaAttn): link

DeepCache: link

Should I align my current implementation with the style and structure of these existing caches? Any specific one I should prioritize as a reference?

Yeah, I think maybe we need to align the current implementation if we would like to scale up the codebase and support more cache mechanisms in the future. Better check how diffusers maintain the codebase.

@Bluear7878

Copy link
Copy Markdown
Contributor Author

Could you provide the reference cache implementation in diffusers?

I looked into implementing caching in diffusers. FasterCache: link
FBCache (in ParaAttn): link
DeepCache: link
Should I align my current implementation with the style and structure of these existing caches? Any specific one I should prioritize as a reference?

Yeah, I think maybe we need to align the current implementation if we would like to scale up the codebase and support more cache mechanisms in the future. Better check how diffusers maintain the codebase.

I got it. i will make those caches to the format that diffuser supports

@Bluear7878 Bluear7878 closed this Aug 19, 2025
@Bluear7878 Bluear7878 reopened this Aug 24, 2025
Comment thread nunchaku/caching/utils_v2.py Outdated
import torch
from diffusers.models.modeling_outputs import Transformer2DModelOutput

from nunchaku.caching.FBCache import check_and_apply_cache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Python file, we'd better call it fbcache instead of FBCache.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename the file to fbcache.py.

@lmxyy lmxyy closed this Aug 27, 2025
@lmxyy lmxyy reopened this Aug 27, 2025
@lmxyy lmxyy closed this Aug 27, 2025
@lmxyy lmxyy reopened this Aug 27, 2025
@lmxyy lmxyy changed the title Feat: Implement V2 FB Caching and Optimize Existing FBCache feat: Implement V2 FBCaching and Optimize Existing FBCache Aug 27, 2025
@lmxyy lmxyy closed this Aug 27, 2025
@lmxyy lmxyy reopened this Aug 27, 2025
@lmxyy
lmxyy merged commit 882aa07 into nunchux-ai:main Aug 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants