HF support multiple precision options#361
Merged
HenryNdubuaku merged 6 commits intomainfrom Feb 17, 2026
Merged
Conversation
jakmro
force-pushed
the
hf_publish_precisions
branch
from
edb8fce to
b5b3a22
Compare
Collaborator
|
@jakmro is this ready? |
Collaborator
Author
|
@HenryNdubuaku need a few more minutes to do final testing and will be ready to go |
…nd streamline configuration handling Signed-off-by: jakmro <[email protected]>
* Add optional cloud key prompt for transcribe Prompts user for cloud API key on first run or whenever key is not set. If provided, key is saved to ~/.cactus/config.json and exported as CACTUS_CLOUD_API_KEY env var for C++ cloud code. If skipped, user is prompted again on next run. Env var takes precedence over config file. Signed-off-by: Roman Shemet <[email protected]> * Structure code properly Signed-off-by: HenryNdubuaku <[email protected]> * More changes padded in Signed-off-by: HenryNdubuaku <[email protected]> --------- Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: HenryNdubuaku <[email protected]> Signed-off-by: jakmro <[email protected]>
* Add optional cloud key prompt for transcribe Prompts user for cloud API key on first run or whenever key is not set. If provided, key is saved to ~/.cactus/config.json and exported as CACTUS_CLOUD_API_KEY env var for C++ cloud code. If skipped, user is prompted again on next run. Env var takes precedence over config file. Signed-off-by: Roman Shemet <[email protected]> * Structure code properly Signed-off-by: HenryNdubuaku <[email protected]> * More changes padded in Signed-off-by: HenryNdubuaku <[email protected]> * pauth Signed-off-by: HenryNdubuaku <[email protected]> --------- Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: Roman Shemet <[email protected]> Signed-off-by: jakmro <[email protected]>
Signed-off-by: jakmro <[email protected]>
Signed-off-by: jakmro <[email protected]>
jakmro
force-pushed
the
hf_publish_precisions
branch
from
3045a5d to
3b0cc78
Compare
Signed-off-by: jakmro <[email protected]>
Collaborator
Author
|
@HenryNdubuaku ready |
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds support for exporting models in multiple precision formats (INT4, INT8, FP16) simultaneously in the HuggingFace publishing workflow. It also modernizes the Silero-VAD model loading by replacing torch.hub.load() with a dedicated silero-vad package.
Changes:
- Refactored model export to support multiple precisions per model instead of single precision
- Changed CLI arguments from
--precisionand--bitsto boolean flags--int4,--int8,--fp16 - Updated GitHub Actions workflow to use JSON configuration with per-model precision settings
- Replaced Silero-VAD loading from torch.hub with
silero-vadpackage and removedtorchaudiodependency
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| python/src/publish_to_hf.py | Refactored export logic to iterate over multiple precisions, removed stage_model function, changed CLI arguments to boolean flags |
| python/src/converter.py | Updated Silero-VAD loading to use load_silero_vad() from silero-vad package |
| python/src/cli.py | Updated Silero-VAD loading and removed torchaudio import check |
| python/requirements.txt | Added silero-vad==6.2.0, removed torchaudio==2.8.0 |
| .github/workflows/publish_to_hf.yml | Restructured to use JSON config for model-specific precision settings, updated bash script logic |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| except Exception: | ||
| print(f"Failed to export Apple weights for {precision}") | ||
|
|
||
| shutil.rmtree(exported) |
There was a problem hiding this comment.
The exported directory is being deleted at line 160 before the loop continues to the next precision. However, if export_model returns the same directory path for different precisions (which it likely does based on the get_weights_dir implementation), this will cause the export of subsequent precisions to fail because the directory was already deleted. The export directory should only be deleted once, after all precisions have been processed, or each precision should export to a unique directory.
ncylich
pushed a commit
that referenced
this pull request
Feb 24, 2026
* Refactor model export process to support multiple precision options and streamline configuration handling Signed-off-by: jakmro <[email protected]> * Add optional cloud key prompt for transcribe (#359) * Add optional cloud key prompt for transcribe Prompts user for cloud API key on first run or whenever key is not set. If provided, key is saved to ~/.cactus/config.json and exported as CACTUS_CLOUD_API_KEY env var for C++ cloud code. If skipped, user is prompted again on next run. Env var takes precedence over config file. Signed-off-by: Roman Shemet <[email protected]> * Structure code properly Signed-off-by: HenryNdubuaku <[email protected]> * More changes padded in Signed-off-by: HenryNdubuaku <[email protected]> --------- Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: HenryNdubuaku <[email protected]> Signed-off-by: jakmro <[email protected]> * Cactus auth command (#360) * Add optional cloud key prompt for transcribe Prompts user for cloud API key on first run or whenever key is not set. If provided, key is saved to ~/.cactus/config.json and exported as CACTUS_CLOUD_API_KEY env var for C++ cloud code. If skipped, user is prompted again on next run. Env var takes precedence over config file. Signed-off-by: Roman Shemet <[email protected]> * Structure code properly Signed-off-by: HenryNdubuaku <[email protected]> * More changes padded in Signed-off-by: HenryNdubuaku <[email protected]> * pauth Signed-off-by: HenryNdubuaku <[email protected]> --------- Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: Roman Shemet <[email protected]> Signed-off-by: jakmro <[email protected]> * fix rate limiter Signed-off-by: jakmro <[email protected]> * remove torchaudio Signed-off-by: jakmro <[email protected]> --------- Signed-off-by: jakmro <[email protected]> Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: rshemet <[email protected]> Co-authored-by: HenryNdubuaku <[email protected]> Co-authored-by: Roman Shemet <[email protected]>
cattermelon1234
pushed a commit
to cattermelon1234/cactus
that referenced
this pull request
Feb 28, 2026
* Refactor model export process to support multiple precision options and streamline configuration handling Signed-off-by: jakmro <[email protected]> * Add optional cloud key prompt for transcribe (cactus-compute#359) * Add optional cloud key prompt for transcribe Prompts user for cloud API key on first run or whenever key is not set. If provided, key is saved to ~/.cactus/config.json and exported as CACTUS_CLOUD_API_KEY env var for C++ cloud code. If skipped, user is prompted again on next run. Env var takes precedence over config file. Signed-off-by: Roman Shemet <[email protected]> * Structure code properly Signed-off-by: HenryNdubuaku <[email protected]> * More changes padded in Signed-off-by: HenryNdubuaku <[email protected]> --------- Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: HenryNdubuaku <[email protected]> Signed-off-by: jakmro <[email protected]> * Cactus auth command (cactus-compute#360) * Add optional cloud key prompt for transcribe Prompts user for cloud API key on first run or whenever key is not set. If provided, key is saved to ~/.cactus/config.json and exported as CACTUS_CLOUD_API_KEY env var for C++ cloud code. If skipped, user is prompted again on next run. Env var takes precedence over config file. Signed-off-by: Roman Shemet <[email protected]> * Structure code properly Signed-off-by: HenryNdubuaku <[email protected]> * More changes padded in Signed-off-by: HenryNdubuaku <[email protected]> * pauth Signed-off-by: HenryNdubuaku <[email protected]> --------- Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: Roman Shemet <[email protected]> Signed-off-by: jakmro <[email protected]> * fix rate limiter Signed-off-by: jakmro <[email protected]> * remove torchaudio Signed-off-by: jakmro <[email protected]> --------- Signed-off-by: jakmro <[email protected]> Signed-off-by: Roman Shemet <[email protected]> Signed-off-by: HenryNdubuaku <[email protected]> Co-authored-by: rshemet <[email protected]> Co-authored-by: HenryNdubuaku <[email protected]> Co-authored-by: Roman Shemet <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.