Skip to content

Make sure download stats work on Hugging Face, improve metadata#6

Merged
wangbinDL merged 1 commit intoopendatalab:mainfrom
NielsRogge:feature/add_hf
Oct 22, 2024
Merged

Make sure download stats work on Hugging Face, improve metadata#6
wangbinDL merged 1 commit intoopendatalab:mainfrom
NielsRogge:feature/add_hf

Conversation

@NielsRogge
Copy link
Copy Markdown
Contributor

@NielsRogge NielsRogge commented Oct 19, 2024

Hi @wangbinDL,

Thanks for this nice work, great to see the models being released on the hub! Your paper was featured on the daily papers: https://huggingface.co/papers/2410.12628. Would be great to link the models to the paper.

To make download stats work, I wrote a PR similar to THU-MIG/yolov10#168 which I did for the YOLOv10 repository.

It leverages the PyTorchModelHubMixin developed by the 🤗 team to make sure a custom PyTorch model like yours:

  • automatically inherits from_pretrained and push_to_hub methods
  • have better metadata
  • make sure download stats work.

Usage is as follows:

from doclayout_yolo import YOLOv10
from huggingface_hub import hf_hub_download

# Load a pre-trained model
filepath = hf_hub_download(repo_id="juliozhao/DocLayout-YOLO-D4LA-from_scratch", filename="doclayout_yolo_d4la_imgsz1600_from_scratch.pt")
model = YOLOv10(filepath)

# One can optionally push this to the hub
model.push_to_hub("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

# Can now be reloaded as follows (and will increment download count)
model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

Besides that, some more suggestions to improve the HF release:

Let me know whether you need any help!

Kind regards,

Niels from HF

@JulioZhao97
Copy link
Copy Markdown
Member

Thank you very much for your suggestions! If there are updates I will let you know immediately.

@wangbinDL
Copy link
Copy Markdown
Member

Hi @NielsRogge

Thanks for your valuable suggestions. We've now:

Your assistance is greatly appreciated!

Best,
Bin

@wangbinDL wangbinDL merged commit 966f1bb into opendatalab:main Oct 22, 2024
@NielsRogge
Copy link
Copy Markdown
Contributor Author

Thanks, looking great!

Note that for download stats to work, you will need to run the code snippet above for each of the model repos (basically overwriting them). As the class now inherits from PyTorchModelHubMixin, a config.json along with safetensors weights will be pushed.

@JulioZhao97
Copy link
Copy Markdown
Member

Thanks, looking great!

Note that for download stats to work, you will need to run the code snippet above for each of the model repos (basically overwriting them). As the class now inherits from PyTorchModelHubMixin, a config.json along with safetensors weights will be pushed.

Noted, thanks for your contribution again!

@NielsRogge
Copy link
Copy Markdown
Contributor Author

Hi @JulioZhao97 are you planning to run the script above?

Cause I see downloads aren't working yet

@JulioZhao97
Copy link
Copy Markdown
Member

@NielsRogge Hello! I tested the script above at the time you submit the PR and I find

model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

requires yolov10n.pt, and the model weights I upload to huggingface reposity have not been re-uploaded. But other in the script above is fine. I will see to it in these two days.

@JulioZhao97
Copy link
Copy Markdown
Member

@NielsRogge Hello! I tried model.push_to_hub but no model weights found in the repository, could you please tell me where is wrong?
image

Follow is the target repository:
image

@NielsRogge
Copy link
Copy Markdown
Contributor Author

@JulioZhao97 JulioZhao97 mentioned this pull request Nov 13, 2024
@agombert
Copy link
Copy Markdown

Hey @JulioZhao97 how did you solve the pb to push to the hub ? Do you need to download the repo to do so as in Niels example ?

@NielsRogge
Copy link
Copy Markdown
Contributor Author

@agombert which error are you encountering?

@agombert
Copy link
Copy Markdown

@NielsRogge thanks for the quick answer

model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[91], [line 1](vscode-notebook-cell:?execution_count=91&line=1)
----> [1](vscode-notebook-cell:?execution_count=91&line=1) model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

AttributeError: type object 'YOLOv10' has no attribute 'from_pretrained'

It happens when I install doclayout-yolo = "^0.0.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants