Skip to content

Add support for BLIP and GIT in image-to-text and VQA pipelines #21110

Description

@NielsRogge

Feature request

BLIP and GIT are 2 recent additions in the library, providing state-of-the-art performance for tasks like image captioning and visual question answering (VQA). GIT is even capable of video captioning and video QA.

Hence it makes sense to support them in our image-to-text and VQA pipelines.

Motivation

Having support for better models in pipelines is very desired!

See also a request for it here: https://discuss.huggingface.co/t/support-for-different-models-in-text-to-image-pipeline/29504

Your contribution

I can assist in adding support, see #18446 as a very similar case

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions