Skip to content

Enable Qlora scripts on Intel GPUs#219

Merged
artidoro merged 2 commits into
artidoro:mainfrom
abhilash1910:main
Jul 24, 2023
Merged

Enable Qlora scripts on Intel GPUs#219
artidoro merged 2 commits into
artidoro:mainfrom
abhilash1910:main

Conversation

@abhilash1910

Copy link
Copy Markdown
Contributor

Motivation:

  • Thanks for the qlora repo, this PR is for adding intel gpu support for quantized inference in the qlora script.
  • There is an internal progress on B&B enabling for intel gpus which will make the support complete.

@artidoro could you please review . Would appreciate your thoughts on this. Thanks

@artidoro artidoro left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Comment thread qlora.py Outdated
@abhilash1910

Copy link
Copy Markdown
Contributor Author

@artidoro requesting re-review , let me know if any changes needed else can this be merged?

@artidoro

Copy link
Copy Markdown
Owner

So it looks good to me, however, I cannot test the code as I don't have an Intel GPU. Could you let me know how you verified that it works?
@TimDettmers do you have thoughts on adding this to the repo? It could be helpful to people using Intel GPUs and it seems relatively few changes.

@abhilash1910

Copy link
Copy Markdown
Contributor Author

@artidoro @TimDettmers at Intel we are planning to integrate our kernel code for the quantization cu kernels . This is a work in progress internally and after validation I will flag a PR at b&b. The pythonic frontend (here) has relatively easier changes but the core of the task is in the kernel adaptation for our gpu devices which we are integrating.

@artidoro

Copy link
Copy Markdown
Owner

Sounds good! I am happy to merge this PR. Might be useful to add some documentation too.

@artidoro
artidoro merged commit 76d8664 into artidoro:main Jul 24, 2023
LagPixelLOL pushed a commit to LagPixelLOL/qlora that referenced this pull request Feb 8, 2024
Enable Qlora scripts on Intel GPUs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants