Skip to content

add support for Orion-14B#5118

Merged
ggerganov merged 12 commits intoggml-org:masterfrom
sharpHL:Orion-14B-support
Jan 28, 2024
Merged

add support for Orion-14B#5118
ggerganov merged 12 commits intoggml-org:masterfrom
sharpHL:Orion-14B-support

Conversation

@sharpHL
Copy link
Copy Markdown
Contributor

@sharpHL sharpHL commented Jan 24, 2024

@arch-btw
Copy link
Copy Markdown
Contributor

arch-btw commented Jan 25, 2024

Can confirm that it works with https://huggingface.co/OrionStarAI/Orion-14B-Chat/blob/main/Orion-14B-Chat.gguf (converted to Q5_K_M).

Although, it is not clear what the correct prompt format is, -i -ins seems to work.

@sorasoras
Copy link
Copy Markdown

sorasoras commented Jan 25, 2024

Can confirm working on rocm

@LostRuins LostRuins mentioned this pull request Jan 26, 2024
Closed
Tangweirui2021
Tangweirui2021 approved these changes Jan 26, 2024
View reviewed changes
Copy link
Copy Markdown

@Tangweirui2021 Tangweirui2021 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes do can fix the convert problem. And it also enables the model to run correctly.

ggerganov
ggerganov approved these changes Jan 26, 2024
View reviewed changes
Comment thread llama.cpp
Comment thread llama.cpp
Comment thread llama.cpp
Comment thread llama.cpp Outdated
Comment thread llama.cpp
sharpHL
sharpHL commented Jan 27, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor Author

@sharpHL sharpHL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orion-14B-support

sharpHL and others added 5 commits January 27, 2024 19:19
@sharpHL @ggerganov
Update llama.cpp
0185aa7 
Co-authored-by: Georgi Gerganov <[email protected]>
@sharpHL @ggerganov
Update llama.cpp
db44ddf 
Co-authored-by: Georgi Gerganov <[email protected]>
@sharpHL @ggerganov
Update llama.cpp
aac36f9 
Co-authored-by: Georgi Gerganov <[email protected]>
@sharpHL
Merge branch 'ggerganov:master' into Orion-14B-support
220f917
@sharpHL @ggerganov
Update llama.cpp
82f5d56 
Co-authored-by: Georgi Gerganov <[email protected]>
slaren
slaren reviewed Jan 27, 2024
View reviewed changes
Comment thread llama.cpp Outdated
ggerganov
ggerganov reviewed Jan 28, 2024
View reviewed changes
Comment thread llama.cpp
ggerganov
ggerganov reviewed Jan 28, 2024
View reviewed changes
Comment thread llama.cpp
@ggerganov ggerganov merged commit f2e69d2 into ggml-org:master Jan 28, 2024
@prusnak prusnak mentioned this pull request Jan 28, 2024
Closed
@zyxcambridge
Copy link
Copy Markdown

zyxcambridge commented Jan 29, 2024

llm_load_print_meta: BOS token = 1 ''
llm_load_print_meta: EOS token = 2 ''
llm_load_print_meta: UNK token = 0 ''
llm_load_print_meta: PAD token = 0 ''
llm_load_print_meta: LF token = 64 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.34 MiB
ggml_backend_metal_buffer_from_ptr: error: failed to allocate buffer, size = 0.00 MiB
llama_model_load: error loading model: failed to allocate buffer
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model 'Orion-14B-Chat.gguf'
main: error: unable to load model
(base) zhangyixin@zhangyixin llama.cpp %

ggerganov added a commit that referenced this pull request Jan 31, 2024
@ggerganov
llama : reorder build_orion() at correct place (#5118)
d3bac7d
jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Feb 3, 2024
@sharpHL @ggerganov
@slaren @jordankanter
llama : add support for Orion-14B (ggml-org#5118)
1bfa876 
* add support for Orion-14B(https://huggingface.co/OrionStarAI/Orion-14B-Chat)

* flake8 support

* Update llama.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

* Update llama.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

* Update llama.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

* Update llama.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

* Update llama.cpp

Co-authored-by: slaren <[email protected]>

* Update llama.cpp

* Update llama.cpp

---------

Co-authored-by: lixiaopu <[email protected]>
Co-authored-by: Georgi Gerganov <[email protected]>
Co-authored-by: slaren <[email protected]>
jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Feb 3, 2024
@ggerganov @jordankanter
llama : reorder build_orion() at correct place (ggml-org#5118)
3196e58
@sharpHL sharpHL deleted the Orion-14B-support branch April 8, 2025 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

@slaren slaren slaren left review comments

+2 more reviewers

@cebtenzzre cebtenzzre cebtenzzre left review comments

@Tangweirui2021 Tangweirui2021 Tangweirui2021 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@sharpHL @arch-btw @sorasoras @zyxcambridge @ggerganov @slaren @cebtenzzre @Tangweirui2021