Skip to content

Misc. bug: server/rerank output result is wrong with most models include qwen3-Rerank #16407

New issue
New issue
Closed
Closed
Misc. bug: server/rerank output result is wrong with most models include qwen3-Rerank#16407
Labels
bug-unconfirmed

Description

@YannFollet
YannFollet
opened on Oct 3, 2025

Name and Version

llama-server --version

ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
version: 6674 (5113efd3)
built with cc (Ubuntu 12.4.0-2ubuntu1~24.04) 12.4.0 for x86_64-linux-gnu

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

llama-server -m Qwen3-Reranker-8B-q4_k_s.gguf -c 4096 -ngl 99  --host 0.0.0.0 --port 8181 --prio 2 --no-webui -ctk q4_0 -ctv q4_0 -fa auto --rerank

Problem description & steps to reproduce

the reranking give bad result with qwen3-reranker (0.6B 4B 8B), bge, mxbai
llama-server -m Qwen3-Reranker-8B-q4_k_s.gguf -c 4096 -ngl 99 --host 0.0.0.0 --port 8181 --prio 2 --no-webui -ctk q4_0 -ctv q4_0 -fa auto --rerank

I test with this sh script :

#!/bin/bash

# Set default URL if not provided
URL=${1:-http://127.0.0.1:8181}

curl "$URL/v1/rerank" -H "Content-Type: application/json" \
 -d '{ "model": "M", "query": "What is the recipe to make bread ?",
 "return_text" : false,
 "texts" : true,
 "top_n": 6,"documents": [
 "voici la recette pour faire du pain, il faut de la farine de l eau et du levain et du sel",
 "it is a bear",
 "bread recipe : floor, water, yest, salt",
 "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.",
 "here is the ingedients to bake bread : 500g floor, 350g water, 120g fresh refresh yest, 15g salt",
 "recipe to make cookies : floor, eggs, water, chocolat",
 "here is the recipe to make bread : 500g floor, 350g water, 120g fresh refresh yest, 15g salt",
 "il fait tres beau aujourd hui",
 "je n ai pas faim, je ne veux pas manger",
 "je suis a paris"
 ] }' | jq

I alway get result like this :

[
  {
    "index": 5,
    "score": 1.1353239058953662E-28
  },
  {
    "index": 3,
    "score": 3.111864641067425E-29
  },
  {
    "index": 8,
    "score": 2.3408178355156106E-29
  },
  {
    "index": 9,
    "score": 2.6804792039427738E-30
  },
  {
    "index": 1,
    "score": 6.065239931987211E-32
  },
  {
    "index": 2,
    "score": 5.335152733382172E-32
  }
]

the only models that seems works with llama-server is jina-reranker
llama-server -m jina-reranker-v2-base-multilingual-Q8_0.gguf -c 16000 -ngl 99 --host 0.0.0.0 --port 8181 --prio 2 --no-webui -ctk q4_0 -ctv q4_0 -fa auto --rerank
the result is not too bad but is very differant from https://jina.ai/reranker/
result with llama-server :

[
  {
    "index": 6,
    "score": 0.7979143261909485
  },
  {
    "index": 0,
    "score": 0.3886369466781616
  },
  {
    "index": 2,
    "score": 0.2865810990333557
  },
  {
    "index": 4,
    "score": -0.5105927586555481
  },
  {
    "index": 5,
    "score": -1.9573085308074951
  },
  {
    "index": 8,
    "score": -3.1544036865234375
  }
]

result in https://jina.ai/reranker/ same model, same content

[
    {
      "index": 6,
      "relevance_score": 0.69595832
    },
    {
      "index": 0,
      "relevance_score": 0.60346454
    },
    {
      "index": 2,
      "relevance_score": 0.5559175
    },
    {
      "index": 4,
      "relevance_score": 0.3684057
    },
    {
      "index": 5,
      "relevance_score": 0.12085322
    },
    {
      "index": 7,
      "relevance_score": 0.04146227
    }
  ]

First Bad Commit

No response

Relevant log output

Activity
  1. YannFollet
    added 
    bug-unconfirmed
     on Oct 3, 2025
  2. ggerganov commented on Oct 3, 2025 
    @ggerganov
    ggerganov
    on Oct 3, 2025
    Member
    Most likely your GGUF is not converted properly. I just uploaded a correct one:
    

    https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF
    

    You can use it like this:
    

    llama-server -hf ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF -c 4096 -ngl 99 --host 0.0.0.0 --port 8181 --no-webui --rerank
    

    [
  {
    "index": 0,
    "score": 0.999091386795044
  },
  {
    "index": 1,
    "score": 0.00010501546057639644
  },
  {
    "index": 2,
    "score": 0.9930423498153687
  },
  {
    "index": 3,
    "score": 0.0001045576500473544
  },
  {
    "index": 4,
    "score": 0.9962399005889893
  },
  {
    "index": 5,
    "score": 0.059775762259960175
  },
  {
    "index": 6,
    "score": 0.9997950196266174
  },
  {
    "index": 7,
    "score": 0.000041759467421798036
  },
  {
    "index": 8,
    "score": 0.00004209155304124579
  },
  {
    "index": 9,
    "score": 0.00006368845788529143
  }
]

    

    If you want to use parallel processing, use this:
    

    llama-server -hf ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF -c 65536 -np 16 -ngl 99 --host 0.0.0.0 --port 8181 --no-webui --rerank
  3. ggerganov commented on Oct 3, 2025 
    @ggerganov
    ggerganov
    on Oct 3, 2025
    Member
    Regarding the Jina scores: llama-server provides raw scores while the Jina API applies normalization of the scores to map them in the [0, 1] range.
  4. YannFollet commented on Oct 3, 2025 
    @YannFollet
    YannFollet
    on Oct 3, 2025
    ContributorAuthor
    @ggerganov thanks for the info
    
the result you post looks great I will try it next week
    

    I have make a PR to sort and use the top_n
  5. YannFollet commented on Oct 4, 2025 
    @YannFollet
    YannFollet
    on Oct 4, 2025
    ContributorAuthor
    is there a reason that the results are differant with -np 16
    
value and order are differant
    

    without -np 16
    

    [
  {
    "index": 6,
    "score": 0.8931634426116943
  },
  {
    "index": 0,
    "score": 0.7729564905166626
  },
  {
    "index": 2,
    "score": 0.6056272983551025
  },
  {
    "index": 4,
    "score": 0.39953315258026123
  },
  {
    "index": 5,
    "score": 0.20445697009563446
  },
  {
    "index": 1,
    "score": 0.11601262539625168
  },
  {
    "index": 8,
    "score": 0.0948135182261467
  },
  {
    "index": 7,
    "score": 0.017812632024288177
  },
  {
    "index": 9,
    "score": 0.009248962625861168
  },
  {
    "index": 3,
    "score": 0.0010981553932651877
  }
]
    

    with -np 16
    

    [
  {
    "index": 6,
    "score": 0.9517623782157898
  },
  {
    "index": 0,
    "score": 0.9303007125854492
  },
  {
    "index": 2,
    "score": 0.7105339765548706
  },
  {
    "index": 4,
    "score": 0.6309738159179688
  },
  {
    "index": 5,
    "score": 0.47012484073638916
  },
  {
    "index": 1,
    "score": 0.3791893422603607
  },
  {
    "index": 8,
    "score": 0.02607090212404728
  },
  {
    "index": 9,
    "score": 0.014488549903035164
  },
  {
    "index": 7,
    "score": 0.005708911456167698
  },
  {
    "index": 3,
    "score": 0.00030652410350739956
  }
]
  6. ggerganov commented on Oct 6, 2025 
    @ggerganov
    ggerganov
    on Oct 6, 2025
    Member
    

    is there a reason that the results are differant with -np 16
    
value and order are differant
    

    

    The results should be the same (within some rounding errors). I am not able to reproduce - what commands are you using?
  7. YannFollet commented on Oct 7, 2025 
    @YannFollet
    YannFollet
    on Oct 7, 2025
    ContributorAuthor
    llama-server -m qwen3-reranker-0.6b-q8_0.gguf -c 16000 -ngl 99  --host 0.0.0.0 --port 8181 --prio 2 --no-webui -ctk q4_0 -ctv q4_0 -fa auto --rerank -np 16
    

    llama-server -m qwen3-reranker-0.6b-q8_0.gguf -c 16000 -ngl 99  --host 0.0.0.0 --port 8181 --prio 2 --no-webui -ctk q4_0 -ctv q4_0 -fa auto --rerank
    

    is using -ctk q4_0 -ctv q4_0 is bad for embedding / rerank ? should I use q8_0, I try fa on / off, I still have a an order ranking diff in the lowest ranked
  8. ggerganov commented on Oct 7, 2025 
    @ggerganov
    ggerganov
    on Oct 7, 2025
    Member
    In general, not just for embeddings, I don't recommend using -ctk q4_0 -ctv q4_0. Using q8_0 could sometimes be OK, but also not recommended.
    

    Some small variations in the results can be expected with FA on/off.
    

    Note that soon embedding models will not require a KV cache at all, so the -ctk, -ctv arguments will become irrelevant for this use case.
  9. YannFollet
    closed this as completedon Oct 9, 2025
  10. crazyn2 commented on Oct 16, 2025 
    @crazyn2
    crazyn2
    on Oct 16, 2025
    

    Most likely your GGUF is not converted properly. I just uploaded a correct one:
    

    https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF
    

    You can use it like this:
    

    llama-server -hf ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF -c 4096 -ngl 99 --host 0.0.0.0 --port 8181 --no-webui --rerank
    

    [
  {
    "index": 0,
    "score": 0.999091386795044
  },
  {
    "index": 1,
    "score": 0.00010501546057639644
  },
  {
    "index": 2,
    "score": 0.9930423498153687
  },
  {
    "index": 3,
    "score": 0.0001045576500473544
  },
  {
    "index": 4,
    "score": 0.9962399005889893
  },
  {
    "index": 5,
    "score": 0.059775762259960175
  },
  {
    "index": 6,
    "score": 0.9997950196266174
  },
  {
    "index": 7,
    "score": 0.000041759467421798036
  },
  {
    "index": 8,
    "score": 0.00004209155304124579
  },
  {
    "index": 9,
    "score": 0.00006368845788529143
  }
]

    

    If you want to use parallel processing, use this:
    

    llama-server -hf ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF -c 65536 -np 16 -ngl 99 --host 0.0.0.0 --port 8181 --no-webui --rerank
    

    

    Can you provide more type of Qwen3-Reranker-GGUF such as 8B-fp16?
  11. jeffutter commented on Nov 28, 2025 
    @jeffutter
    jeffutter
    on Nov 28, 2025
    👋 Thanks for this. Any chance you could share how your version was converted? I’d love a f16 version of the 0.6b model. I had been trying this one: https://huggingface.co/mradermacher/Qwen3-Reranker-0.6B-GGUF but it was giving whacky results (your q8 seems much more accurate)
  12. heibaidaolx123 commented on Feb 2, 2026 
    @heibaidaolx123
    heibaidaolx123
    on Feb 2, 2026
    Hi, @ggerganov
    
I have the same problem: the ranking from llama.cpp (macos with M1) is not the same as that from vllm (linux with cuda).
    

    Model I use
    

    a fine-tuned bge-reranker-v2-m3, with fp16 precision
    

    The command to convert
    

    python ./convert_hf_to_gguf.py /xxx/bge-reranker-v2-m3_epoch3/ --outtype f16 --outfile /xxx/bge-reranker-v2-m3_epoch3-GGUF/bge-reranker-v2-m3-FP16.gguf

    

    The serving command
    

    With llama.cpp
    

    MODEL_FILE=/xxx/bge-reranker-v2-m3_epoch3-GGUF/bge-reranker-v2-m3-FP16.gguf
MODEL_NAME=bge-reranker-v2-m3
PORT=12345
TIMEOUT=600

llama-server -m $MODEL_FILE --reranking \
 -ub 8192 -np 4 --port $PORT \
 --alias $MODEL_NAME \
 --timeout $TIMEOUT

    

    With vllm
    

    docker run xxxxx/vllm-openai:v0.11.2 \
 .......... \
 --model /model/bge-reranker-v2-m3 \
 --served-model-name $MODEL_NAME \
 --max-num-seqs=16 \
 --max-model-len=8192 \
 --max-num-batched-tokens=32768 \
 --tensor-parallel-size=1 \
 --host=0.0.0.0 \
 --port=9001 \
 --gpu-memory-utilization=0.5 \
 --trust-remote-code \
 --no-enable-prefix-caching

    

    The results
    

    For llama.cpp
    

        {
        "model": "bge-reranker-v2-m3",
        "object": "list",
        "usage": {
            "prompt_tokens": 5120,
            "total_tokens": 5120
        },
        "results": [
            {
                "index": 5,
                "relevance_score": 4.517701148986816
            },
            {
                "index": 9,
                "relevance_score": 4.305874347686768
            },
            {
                "index": 4,
                "relevance_score": 4.3025712966918945
            },
            {
                "index": 7,
                "relevance_score": 4.013558387756348
            },
            {
                "index": 3,
                "relevance_score": 3.952303171157837
            },
            {
                "index": 0,
                "relevance_score": 3.527942180633545
            },
            {
                "index": 1,
                "relevance_score": 3.4603500366210938
            },
            {
                "index": 6,
                "relevance_score": 3.201193332672119
            },
            {
                "index": 8,
                "relevance_score": 2.1809701919555664
            },
            {
                "index": 2,
                "relevance_score": 1.9854843616485596
            }
        ]
    }

    

    For vllm
    

        {
        "model": "bge-reranker-v2-m3-ft",
        "usage": {
            "total_tokens": 5140
        },
        "results": [
            {
                "index": 9,
                "relevance_score": 0.9633170366287231
            },
            {
                "index": 5,
                "relevance_score": 0.9595071077346802
            },
            {
                "index": 4,
                "relevance_score": 0.9585697650909424
            },
            {
                "index": 7,
                "relevance_score": 0.9577140808105469
            },
            {
                "index": 3,
                "relevance_score": 0.957077145576477
            },
            {
                "index": 1,
                "relevance_score": 0.9432849884033203
            },
            {
                "index": 6,
                "relevance_score": 0.9383541345596313
            },
            {
                "index": 0,
                "relevance_score": 0.9130572080612183
            },
            {
                "index": 2,
                "relevance_score": 0.8902527689933777
            },
            {
                "index": 8,
                "relevance_score": 0.8439629077911377
            }
        ]
    }

    

    As the results show, for a 10-doc request, the rankings  are diffrent, and seems ok with just 4 positions shiftted.
    
But when testing with a 100-doc request, the ranking are far more different, like randomly shuffled. The results are too long to paste here.
  13. lucasmcoleman
    added a commit that references this issue on Jul 28, 2026
    Add qwen3-reranker-8b recipe overlay
    3f0449e
  14. GabryeleSantoro
    added a commit that references this issue on Sep 18, 2026
    fix: point fetch-models.sh at a reranker-aware Qwen3-Reranker GGUF
    ccd26ed
  15. OrlandoLuque
    added a commit that references this issue on Sep 25, 2026
    N96: un cross-encoder de verdad, por el motor que ya llevabamos (0.2.…
    ce89285
  16. lucasmcoleman
    added a commit that references this issue on Oct 22, 2026
    Add qwen3-reranker-8b recipe overlay
    e66dbca
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Metadata
Assignees
No one assigned
    Labels
    bug-unconfirmed
    Type
    No type
    Projects
    No projects
      Milestone
      No milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions