Test results of ./build-cuda/bin/test-backend-ops -o GET_REL_POS:

@AgainstEntropy Please take a look at the error message for the failures and feel free to tell me what I can help you.

@AgainstEntropy Please take a look at the error message for the failures and feel free to tell me what I can help you.

Seems the error was raised from this line:

I think we should be able to fix it by replacing the current *((T *) dp) = 0; with *((T *) dp) = 0.0;

@AgainstEntropy Please take a look at the error message for the failures and feel free to tell me what I can help you.

Seems the error was raised from this line:

llama.cpp/ggml/src/ggml-cuda/win.cu

Line 141 in 4813779

*((T *) dp) = 0;

I think we should be able to fix it by replacing the current *((T *) dp) = 0; with *((T *) dp) = 0.0;

Hi @bluebread ,I’ve fixed the compilation error in the HIP backend. The CI workflow currently uses ROCm 6.1.2, which doesn’t yet support constructing bfloat16 values from float or double. With the new change it builds successfully on my machine and passes test-backend-ops -o WIN_PART

I also noticed that you’re using existing ggml ops to construct win_part, win_unpart, and get_rel_pos.
Could you let me know if you still plan to continue with ggml-org#17383 ? If there’s anything I can help with to move it forward, I’d be happy to assist.

@AgainstEntropy Please take a look at the error message for the failures and feel free to tell me what I can help you.

Seems the error was raised from this line:

llama.cpp/ggml/src/ggml-cuda/win.cu

Line 141 in 4813779

*((T *) dp) = 0;

I think we should be able to fix it by replacing the current *((T *) dp) = 0; with *((T *) dp) = 0.0;

Hi @bluebread ,I’ve fixed the compilation error in the HIP backend. The CI workflow currently uses ROCm 6.1.2, which doesn’t yet support constructing bfloat16 values from float or double. With the new change it builds successfully on my machine and passes test-backend-ops -o WIN_PART

I also noticed that you’re using existing ggml ops to construct win_part, win_unpart, and get_rel_pos.

Could you let me know if you still plan to continue with ggml-org#17383 ? If there’s anything I can help with to move it forward, I’d be happy to assist.

@AgainstEntropy To be honest, I don't know if we should continue with that PR or not. I'm still discussing what is the best way to implement the window attention with the maintainer. It is actually inefficient to construct it through existing ggml ops despite the convenience. So I guess there is still a chance for ggml_win_part & ggml_win_unpart being accepted. I really appreciate your help on the PR but it really needs some time for us to make decisions. Sorry.

@AgainstEntropy To be honest, I don't know if we should continue with that PR or not. I'm still discussing what is the best way to implement the window attention with the maintainer. It is actually inefficient to construct it through existing ggml ops despite the convenience. So I guess there is still a chance for ggml_win_part & ggml_win_unpart being accepted. I really appreciate your help on the PR but it really needs some time for us to make decisions. Sorry.

I’m always happy to contribute, so no need to apologize at all🤣! I agree with the inefficiency, and the same for ggml_get_rel_pos.
Just lmk if anything I can help with as you work out the best solution.