ref #249 #95

See #407 (comment)

In the mul_mat() implementation currently we have 2 main branches:

• src0 is contiguous in memory (code)
• src0 is not contiguous in memory (code)

In the first branch we parallelize the computation along the src0 rows. Each thread computes a dot product of src0 row with src1 column and writes the result into a cell of dst.

In the second branch we parallelize along the src1 columns. Each thread computes multiply + add (mad) of a src0 column with an element from src1 and writes the result into a per-thread temporary buffer row. At the end of the multiplication, the results from all temporary buffers are accumulated into dst.

The second branch produces variation in the final result based on the used number of threads, since the result into a single dst cell is computed by adding different number of floating point terms, based on the used number of threads. It is a bit more efficient, but also uses a lot more memory due to the temporary buffers.

I am thinking that in view of having more stable results and also simplifying significantly the code in ggml.c, we should eliminate this second branch. The solution is to always make sure that src0 is contiguous, which the user can always achieve with a simple ggml_cpy() call.

The benefits are quite a lot:

• no need to maintain ggml_vec_mad_xxx() functions - can be simply deleted
• reproducible results for different number of threads
• simpler ggml_forward_mul_mat_xxx() implementations
• less memory usage

The drawbacks:

• very slight performance degradation