Jamii Afya

Inspiration

In much of rural Africa, a community health worker is the only medical presence for an entire village. They triage childhood fever, malaria, diarrhoea, pneumonia and pregnancy danger signs — usually with no doctor on site, unreliable electricity, and no internet.

Cloud AI is useless to them. It needs connectivity they don't have, subscriptions priced in dollars, and it sends patient data somewhere else. Whatever helps has to run on the laptop already in the room.

Jamii Afya ("community health" in Kiswahili) is a fully offline clinical decision-support assistant. It answers in English and Kiswahili, grounds every answer in WHO/IMCI guidelines stored on the device, and always surfaces danger signs and when to refer.

What it does

Ask a clinical question in either language. A BM25 retriever pulls the relevant guidelines from a curated offline corpus, and a fine-tuned 0.6B model answers from that grounding — with the source guidelines shown alongside every answer.

It runs on a commodity 8 GB laptop, CPU only, with zero network calls after the one-time model download.

How we built it

We worked backwards from how the system is actually measured, and let measurement overrule intuition at every step.

• Base model: Qwen3-0.6B-Base (Apache-2.0), chosen because it's the only small commercially-usable family that officially covers Kiswahili. Base rather than Instruct because the evaluator ranks answers by raw loglikelihood with no chat template — a regime where base checkpoints beat instruct-tuned ones.
• Training objective: a custom listwise ranking loss, character-length-normalised to mirror the acc_norm metric exactly, rather than ordinary fine-tuning. Research on this exact model family found gold-only tuning is the worst option below 3B.
• Retrieval: pure-stdlib BM25 over WHO/IMCI guidelines — no embeddings, no vector DB, negligible RAM, and it works for both languages since both are Latin-script.
• Data: public MCQA train splits, open medical corpora (MedQuAD, PubMedQA, medical flashcards), plus bilingual clinical data distilled from our own 32 hand-verified guidelines — the teacher was constrained to elaborate on verified text only, never to invent clinical claims.

Challenges we ran into

The grading machine runs with CPU acceleration disabled. We found this reading the profiler source. Every benchmark from a normal laptop is therefore meaningless. We rebuilt our entire testing process around a no-SIMD build — a detail we suspect sinks teams who benchmark on fast hardware.

Our first plan was wrong. We inherited a 14B design. Running the actual scoring math against real measurements showed it would forfeit roughly half the available score. We rebuilt around 0.6B, and let a head-to-head sweep pick the quantization rather than defaulting to the "obvious" choice.

Our accuracy checker was broken before our model was. It reported near-random results. Instead of accepting "the model is bad," we tested the checker itself against a question with an obvious answer — it ranked "banana" above "Paris" for the capital of France. The library's echoed logprobs are misaligned; we rewrote it to read raw logits.

A weak retrieval match caused unsafe advice. For a pre-eclampsia case, BM25 correctly returned the two right guidelines, then padded the context with "Diabetes Basics" — and the model answered about blood glucose. A weak match is worse than no match, because a small model can't tell which part of its context to ignore. We now drop hits below 45% of the top score, a threshold set by sweeping real queries.

The model ranked brilliantly and wrote terribly. The cause was our own data: it had seen ~93,000 ranking examples against 80 examples of writing an answer. It learned exactly what we taught it. Correcting that ratio fixed the prose.

Accomplishments we're proud of

Measured by the official profiler on a scalar x86 build:

That's 48.53 of 50 banked from speed and efficiency alone.

What we learned

Distrust your instruments before you distrust your model. Two of our biggest "model problems" turned out to be a broken evaluator and a bad retrieval threshold.

And a small model becomes exactly what your data ratio makes it — architecture was never the constraint.

What's next

Kiswahili generation is still our weakest area: the model handles it well when strongly grounded, but degrades on novel long-form output. Three fine-tuning rounds didn't resolve it, which points to a base-capability limit rather than a data shortage. Next steps are grammar-constrained (GBNF) structured output and a larger Kiswahili corpus. Our full limitations are documented openly in REPORT.md §6 — a clinical tool should be judged on what it gets wrong, not just its best numbers.

Built With

• gguf
• llama.cpp
• lora
• peft
• python
• pytorch
• qlora
• qwen3
• transformers