The exact sentence from a document returned zero hits
A verbatim match_phrase found nothing because two query paths were dropping a positional gap the Korean analyzer had created.
I trace failures in Korean search, ranking, and ML systems to reproducible tests — then to an upstream fix or a measured decision.
5.5 years in Korean search and question answering at 42Maru · production ML at MetLife · 18 patches merged into Lucene, Elasticsearch, sentence-transformers, and Transformers.
Search correctness
Where a transformation that is correct on its own — a normalizer, a decompounder, a dictionary — still makes the search return the wrong thing.
Identical Korean text produced different search results
A Korean word stored as decomposed Unicode looked normal on screen but bypassed Nori's dictionary and became one unknown token. I traced the silent mismatch to the analyzer boundary, then added a Hangul composition filter that restores the dictionary lookup without adding ICU and preserves offsets for highlighting.
Merged into Apache Lucene · reviewed and approved by a core maintainer (Robert Muir) · #16242
Searching 비급여 (non-covered) returned 급여 (covered)
The default Korean analyzer removes negation prefixes, merging opposite words at index time.
A normalizer turned fullwidth characters into wildcard operators
A keyword normalizer folded fullwidth forms to ASCII after wildcard escaping, so characters that were meant as literals became query operators.
Ranking & retrieval
Learning-to-rank losses and retrieval quality — from a gradient that was quietly wrong to a number a maintainer could measure.
A ranking model trained differently depending on the batch
To train a ranking model quickly, many search queries are scored together in one batch — but each query has a different number of documents, so the shorter lists are filled with empty placeholder slots to make them the same length. Those empty slots were being counted as if they were real documents, so the same query trained differently depending on which other queries happened to share its batch. I excluded the padding from the calculation so the result no longer depends on batch composition.
Merged into sentence-transformers · controlled maintainer benchmark: PListMLE nDCG@10 0.514 → 0.525 · #3827
Two correctness fixes and a scalability fix in v5.6.0
A full similarity-matrix materialization, a missing rank offset in multi-GPU masking, and a sign-dependent relative margin — merged in one release.
ML systems & evaluation
Serving state, dataset validity, and label policy — where the right call is a reproducible test or a measured decision, not a bigger model.
The fraud labels were visible in 48 transaction amounts
Before comparing FDS models, I audited 6.38 million AI-Hub rows. The 4.43M-row electronic-network segment used only 48 transaction amounts, and the range covering 98.97% of its rows contained no positives. A high score could therefore reward recovery of the synthesis template rather than behavior-level fraud detection.
Published as a reproducible audit · benchmark rejected for behavior-model comparison · code and evidence public
Treating unreviewed cases as a third label
Labeling cases that are merely awaiting review as negatives turns past investigation policy into the model's ground truth, so I split "unreviewed" into its own label and measured the selection bias.
Choosing a model when only 0.5% of cases are positive
The investigation team can process about 100 cases a day, which turns abstract model metrics into an actual investigation policy under that capacity limit.
Streaming text changed after it had already been sent
In batched generation, chunks already streamed to the user could change retroactively because the output read from live shared buffers.