Repository navigation
Adopt upstream #5365: identity distance with skip_letters - #292
Conversation
DistanceCalculator._pairwise scored only non-skipped positions but used len(seq1) as the maximum, so gap columns inflated the reported distance: identical sequences with gaps returned > 0 and the identity branch disagreed with the scoring-matrix branch on the same data. Count only non-skipped aligned positions in max_score, matching the set the score is computed over. The default identity path (empty skip_letters) is unchanged. Closes #5332. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 4e894b440b17037b2ddeadc5445a926af47628a2)
(cherry picked from commit 69921cb0094ee534a02116e493dbf4036bed2ff0)
The adopted test uses identical gapped rows, and Align.read drops the columns that are gaps in every row, so it never exercised skipping with a Bio.Align.Alignment. The new test puts a gap in one row only and checks both alignment classes; AC--GT against ACTTGT gave 1/3 before the fix and now gives 0. Also correct the comment above the identity loop, which said no letters were skipped (stale since skip_letters was added in 2017), and cite the upstream issue as biopython/biopython#5332, since #5332 in this repository would be a different number. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…lumns Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The examples said identical rows AC--GT were 0.333 apart "in a MultipleSeqAlignment", which reads as though a Bio.Align.Alignment was spared. It was not: with a third row ACTTGT filling the gap columns, every pair was 0.333 apart in either class. An Alignment escapes only the columns that are gaps in every row, which Align.read drops. ADOPTED.md also gave the wrong reason why the pull request's test never covered an Alignment: it reads only with AlignIO. The column dropping is why #5401's Alignment test passed without the fix. After the rebase onto main, the #5363 adoption already has the author's CONTRIB.rst line, so the two cherry-picked commits leave that file unchanged; say so. Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Codex review (CLI) There is nothing worth fixing in this PR. I checked the identity denominator change, unchanged default behavior, custom skipped letters, all-skipped inputs, both alignment classes, public signatures/keywords, and the NEWS/PR explanation. Both cherry-picked commits preserve the upstream author's name, email and author date; the fix also retains its co-author attribution. Validation on Python 3.12: the scratch-installed PR passes all 71 tests in |
Adopts biopython/biopython#5365 by Angus Barlow, which fixes biopython/biopython#5332. Both upstream commits are cherry-picked with
-x, so they keep their author. Rebase-merge, do not squash.The defect
DistanceCalculatorwith theidentitymodel counted matches only over the columns where neither letter is inskip_letters, but divided bylen(seq1), which counts every column. So with an explicitskip_letters, every skipped column made the distance larger. Withskip_letters=("-", "*"):AC--GT/AC--GTAC--GT/AC--GAACGT/ACGA)AC--GT/ACTTGTThese numbers hold for a
MultipleSeqAlignment. They also hold for aBio.Align.Alignmentwhenever another row has letters in the gap columns, asACTTGTdoes, which is the usual case in a real multiple alignment. AnAlignmentwas spared only the columns that are gaps in every row, becauseAlign.readdrops them. So a two-rowAlignmentof justAC--GTandAC--GTwas already 0.The scoring-matrix branch already left skipped columns out of its total.
skip_letterswas added in 2017 (f25c05203, for biopython/biopython#491) so thatidentitycould skip gaps the way the other models do. That commit changed how matches were counted but not the total, so this looks like an oversight. The defaultidentitymodel skips no letters and gives the same numbers as before. Biopython 1.88 and upstream master give the same wrong numbers as fork main; upstream has not merged the PR.Commits
Rebased onto main at
8f9e0ffec.4e894b440b(upstream): the fix, its regression test, and the author'sCONTRIB.rstline. The only conflict was inNEWS.rst. The upstream hunk added a contributor list to upstream's in-progress 1.89 section, which this tree keeps as upstream history, so I left that section unchanged, as was done for Write PDBIO TER records within 80 columns and with their own serial biopython/biopython#5391. Main already has the sameCONTRIB.rstline in the same place, from the adoption of Fix protein_scale for window 1 and even window sizes biopython/biopython#5363. This commit and the next together leaveCONTRIB.rstunchanged.69921cb009(upstream): moves theCONTRIB.rstline into alphabetical order. On this base, that removes the second copy that the first commit adds.AlignIO, so it never passes aBio.Align.Alignment. Its identical gapped rows would not have helped there, becauseAlign.readdrops the columns that are gaps in every row. The new test puts a gap in one row only and checks both alignment classes. This commit also corrects the comment above the loop (it said no letters were skipped) and cites the issue asbiopython/biopython#5332. Its commit message gives the column-dropping as the reason the PR's test missedAlignment; theADOPTED.mdentry gives the correct reason.Alignmenttoo, the reason the PR's test missedAlignmentis now the correct one, and the entry notes theCONTRIB.rstoverlap.Verified
Bio/Phylo/TreeConstruction.pyreverted to main, exactly the two new tests fail (5 failures, counting subtests). Everything else intest_TreeConstruction.py,test_Consensus.pyandtest_ProtParam.pypasses. With the fix, all pass: 71 tests and 65 subtests.ACTTGT.Bio/Phylo/TreeConstruction.py:test_mmtf_online: the MMTF host does not resolve.test_UniProt: the live response has an extra field.test_KEGG_online: KEGG returned HTTP 400.test_NCBI_qblast: timed out after 300 s.The first three fail the same way on main. I did not run
test_NCBI_qblaston main.pre-commit run --fileson every changed file: all hooks pass.[X]). The PR has no comments or reviews as of 2026-11-09, and DistanceCalculator identity model inflates the distance when skip_letters is used biopython/biopython#5332 is still open.Not checked: other Python versions or platforms (this is a pure-Python change), and a local Sphinx build (rstcheck and doc8 pass on the NEWS text).
Known and left for a separate fix: when a scoring-matrix model such as
blosum62is given aBio.Align.Alignmentwith a letter outside the matrix, it raisesAttributeErrorinstead of the intendedValueError. The error message readsseq1.id, butAlignmentrows are plain strings. Biopython 1.88 and fork main do the same, and this PR does not touch that code.🤖 Generated with Claude Code