Skip to content

IndexError when snakefmt parses pyproject.toml (upstream toml bug) #255

@thisIsMikeKane

Description

@thisIsMikeKane

Summary

Running snakefmt on a project whose pyproject.toml contains certain mixed-type arrays crashes with IndexError: list index out of range.

The same symptom was reported in the now-closed issue #112 but never resolved; the original reporter said "the error is gone". After debugging I’ve traced the root cause to the third-party toml library that snakefmt pulls in to parse pyproject.toml.

Steps to reproduce

1. Create a minimal pyproject.toml that triggers the bug

[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "sample-package"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
  "requests>=2.31",
]
[dependency-groups]
test = ["pytest>=8.2"]
dev = [
    "ruff>=0.4",
	# Commenting out this next line hides the issue
    { include-group = "test" },
]

2. Run snakefmt (version 0.10.2 here)

snakefmt

Actual behaviour

Traceback (most recent call last):
  ...
  File ".../site-packages/toml/decoder.py", line 1002, in load_array
    ... = ...  # → IndexError here
IndexError: list index out of range

Expected behaviour

snakefmt should either format the file or, at worst, emit a helpful error message—not crash.

Environment

  • snakefmt 0.11.0
  • Python 3.12.10
  • toml 0.10.2
  • OS – Ubuntu 22.04 (WSL2 on Windows 10)

snakefmt reads pyproject.toml for configuration, which is why the crash occurs before any files are formatted. (PyPI)

Root cause analysis

  • In the failing code path, snakefmt calls toml.load().
  • The error propagates from toml/decoder.py where the parser cannot handle arrays that mix types.
  • Very similar stack traces are discussed in these toml issues:
    • uiri/toml #449 - "Error parsing PEP 735 dependent dependency groups"
    • uiri/toml #444 - "Unable to parse array of mixed types (string + array) - IndexError: list index out of range"
    • uiri/toml #270 - "mixed types in array don't work"

Proposed fix

Because the toml library is effectively unmaintained and Python 3.11+ ships with the standard-library tomllib:

  1. Replace the dependency on toml with tomllib (and tomli as a back-port for < 3.11).
    Pros: no external dependency for modern Pythons, avoids the upstream bug entirely.
    Cons: minimal shim code to keep the interface identical.

Work-around

For now I can prevent the crash by keeping my pyproject.toml free of mixed-type arrays or supplying a minimal "dummy" snakefmt.toml.

Why this matters

  • Prevents unexpected crashes for anyone with PEP 735 dependent dependency groups.
  • Removes an unmaintained dependency.
  • Aligns snakefmt with Python’s standard-library solution going forward.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions