Repository navigation
Description
Warning
Before submitting a PR, please make sure that:
- A maintainer has triaged this issue and applied the
readylabel - This issue has no assignee
- No duplicate PR exists
PRs not meeting these requirements may be automatically closed.
Issues Policy acknowledgement
- I have read and agree to submit bug reports in accordance with the issues policy
Where did you encounter this bug?
Local machine
MLflow version
Source checkout at master, 3.16.1.dev0
commit 7b33d82
System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)OS: Windows 11 (Windows-11-10.0.26200-SP0)
Python: 3.13.15
Installed from: source
git config core.symlinks: false (the default for a non-elevated Windows user)
setuptools: per libs/skinny's own build-system requirement (<=82.0.1)
Shell: non-elevated, Developer Mode off: - Python version:
Describe the problem
libs/skinny/ and libs/tracing/ reach the package source through symlinks committed to git — git ls-files -s reports mode 120000:
libs/skinny/mlflow -> ../../mlflow
libs/tracing/mlflow -> ../../mlflow
On Windows git sets core.symlinks=false unless the user is elevated or has Developer Mode enabled, and materialises those entries as regular text files containing the link target:
type libs\skinny\mlflow
../../mlflow
Both packages then discover their modules with:
[tool.setuptools.packages.find]
where = ["."]
include = ["mlflow", "mlflow.*"]
libs/skinny/mlflow is a file, not a package directory, so discovery finds nothing:
find_packages(where="libs/skinny", include=["mlflow", "mlflow.*"])
[]
Expected: the build fails, or resolves the real package tree.
Actual: the build exits 0 and produces a wheel containing no Python modules at all. The wheel still declares the mlflow console entry point, so installing it yields a working-looking install whose CLI cannot import anything.
Published PyPI artifacts are unaffected — they are built on Linux. The impact is on anyone building or installing from source on Windows.
Why CI does not catch it
.github/workflows/build-wheel.yml builds the skinny and tracing variants on runs-on: ubuntu-latest, where symlinks resolve normally. No job builds either variant on Windows, so this is invisible to CI by construction — it is not a gap that a green Windows job would close, because the wheel build never runs there at all.
Note on a recent migration
These two packages moved from setup.py to a generated pyproject.toml, and the behaviour carried straight through the migration: [tool.setuptools.packages.find] with where = ["."] fails exactly the way the previous build did.
Tracking information
Not applicable — this is a build/packaging defect, not a tracking one.
Code to reproduce issue
On Windows, as a normal (non-elevated) user with Developer Mode off:
git clone https://github.com/mlflow/mlflow
cd mlflow\libs\skinny
python -c "from setuptools import build_meta; print(build_meta.build_wheel('dist'))"
python -c "import zipfile,glob; w=glob.glob('dist/*.whl')[0]; n=zipfile.ZipFile(w).namelist(); print(len([x for x in n if x.endswith('.py')]), 'py files')"
Prints 0 py files. pip install ./libs/skinny reproduces the same way, as does libs/tracing.
Stack trace
There is no stack trace, and that is the defect being reported: the build does not raise, it exits 0 and silently produces an empty wheel. The closest thing to a traceback is the build output and the resulting wheel contents:
built: mlflow_skinny-3.16.1.dev0-py3-none-any.whl
entries: 6 | .py files: 0
mlflow_skinny-3.16.1.dev0.dist-info/licenses/LICENSE.txt
mlflow_skinny-3.16.1.dev0.dist-info/METADATA
mlflow_skinny-3.16.1.dev0.dist-info/WHEEL
mlflow_skinny-3.16.1.dev0.dist-info/entry_points.txt
mlflow_skinny-3.16.1.dev0.dist-info/top_level.txt
mlflow_skinny-3.16.1.dev0.dist-info/RECORD
A failing import after installing that wheel:
import mlflow
ModuleNotFoundError: No module named 'mlflow'
Other info / logs
Suggested fixes
A — fail loudly instead of silently. The empty wheel is the real defect; a build that reports success while producing nothing is worse than one that stops. A small setup.py beside each pyproject.toml asserting the linked source is present:
from pathlib import Path
from setuptools import setup
if not (Path(file).parent / "mlflow").is_dir():
raise SystemExit(
"libs/skinny/mlflow is not a directory, so this build would produce a wheel "
"with no code in it.\n"
"On Windows, git materialises symlinks as text files unless you enable "
"Developer Mode\n"
"or clone with: git -c core.symlinks=true clone https://github.com/mlflow/mlflow"
)
setup()
setuptools still executes setup.py when present alongside PEP 621 metadata, so the guard runs at build time without changing any packaging config. Note pyproject.toml itself is auto-generated by dev/pyproject.py and marked "do not edit manually" — this approach deliberately avoids touching it.
B — remove the dependence on symlinks, e.g. pointing setuptools at the real tree. Better outcome, larger change, needs care with MANIFEST.in and package-data globs.
C — document it. A contributor-docs line about Developer Mode or core.symlinks=true. Cheapest, leaves the silent failure in place.
I would suggest A regardless of whether B or C is also adopted — no build should report success while producing an empty wheel. Happy to submit A as a self-contained PR once this is triaged and labelled ready.
Willingness to contribute
Yes — willing to contribute a fix.
What component(s) does this bug affect?
None of the ten listed components fit (this is build/packaging; there is no area/build or area/packaging option). Leave unchecked and let a maintainer route it.
Willingness to contribute
Yes. I can contribute a fix for this bug independently.
What component(s) does this bug affect?
-
area/tracking: Tracking Service, tracking client APIs, autologging -
area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry -
area/scoring: MLflow model serving, deployment tools, Spark UDFs -
area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows -
area/prompt: MLflow prompt engineering features, prompt templates, and prompt management -
area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality -
area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations -
area/projects: MLproject format, project running backends -
area/uiux: Front-end, user experience, plotting -
area/docs: MLflow documentation pages
Activity
siva23367 commentedon Sep 7, 2026
I want to work on this issue , could you please assign this issue to me @iam-kira
kriscon-db commentedon Sep 8, 2026
Confirmed.
libs/skinny/mlflowandlibs/tracing/mlflow(plusLICENSE.txt/MANIFEST.in) are committed as git symlinks (mode 120000). On a Windows checkout without symlink privilege (core.symlinks=false, the default for a non-elevated user), git materializes them as plain text files containing the link target rather than real links, so the packaged source is missing andmlflow-skinnybuilds an empty wheel. Thanks for the precise repro.Marked
ready. @siva23367 we don't assign issues on request, but this is now triaged and open to contributions - feel free to open a PR following the contributing guide.-- Claude-drafted, reviewed before posting.
siddharthgaur1 commentedon Sep 8, 2026
@siva23367 — you said on 09-07 you wanted this one, and @kriscon-db noted issues aren't assigned here. I happen to be on the exact reported setup (Windows 11, non-elevated, Developer Mode off,
core.symlinks=false), so I've verified the repro and a fix end to end and opened a PR. If you already have work in progress, say so and I'll close mine — I'd rather not duplicate your effort.Reproduced on a fresh
--depth=1clone at master:git ls-files -sreports mode120000forlibs/skinny/mlflowandlibs/tracing/mlflow; on disk each is a 12-byte regular file containing../../mlflow.find_packages(where='libs/skinny', include=['mlflow','mlflow.*'])returns[].build_meta.build_wheel('dist')exits 0 and produces a 6-entry wheel with 0 .py files, still declaring themlflowconsole entry point.I went with your option A (fail loudly), since
pyproject.tomlis generated bydev/pyproject.pyand marked do-not-edit. What I could add is evidence that it costs the working build nothing: with a junction standing in for the resolved symlink, the wheel built with the guard and the wheel built without it have identical file lists (1190 entries, 1173 .py files), and metadata still comes frompyproject.toml(Name: mlflow-skinny,Version: 3.16.1.dev0). Sosetup.pybeside PEP 621 metadata runs the check without taking over packaging.