Skip to content

Fix/s3 no file#3215

Merged
romain-intel merged 3 commits into
masterfrom
fix/s3_no_file
May 29, 2026
Merged

Fix/s3 no file#3215
romain-intel merged 3 commits into
masterfrom
fix/s3_no_file

Conversation

@romain-intel

Copy link
Copy Markdown
Contributor

PR Type

  • [ X] Bug fix
  • New feature
  • Core Runtime change (higher bar -- see CONTRIBUTING.md)
  • Docs / tooling
  • Refactoring

Summary

Small fix for the handling of .mf_code.

…directory

When a code package is extracted to dest_dir/ and metaflow is loaded via
PYTHONPATH pointing to dest_dir/.mf_code/, get_metaflow_root() returns
dest_dir/.mf_code rather than dest_dir/. The MFCONTENT_MARKER (.mf_install)
is placed at the archive root (dest_dir/), so the lookup missed it, leaving
mfcontent_info=None and bypassing the V1 INFO file mechanism.

Add a fallback in _extract_mfcontent_info() to also check the parent of
get_metaflow_root() when METAFLOW_EXTRACTED_ROOT is not set. This allows
the V1 class to be selected and _get_otherfile_path() to locate
dest_dir/.mf_meta/INFO, triggering the early return in
_get_extension_packages() that skips the implicit namespace package check.
@greptile-apps

greptile-apps Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a targeted fallback path in _extract_mfcontent_info to reliably locate the .mf_install marker file when metaflow is loaded from a code package's .mf_code/ subdirectory.

  • When running inside a .mf_code/ layout, get_metaflow_root() resolves to dest_dir/.mf_code while the marker lives at dest_dir/. The new elif branch checks os.path.dirname(get_metaflow_root()) as a reliable second guess, intentionally bypassing METAFLOW_EXTRACTED_ROOT which can hold a relative path (e.g. ".") that breaks in subprocesses that change their working directory.
  • The change is gated on target_dir == "_local", so it has no effect on non-local (archive-based) lookups.

Confidence Score: 5/5

Safe to merge; the change is a narrow, well-commented fallback that is unreachable unless the primary path lookup already failed.

The fix is a one-level parent-directory check behind an elif guard. It cannot affect callers that supply a non-_local target_dir, cannot affect runs where METAFLOW_EXTRACTED_ROOT already resolves correctly, and has no write-side effects. The only theoretical downside—a spurious .mf_install file existing in an unrelated ancestor directory—is not a realistic concern for this generated marker file.

No files require special attention.

Important Files Changed

Filename Overview
metaflow/packaging_sys/init.py Adds an elif fallback in _extract_mfcontent_info to check the parent of get_metaflow_root() for the .mf_install marker when loading from a .mf_code/ subdirectory with a stale or relative METAFLOW_EXTRACTED_ROOT.

Reviews (1): Last reviewed commit: "Revert a fix that is causing more issues" | Re-trigger Greptile

@romain-intel
romain-intel merged commit cd44dfc into master May 29, 2026
45 checks passed
@romain-intel
romain-intel deleted the fix/s3_no_file branch May 29, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant