You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 5894ba8
Browse filesBrowse the repository at this point in the historyBrowse files
Add portable random-access file reads to Env (#32503)
## Description
Fixes#32500.
Add `Env::OpenRandomAccessFile()` and an owned `RandomAccessFile`
interface with `GetLength()` and synchronous, thread-safe positional
`Read()` operations. All operations use the same open file identity
rather than reopening its pathname for each range.
- POSIX uses `pread`, retries interrupted operations, handles short
reads, and rejects non-regular files without blocking on FIFOs.
- Windows uses an overlapped disk handle and per-call event/OVERLAPPED
state, handling pending operations and draining outstanding I/O before
returning. Read sharing and delete sharing permit pathname replacement
while retaining the original file identity; write sharing is denied.
- Both implementations validate offsets and range overflow and preserve
output ownership on failed opens.
- Existing `ReadFileIntoBuffer` remains available and unchanged. The new
Env virtual is appended, and its default implementation explicitly
returns NOT_IMPLEMENTED so custom environments are not silently
bypassed.
This is not protection against in-place file modification under POSIX,
or a snapshot across multiple files. Callers must retain the object for
the entire sequence of reads that needs a stable identity and keep it
alive until all readers finish.
## Scope
This PR is based directly on main. It provides the platform abstraction
requested during #32437; migration of that CUDA loader is not included.
Loader lifecycle cleanup is tracked separately in #32502.
## Platform compatibility
Android exposes fortified overloads of `open`, which prevented
`TempFailureRetry` from deducing its callable type and caused all four
failing Android CI jobs to stop at compilation. The call to `open` is
now wrapped in a lambda, preserving the fortified call and EINTR retry
behavior. After the compilation fixes, Android minimal-baseline CI
measured 1,587,406 bytes of ELF sections. The active workflow budget is
now 1,589,248 bytes (a 4 KiB increase), leaving 1,842 bytes of headroom
while keeping the size check enabled.
The concurrent-read tests use `std::thread` with scope-bound joins for
Android and older libc++ implementations that lack `std::jthread`. On
Windows, the atomic replacement test uses `FileRenameInfoEx` with POSIX
replacement semantics because legacy filesystem rename rejects an open
destination; the production write-sharing restriction is unchanged.
## Coverage
Adds platform tests for concurrent reads, file size, invalid ranges, EOF
after a partial read, empty files, failed-open output preservation, the
default unsupported implementation, atomic pathname replacement,
platform-specific truncation behavior, FIFO rejection, and sparse files
beyond 4 GiB on 64-bit POSIX and Windows. Existing path-based reads and
length queries are also exercised.
## Validation
All 12 selected Linux CPU tests passed:
```text
onnxruntime_test_all --gtest_filter="RandomAccessFileTest.*:PlatformEnvTest.*"
```
Windows code and Windows-specific test branches require Windows CI; no
Windows compiler/runtime was available locally.
The local CPU build has contrib ops disabled and encountered an existing
unrelated unused-function warning in nhwc_transformer_test.cc. Only that
translation unit was compiled with -Wno-error=unused-function; no
unrelated source or global warning settings were changed.
---------
Co-authored-by: Copilot <[email protected]>
0 commit comments