Skip to content

Commit 93c0065

Browse files
Merge current harness base
2 parents ab01182 + 222cb44 commit 93c0065

49 files changed

Lines changed: 2159 additions & 177 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/codex-python-qualify.yml‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,7 @@ jobs:
216216
"$python" -I -B -m pytest -q -o "pythonpath=$stage/packages/codex" -o asyncio_mode=auto \
217217
packages/codex/tests/test_canonical_storage.py packages/codex/tests/test_canonical_storage_example.py \
218218
packages/codex/tests/test_custom_storage_durability.py \
219+
packages/codex/tests/test_checkpoint_upgrades.py \
219220
packages/codex/tests/test_native_local_descendant_receipts.py::test_local_descendant_admission_checks_receipts_without_resuming_work \
220221
packages/codex/tests/test_native_local_descendant_receipts.py::test_local_reopen_refuses_cold_active_child_without_tool_receipt
221222
if [[ "$RUNNER_OS" == "Linux" ]]; then
@@ -288,9 +289,10 @@ jobs:
288289
""".split())
289290
env = {key: value for key, value in os.environ.items() if key.upper() in keep}
290291
subprocess.run(
291-
[stage / "venv/Scripts/python.exe", "-I", "-B", "-c",
292-
"import os, sys, pytest; sys.path.insert(0, os.path.join(os.getcwd(), 'packages/codex')); "
293-
"sys.exit(pytest.main(['-q', 'packages/codex/tests/qualification/unix_peer_approval.py']))"],
292+
[stage / "venv/Scripts/python.exe", "-I", "-B", "-u", "-c",
293+
"import faulthandler, os, sys; faulthandler.dump_traceback_later(45); "
294+
"import pytest; sys.path.insert(0, os.path.join(os.getcwd(), 'packages/codex')); "
295+
"sys.exit(pytest.main(['-vv', '-s', 'packages/codex/tests/qualification/unix_peer_approval.py']))"],
294296
cwd=stage, env=env, check=True, timeout=90,
295297
)
296298
subprocess.run(

‎examples/codex/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ These programs use the current SDK and model credentials:
4545
| --- | --- | --- |
4646
| Own native Session storage | [Application-owned storage](canonical_storage/README.md) | A persistent POSIX reference adapter, writer ownership, and verified tool-result recovery. |
4747
| Test application behavior | [Task recipe chooser](../../docs/sdk/recipes.md) | Typed tools, request inspection, concurrency, safe effects, specialists, local MCP, images, retrieval, and application memory. Each entry identifies scripted or live execution. |
48+
| Read or revise selected text files | [Selected documents](selected_documents.md) | Bounded pages and search, application-selected writable aliases, and serialized revision checks. The application extracts PDF text. |
4849
| Edit a sampling request after a tool result | [prepared_request.py](recipes/prepared_request.py) | Synthetic Responses HTTP; requires a development build newer than alpha.6. Follow [matching-source setup](../../docs/sdk/installation.md#examples-and-upgrades) and [request-editing limits](../../docs/sdk/configuration.md#edit-the-next-sampling-request). |
4950
| Build a browser application | [Service and browser starter](web_starter/README.md) | Request ownership, approvals, bounded admission, typed results, and authorized artifact downloads. Install its pinned browser dependencies. |
5051
| Stream Vercel UI frames | [vercel_stream.py](vercel_stream.py) | The application supplies its HTTP response and owns the Session. |

‎examples/codex/a2a/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@ Known receipts support observation without creating another task. Lost create, i
4141

4242
`server.py` composes `NativeA2AExecutor` with official routes and an application-provided `NativeA2AStore`. The host supplies authentication middleware and a `ServerCallContextBuilder` that includes the authenticated user and the incoming `a2a-version` header in `state['headers']`. The host factory receives the trusted principal/task/context binding and current official `ServerCallContext`. It returns current `NativeA2AAgent` configuration and application context. Filter incoming trace headers from that call context and supply the resulting `parent_trace`; a fresh worker supplies a fresh parent. It must not reconstruct authority from peer metadata or checkpoint contents.
4343

44+
With the current source build, set `NativeA2AAgent.environment` to a current `Environment` or an `EnvironmentSource` managed async context manager. The host factory runs again for every execution and resume. Return a fresh managed context manager each time. The current host configuration supplies executor tools, write and network policy, and existing Environment approval or result callbacks. Peer input and checkpoint metadata supply none of that authority. Restore still requires a matching saved executor policy.
45+
46+
The adapter retains ownership of Session state, cancellation, and publication. The factory has no general Session or Turn options bag. Root `before_step` and `prepare_request` callbacks and portable checkpoints with an active `RunBudget` are unsupported by this adapter profile. Use direct Agent or Session calls when those controls are required.
47+
4448
Data parts must preserve JSON values exactly. The executor fails a task if encoding its result or approval proposal would round a number. The job provider rejects such replies before reserving an action. Use strings for large integer identifiers that must retain every digit.
4549

4650
Each A2A task owns one native Session. Reusing a conversation context ID does not combine Sessions. Core owns model requests, tools and accepted history. Tool progress is passive; successful final artifacts follow validated `get_result()` and joined Session cleanup. Disconnecting an SSE observer leaves remote work running. Explicit CancelTask joins owned native work and reports settled state; it does not roll back completed external effects.
Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
"""Bounded tools over application-selected decoded UTF-8 documents."""
2+
3+
from __future__ import annotations
4+
5+
import asyncio
6+
import hashlib
7+
import os
8+
import stat
9+
import tempfile
10+
from collections.abc import Callable, Mapping, Sequence
11+
from contextlib import suppress
12+
from dataclasses import dataclass
13+
from pathlib import Path
14+
from typing import TypeVar
15+
16+
from typing_extensions import TypedDict
17+
18+
from agents import ToolOutput, function_tool
19+
20+
PAGE_CHARS = 4096
21+
MAX_FILE_BYTES = 16 * 1024 * 1024
22+
T = TypeVar("T")
23+
24+
25+
class RevisionConflict(ValueError):
26+
"""The caller must read again before continuing or editing."""
27+
28+
29+
class DocumentPage(TypedDict):
30+
document: str
31+
revision: str
32+
offset: int
33+
text: str
34+
next_offset: int | None
35+
36+
37+
class SearchMatch(TypedDict):
38+
offset: int
39+
text: str
40+
41+
42+
class SearchPage(TypedDict):
43+
document: str
44+
revision: str
45+
matches: list[SearchMatch]
46+
next_offset: int | None
47+
48+
49+
class SavedDocument(TypedDict):
50+
document: str
51+
revision: str
52+
status: str
53+
54+
55+
@dataclass(frozen=True)
56+
class _Document:
57+
path: Path
58+
text: str
59+
mode: int
60+
revision: str
61+
62+
63+
class SelectedDocuments:
64+
"""The application chooses aliases, paths, and permission to write.
65+
66+
Supply decoded text, including text extracted from a PDF. Keep the root
67+
and its parent directories under application control. External writers
68+
must share an application lock with this owner.
69+
"""
70+
71+
def __init__(
72+
self,
73+
*,
74+
root: Path,
75+
selected: Mapping[str, str],
76+
writable: Sequence[str] = (),
77+
) -> None:
78+
self._root = root.resolve(strict=True)
79+
self._selected = dict(selected)
80+
self._writable = frozenset(writable)
81+
if not self._writable.issubset(self._selected):
82+
raise ValueError("Writable documents must be selected by the application")
83+
self._lock = asyncio.Lock()
84+
85+
async def _exclusive(self, operation: Callable[[], T]) -> T:
86+
async with self._lock:
87+
worker = asyncio.create_task(asyncio.to_thread(operation))
88+
try:
89+
return await asyncio.shield(worker)
90+
except asyncio.CancelledError:
91+
# Join file work before releasing ownership, even after repeated cancellation.
92+
while not worker.done():
93+
with suppress(asyncio.CancelledError):
94+
await asyncio.wait((worker,))
95+
worker.exception()
96+
raise
97+
98+
def _load(self, alias: str) -> _Document:
99+
if alias not in self._selected:
100+
raise ValueError("The application did not select this document")
101+
path = Path(os.path.abspath(self._root / self._selected[alias]))
102+
if (
103+
path == self._root
104+
or not path.is_relative_to(self._root)
105+
or path.resolve(strict=True) != path
106+
):
107+
raise ValueError("Selected document must remain at its selected path")
108+
flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
109+
with os.fdopen(os.open(path, flags), "rb") as file:
110+
details = os.fstat(file.fileno())
111+
if not stat.S_ISREG(details.st_mode) or details.st_size > MAX_FILE_BYTES:
112+
raise ValueError("Select a regular text file of at most 16 MiB")
113+
content = file.read(MAX_FILE_BYTES + 1)
114+
if len(content) > MAX_FILE_BYTES:
115+
raise ValueError("Selected document grew beyond 16 MiB")
116+
return _Document(
117+
path,
118+
content.decode("utf-8"),
119+
stat.S_IMODE(details.st_mode),
120+
hashlib.sha256(content).hexdigest(),
121+
)
122+
123+
async def read(self, alias: str, offset: int = 0, revision: str | None = None) -> DocumentPage:
124+
if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0:
125+
raise ValueError("Offset must be a nonnegative integer")
126+
127+
def read_page() -> DocumentPage:
128+
document = self._load(alias)
129+
if revision is not None and revision != document.revision:
130+
raise RevisionConflict("Stale revision; restart the read")
131+
end = offset + PAGE_CHARS
132+
return {
133+
"document": alias,
134+
"revision": document.revision,
135+
"offset": offset,
136+
"text": document.text[offset:end],
137+
"next_offset": end if end < len(document.text) else None,
138+
}
139+
140+
return await self._exclusive(read_page)
141+
142+
async def search(
143+
self, alias: str, query: str, offset: int = 0, revision: str | None = None
144+
) -> SearchPage:
145+
if not 1 <= len(query) <= 256:
146+
raise ValueError("Search text must contain 1 to 256 code points")
147+
page = await self.read(alias, offset, revision)
148+
matches: list[SearchMatch] = []
149+
cursor = 0
150+
while cursor <= len(page["text"]) - len(query) and len(matches) < 16:
151+
found = page["text"].find(query, cursor)
152+
if found < 0:
153+
break
154+
matches.append({"offset": offset + found, "text": page["text"][found : found + 128]})
155+
cursor = found + len(query)
156+
next_offset = (
157+
offset + cursor
158+
if len(matches) == 16
159+
else None
160+
if page["next_offset"] is None
161+
else offset + len(page["text"]) - len(query) + 1
162+
)
163+
return {
164+
"document": alias,
165+
"revision": page["revision"],
166+
"matches": matches,
167+
"next_offset": next_offset,
168+
}
169+
170+
async def revise(
171+
self, alias: str, revision: str, old_text: str, new_text: str
172+
) -> SavedDocument:
173+
if alias not in self._writable:
174+
raise ValueError("The application did not select this document for writing")
175+
if not old_text or len(old_text) > PAGE_CHARS or len(new_text) > PAGE_CHARS:
176+
raise ValueError("Select one nonempty edit of at most one page")
177+
178+
def save() -> SavedDocument:
179+
before = self._load(alias)
180+
if before.revision != revision:
181+
raise RevisionConflict("Stale revision; read before editing")
182+
first = before.text.find(old_text)
183+
if first < 0 or before.text.find(old_text, first + 1) >= 0:
184+
raise ValueError("Edit text must occur exactly once")
185+
after = before.text[:first] + new_text + before.text[first + len(old_text) :]
186+
content = after.encode("utf-8")
187+
if len(content) > MAX_FILE_BYTES:
188+
raise ValueError("Edited document exceeds 16 MiB")
189+
descriptor, name = tempfile.mkstemp(prefix=".agents-document-", dir=before.path.parent)
190+
temporary = Path(name)
191+
try:
192+
with os.fdopen(descriptor, "wb") as file:
193+
file.write(content)
194+
file.flush()
195+
os.fsync(file.fileno())
196+
os.chmod(temporary, before.mode)
197+
if self._load(alias).revision != before.revision:
198+
raise RevisionConflict("Document changed before the write")
199+
os.replace(temporary, before.path)
200+
saved = self._load(alias)
201+
if saved.text != after:
202+
raise OSError("Artifact verification failed")
203+
return {
204+
"document": alias,
205+
"revision": saved.revision,
206+
"status": "saved and reopened",
207+
}
208+
finally:
209+
temporary.unlink(missing_ok=True)
210+
211+
return await self._exclusive(save)
212+
213+
def tools(self):
214+
"""Expose aliases and bounded operations through ordinary function tools."""
215+
216+
@function_tool
217+
async def read_document(
218+
document: str, offset: int, revision: str | None
219+
) -> DocumentPage | ToolOutput:
220+
"""Read a selected page; continue with its next_offset and revision."""
221+
try:
222+
return await self.read(document, offset, revision)
223+
except RevisionConflict:
224+
return _revision_retry()
225+
226+
@function_tool
227+
async def search_document(
228+
document: str, query: str, offset: int, revision: str | None
229+
) -> SearchPage | ToolOutput:
230+
"""Search one selected page; follow next_offset until it is null."""
231+
try:
232+
return await self.search(document, query, offset, revision)
233+
except RevisionConflict:
234+
return _revision_retry()
235+
236+
@function_tool
237+
async def revise_document(
238+
document: str, revision: str, old_text: str, new_text: str
239+
) -> SavedDocument | ToolOutput:
240+
"""Replace unique text in a writable document using its current revision."""
241+
try:
242+
return await self.revise(document, revision, old_text, new_text)
243+
except RevisionConflict:
244+
return _revision_retry()
245+
246+
return [read_document, search_document, revise_document]
247+
248+
249+
def _revision_retry() -> ToolOutput:
250+
return ToolOutput.error(
251+
"The document changed. Restart read_document at offset 0 with revision null, "
252+
"then use the returned revision for later reads or edits."
253+
)
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Read and revise selected documents
2+
3+
Give an Agent access to specific decoded UTF-8 files by assigning each file an alias. The application chooses the trusted root, paths, and writable aliases. The model uses those aliases in tool calls.
4+
5+
## Run the example
6+
7+
Install the SDK preview wheel and use the source archive from the same release. Follow the [installation guide](../../docs/sdk/installation.md) and configure `OPENAI_API_KEY` through your usual credential setup. Live model requests can incur charges. From the checkout root, run:
8+
9+
```sh
10+
PYTHONPATH=. python -m examples.codex.selected_documents
11+
```
12+
13+
The [example](selected_documents.py) creates a temporary text file, exposes it as `notes`, and asks the Agent to revise its status. It prints the answer and the status line read back from disk, then removes the temporary directory.
14+
15+
## Select files and connect tools
16+
17+
Import [`SelectedDocuments`](document_workspace.py) and supply `root`, a `selected` mapping from aliases to paths, and optional `writable` aliases. Pass the helper and an open `NativeProvider` to `document_agent()` in the example. The caller owns the provider and closes it after its Sessions. The helper uses ordinary `function_tool` callbacks; it needs no executor or hooks.
18+
19+
| Tool | What it returns or changes |
20+
| --- | --- |
21+
| `read_document` | Up to 4,096 Unicode code points, a SHA-256 content revision, and `next_offset`. |
22+
| `search_document` | Up to 16 matches of up to 128 code points each. Continue with the same revision and `next_offset` to find phrases across page boundaries. |
23+
| `revise_document` | Replaces one unique occurrence in a writable document. Supply its current revision and up to one page each of old and new text. The helper saves and reopens the file before reporting success. |
24+
25+
Follow the returned offset and revision to continue a read or search. A changed revision requires a fresh read. Stale tool calls return a failed `ToolOutput` so the model can try again. Direct helper methods raise `RevisionConflict`. Missing or ambiguous replacement text fails the run without changing the file. Unexpected filesystem errors also fail the run.
26+
27+
## File and concurrency limits
28+
29+
Supply application-created working copies as regular decoded UTF-8 files up to 16 MiB each. The helper reads each selected file into application memory. Extract text from PDF, Word, or Excel files before selecting it. The helper preserves UTF-8 BOM bytes and newline characters when it edits text. Unknown aliases, disallowed writes, paths outside the selected root, and selected paths that become symlinks are rejected.
30+
31+
The application must control the root and its parent directories. This recipe does not protect against a hostile process changing those directories. The helper serializes its own file operations. If other components write the same files, hold a shared application lock around the complete helper call or Agent run. Every external writer must use that same lock. Keep it held through replacement and verification; the helper's private lock coordinates only its own operations.
32+
33+
Each edit writes a temporary file next to the selected file and checks the revision again. It then replaces the original file and reopens it to verify the result. Replacement preserves text and permission bits, but not ownership, access-control lists (ACLs), extended attributes, or security labels. Select only working copies whose security permits replacement under the parent directory's policy. Use an application editor that preserves those properties when editing files that require them.
34+
35+
Cancellation waits for an in-flight file operation to finish before releasing the helper's lock. An edit may finish even when its caller is cancelled. Read the current revision and saved file before retrying; cancellation does not roll back a saved edit.
36+
37+
The page and match limits bound text returned by each call. Previous results can remain in model context, and JSON adds overhead. This recipe does not enforce a global context budget or search unselected project files.

0 commit comments

Comments
 (0)