When working with LLMs for building Agentic Apps, one thing quickly becomes obvious:

  • You want structured outputs.
  • The model streams tokens.
  • Your code waits for a finished JSON blob.

That mismatch accumulates latency. jsontap closes that gap.

This post explains the design behind jsontap.

JSON is a tree

JSON is not just a string. It is a hierarchical structure.

{
  "user": {
    "name": "Alice",
    "scores": [10, 20, 30],
    "friends": [
      {
        "name": "Bob",
        "email": "[email protected]"
      }
    ]
  }
}

As a tree:

(root)
 └── user
      ├── name -> "Alice"
      ├── scores
           ├── 0 -> 10
           ├── 1 -> 20
           └── 2 -> 30
      └── friends
           ├── 0
              ├── name -> "Bob"
              └── email -> "[email protected]"

Every value in JSON can be addressed by a unique path.

  • /user
  • /user/name
  • /user/scores
  • /user/friends/0
  • /user/friends/0/name

Traditional JSON parsing gives you the whole tree at once. Only then can you walk/access it.

But what if the tree is being built token after token?

The streaming problem

With LLMs, JSON arrives progressively:

{
  "reasoning": "Let me think...",
  "tool-call" ...

If "reasoning" has already materialized in the internal JSON tree, why wait for the rest of the JSON to finish before accessing it?

Standard JSON libraries do not allow this. They require the entire JSON to be parsed before giving you access to any node.

That’s the core problem jsontap solves, using the ijson iterative parser.

ijson

ijson emits a stream of (path, event, value) triples while input bytes are still arriving.

For the JSON example above, you can inspect this with:

pbpaste | uv run python -m ijson.dump -m parse

#:   path,                     name,         value
0:   ,                         start_map,    None
1:   ,                         map_key,      user
2:   user,                     start_map,    None
3:   user,                     map_key,      name
4:   user.name,                string,       Alice
5:   user,                     map_key,      scores
6:   user.scores,              start_array,  None
7:   user.scores.item,         number,       10
8:   user.scores.item,         number,       20
9:   user.scores.item,         number,       30
10:  user.scores,              end_array,    None
11:  user,                     map_key,      friends
12:  user.friends,             start_array,  None
13:  user.friends.item,        start_map,    None
14:  user.friends.item,        map_key,      name
15:  user.friends.item.name,   string,       Bob
16:  user.friends.item,        map_key,      email
17:  user.friends.item.email,  string,       [email protected]
18:  user.friends.item,        end_map,      None
19:  user.friends,             end_array,    None
20:  user,                     end_map,      None
21:  ,                         end_map,      None

This event stream is exactly what jsontap uses: when a path event arrives, the corresponding awaiter can be resolved.

Any path can be awaited

The core abstraction in jsontap is the AsyncJsonNode:

from jsontap import jsontap

root = jsontap(stream)
reasoning = root["reasoning"] # returns an AsyncJsonNode
await reasoning # suspends just until that value is available

Even if the key "reasoning" has not been encountered by the parser yet, this works.

Under the hood, AsyncJsonNode implements the Awaitable and AsyncIterator protocols.

When you write:

node = root["user"]["scores"][1]

You are not indexing into a dict.

You are constructing a new node handle a.k.a AsyncJsonNode pointing at the path:

("user", "scores", 1)

That AsyncJsonNode handle:

  • can be awaited
  • can be iterated (if it is an array)
  • can throw if parsing fails

The wrapper exists to preserve lineage information.

The PathStore

Internally, jsontap does not store a tree in the traditional sense (for simplicity).

It stores a mapping from Path -> PathState in a PathStore.

1) PathState

A PathState contains:

  • a future to be resolved when the value is available
  • the current val
  • completion flags like sealed (mostly for handling arrays)

2) Resolving futures as data arrives

When ijson parses a node:

"answer": 42

jsontap resolves the path:

store.get(("answer",)).future.set_result(42)

If someone previously did:

await root["answer"]
# equivalent to self.store.get(("answer",)).future.__await__()

The waiting coroutine resumes immediately.

3) Supporting progressive array iteration

Arrays are more complex.

You do not just want to wait for the entire array:

friends = await root["friends"]

You want to iterate over the array items as they arrive:

async for friend in root["friends"]:
    name = await friend["name"]

It’s worth discussing this in more detail.

The iterator above isn’t waiting for the array item to fully materialize before the loop body is executed.

Instead, it yields an AsyncJsonNode handle the moment the parser recognizes the start of an array item. At that point, friends[i] might be half-parsed – "name" might exist but "email" is still mid-stream. This is important since items can be deeply nested objects you’d rather not wait to be fully parsed.

However, sometimes you may want to wait for the array item to be fully available. Here’s how you do it:

async for friend in root["friends"]:
    friend = await friend
    print(friend["name"])

Summary

Normally, JSON is treated as static data.

jsontap turns it into a tree of awaitable nodes.

This enables:

  • Lower latency LLM pipelines
  • Early extraction
  • Progressive UI updates

The model keeps generating while your code keeps unfolding.

uv add jsontap

Enjoy!