The latest (1.28) spec requires emitting an event for every individual message. This is problematic in a variety of ways:
- Messages that are part of the same request are not individual entities: they are only meaningful as part of the overall request being submitted to the LLM, which functionally turns the totality of those messages into a single prompt. The whole set of messages / the whole request should be logged, not each tiny piece of it individually.
- A request might contain dozens or hundreds of messages. Logging each individually spams the listener, which then needs to do work to reconstitute the original request.
- Any sampling being performed also then needs to do such work, as it otherwise might drop portions of a request.
- The same message ends up getting logged on its own many times, each time a new request with the same history is resubmitted (augmented with additional messages). That's easy to understand when the messages are grouped, but when they're logged individually, it's much harder to reason about the duplication.
The latest (1.28) spec requires emitting an event for every individual message. This is problematic in a variety of ways: