Pull Request Summary

• Partial open source of tool_completion
• Adapt batch inference to run tool completion loop (generations -> tool run)
• Add tool config to batch inference API

for notekeeping, some things to fix:

1. vLLM stop sequence does not cut token logprobs so there's extra stop sequence token logprobs
2. tool should not use a hardcoded tokenizer, should use whatever provided together with model weights
3. token log prob not captured for tool output

Test Plan and Usage Guide

some local test
tested running jobs in training cluster
added unit tests
vllm batch image deployed