Hi,
Are there plans for supporting the Realtime API in the Python lib? Currently the API Docs refer to javascript only and suggest sending low lever websocket messages.
Also are there plans for integrating Realtime with the Assistant API? Assistant can use code interpreter, file search, that is not available in the Realtime API as of now.
I am asking these so we can plan ahead within our product accordingly.
Hi @VSZM !
So in their Realtime API examples they just use the standard websockets ws library from Node.js. You can try equivalent libraries in Python, e.g.
Not sure if there is plan to have something built-in into openai Python package though. The Realtime is in Beta right now, so we’ll see?
But ultimately if you use lower level ws libs in Python, you will be future-proofed anyway, right?
There is an example here: aoai-realtime-audio-sdk/python/samples/low_level_sample.py at main · Azure-Samples/aoai-realtime-audio-sdk · GitHub
Should it be simple to convert this code to connect directly to OpenAI API, i.e., not using Azure?
You can use it without Azure, yes. I haven’t tried yet though.
I have got a partial hack in python using websockets. It’s going to take a day or two more (given my skills) to hopefully get it working. I have mostly relied on desktop ChatGPT guiding me through converting the JS sample to Python. Then used pydub to assemble the audio chunks back into the response.
But my bigger concern is after all this hacking is done… How can I incorporate the Assistant API? Without RAG, what’s the point?
The only way I can see doing it is to turn my Assistants API implementation into a web service and access it as a tool. Pretty darn roundabout (and days of coding).