WebSocket Connection Closed After Short Inactivity in Realtime API

[hf75]

OpenAIRealtimeChat.zip

Service

OpenAI

Describe the bug

I am using the OpenAI Realtime API to develop a small WPF chat application that exchanges voice messages in a ping/pong manner (with ConversationTurnDetectionOptions.CreateDisabledTurnDetectionOptions()). The voice messages are successfully transmitted/received, but I am encountering an issue where the WebSocket connection is closed automatically after a few seconds of inactivity (lets say 5 to 10 seconds). When I conduct a chat with quickly consecutive short voice messages, I can chat for as long as I want.

What I can see in the API is that in AsyncWebsocketMessageResultEnumerator::MoveNextAsync
receiveResult.CloseStatus.HasValue is true and the closeStatus is InternalServerError with
receiveResult.CloseStatusDescription is "keepalive ping timeout".

So the webSocket is closed and the AsyncWebsocketMessageResultEnumerator is disposed.

Is this expected behaviour? If so, what is the expected way to reestablich the connection?

Steps to reproduce

await session.SendAudioAsync(recording);

// Do something for a couple of seconds, e. g. play the BinaryData result in an audio player etc.

// might fail because the internal webSocket is disposed.
await session.SendAudioAsync(recording);

Code snippets

In the attached code you only need to insert your key in the method MainWindow::InitOpenAI(). Then it should compile and run. Please note that I do not use the nugget package for the OpenAI-API but compile it myself (also included in the solution). I need to do that to fix #268.

OS

Win11

.NET version

8

Library version

v2.1.0-beta.1

[jkears]

Are you able to show your code?

[hf75]

Turns out it was a user error. Works like a charm now after fixing some bugs.

I attached the code. Just insert your key in the method MainWindow::InitOpenAI() and it should compile and run. I do not use the OpenAI-API nugget package but the source which is also included in the solution. I need to do this to fix #268. I hope it does not turn out to be complete incompetence on my side :)