CD
Cloudflare Developers•3w ago
Dave M

Connect via WebSocket connection to OpenAI Realtime API - wss://api.openai.com/v1/realtime

I watched @Craig 's YT video connecting WebRTC to the OpenAI Realtime API. Exciting stuff! However, I tried my own Worker solution and was not as successful. Why? Cloudflare workers does not support NPM WS library. Why can't I just use the Worker-supported WebSocket class? Well, because the OpenAI realtime API requires HTTP header Authorization : Bearer $OAI_API_KEY On OAI Realtime API Docs page, they just added this code to establish vanilla javascript WebSocket connection. Thank the higher powers! I was stuck on this for a week!! 🤪 
/*
Note that in client-side environments like web browsers, we recommend
using WebRTC instead. It is possible, however, to use the standard 
WebSocket interface in browser-like environments like Deno and 
Cloudflare Workers.
*/

const ws = new WebSocket(
  "wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-12-17",
  [
    "realtime",
    // Auth
    "openai-insecure-api-key." + OPENAI_API_KEY, 
    // Optional
    "openai-organization." + OPENAI_ORG_ID,
    "openai-project." + OPENAI_PROJECT_ID,
    // Beta protocol, required
    "openai-beta.realtime-v1"
  ]
);

ws.on("open", function open() {
  console.log("Connected to server.");
});

ws.on("message", function incoming(message) {
  console.log(message.data);
});
/*
Note that in client-side environments like web browsers, we recommend
using WebRTC instead. It is possible, however, to use the standard 
WebSocket interface in browser-like environments like Deno and 
Cloudflare Workers.
*/

const ws = new WebSocket(
  "wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-12-17",
  [
    "realtime",
    // Auth
    "openai-insecure-api-key." + OPENAI_API_KEY, 
    // Optional
    "openai-organization." + OPENAI_ORG_ID,
    "openai-project." + OPENAI_PROJECT_ID,
    // Beta protocol, required
    "openai-beta.realtime-v1"
  ]
);

ws.on("open", function open() {
  console.log("Connected to server.");
});

ws.on("message", function incoming(message) {
  console.log(message.data);
});
Cloudflare Developers
YouTube
Client Side Tool Calling with the OpenAI WebRTC Realtime API
OpenAI recently released the WebRTC interface for their realtime API. It supports tool calling, which means you can use it to do client side JavaScript function calling. This opens up a whole new set of use cases. In this video, Craig shows off how he used Cloudflare Worker to relay the WebRTC call securely. Then he uses function calling to co...
MDN Web Docs
WebSocket - Web APIs | MDN
The WebSocket object provides the API for creating and managing a WebSocket connection to a server, as well as for sending and receiving data on the connection.
1 Reply
Quicksilver
Quicksilver•2w ago
@Dave M Are you having any server 500 errors now?

Did you find this page helpful?