Streaming an AI Tutor Over WebSockets in Laravel

When we added an AI conversation tutor to Endaca, the first version felt wrong. The student typed a sentence, then stared at a spinner while the whole reply was generated. Three seconds of silence is an eternity in a chat.

The fix was streaming — but with a twist. Most streaming tutorials assume Server-Sent Events straight from your API. Our frontend already had a WebSocket connection for everything else, so I reused it.

The shape of it

The flow is simple once you see it:

1. The student's message hits a normal API endpoint.
2. Laravel calls OpenAI with createStreamed().
3. Every token that comes back is re-broadcast on a private channel that belongs to that student.
4. The Vue client appends tokens as they arrive and flips an is_final flag on the last one.

$stream = $client->chat()->createStreamed([...]);

foreach ($stream as $response) {
    $token = $response->choices[0]->delta->content ?? '';
    broadcast(new ChatTutorTokenReceived($userId, $token, false));
}
broadcast(new ChatTutorTokenReceived($userId, '', true));

The channel is private (chat-tutor.{userId}), so nobody else can listen in. And because it rides the existing socket connection, there was zero new infrastructure.

The part nobody warns you about

The model's reply wasn't just prose. We prompted it to append a structured grammar-correction block after its conversational answer. That block should never reach the student raw.

So the stream gets post-processed: a parser extracts the corrections into structured feedback, and a second function strips the block from the visible text. The student sees a natural reply, and separately, a tidy list of what they got wrong.

Two lessons from shipping this:

• Put a daily limit on it from day one. We added a message-per-day guard early, and I'm glad we did. Streaming makes the AI feel cheap to use, and usage explodes.
• Buffer smart, not eager. Broadcasting every single token is wasteful. Batching every few tokens looks identical to the user and cuts the message volume dramatically.

The feature that felt broken became the one students mention most. Latency wasn't the problem — silence was.