FastAPI Chat WebSocket: Bidirectional LLM Streaming, Rooms, and Connection Management for 2026
TL;DR — FastAPI chat WebSocket: bidirectional LLM streaming, rooms, connection management. WebSocket.org: "SSE for simple one-way token streaming. WebSockets for tool use, interrupts, multi-turn agents, collaborative AI with bidirectional communication." FastAPI Docs: "WebSockets with FastAPI. WebSocket endpoints with async receive and send." BuildMVPFast: "WebSockets: manual reconnection, message ID tracking, replay. HTTP/1.1: 6 SSE per domain limit." Render: "SSE delivers real-time feel with minimal complexity. WebSockets for bidirectional AI chat." Learn more with FastAPI AI tutorial, streaming SSE, async backend, and Docker AI API.
WebSocket.org frames the decision: "Use SSE for simple one-way token streaming where the user sends a prompt and waits for a response. Use WebSockets when you need tool use, interrupts, multi-turn agents, or collaborative AI features that require bidirectional communication."
BuildMVPFast adds the tradeoff: "With WebSockets, you're writing that reconnection logic yourself, tracking message IDs, replaying missed events. Most teams skip this and just restart the generation, which wastes tokens and money."
WebSocket Chat Architecture
new WebSocket(url)
ws.send(message)
ws.onmessage handler
auto reconnection logic"] Features["Client Features
typing indicator
message history
multi-turn context
interrupt/cancel"] end subgraph FastAPI["FastAPI WebSocket"] Endpoint["WebSocket Endpoint
@app.websocket('/ws/{room_id}')
await websocket.accept()
receive_text / send_text
send_json for structured"] Manager["ConnectionManager
active_connections dict
connect / disconnect
broadcast to room
send_personal to client"] Rooms["Chat Rooms
room-based isolation
multi-tenant
connection limit per room
message history per room"] end subgraph Streaming["LLM Token Streaming"] Ollama["Ollama Streaming
stream=true
async for line in
response.aiter_lines()
parse JSON chunks"] TokenSend["Token Delivery
websocket.send_text(token)
per token as generated
send_json done event
send_json error event"] Context["Conversation Context
message history
multi-turn support
system prompt
context window management"] end subgraph Reliability["Production Reliability"] Heartbeat["Heartbeat
ping/pong every 30s
detect stale connections
close idle connections"] Reconnect["Reconnection
message ID tracking
replay buffer
Last-Message-ID
exponential backoff"] Cleanup["Cleanup
WebSocketDisconnect handler
finally block
remove from manager
stop LLM generation"] end subgraph Nginx["Nginx WebSocket Proxy"] Proxy["WebSocket Upgrade
proxy_http_version 1.1
proxy_set_header Upgrade
proxy_set_header Connection
proxy_read_timeout 300s"] end WSClient --> Endpoint Features --> Endpoint Endpoint --> Manager Manager --> Rooms Endpoint --> Ollama Ollama --> TokenSend TokenSend --> WSClient Context --> Ollama Endpoint --> Heartbeat Endpoint --> Reconnect Endpoint --> Cleanup Proxy --> Endpoint style FastAPI fill:#4169E1,color:#fff style Streaming fill:#39FF14,color:#000 style Reliability fill:#2D1B69,color:#fff
WebSocket vs SSE Comparison
| Feature | WebSocket | SSE |
|---|---|---|
| Direction | Bidirectional | Server→Client only |
| Browser API | WebSocket API | EventSource (GET only) |
| Reconnection | Manual | Built-in automatic |
| Message IDs | Manual tracking | Built-in Last-Event-ID |
| Use case | Tool use, interrupts, multi-turn | Simple prompt→response |
| HTTP/1.1 limit | No limit | 6 per domain |
| Complexity | Higher | Lower |
| Best for | Collaborative AI chat | Token streaming |
Implementation
from dataclasses import dataclass
from typing import Optional
from enum import Enum
class WSFeature(Enum):
ENDPOINT = "endpoint"
MANAGER = "manager"
STREAMING = "streaming"
RELIABILITY = "reliability"
@dataclass
class FastAPIChatWSGuide:
"""FastAPI chat WebSocket implementation guide."""
def get_connection_manager(self) -> str:
"""ConnectionManager for WebSocket rooms."""
return (
"# === CONNECTION MANAGER ===\n"
"from fastapi import WebSocket\n"
"from typing import Dict, List\n"
"import json\n"
"\n"
"class ConnectionManager:\n"
" '''Manage WebSocket\n"
" connections by room.''' \n"
" \n"
" def __init__(self):\n"
" # room_id -> connections\n"
" self.active: Dict[str,\n"
" List[WebSocket]] = {}\n"
" # ws -> client_id\n"
" self.client_ids: Dict[\n"
" WebSocket, str] = {}\n"
" # room message history\n"
" self.history: Dict[str,\n"
" List[dict]] = {}\n"
" \n"
" async def connect(\n"
" self, websocket,\n"
" room_id, client_id):\n"
" await websocket.accept()\n"
" if room_id not in (\n"
" self.active):\n"
" self.active[room_id] = []\n"
" self.active[room_id]\n"
" .append(websocket)\n"
" self.client_ids[\n"
" websocket] = client_id\n"
" \n"
" def disconnect(\n"
" self, websocket,\n"
" room_id):\n"
" if room_id in (\n"
" self.active):\n"
" self.active[\n"
" room_id].remove(\n"
" websocket)\n"
" self.client_ids.pop(\n"
" websocket, None)\n"
" \n"
" async def broadcast(\n"
" self, message,\n"
" room_id,\n"
" exclude: WebSocket = None):\n"
" if room_id in (\n"
" self.active):\n"
" for ws in (\n"
" self.active[room_id]):\n"
" if ws != exclude:\n"
" await ws.send_text(\n"
" json.dumps(\n"
" message))\n"
" \n"
" async def send_personal(\n"
" self, message,\n"
" websocket):\n"
" await websocket.send_text(\n"
" json.dumps(message))\n"
" \n"
" def add_to_history(\n"
" self, room_id,\n"
" message):\n"
" if room_id not in (\n"
" self.history):\n"
" self.history[room_id]\n"
" = []\n"
" self.history[room_id]\n"
" .append(message)\n"
" # Keep last 100\n"
" if len(self.history[\n"
" room_id]) > 100:\n"
" self.history[\n"
" room_id] = (\n"
" self.history[\n"
" room_id][-100:])\n"
"\n"
"manager = ConnectionManager()"
)
def get_websocket_endpoint(self) -> str:
"""WebSocket endpoint with LLM streaming."""
return (
"# === WEBSOCKET ENDPOINT ===\n"
"from fastapi import FastAPI,\n"
" WebSocket, WebSocketDisconnect\n"
"import httpx, json, asyncio\n"
"\n"
"app = FastAPI()\n"
"OLLAMA_URL = (\n"
" 'http://localhost:11434')\n"
"\n"
"@app.websocket(\n"
" '/ws/{room_id}/{client_id}')\n"
"async def chat_websocket(\n"
" websocket: WebSocket,\n"
" room_id: str,\n"
" client_id: str):\n"
" '''WebSocket chat endpoint\n"
" with LLM token streaming.''' \n"
" \n"
" await manager.connect(\n"
" websocket, room_id,\n"
" client_id)\n"
" \n"
" try:\n"
" # Send history on connect\n"
" if room_id in (\n"
" manager.history):\n"
" await websocket.send_text(\n"
" json.dumps({\n"
" 'type': 'history',\n"
" 'messages': (\n"
" manager.history[\n"
" room_id])}))\n"
" \n"
" while True:\n"
" # Receive user message\n"
" data = await (\n"
" websocket.receive_text())\n"
" msg = json.loads(data)\n"
" \n"
" # Broadcast user message\n"
" user_msg = {\n"
" 'type': 'message',\n"
" 'role': 'user',\n"
" 'client_id':\n"
" client_id,\n"
" 'content': msg[\n"
" 'content'],\n"
" 'id': msg.get('id'),\n"
" }\n"
" manager.add_to_history(\n"
" room_id, user_msg)\n"
" await manager.broadcast(\n"
" user_msg, room_id)\n"
" \n"
" # Typing indicator\n"
" await manager.broadcast({\n"
" 'type': 'typing',\n"
" 'client_id': 'ai'\n"
" }, room_id)\n"
" \n"
" # Stream LLM response\n"
" conversation = (\n"
" manager.history[\n"
" room_id])\n"
" \n"
" async with httpx.AsyncClient(\n"
" timeout=300.0\n"
" ) as client:\n"
" async with client.stream(\n"
" 'POST',\n"
" f'{OLLAMA_URL}'\n"
" '/api/chat',\n"
" json={\n"
" 'model': msg.get(\n"
" 'model',\n"
" 'llama3.2'),\n"
" 'messages': [\n"
" {'role': m['role'],\n"
" 'content': m[\n"
" 'content']}\n"
" for m in\n"
" conversation],\n"
" 'stream': True,\n"
" }\n"
" ) as response:\n"
" full_response = ''\n"
" async for line in (\n"
" response\n"
" .aiter_lines()):\n"
" if not line:\n"
" continue\n"
" chunk = (\n"
" json.loads(\n"
" line))\n"
" token = (\n"
" chunk.get(\n"
" 'message',{})\n"
" .get(\n"
" 'content',''))\n"
" if token:\n"
" full_response\n"
" += token\n"
" # Send token\n"
" await (\n"
" manager\n"
" .broadcast({\n"
" 'type': 'token',\n"
" 'content': token\n"
" }, room_id))\n"
" if chunk.get('done'):\n"
" break\n"
" \n"
" # Store AI response\n"
" ai_msg = {\n"
" 'type': 'message',\n"
" 'role': 'assistant',\n"
" 'content':\n"
" full_response,\n"
" }\n"
" manager.add_to_history(\n"
" room_id, ai_msg)\n"
" await (\n"
" manager.broadcast({\n"
" 'type': 'done',\n"
" 'content':\n"
" full_response\n"
" }, room_id))\n"
" \n"
" except WebSocketDisconnect:\n"
" manager.disconnect(\n"
" websocket, room_id)\n"
" await manager.broadcast({\n"
" 'type': 'leave',\n"
" 'client_id':\n"
" client_id\n"
" }, room_id)"
)
def get_heartbeat(self) -> str:
"""WebSocket heartbeat for stale connection detection."""
return (
"# === HEARTBEAT ===\n"
"\n"
"import asyncio\n"
"from fastapi import WebSocket\n"
"\n"
"async def heartbeat(\n"
" websocket: WebSocket,\n"
" interval: int = 30):\n"
" '''Send periodic pings to\n"
" detect stale connections.''' \n"
" while True:\n"
" await asyncio.sleep(interval)\n"
" try:\n"
" await websocket.send_json({\n"
" 'type': 'ping'\n"
" })\n"
" except Exception:\n"
" break\n"
"\n"
"# In endpoint:\n"
"# heartbeat_task = (\n"
"# asyncio.create_task(\n"
"# heartbeat(websocket)))\n"
"# ... message loop ...\n"
"# heartbeat_task.cancel()\n"
"\n"
"# Client responds with pong:\n"
"# if msg['type'] == 'ping':\n"
"# await websocket.send_json({\n"
"# 'type': 'pong'})"
)
def get_reconnection(self) -> str:
"""Reconnection with message ID tracking."""
return (
"# === RECONNECTION ===\n"
"\n"
"class ReconnectionManager:\n"
" '''Track message IDs for\n"
" replay on reconnect.''' \n"
" \n"
" def __init__(self):\n"
" # room -> msg_id -> msg\n"
" self.replay_buffer: (\n"
" Dict[str, Dict[int,\n"
" dict]]) = {}\n"
" self.msg_counters: (\n"
" Dict[str, int]) = {}\n"
" \n"
" def next_id(self, room_id):\n"
" if room_id not in (\n"
" self.msg_counters):\n"
" self.msg_counters[\n"
" room_id] = 0\n"
" self.msg_counters[\n"
" room_id] += 1\n"
" return self.msg_counters[\n"
" room_id]\n"
" \n"
" def store(self, room_id,\n"
" msg_id, message):\n"
" if room_id not in (\n"
" self.replay_buffer):\n"
" self.replay_buffer[\n"
" room_id] = {}\n"
" self.replay_buffer[\n"
" room_id][msg_id] = (\n"
" message)\n"
" # Keep last 200\n"
" if len(self.replay_buffer[\n"
" room_id]) > 200:\n"
" oldest = min(\n"
" self.replay_buffer[\n"
" room_id].keys())\n"
" del self.replay_buffer[\n"
" room_id][oldest]\n"
" \n"
" def get_since(self, room_id,\n"
" last_msg_id):\n"
" if room_id not in (\n"
" self.replay_buffer):\n"
" return []\n"
" return [\n"
" self.replay_buffer[\n"
" room_id][k]\n"
" for k in sorted(\n"
" self.replay_buffer[\n"
" room_id].keys())\n"
" if k > last_msg_id]\n"
"\n"
"# On reconnect:\n"
"# last_id = msg.get('last_id', 0)\n"
"# missed = replay.get_since(\n"
"# room_id, last_id)\n"
"# for m in missed:\n"
"# await ws.send_json(m)"
)
def get_nginx_ws(self) -> str:
"""Nginx WebSocket proxy configuration."""
return (
"# === NGINX WEBSOCKET PROXY ===\n"
"\n"
"location /ws/ {\n"
" proxy_pass http://api:8000;\n"
" \n"
" # WebSocket upgrade headers\n"
" proxy_http_version 1.1;\n"
" proxy_set_header Upgrade\n"
" $http_upgrade;\n"
" proxy_set_header Connection\n"
" 'upgrade';\n"
" \n"
" # Long timeout for WS\n"
" proxy_read_timeout 300s;\n"
" proxy_send_timeout 300s;\n"
" \n"
" # Don't buffer\n"
" proxy_buffering off;\n"
" \n"
" # Pass headers\n"
" proxy_set_header Host $host;\n"
" proxy_set_header X-Real-IP\n"
" $remote_addr;\n"
" proxy_set_header X-Forwarded-For\n"
" $proxy_add_x_forwarded_for;\n"
" proxy_set_header X-Forwarded-Proto\n"
" $scheme;\n"
"}"
)
def get_client_js(self) -> str:
"""JavaScript WebSocket client."""
return (
"// === JAVASCRIPT WS CLIENT ===\n"
"\n"
"const ws = new WebSocket(\n"
" 'ws://localhost:8000'\n"
" + '/ws/room1/user1');\n"
"\n"
"ws.onopen = () => {\n"
" console.log('Connected');\n"
"};\n"
"\n"
"ws.onmessage = (e) => {\n"
" const msg = JSON.parse(\n"
" e.data);\n"
" switch(msg.type) {\n"
" case 'token':\n"
" // Append token to UI\n"
" appendToken(\n"
" msg.content);\n"
" break;\n"
" case 'done':\n"
" // Complete response\n"
" finishResponse();\n"
" break;\n"
" case 'typing':\n"
" showTypingIndicator();\n"
" break;\n"
" case 'history':\n"
" // Load history\n"
" loadHistory(\n"
" msg.messages);\n"
" break;\n"
" case 'ping':\n"
" ws.send(JSON.stringify({\n"
" type: 'pong'}));\n"
" break;\n"
" }\n"
"};\n"
"\n"
"ws.onclose = (e) => {\n"
" console.log(\n"
" 'Disconnected:',\n"
" e.code, e.reason);\n"
" // Reconnect with backoff\n"
" setTimeout(() => {\n"
" // Reconnect logic\n"
" // Include last_msg_id\n"
" }, 1000);\n"
"};\n"
"\n"
"// Send message\n"
"function send(text) {\n"
" ws.send(JSON.stringify({\n"
" content: text,\n"
" model: 'llama3.2',\n"
" id: Date.now()\n"
" }));\n"
"}"
)
def get_production_tips(self) -> dict:
"""Production tips."""
return {
"ws_vs_sse": "SSE for simple one-way streaming. WebSocket for bidirectional, tool use, interrupts, multi-turn agents, collaborative AI.",
"connection_manager": "ConnectionManager class: track active connections by room, broadcast, send_personal, disconnect cleanup.",
"rooms": "Room-based isolation for multi-tenant chat. Connection limit per room for resource management.",
"streaming": "Call Ollama stream=true, send each token via websocket.send_text() as it arrives. Send done event when complete.",
"typing_indicator": "Send typing event before LLM response starts. Clients show indicator while waiting.",
"disconnect": "WebSocketDisconnect exception handler in finally block. Remove from manager, stop LLM generation, broadcast leave.",
"heartbeat": "Ping/pong every 30s to detect stale connections. Close idle connections. Client responds with pong.",
"reconnection": "Message ID tracking, replay buffer, Last-Message-ID on reconnect. Exponential backoff on client.",
"history": "Store message history per room (last 100). Send on connect for new clients joining mid-conversation.",
"nginx": "WebSocket upgrade: proxy_http_version 1.1, Upgrade header, Connection upgrade, proxy_read_timeout 300s.",
}
FastAPI Chat WebSocket Checklist
- [ ] WebSocket: bidirectional, full-duplex communication — ideal for interactive real-time AI chat
- [ ] SSE for simple one-way token streaming where user sends prompt and waits for response
- [ ] WebSocket for tool use, interrupts, multi-turn agents, collaborative AI with bidirectional communication
- [ ] WebSocket: manual reconnection logic, message ID tracking, replay missed events
- [ ] SSE: built-in automatic reconnection via EventSource API, Last-Event-ID header
- [ ] HTTP/1.1: browsers limit 6 SSE connections per domain — no such limit for WebSocket
- [ ] HTTP/2 multiplexing removes SSE per-domain limit
- [ ] Most teams skip WebSocket reconnection and restart generation — wastes tokens and money
- [ ] FastAPI WebSocket:
from fastapi import WebSocket,@app.websocket('/ws/{room_id}') - [ ]
await websocket.accept()to accept connection - [ ]
while True: data = await websocket.receive_text()for message loop - [ ]
await websocket.send_text(response)orawait websocket.send_json({...})to send - [ ]
WebSocketDisconnectexception for cleanup - [ ] ConnectionManager class: track active connections, group by room, broadcast, send_personal
- [ ] ConnectionManager:
active_connections: dict[str, list[WebSocket]]for rooms - [ ] ConnectionManager:
connect(websocket, room_id, client_id),disconnect(websocket, room_id) - [ ] ConnectionManager:
broadcast(message, room_id, exclude=None)to all in room - [ ] ConnectionManager:
send_personal(message, websocket)to specific client - [ ] Chat rooms: room-based isolation for multi-tenant chat
- [ ] Chat rooms: connection limit per room for resource management
- [ ] Chat rooms: message history per room (last 100 messages)
- [ ] Send message history on connect for new clients joining mid-conversation
- [ ] LLM streaming: call Ollama with stream=true via httpx.AsyncClient
- [ ] LLM streaming:
async for line in response.aiter_lines(): parse JSON, send token - [ ] LLM streaming:
await websocket.send_text(token)for each token as it arrives - [ ] LLM streaming: send
{'type': 'done'}event when generation complete - [ ] LLM streaming: send
{'type': 'error'}event on failure - [ ] Typing indicator: send
{'type': 'typing'}before LLM response starts - [ ] Conversation context: track message history for multi-turn support
- [ ] Conversation context: system prompt, context window management
- [ ] Heartbeat: ping/pong every 30s to detect stale connections
- [ ] Heartbeat:
asyncio.create_task(heartbeat(websocket)), cancel on disconnect - [ ] Heartbeat: client responds with pong, server detects stale if no response
- [ ] Close idle connections to free resources
- [ ] Reconnection: sequential message IDs per conversation
- [ ] Reconnection: replay buffer stores recent messages (last 200)
- [ ] Reconnection: Last-Message-ID on reconnect, replay missed messages
- [ ] Reconnection: client exponential backoff (1s, 2s, 4s, 8s)
- [ ] Reconnection: idempotent message handling for safe replay
- [ ] WebSocket close codes: 1000 normal, 1001 going away, 1011 server error
- [ ] Nginx WebSocket:
proxy_http_version 1.1,proxy_set_header Upgrade $http_upgrade - [ ] Nginx WebSocket:
proxy_set_header Connection 'upgrade',proxy_read_timeout 300s - [ ] Nginx WebSocket:
proxy_buffering offfor real-time delivery - [ ] JavaScript client:
new WebSocket(url),ws.onmessage,ws.onclosewith reconnect - [ ] JavaScript client: message types: token, done, typing, history, ping/pong
- [ ] Read FastAPI AI tutorial for base setup
- [ ] Read streaming SSE for SSE comparison
- [ ] Read async backend for async patterns
- [ ] Read Docker AI API for deployment
- [ ] Test: WebSocket connection establishes and accepts
- [ ] Test: messages broadcast to all clients in room
- [ ] Test: LLM tokens stream token-by-token to all clients
- [ ] Test: typing indicator shows before LLM response
- [ ] Test: disconnect removes from manager and broadcasts leave
- [ ] Test: heartbeat detects and closes stale connections
- [ ] Test: reconnection replays missed messages from Last-Message-ID
- [ ] Test: message history loads on connect for new clients
- [ ] Document WebSocket vs SSE decision, ConnectionManager, room strategy, reconnection, Nginx config
FAQ
When should you use WebSockets vs SSE for LLM streaming?
Use SSE for simple one-way token streaming; WebSockets for bidirectional, multi-turn, tool use, and collaborative AI. WebSocket.org: "Use SSE for simple one-way token streaming where user sends prompt and waits for response. Use WebSockets when you need tool use, interrupts, multi-turn agents, or collaborative AI features that require bidirectional communication." BuildMVPFast: "With WebSockets, you are writing reconnection logic yourself, tracking message IDs, replaying missed events. Most teams skip this and just restart generation, wasting tokens and money. HTTP/2 multiplexing: under HTTP/1.1, browsers limit 6 SSE connections per domain." Render: "SSE delivers fluid real-time feel with minimal engineering complexity. Modern browsers support SSE natively through EventSource API with automatic reconnection." Decision: (1) SSE: one-way streaming, simple prompt-response, native reconnection, GET only. (2) WebSocket: bidirectional, tool use, interrupts, multi-turn agents, typing indicators, presence, collaborative. (3) HTTP/1.1: 6 SSE per domain limit. (4) WebSocket: manual reconnection, message ID tracking. (5) SSE: simpler, less code. (6) WebSocket: more control, more complexity.
How do you implement WebSocket chat with FastAPI?
Use FastAPI WebSocket endpoint with async receive/send and a connection manager. FastAPI Docs: "You can use WebSockets with FastAPI. from fastapi import FastAPI, WebSocket. WebSocket endpoints with async receive and send." Implementation: (1) @app.websocket('/ws/{client_id}') async def websocket_endpoint(websocket, client_id). (2) await websocket.accept() to accept connection. (3) while True: data = await websocket.receive_text() for message loop. (4) await websocket.send_text(response) to send. (5) await websocket.send_json({...}) for structured data. (6) ConnectionManager class to track active connections. (7) await manager.connect(websocket, client_id). (8) manager.broadcast(message) to all clients. (9) manager.send_personal(message, client_id) to specific client. (10) try/except WebSocketDisconnect for cleanup.
How do you stream LLM tokens over WebSocket in FastAPI?
Call Ollama streaming API and send each token via websocket.send_text() as it arrives. WebSocket.org: "WebSockets for tool use, interrupts, multi-turn agents, or collaborative AI features that require bidirectional communication." Render: "Real-time AI chat infrastructure with WebSockets and LLM streaming." Streaming: (1) Receive user message via websocket.receive_text(). (2) Call Ollama with stream=true via httpx.AsyncClient. (3) async for line in response.aiter_lines(): parse JSON. (4) await websocket.send_text(token) for each token. (5) await websocket.send_json({'type':'done'}) when complete. (6) Check websocket.client_state for disconnect. (7) Handle WebSocketDisconnect exception for cleanup. (8) Send typing indicator before LLM response. (9) Send error events on failure. (10) Track conversation history for multi-turn.
How do you manage WebSocket connections and chat rooms in FastAPI?
Use a ConnectionManager class to track active connections, group by room, and broadcast messages. ConnectionManager: (1) active_connections: dict[str, list[WebSocket]] for rooms. (2) async def connect(websocket, room_id): accept and append. (3) async def disconnect(websocket, room_id): remove from list. (4) async def broadcast(message, room_id): send to all in room. (5) async def send_personal(message, client_id): send to specific. (6) Track client_id per connection. (7) Handle WebSocketDisconnect in finally block. (8) Clean up on disconnect. (9) Room-based isolation for multi-tenant chat. (10) Connection limit per room for resource management.
How do you handle WebSocket reconnection and message ordering in production?
Track message IDs, implement replay buffer, and use heartbeat pings. BuildMVPFast: "With WebSockets, you are writing reconnection logic yourself, tracking message IDs, replaying missed events. Most teams skip this and just restart generation, wasting tokens and money." Reconnection: (1) Message ID: sequential per conversation. (2) Last-Message-ID header on reconnect. (3) Replay buffer: store recent messages for replay. (4) Heartbeat: ping/pong every 30s to detect stale connections. (5) Client reconnect with exponential backoff. (6) Resume from last message ID. (7) Idempotent message handling. (8) WebSocket close codes: 1000 normal, 1001 going away, 1011 server error. (9) Connection timeout: close idle connections. (10) Nginx: proxy_read_timeout 300s for WebSocket upgrade.
Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →