AI integration

Mlola's AI components speak one small event stream. Your server translates any model into it; the chat, the Bot and the agent views never know which provider answered.

The event stream

Reasoning and text arrive as deltas; tool calls and sources arrive whole and can be sent again as their status changes. A responder is an async generator of these events, cancelled through signal when the person presses stop.

type ChatEvent =
  | { type: "reasoning"; delta: string }
  | { type: "text"; delta: string }
  | { type: "tool"; id: string; name: string; title?: string; status: ToolCallStatus; input?: unknown; output?: unknown; duration?: number }
  | { type: "sources"; sources: CitationSource[] };

type ChatResponder = (
  history: { role: "user" | "assistant"; text: string }[],
  options: { signal: AbortSignal; model: string; search: boolean },
) => AsyncIterable<ChatEvent>;

1. Translate the model on the server

Keep keys and prompts server-side. Stream newline-delimited JSON so the browser can act on every event as it lands.

app/api/chat/route.ts

// app/api/chat/route.ts — the model stays on the server; the browser gets Mlola events.
import OpenAI from "openai";

const client = new OpenAI();

export async function POST(request: Request) {
  const { history, model } = await request.json();
  const stream = await client.responses.create({
    model,
    input: history.map((turn) => ({ role: turn.role, content: turn.text })),
    stream: true,
  });

  const encoder = new TextEncoder();
  const body = new ReadableStream({
    async start(controller) {
      const send = (event: object) => controller.enqueue(encoder.encode(JSON.stringify(event) + "\n"));
      for await (const chunk of stream) {
        if (chunk.type === "response.reasoning_summary_text.delta") send({ type: "reasoning", delta: chunk.delta });
        if (chunk.type === "response.output_text.delta") send({ type: "text", delta: chunk.delta });
      }
      controller.close();
    },
  });
  return new Response(body, { headers: { "Content-Type": "application/x-ndjson" } });
}

2. Hand the chat a responder

The AI Chat block takes it as respond. StreamingText paces the bursts, Thinking folds the reasoning, ToolCall shows each tool, and Citation numbers the sources.

lib/respond.ts

// A responder: read the server's newline-delimited events and yield them.
import type { ChatResponder } from "@/components/blocks/ai-chat";

export const respond: ChatResponder = async function* (history, { signal, model }) {
  const response = await fetch("/api/chat", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ history, model }),
    signal,
  });
  const reader = response.body!.pipeThrough(new TextDecoderStream()).getReader();
  let buffer = "";
  for (;;) {
    const { value, done } = await reader.read();
    buffer += value ?? "";
    let newline;
    while ((newline = buffer.indexOf("\n")) >= 0) {
      const line = buffer.slice(0, newline).trim();
      buffer = buffer.slice(newline + 1);
      if (line) yield JSON.parse(line);
    }
    if (done) return;
  }
};

// <AiChat respond={respond} />

Voice

useAudioMeter reads a microphone stream or an <audio> element. Orb, Bot and VoiceWave read the meter every frame without re-rendering React.

const [stream, setStream] = useState<MediaStream | null>(null);
const microphone = useAudioMeter(stream);          // the person
const speaker = useAudioMeter(audioElement);        // the assistant's <audio>

<Bot state={speaking ? "speaking" : "listening"} meter={speaking ? speaker : microphone} />
<VoiceWave state={speaking ? "speaking" : "listening"} meter={speaking ? speaker : microphone} />

Mapping state to the Bot

State	When
`idle`	Waiting for the person
`listening`	Recording or the person is typing
`thinking`	Reasoning events are arriving
`working`	A tool call is running
`speaking`	Text or audio is streaming
`happy`	A task finished
`confused`	The model asked a clarifying question
`error`	The request failed
`sleeping`	The session is paused or idle for long

Make answers checkable

Send sources for Citation and SourceList, retrieved passages for Context Cards, and per-span confidence for Confidence Text. A person should be able to see why an answer says what it says.