UI tools and companion plugins

The host's agent can drive the visitor's UI through UI event tools — tools built with the Vercel AI SDK that emit DARTC CUSTOM events over the active peer session. There are two categories:

Generic UI tools (always available)

When you provide a sendUiEvent callback to buildAgentTools, the host automatically registers three generic tools:

show_presentation(title, body?, items?, status?)

Emits CUSTOM name="presentation.show" with a title, optional body, bullet items, and a status (working | ready).

set_state(mode, data)

Emits STATE_SNAPSHOT or STATE_DELTA (RFC 6902 JSON Patch). The runtime auto-applies these to runtime.state.

send_custom_event(name, value?)

Emits CUSTOM with any name and JSON-serializable payload. This is the escape hatch for host-specific UI updates.

Companion-specific tools (opt-in)

react_companion and say_companion are not registered by default. They assume the host page has a 3D avatar that understands moods (walking, thinking, happy, …), stages (center, peek, presenting-left), and expressions (neutral, happy, talking, …).

If your host has such an avatar, register the companion tools explicitly:

import { buildCompanionTools } from "@gemmapod/host";

// Companion tools are built from a sendUiEvent callback
const uiTools = buildCompanionTools(mySendUiEvent);

// Pass them to buildAgentTools alongside the regular config
const tools = buildAgentTools({
  systemPrompt: "You are a helpful assistant.",
  model: "gemma4:e4b",
  ollamaUrl: "http://localhost:11434",
  manifest: verifiedManifest,
  toolRuntime,
  sendUiEvent: mySendUiEvent,
  uiTools, // includes react_companion, say_companion
});

// streamText uses the composed tools
const response = await streamText({
  model: createLanguageModel("gemma4:e4b", "http://localhost:11434"),
  system: buildSystemPrompt({ systemPrompt: "...", sendUiEvent: mySendUiEvent, uiTools }),
  messages: [{ role: "user", content: "Hello!" }],
  tools,
});

The agent's system prompt will automatically list only the tools that are registered (via buildSystemPrompt). If you don't pass uiTools, the prompt will not mention react_companion or say_companion.

Building your own UI tools

You can create custom UI tools that emit any CUSTOM event your host understands. Custom tools are built with the Vercel AI SDK's tool() function and zod for input schema validation:

import { tool } from "ai";
import { z } from "zod";

function buildMyDashboardTools(sendUiEvent: (event: any) => Promise<void>) {
  return {
    update_chart: tool({
      description: "Update a chart on the visitor's dashboard.",
      inputSchema: z.object({
        chartId: z.string(),
        data: z.array(z.record(z.unknown())),
      }),
      execute: async ({ chartId, data }) => {
        await sendUiEvent({
          type: "CUSTOM",
          threadId: "default",
          name: "dashboard.chart.update",
          value: { chartId, data },
        });
        return { ok: true };
      },
    }),
  };
}

// Register them alongside generic tools
const tools = buildAgentTools({
  sendUiEvent: mySendUiEvent,
  uiTools: {
    ...buildCompanionTools(mySendUiEvent),
    ...buildMyDashboardTools(mySendUiEvent),
  },
});

How the agent discovers UI tools

The agent's system prompt includes an "Event Capabilities" section that lists every registered UI tool with its signature. This is generated dynamically by buildSystemPrompt (called internally by buildAgentTools), which inspects the registered tool names and appends a signature guide for each one.

This means:

• The LLM only knows about tools you registered.
• It won't hallucinate react_companion on a text-only host.
• It won't try update_chart unless you registered it.

Without a 3D companion

If you're embedding the host and don't have a 3D companion, simply omit uiTools — only the three generic tools (show_presentation, set_state, send_custom_event) will be registered:

const tools = buildAgentTools({
  systemPrompt: "You are a helpful assistant.",
  model: "gemma4:e4b",
  ollamaUrl: "http://localhost:11434",
  manifest: verifiedManifest,
  toolRuntime,
  sendUiEvent: mySendUiEvent,
  // No uiTools — only generic tools registered
});

See also

• Custom UI driven by events
• State + snapshots guide
• Add a tool guide
• DARTC UI events reference