When we started building ClawShell, the architecture was simple to the point of being boring. Record audio from the microphone. Transcribe it with Whisper. POST the text to the OpenClaw gateway’s chat completions endpoint. Wait for a response. Synthesize speech. Play it back. A walkie-talkie with extra steps.

It worked. You could talk to your agent. Your agent could talk back. For a first version, that was enough. But it felt thin. We were interacting with a platform through its narrowest interface — the HTTP API — and treating it like a dumb pipe. Send text in, get text out.

We knew there had to be more. OpenClaw is not a simple API wrapper. It runs agents with tools, manages sessions, handles multiple channels. There had to be richer integration points. So we started reading the source.

Down the WebSocket Rabbit Hole

The OpenClaw codebase is not small. We were looking for anything that would let ClawShell be more than an HTTP client — streaming, push notifications, real-time updates. That search led us to a WebSocket server running alongside the HTTP API.

The WebSocket protocol handles several things, but one caught our attention immediately: node registration. The code showed a handshake where a connecting client declares itself as a “node” — a paired device with capabilities. The gateway acknowledges the pairing and keeps the connection alive.

This was not documented anywhere we could find. We pieced it together from the server-side handlers, the message schemas, and eventually from traces of the official mobile apps connecting to test gateways.

What a Node Is

A node, in OpenClaw’s model, is a device that has been paired with a gateway and exposes hardware capabilities to agents. Think of it as the inverse of a typical client-server relationship. Instead of the client asking the server for things, the server (via agents) can ask the client for things.

The capabilities we found in the protocol include:

• Camera — an agent can request a photo from the device’s front or back camera
• Location — an agent can request the device’s current GPS coordinates
• Canvas — an agent can push rich visual content to a rendering surface on the device
• Notifications — an agent can surface alerts through the device’s notification system

Each capability has a request-response pattern. The agent sends a request through the gateway. The gateway routes it over the WebSocket to the paired node. The node handles it and sends a response back.

The key detail: the node decides what to do with each request. The protocol does not assume automatic compliance. It is a request, not a command.

The Aha Moment

We had been building a walkie-talkie. The platform supported a full sensory extension.

Consider what this means practically. An agent running on a gateway is, by default, a brain in a jar. It can think, it can talk, but it cannot see, it cannot know where you are, it cannot show you anything beyond text. A node gives it senses and a screen. A paired phone becomes the agent’s eyes, its sense of place, its visual output.

But — and this is the part that matters to us — only if the user allows it.

The Mediation Layer

When we saw the node protocol, our first instinct was excitement. Our second was caution. The protocol is powerful precisely because it gives agents access to hardware. That power needs a filter.

This is where ClawShell’s philosophy crystallized. We do not want to be a dumb proxy that silently forwards every agent request to the hardware. We want to be a smart mediator. The agent proposes; the app disposes.

When an agent requests a camera snap, ClawShell should not silently take a photo and send it back. It should show the user: “Your Chinese Teacher agent is requesting access to your camera. Allow?” The user taps yes or no. Same for location. Same for any capability.

This is the iOS permission model applied to agent interactions. Users already understand it. They already trust it. And it keeps the human in the loop where they need to be — between the AI and the hardware.

What This Unlocks

The combination of voice and canvas, mediated by the node protocol, opens up interactions we could not have built with HTTP alone.

Take language learning. P’s kid learns Chinese with an agent. With our walkie-talkie architecture, the interaction was purely auditory. The agent speaks Mandarin, the kid responds, the agent corrects pronunciation. Useful, but limited. Chinese is a language where the written form carries meaning that speech cannot convey. Tones are hard enough — but the difference between characters that sound similar is only visible on the page.

With canvas, the Chinese Teacher agent can push characters to the screen in real time while speaking. The kid hears the pronunciation and sees the character simultaneously. The agent can highlight tone marks, show stroke order, display example sentences. Voice carries the conversation; canvas carries the context.

Or consider expense tracking. An agent that summarizes your monthly spending is helpful over voice. An agent that speaks the summary while simultaneously pushing a breakdown chart to your screen is dramatically more useful. You hear “dining is up 40% this month” while seeing exactly where the money went.

These are not hypothetical features. They are direct applications of a protocol that already exists. We just need to implement the client side.

What the Protocol Looks Like

The technical shape is straightforward. The node connects via WebSocket and sends a registration message containing a device identifier and a list of capabilities. The gateway stores the pairing. From that point, capability requests arrive as JSON messages on the WebSocket, and responses go back the same way.

Canvas content arrives as structured JSONL — a sequence of rendering instructions that the client interprets. This is intentionally not raw HTML. The agent cannot run arbitrary code on your device. It can push structured content that the app knows how to render safely.

The WebSocket reconnects automatically on network changes, and the pairing persists across gateway restarts. The protocol is designed for mobile devices that come and go.

Where We Go From Here

We are not abandoning the voice-first approach. Voice is still primary. But voice alone was a ceiling, and the node protocol removes it.

Our immediate next steps are implementing the WebSocket connection and node registration, then building the capability request UI — the permission dialogs that keep the user in control. Canvas rendering comes after that.

We started this project thinking we were building a voice client. We are building something more interesting: a bridge between an AI agent and the physical world, with a human standing at the gate.

That feels right.