Vessel
An AI agent that controls a sandboxed Linux desktop, live in your browser
THE CHALLENGE
Vercel's computer-use demo shows the idea, but it is a minimal proof of concept: no session history, no visibility into individual tool calls, and a model integration that did not work for computer use on AWS Bedrock. I wanted a version that runs on Bedrock, keeps credentials on the server, and lets someone watch the agent work and review what it did.
TECH STACK
Engineering Approach
Server-Side Agent Loop
The whole tool loop runs inside one POST /api/chat request: stream from Claude, execute each tool_use against the sandbox, feed the tool_result back, and repeat until the model stops. The frontend still sees a standard AI SDK data stream, so useChat needed no changes.
Native Anthropic SDK on Bedrock
Replaced the AI SDK's Bedrock provider with @anthropic-ai/bedrock-sdk (InvokeModel) so Anthropic's computer-use beta flag reaches the model.
Explicit History Conversion
toAnthropicMessages() converts AI SDK UIMessage history into the alternating user/assistant turns Anthropic requires, dropping incomplete tool calls and converting screenshot results into the image format the model expects.
Separated State
Chat sessions and the tool-call event timeline live in separate Zustand stores, and message history is read through a ref inside onFinish so callbacks never act on stale values.
Key Technical Decisions
The choices that shaped the architecture and how easy it is to maintain.
Native SDK instead of the AI SDK Bedrock provider
Calls go through @anthropic-ai/bedrock-sdk using InvokeModel.
Through the AI SDK's Bedrock provider, Claude received the tool definitions but described actions in text instead of emitting tool_use blocks. The native SDK sends the beta flag in the request body, and the model started acting.
Run the loop on the server
Tool execution happens inside the API route, not in the browser.
Client-side execution would add a browser round-trip per tool call and put sandbox credentials in the client. Server-side keeps both on the server.
maxSteps: 1 in useChat
The multi-step loop stays inside one server request.
With maxSteps above 1, useChat posts the history again after each turn, and screenshot results lose their image format in JSON serialization and come back as plain strings, so the model can no longer see them.
Two separate stores
Event pipeline and session management are separate Zustand stores.
They change at different rates and for different reasons. Keeping them apart keeps each one simple.
Challenges & Solutions
On AWS Bedrock, Claude described actions in text instead of performing them.
Traced it to the computer-use beta flag not reaching the model through the AI SDK's Bedrock provider, and moved to the native Anthropic SDK.
Screenshot results stopped working after the first turn, because re-posted history returned them as plain strings.
Kept the multi-step loop inside a single server request with maxSteps: 1, and converted history explicitly in toAnthropicMessages().
Anthropic requires every tool_use to be followed by a tool_result, and interrupted runs left orphaned tool calls.
Drop assistant turns with incomplete tool invocations before sending, and mark aborted calls as results when the user stops a run.
Outcomes
Vessel runs browser and terminal tasks end to end in an isolated Linux desktop, streamed live in the browser, with every tool call visible in a debug timeline. It is open source and deployed on Vercel.
KEY TAKEAWAYS
The hardest bugs were at boundaries: between the SDK and the provider, and between UI message formats and the model's turn rules. Reading the wire format directly was faster than guessing from symptoms.
