tensorlake/ubuntu-vnc is a managed desktop image for browser automation and computer-use agents. It boots XFCE, TigerVNC, and Firefox for you, and the SDK connects through the authenticated sandbox proxy so you can drive the desktop without manually exposing port 5901.
This guide builds on Sandboxes. If you already have a Tensorlake API key, you can create a desktop sandbox, capture screenshots, and send mouse and keyboard input in just a few lines.
If you specifically want to automate Chrome rather than the desktop, see Drive Chrome over CDP — it pairs tensorlake/ubuntu-vnc with a tunnel so Playwright, Puppeteer, or chrome-devtools-mcp can drive the in-sandbox browser as if it were running locally.
Prerequisites
- Python
- JavaScript
- CLI
tensorlake/ubuntu-vnc image uses tensorlake as its VNC password.
Launch a Desktop Sandbox
Usetensorlake/ubuntu-vnc when you want a full Linux desktop instead of a shell-only environment.
- Python
- JavaScript
- CLI
Sandbox object back, so computer use fits naturally alongside run(), file operations, PTY sessions, snapshots, and tunnels.
Capture Screenshots
Once the sandbox is running, attach to the desktop and save a PNG. This is the easiest way to inspect the layout and discover click coordinates before sending pointer events. Fresh desktop sandboxes can take a few seconds to finish starting XFCE and other desktop services. If your first screenshot is blank or input does not land where you expect, wait briefly after connecting and then retry.- Python
- JavaScript
Send Keyboard and Mouse Input
The desktop client supports keyboard shortcuts, typed input, clicks, double-clicks, mouse movement, and scrolling. The example below uses a reliable keyboard-driven flow: open a terminal, type a command, and then verify the result from the sandbox shell.- Python
- JavaScript
- Take a screenshot.
- Inspect the desktop layout and note the coordinates you care about.
- Use
move_mouse()/moveMouse(),click(),double_click()/doubleClick(), andscroll()with those coordinates.
Reconnect to an Existing Sandbox
If a sandbox is already running, connect by sandbox ID and attach to the desktop without creating a new VM.- Python
- JavaScript
Connect with a VNC Client
If you want to drive the desktop from a real VNC viewer (Screen Sharing on macOS, TigerVNC, RealVNC, Remmina, etc.) rather than the SDK, open a TCP tunnel to the sandbox’s VNC port and point your client at the local end. The tunnel keeps sandbox-proxy authentication local — you do not need to expose5901 publicly.
Open the tunnel with tl sbx tunnel. Replace <sandbox-id> with the id printed by tl sbx create (or tl sbx ls):
127.0.0.1:15901 on your machine to port 5901 inside the sandbox over an authenticated WebSocket. Then connect any VNC client to localhost:15901 using the desktop password tensorlake:
- macOS
- Linux (TigerVNC)
- Other
Use the built-in Screen Sharing client:Enter
tensorlake when macOS prompts for the password.Ctrl+C when you are done. Closing the tunnel does not terminate the sandbox; reopen it any time with the same command.
Use noVNC in the Browser
If you want a human to interact with the sandbox desktop in real time, use a real VNC client in the browser instead of polling screenshots.noVNC is a good fit here.
The recommended architecture is:
- Keep the Tensorlake API key on your backend.
- Use the backend to open a TCP tunnel to the sandbox’s VNC port
5901. - Bridge that local tunnel to a browser WebSocket endpoint such as
/vnc/<session-id>. - Point
noVNCat your backend WebSocket and authenticate with the desktop passwordtensorlake.
5901 publicly yourself.
If you are also running an agent loop, a good pattern is to use:
noVNCfor the live human-facing desktop streamsandbox.connectDesktop()for screenshots and high-level computer-use actions on the backend
Browser Client with noVNC
InstallnoVNC in your frontend:
Desktop API Surface
Python usessnake_case, while JavaScript uses camelCase, but both SDKs expose the same core capabilities:
- Screenshots:
screenshot() - Mouse input:
move_mouse()/moveMouse(),mouse_press()/mousePress(),mouse_release()/mouseRelease(),click(),double_click()/doubleClick(),scroll(),scroll_up()/scrollUp(), andscroll_down()/scrollDown() - Keyboard input:
key_down()/keyDown(),key_up()/keyUp(),press(), andtype_text()/typeText() - Desktop size:
widthandheight
connect_desktop() and connectDesktop() go through the authenticated sandbox proxy, so you do not need to bind or expose the VNC port yourself. For interactive debugging through a real VNC viewer, see Connect with a VNC Client above.
Related Guides
- Drive Chrome over CDP — point Playwright, Puppeteer, or
chrome-devtools-mcpat the Chrome that ships intensorlake/ubuntu-vnc. - Local Tunnels — the tunneling primitive used by both the VNC viewer and Chrome CDP workflows.
- Snapshots — fork warm desktops to parallelize agent runs without re-launching XFCE.