> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensorlake.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Computer Use

> Launch ubuntu-vnc sandboxes and drive their desktop from Python or JavaScript.

`tensorlake/ubuntu-vnc` is a managed desktop image for browser automation and computer-use agents. It boots XFCE, TigerVNC, and Firefox for you, and the SDK connects through the authenticated sandbox proxy so you can drive the desktop without manually exposing port `5901`.

This guide builds on [Sandboxes](/sandboxes/introduction). If you already have a Tensorlake API key, you can create a desktop sandbox, capture screenshots, and send mouse and keyboard input in just a few lines.

If you specifically want to automate **Chrome** rather than the desktop, see [Drive Chrome over CDP](/sandboxes/chrome-cdp) — it pairs `tensorlake/ubuntu-vnc` with a [tunnel](/sandboxes/tunnels) so Playwright, Puppeteer, or `chrome-devtools-mcp` can drive the in-sandbox browser as if it were running locally.

## Prerequisites

<Tabs>
  <Tab title="Python">
    ```bash theme={null}
    pip install tensorlake
    export TENSORLAKE_API_KEY=your-api-key
    ```
  </Tab>

  <Tab title="JavaScript">
    ```bash theme={null}
    npm install tensorlake
    export TENSORLAKE_API_KEY=your-api-key
    ```
  </Tab>

  <Tab title="CLI">
    ```bash theme={null}
    curl -fsSL https://tensorlake.ai/install | sh
    export TENSORLAKE_API_KEY=your-api-key
    ```

    Prefer an interactive login? Run `tl login` instead of setting
    `TENSORLAKE_API_KEY`; it stores a Personal Access Token in
    `~/.config/tensorlake/credentials.toml`.
  </Tab>
</Tabs>

The current managed `tensorlake/ubuntu-vnc` image uses `tensorlake` as its VNC password.

## Launch a Desktop Sandbox

Use `tensorlake/ubuntu-vnc` when you want a full Linux desktop instead of a shell-only environment.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    from tensorlake.sandbox import Sandbox


    sandbox = Sandbox.create(image="tensorlake/ubuntu-vnc")
    print(sandbox.sandbox_id)
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import { Sandbox } from "tensorlake";


    const sandbox = await Sandbox.create({
      image: "tensorlake/ubuntu-vnc",
    });

    try {
      console.log(sandbox.sandboxId);
    } finally {
      await sandbox.terminate();
    }
    ```
  </Tab>

  <Tab title="CLI">
    ```bash theme={null}
    tl sbx create -i tensorlake/ubuntu-vnc
    ```

    `tl sbx create` prints the sandbox id on stdout. Reuse it with
    `tl sbx tunnel`, `tl sbx ssh`, `tl sbx exec`, and the rest of the
    `tl sbx ...` subcommands. List running sandboxes with `tl sbx ls` and
    terminate one with `tl sbx terminate <sandbox-id>`.
  </Tab>
</Tabs>

You still get a normal `Sandbox` object back, so computer use fits naturally alongside `run()`, file operations, PTY sessions, snapshots, and tunnels.

## Capture Screenshots

Once the sandbox is running, attach to the desktop and save a PNG. This is the easiest way to inspect the layout and discover click coordinates before sending pointer events.

Fresh desktop sandboxes can take a few seconds to finish starting XFCE and other desktop services. If your first screenshot is blank or input does not land where you expect, wait briefly after connecting and then retry.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import time
    from pathlib import Path
    from tensorlake.sandbox import Sandbox


    sandbox = Sandbox.create(image="tensorlake/ubuntu-vnc")
    with sandbox.connect_desktop(password="tensorlake") as desktop:
        time.sleep(4.0)
        screenshot = desktop.screenshot()
        Path("sandbox-desktop.png").write_bytes(screenshot)

        print(desktop.width, desktop.height)
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import { writeFile } from "node:fs/promises";
    import { Sandbox } from "tensorlake";


    const sandbox = await Sandbox.create({
      image: "tensorlake/ubuntu-vnc",
    });

    try {
      const desktop = await sandbox.connectDesktop({
        password: "tensorlake",
      });

      try {
        await new Promise((resolve) => setTimeout(resolve, 4000));
        const screenshot = await desktop.screenshot();
        await writeFile("sandbox-desktop.png", screenshot);

        console.log(desktop.width, desktop.height);
      } finally {
        await desktop.close();
      }
    } finally {
      await sandbox.terminate();
    }
    ```
  </Tab>
</Tabs>

## Send Keyboard and Mouse Input

The desktop client supports keyboard shortcuts, typed input, clicks, double-clicks, mouse movement, and scrolling. The example below uses a reliable keyboard-driven flow: open a terminal, type a command, and then verify the result from the sandbox shell.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import time
    from tensorlake.sandbox import Sandbox


    sandbox = Sandbox.create(image="tensorlake/ubuntu-vnc")
    with sandbox.connect_desktop(password="tensorlake") as desktop:
        # Give XFCE a moment to finish initializing the keybind daemon and
        # window manager. On a freshly-restored snapshot the in-VM `vncserver`
        # is up before XFCE has finished settling, so the very first
        # `Ctrl+Alt+T` can be lost if it lands before the keybind handler
        # registers.
        time.sleep(5.0)
        desktop.press(["ctrl", "alt", "t"])
        time.sleep(4.0)

        desktop.type_text("echo docs-test > /tmp/desktop-test.txt")
        desktop.press("enter")
        time.sleep(3.0)

        # Mouse helpers are also available when you know the coordinates.
        desktop.move_mouse(640, 400)
        desktop.scroll_down()

    result = sandbox.run("bash", ["-lc", "cat /tmp/desktop-test.txt"])
    print(result.stdout.strip())  # docs-test
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import { Sandbox } from "tensorlake";


    const sandbox = await Sandbox.create({
      image: "tensorlake/ubuntu-vnc",
    });

    try {
      const desktop = await sandbox.connectDesktop({
        password: "tensorlake",
      });

      try {
        // Give XFCE a moment to finish initializing the keybind daemon and
        // window manager. On a freshly-restored snapshot the in-VM `vncserver`
        // is up before XFCE has finished settling, so the very first
        // `Ctrl+Alt+T` can be lost if it lands before the keybind handler
        // registers.
        await new Promise((resolve) => setTimeout(resolve, 5000));
        await desktop.press(["ctrl", "alt", "t"]);
        await new Promise((resolve) => setTimeout(resolve, 4000));

        await desktop.typeText("echo docs-test > /tmp/desktop-test.txt");
        await desktop.press("enter");
        await new Promise((resolve) => setTimeout(resolve, 3000));

        // Mouse helpers are also available when you know the coordinates.
        await desktop.moveMouse(640, 400);
        await desktop.scrollDown();
      } finally {
        await desktop.close();
      }

      const result = await sandbox.run("bash", {
        args: ["-lc", "cat /tmp/desktop-test.txt"],
      });
      console.log(result.stdout.trim()); // docs-test
    } finally {
      await sandbox.terminate();
    }
    ```
  </Tab>
</Tabs>

Coordinate-based actions are screen-relative. A common workflow is:

1. Take a screenshot.
2. Inspect the desktop layout and note the coordinates you care about.
3. Use `move_mouse()` / `moveMouse()`, `click()`, `double_click()` / `doubleClick()`, and `scroll()` with those coordinates.

## Reconnect to an Existing Sandbox

If a sandbox is already running, connect by sandbox ID and attach to the desktop without creating a new VM.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    from pathlib import Path
    from tensorlake.sandbox import Sandbox

    sandbox_id = "your-running-sandbox-id"


    with Sandbox.connect(sandbox_id) as sandbox:
        with sandbox.connect_desktop(password="tensorlake") as desktop:
            Path("existing-sandbox.png").write_bytes(desktop.screenshot())
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import { writeFile } from "node:fs/promises";
    import { Sandbox } from "tensorlake";


    const sandbox = await Sandbox.connect({
      sandboxId: "your-running-sandbox-id",
    });

    try {
      const desktop = await sandbox.connectDesktop({
        password: "tensorlake",
      });

      try {
        const screenshot = await desktop.screenshot();
        await writeFile("existing-sandbox.png", screenshot);
      } finally {
        await desktop.close();
      }
    } finally {
      sandbox.close();
    }
    ```
  </Tab>
</Tabs>

Connecting to an existing sandbox only closes the client connection when you are done. It does not terminate the running VM.

## Connect with a VNC Client

If you want to drive the desktop from a real VNC viewer (Screen Sharing on macOS, TigerVNC, RealVNC, Remmina, etc.) rather than the SDK, open a TCP tunnel to the sandbox's VNC port and point your client at the local end. The tunnel keeps sandbox-proxy authentication local — you do **not** need to expose `5901` publicly.

Open the tunnel with `tl sbx tunnel`. Replace `<sandbox-id>` with the id printed by `tl sbx create` (or `tl sbx ls`):

```bash theme={null}
tl sbx tunnel <sandbox-id> 5901 --listen-port 15901
```

Leave that command running — it forwards `127.0.0.1:15901` on your machine to port `5901` inside the sandbox over an authenticated WebSocket. Then connect any VNC client to `localhost:15901` using the desktop password `tensorlake`:

<Tabs>
  <Tab title="macOS">
    Use the built-in Screen Sharing client:

    ```bash theme={null}
    open vnc://localhost:15901
    ```

    Enter `tensorlake` when macOS prompts for the password.
  </Tab>

  <Tab title="Linux (TigerVNC)">
    ```bash theme={null}
    vncviewer localhost:15901
    ```

    Most distributions ship `vncviewer` in the `tigervnc-viewer` package
    (`apt install tigervnc-viewer` on Debian/Ubuntu,
    `dnf install tigervnc` on Fedora).
  </Tab>

  <Tab title="Other">
    Any RFB-compatible viewer works — RealVNC Viewer, TightVNC, Remmina,
    KRDC, etc. Point it at `localhost:15901` and use `tensorlake` as the
    password.
  </Tab>
</Tabs>

Stop the tunnel with `Ctrl+C` when you are done. Closing the tunnel does not terminate the sandbox; reopen it any time with the same command.

## Use noVNC in the Browser

If you want a human to interact with the sandbox desktop in real time, use a real VNC client in the browser instead of polling screenshots. [`noVNC`](https://novnc.com/info.html) is a good fit here.

The recommended architecture is:

1. Keep the Tensorlake API key on your backend.
2. Use the backend to open a TCP tunnel to the sandbox's VNC port `5901`.
3. Bridge that local tunnel to a browser WebSocket endpoint such as `/vnc/<session-id>`.
4. Point `noVNC` at your backend WebSocket and authenticate with the desktop password `tensorlake`.

This keeps sandbox proxy authentication server-side and gives the browser a low-latency live desktop stream. You do **not** need to expose port `5901` publicly yourself.

If you are also running an agent loop, a good pattern is to use:

* `noVNC` for the live human-facing desktop stream
* `sandbox.connectDesktop()` for screenshots and high-level computer-use actions on the backend

That separation avoids turning the browser view into a screenshot polling loop.

### Browser Client with noVNC

Install `noVNC` in your frontend:

```bash theme={null}
npm install @novnc/novnc
```

Then connect the browser to your own WebSocket bridge:

```ts theme={null}
import RFB from "@novnc/novnc/lib/rfb";

const host = document.getElementById("desktop");
if (!(host instanceof HTMLDivElement)) {
  throw new Error("Missing #desktop container");
}

const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
const url = `${protocol}//${window.location.host}/vnc`;

const rfb = new RFB(host, url, {
  credentials: { password: "tensorlake" },
  shared: true,
});

rfb.scaleViewport = true;
rfb.clipViewport = false;
rfb.showDotCursor = true;
```

Use a fixed-size container for the desktop surface:

```html theme={null}
<div id="desktop" style="width: 1200px; height: 800px; background: black;"></div>
```

## Desktop API Surface

Python uses `snake_case`, while JavaScript uses `camelCase`, but both SDKs expose the same core capabilities:

* Screenshots: `screenshot()`
* Mouse input: `move_mouse()` / `moveMouse()`, `mouse_press()` / `mousePress()`, `mouse_release()` / `mouseRelease()`, `click()`, `double_click()` / `doubleClick()`, `scroll()`, `scroll_up()` / `scrollUp()`, and `scroll_down()` / `scrollDown()`
* Keyboard input: `key_down()` / `keyDown()`, `key_up()` / `keyUp()`, `press()`, and `type_text()` / `typeText()`
* Desktop size: `width` and `height`

`connect_desktop()` and `connectDesktop()` go through the authenticated sandbox proxy, so you do not need to bind or expose the VNC port yourself. For interactive debugging through a real VNC viewer, see [Connect with a VNC Client](#connect-with-a-vnc-client) above.

## Related Guides

* [Drive Chrome over CDP](/sandboxes/chrome-cdp) — point Playwright, Puppeteer, or `chrome-devtools-mcp` at the Chrome that ships in `tensorlake/ubuntu-vnc`.
* [Local Tunnels](/sandboxes/tunnels) — the tunneling primitive used by both the VNC viewer and Chrome CDP workflows.
* [Snapshots](/sandboxes/snapshots) — fork warm desktops to parallelize agent runs without re-launching XFCE.