> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensorlake.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# GPU Sandboxes

> Run CUDA, PyTorch, vLLM, SGLang in GPU sandboxes.

GPU sandboxes are available by request. [Reach out on Slack](https://join.slack.com/t/tensorlakecloud/shared_invite/zt-32fq4nmib-gO0OM5RIar3zLOBm~ZGqKg) to get access.
They run on gVisor rather than the Firecracker and Cloud Hypervisor VMs that regular sandboxes use, which brings the following constraints:

* CPU and memory can't be live resized
* Tensorlake filesystems (TLFS) can't be mounted
* Only work with [CAS images](/sandboxes/cas-images.mdx)

## Example: create and use a vLLM sandbox

This example creates a GPU sandbox from [Tensorlake's vLLM image](/sandboxes/tensorlake-images), starts a vLLM server in it, sends one prompt, and prints the reply.
You can also [build or import your own image](/sandboxes/images).

### Create the sandbox

```bash theme={null}
SANDBOX_ID=$(tl sbx create --image tensorlake/cas/vllm:0.26 --gpu H100 \
  --cpus 4 --memory 32768 --disk_mb 102400)
```

### Start the vLLM server

Start vLLM in the foreground. It downloads the model from Hugging Face, loads it onto the GPU, and streams its output to your terminal:

```bash theme={null}
tl sbx exec $SANDBOX_ID -e PYTHONUNBUFFERED=1 \
  python3 -m vllm.entrypoints.openai.api_server \
    --model HuggingFaceTB/SmolLM2-135M-Instruct \
    --served-model-name smollm2 --host 127.0.0.1 --port 8000 \
    --max-model-len 4096 --gpu-memory-utilization 0.85 --enforce-eager
```

When the output shows `Application startup complete`, press Ctrl+C. That stops the stream, not the server: vLLM keeps running in the sandbox.

### Send a prompt

The server listens on loopback, so send the request from inside the sandbox. Terminate the sandbox afterwards to release its GPU.

```bash theme={null}
tl sbx exec $SANDBOX_ID python3 -c '
import json, urllib.request
request = urllib.request.Request(
    "http://127.0.0.1:8000/v1/chat/completions",
    data=json.dumps({"model": "smollm2", "messages": [
        {"role": "user", "content": "Say hello in exactly three words."}]}).encode(),
    headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(request))["choices"][0]["message"]["content"])
'

tl sbx terminate $SANDBOX_ID
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.