> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensorlake.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# SDK Reference
> Functions, applications, decorators, request context, and lifecycle reference
## Applications
Applications are the top-level decorators that define entry points for your applications. You can define as many applications as you want in
your project. Each one of them will be assigned a unique HTTP entry point based on the name of the Python function.
```python theme={null}
from tensorlake.applications import application, function
# This application's name will be `hello_world`.
@application()
@function()
def hello_world():
print("Hello, world!")
# This application's name will be `hola_mundo`.
@application()
@function()
def hola_mundo():
print("Hola, mundo!")
```
### Configuring Applications
The `@application` decorator allows you to specify the following attributes:
1. `tags` - dict of tags to categorize the application.
2. `retries` - Retry policy for every function in the application unless a function specifies its own retry policy. No retries by default if function failed. See [Retries](/applications/concepts#retries).
3. `region` - The region where every function in the application will be deployed unless a function specifies its own region. Either `us-east-1` or `eu-west-1`. The default is any of the regions.
The following code snippet shows an example of all the function attributes set to custom values.
```python theme={null}
from tensorlake.applications import application, Retries
@application(
tags={"language": "python"},
retries=Retries(max_retries=3),
region="us-east-1",
)
@function()
def hello_world():
print("Hello, world!")
```
### Application inputs and output
Application functions take zero or more arguments which are the current request inputs.
The inputs get deserialized from their JSON representation into Python objects specified
in the arguments' type hints. A reverse process happens for the request output.
The object returned from the application function gets JSON serialized according to the return
type hint of the function. The resulting JSON is returned as the HTTP response body of the application request.
For example if your application function takes a single `str` argument and returns a `str`, then the
request input and output should be JSON strings:
```python theme={null}
from tensorlake.applications import function, application
@application()
@function()
def greet(data: str) -> str:
return data + " from greet!"
```
```json request input theme={null}
"Hello, world!"
```
```json request output theme={null}
"Hello, world! from greet!"
```
If you want to use multiple application request inputs with complex data structures,
you can add more arguments and use type hints with your Pydantic model classes.
Each type hint needs to be [supported in Pydantic JSON mode](https://docs.pydantic.dev/latest/concepts/serialization/#json-mode).
All basic type hints like `str`, `int`, `float`, `bool`, `list`, `dict`, `set`, `tuple`, `None`, `Any`, `|`,
Pydantic model classes and more are supported. If a type hint is a union of multiple Python types,
like `str | int`, then the request JSON input can match any of the types in the union.
```python theme={null}
from pydantic import BaseModel
from tensorlake.applications import application, function
class PersonSearchQuery(BaseModel):
name: str
age: int
class PersonSearchResult(BaseModel):
matches: list[dict]
@application()
@function()
def process_data(query: PersonSearchQuery | list[PersonSearchQuery], limit: int | None = None) -> PersonSearchResult:
if isinstance(query, list):
return PersonSearchResult(
matches=[
{"name": q.name, "age": q.age, "id": i+1} for i, q in enumerate(query) if limit is None or i < limit
]
)
else:
return PersonSearchResult(matches=[{"name": query.name, "age": query.age, "id": 1}])
```
```json request input theme={null}
{"name": "John", "age": 30}
```
```json response output theme={null}
{"matches":[{"name":"John","age":30,"id":1}]}
```
The limit argument is optional so we can omit it from the request input.
If an argument type hint is `Any`, then the corresponding request input can be any valid JSON value
(string, number, object, array, boolean, null). The JSON value gets deserialized into the corresponding
Python object (`str`, `int`/`float`, `dict`, `list`, `bool`, `None`). The same applies to `Any` return type hint
(i.e. a Python dict gets serialized as JSON object, a Python list as JSON array, etc.).
If type hints are not provided then they are treated as `Any`.
### Calling Applications
You can call applications remotely using any HTTP client or Tensorlake Python SDK.
Use empty POST request body if the application takes no arguments.
A JSON serialized request body is passed if the application takes one argument.
Use multipart/form-data request body if the application takes multiple arguments
or one or more files (see [uploading and downloading files](/applications/concepts#uploading-and-downloading-files)).
i.e. to call the `hello_world` application defined above (takes no arguments):
```bash bash theme={null}
curl \
https://api.tensorlake.ai/applications/hello_world \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
--json ''
```
```python python theme={null}
from tensorlake.applications import run_remote_application
run_remote_application("hello_world")
```
i.e. to call the `greet` application defined above (takes a single string argument):
```bash bash theme={null}
curl \
https://api.tensorlake.ai/applications/greet \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
--json '"Hello, John"'
```
```python python theme={null}
from tensorlake.applications import run_remote_application
run_remote_application("greet", "Hello, John")
# Or:
# run_remote_application("greet", data="Hello, John")
```
i.e. to call the `process_data` application defined above (with multiple arguments):
```bash bash theme={null}
query_value='[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 24}]'
limit_value='10'
curl \
https://api.tensorlake.ai/applications/process_data \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
-H "Accept: application/json" \
-F "query=$query_value;type=application/json" \
-F "limit=$limit_value;type=application/json"
```
```python python theme={null}
from tensorlake.applications import run_remote_application
run_remote_application(
"process_data",
query=[
PersonSearchQuery(name="Alice", age=25),
PersonSearchQuery(name="Bob", age=24)
],
limit=10
)
```
If you pass an argument and application function doesn't have it then it's simply ignored.
If an argument has a default value then you can omit it from the request input.
Both of these features make it easy to update application code without breaking existing clients.
### Uploading and downloading files
Application functions can receive files as current request inputs and return a file as a current request output.
This makes it easy to build applications which process input files or produce an output file. File sizes of up
to 5 TB are supported. The file input type is represented by the `File` class in Tensorlake SDK with the following
interface:
```python theme={null}
class File:
content: bytes # Raw bytes of the file
content_type: str # MIME content type of the file
```
When an argument has a `File` type hint, Tensorlake SDK doesn't attempt to deserialize the input from JSON and
instead passes a `File` object with original request input binary content and content type. When the return type
hint is `File`, Tensorlake SDK doesn't JSON serialize the returned `File` object. It instead sets the request
output content type to the `File.content_type` and the HTTP response body to the raw bytes in `File.content`.
```python theme={null}
from tensorlake.applications import function, application, File
@application()
@function()
def process_file(input: File) -> File:
print(
"Got file content type:",
input.content_type,
"size:",
len(input.content),
"bytes"
)
return File(
# HTTP response body is the raw bytes of the input file.
content=input.content,
# HTTP response content type is the same as input file content type.
content_type=input.content_type
)
```
`File.content` field holds the raw bytes of the file uploaded to the application endpoint.
At the moment, the SDK doesn't support lazy loading large files, so the entire file is loaded into memory
when the function is called.
To pass a local file at `/foo/bar/file_name.txt` path to an application, you need to use a multipart/form-data
HTTP request or just use Python SDK.
```bash bash theme={null}
input_value='@/foo/bar/file_name.txt'
curl \
https://api.tensorlake.ai/applications/process_file \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
-H "Accept: application/json" \
-F "input=$input_value"
```
```python python theme={null}
from tensorlake.applications import run_remote_application, File
# Note: File object from Tensorlake SDK is not the same as File object from Python standard library.
with open("/foo/bar/file_name.txt", "rb") as local_file:
local_file_content: bytes = local_file.read()
run_remote_application(
"process_file",
File(
content=local_file_content,
content_type="text/plain"
)
)
```
## Functions
Functions are the building blocks of applications. They are Python functions decorated with the `@function` decorator.
Tensorlake functions can call other Tensorlake functions. The function call blocks until the called function returns
its output to the calling function.
For example, a simple application function which calls another function to process its input:
```python theme={null}
from tensorlake.applications import application, function
# Define an application function which is an HTTP entry point for the application.
@application()
@function()
def greet(name: str) -> str:
if name.startswith("A"):
return "Hello, A-name!"
else:
# Call another function to perform a specific processing of non-A names.
return process_non_a_name(name) + " from greet!"
@function()
def process_non_a_name(name: str) -> str:
return "A" + name[1:]
```
Every Tensorlake function call:
* is executed in its own function container,
* supports durable execution,
* can run in parallel with other function calls,
* can be retried independently if it fails,
* has its own resource limits (CPU, memory, disk, GPU, timeout),
* has its logs available in Tensorlake logging tools,
* has its execution timeline available in Tensorlake tracing tools,
* can report progress updates to extend its timeout,
* can share state with other function calls of the same application request.
Every Python function decorated with `@function` becomes a Tensorlake function and thus gets
all these capabilities automatically.
### Configuring Tensorlake functions
The `@function` decorator allows you to set the following attributes:
1. `description` - A description of the function's purpose and behavior.
2. `cpu` - The number of CPUs available to the function. The default is `1.0` CPU. See [CPU](/applications/concepts#cpu).
3. `memory` - The memory GB available to the function. The default is `1.0` GB. See [Memory](/applications/concepts#memory).
4. `ephemeral_disk` - The ephemeral `/tmp` disk space available to the function in GB. The default is `2.0` GB. See [Ephemeral Disk](/applications/concepts#ephemeral-disk).
5. `gpu` - The GPU model available to the function. The default is `None` (no GPU). Please contact `support@tensorlake.ai` to enable GPU support.
6. `timeout` - The timeout for the function in seconds. The default is 5 minutes. See [Timeouts](/applications/concepts#timeouts).
7. `image` - The image to use for the function container. A basic Debian based image by default. See [Images](/applications/images).
8. `secrets` - The secrets available to the function in its environment variables. No secrets by default. See [Secrets](/applications/secrets).
9. `retries` - Retry policy for the function. No retries by default if function failed. See [Retries](/applications/concepts#retries).
10. `region` - The region where the function will be deployed. Either `us-east-1` or `eu-west-1`. The default is any of the regions.
The following code snippet shows an example of all the function attributes set to custom values.
```python theme={null}
from tensorlake.applications import function, Image, Retries
@function(
# Use Ubuntu as a base image instead of the default Debian
image=Image(base_image="ubuntu:latest"),
# Make my_secret available to the function as an environment variable
secrets=["my_secret"],
# Description of the function in the workflow
description="Measures the string using its length",
# Retry the function twice if it fails
retries=Retries(max_retries=2),
# Function fails if it was running for more than 30 seconds and didn't report any progress
timeout=30,
# 2 CPUs are available to the function
cpu=2,
# 4 GB of memory is available to the function
memory=4,
# 2 GB of ephemeral /tmp disk space is available to the function
ephemeral_disk=2,
# Run the function in a container with GPU support
gpu="H100",
# Run the function in us-east-1 region only
region="us-east-1",
)
def string_length(s: str) -> int:
return len(s)
```
### Function inputs and output
Tensorlake functions are not exposed as HTTP entry points unlike application functions.
Because of this Tensorlake functions have minimal limitations on their signatures.
Arguments and return value don't require any type hints but have to be picklable.
Most Python objects are picklable, except special cases like Processes, Threads,
database connections, etc.
If a Tensorlake function argument or return value is a Tensorlake SDK `File` object, then it
bypasses pickling and is passed as-is to and from Tensorlake functions.
### Application functions and Tensorlake Functions
Every application function decorated with `@application()` decorator is also a Tensorlake function.
This is why every application function needs to be decorated with `@function()` decorator in addition
to `@application()`.
As application functions are HTTP entry points into Tensorlake applications, they have some differences
compared to regular Tensorlake functions. Application functions:
* Require JSON serializable type hints for all arguments and the return value.
* Don't support `/` and `*` in function arguments.
* Don't support `*args` and `**kwargs`.
* Ignore function call arguments that are not defined in the function signature. This simplifies
code migrations, i.e. if HTTP client sends extra arguments that the application function doesn't
take anymore after its code update.
Application functions can be called from regular Tensorlake functions. The call is executed in the current
application request without creating a new one. So it behaves like a regular Tensorlake function call
inside an application.
### Classes
Sometimes a function needs one-time initialization, like loading a large model into memory.
This is achieved by defining a class using the `@cls` decorator. Classes use their `__init__(self)` constructor to run any initialization code once on function container startup.
The constructor can not have any arguments other than `self`. Any number of class methods can be decorated with `@function`.
```python theme={null}
from large_model import load_large_model
from tensorlake.applications import application, cls, function, run_remote_application
@cls()
class MyCompute:
def __init__(self):
# Run initialization code once on function container startup
self.model = load_large_model()
@application()
@function(cpu=4, memory=16)
def run(self, data: str) -> int:
return self.model.run(data)
if __name__ == "__main__":
run_remote_application("MyCompute.run", data="some input data")
```
### Timeouts
When a function runs longer than its timeout, it is terminated and marked as failed. The timeout in seconds is set using the `timeout` attribute.
The default timeout is `300` (5 minutes). Minimum is `1`, maximum is `172800` (48 hours). Progress updates can be sent by the function to extend the
timeout. See [Request Context](/applications/concepts#request-context).
```python theme={null}
from tensorlake.applications import function
# Set a 30 minute timeout for long-running agent tasks
@function(timeout=1800)
def deep_research(prompt: str) -> str:
...
```
### Retries
When a function fails by raising an exception or timing out, it gets retried according to its retry policy.
The default retry policy is to not retry the function call. You can specify a custom retry policy using the `retries` attribute.
If you allow retries, it's typically a best practice to ensure that the function is idempotent unless this is not required for your use case.
```python theme={null}
from tensorlake.applications import function, Retries
# Retry the function once if it failed
@function(retries=Retries(max_retries=1))
def my_function() -> int:
raise Exception("Something went wrong")
```
You can set default retry policy for all the functions in the application decorator. See the [Configuring Applications](/applications/concepts#configuring-applications) guide.
### Request Context
Functions can use a request context to share state between function calls of the same request. The context has information about the current request and provides access
to APIs for the current request. You can access the request context directly from the `RequestContext` class.
```python theme={null}
from tensorlake.applications import RequestContext, function
@function()
def my_function(data: str) -> int:
ctx: RequestContext = RequestContext.get()
print(f"Request ID: {ctx.request_id}")
...
```
#### Request ID
Each request has a unique identifier accessible via `ctx.request_id`. This is useful for logging and debugging.
```python theme={null}
from tensorlake.applications import RequestContext, function
@function()
def my_function(data: str) -> str:
ctx = RequestContext.get()
print(f"Processing request: {ctx.request_id}")
return ctx.request_id
```
#### Request State
The state API allows you to set and get key-value pairs scoped per request. Each new request starts with an empty state. Values can be any picklable object.
| Method | Description |
| ----------------------------------------------------------------- | ------------------------------------------------ |
| `state.set(key: str, value: Any) -> None` | Set a key-value pair |
| `state.get(key: str, default: Any \| None = None) -> Any \| None` | Get a value by key, returns default if not found |
```python theme={null}
from tensorlake.applications import RequestContext, function
@function()
def first_function(data: str) -> int:
ctx = RequestContext.get()
ctx.state.set("user_data", data)
ctx.state.set("processed", True)
return second_function()
@function()
def second_function() -> int:
ctx = RequestContext.get()
data = ctx.state.get("user_data")
return len(data)
```
#### Streaming Progress Updates
The progress API allows you to stream execution progress from your functions. This is useful for monitoring long-running tasks and providing real-time feedback to users.
```python theme={null}
from tensorlake.applications import RequestContext, function
@function()
def process_items(items: list) -> dict:
ctx = RequestContext.get()
results = []
for i, item in enumerate(items):
ctx.progress.update(i + 1, len(items), f"Processing item {i + 1}")
results.append(process(item))
return {"results": results}
```
**Key features:**
* **Automatic timeout reset** - Each progress update resets the function timeout, allowing long-running agents to run indefinitely
* **Real-time streaming** - Stream updates to frontends via Server-Sent Events (SSE)
* **HTTP API access** - Query progress updates programmatically for custom dashboards and monitoring
See the [Streaming Progress guide](/applications/guides/streaming-progress) for detailed API reference, frontend integration examples, and best practices.
#### Request Metrics
The metrics API allows you to record custom metrics for monitoring and debugging.
| Method | Description |
| ------------------------------------------------------- | ------------------------------------------------------------------ |
| `metrics.timer(name: str, value: int \| float) -> None` | Record a duration metric (in seconds) |
| `metrics.counter(name: str, value: int = 1) -> None` | Increment a counter by the given value. Every counter starts at 0. |
```python theme={null}
import time
from tensorlake.applications import RequestContext, function
@function()
def my_function(data: str) -> int:
ctx = RequestContext.get()
start_time = time.monotonic()
result = len(data)
# Record metrics
ctx.metrics.timer("processing_time", time.monotonic() - start_time)
ctx.metrics.counter("items_processed", result)
return result
```
### CPU
{/* TODO: https://github.com/tensorlakeai/indexify/issues/1657 */}
The number of CPUs available to the function is set using the `cpu` attribute. Minimum is `1.0`, maximum is `8.0`.
The default is `1.0`. This is usually sufficient for functions that only call external APIs and do simple data processing.
Adding more CPUs is recommended for functions that do complex data processing or work with large datasets.
If functions use large multi-gigabyte inputs or produce large multi-gigabyte outputs, then at least 3 CPUs are recommended.
This results in the fastest download and upload speeds for the data.
```python theme={null}
from tensorlake.applications import function
# Allocate 4 CPUs for data processing
@function(cpu=4)
def process_data(data: list) -> dict:
...
```
### Memory
{/* TODO: https://github.com/tensorlakeai/indexify/issues/1657 */}
GB memory available to the function is set using the `memory` attribute. Minimum is `1.0`, maximum is `32.0`.
The default is `1.0`. This is usually sufficient for functions that only call external APIs and do simple data processing.
Adding more memory is recommended for functions that do complex data processing or work with large datasets.
It's recommended to set `memory` to at least 2x the size of the largest inputs and outputs of the function.
This is because when the inputs/outputs are deserialized/serialized both serialized and deserialized representations are
kept in memory.
```python theme={null}
from tensorlake.applications import function
# Allocate 8 GB memory for loading large models
@function(memory=8)
def run_model(data: str) -> str:
...
```
### Ephemeral disk
{/* TODO: https://github.com/tensorlakeai/indexify/issues/1657 */}
Ephemeral disk space is a temporary storage space available to functions at `/tmp` path. It gets erased when its
function container gets terminated. It's optimal for storing temporary files that are not needed after the function
execution is completed. Ephemeral disks are backed by fast SSD drives. Using other filesystem paths like `/home/ubuntu`
for storing temporary files will result in slower performance. Temporary files created using Python modules like `tempfile`
are stored in ephemeral disk space inside `/tmp`.
GB of ephemeral disk space available to the function is set using `ephemeral_disk` attribute. Minimum is `2.0`, maximum is `50.0`.
The default is `2.0` GB. This is usually sufficient for functions that only call external APIs and do simple data processing.
If the function needs to temporarily store large files or datasets on disk, then the `ephemeral_disk` attribute should be increased
accordingly.
```python theme={null}
from tensorlake.applications import function
# Allocate 20 GB disk for downloading and processing large files
@function(ephemeral_disk=20)
def process_video(url: str) -> str:
# Download video to /tmp
# Process and return results
...
```