SDK Reference

Applications

Applications are the top-level decorators that define entry points for your applications. You can define as many applications as you want in your project. Each one of them will be assigned a unique HTTP entry point based on the name of the Python function.

from tensorlake.applications import application, function

# This application's name will be `hello_world`.
@application()
@function()
def hello_world():
    print("Hello, world!")

# This application's name will be `hola_mundo`.
@application()
@function()
def hola_mundo():
    print("Hola, mundo!")

Configuring Applications

The @application decorator allows you to specify the following attributes:

tags - dict of tags to categorize the application.
retries - Retry policy for every function in the application unless a function specifies its own retry policy. No retries by default if function failed. See Retries.
region - The region where every function in the application will be deployed unless a function specifies its own region. Either us-east-1 or eu-west-1. The default is any of the regions.

The following code snippet shows an example of all the function attributes set to custom values.

from tensorlake.applications import application, Retries

@application(
    tags={"language": "python"},
    retries=Retries(max_retries=3),
    region="us-east-1",
)
@function()
def hello_world():
    print("Hello, world!")

Application inputs and output

Application functions take zero or more arguments which are the current request inputs. The inputs get deserialized from their JSON representation into Python objects specified in the arguments’ type hints. A reverse process happens for the request output. The object returned from the application function gets JSON serialized according to the return type hint of the function. The resulting JSON is returned as the HTTP response body of the application request. For example if your application function takes a single str argument and returns a str, then the request input and output should be JSON strings:

from tensorlake.applications import function, application

@application()
@function()
def greet(data: str) -> str:
    return data + " from greet!"

request input

"Hello, world!"

request output

"Hello, world! from greet!"

If you want to use multiple application request inputs with complex data structures, you can add more arguments and use type hints with your Pydantic model classes. Each type hint needs to be supported in Pydantic JSON mode. All basic type hints like str, int, float, bool, list, dict, set, tuple, None, Any, |, Pydantic model classes and more are supported. If a type hint is a union of multiple Python types, like str | int, then the request JSON input can match any of the types in the union.

from pydantic import BaseModel
from tensorlake.applications import application, function

class PersonSearchQuery(BaseModel):
    name: str
    age: int

class PersonSearchResult(BaseModel):
    matches: list[dict]

@application()
@function()
def process_data(query: PersonSearchQuery | list[PersonSearchQuery], limit: int | None = None) -> PersonSearchResult:
    if isinstance(query, list):
        return PersonSearchResult(
            matches=[
                {"name": q.name, "age": q.age, "id": i+1} for i, q in enumerate(query) if limit is None or i < limit
            ]
        )
    else:
        return PersonSearchResult(matches=[{"name": query.name, "age": query.age, "id": 1}])

request input

{"name": "John", "age": 30}

response output

{"matches":[{"name":"John","age":30,"id":1}]}

The limit argument is optional so we can omit it from the request input. If an argument type hint is Any, then the corresponding request input can be any valid JSON value (string, number, object, array, boolean, null). The JSON value gets deserialized into the corresponding Python object (str, int/float, dict, list, bool, None). The same applies to Any return type hint (i.e. a Python dict gets serialized as JSON object, a Python list as JSON array, etc.). If type hints are not provided then they are treated as Any.

Calling Applications

You can call applications remotely using any HTTP client or Tensorlake Python SDK. Use empty POST request body if the application takes no arguments. A JSON serialized request body is passed if the application takes one argument. Use multipart/form-data request body if the application takes multiple arguments or one or more files (see uploading and downloading files). i.e. to call the hello_world application defined above (takes no arguments):

curl \
https://api.tensorlake.ai/applications/hello_world \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
--json ''

i.e. to call the greet application defined above (takes a single string argument):

curl \
https://api.tensorlake.ai/applications/greet \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
--json '"Hello, John"'

i.e. to call the process_data application defined above (with multiple arguments):

query_value='[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 24}]'
limit_value='10'
curl \
https://api.tensorlake.ai/applications/process_data \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
-H "Accept: application/json" \
-F "query=$query_value;type=application/json" \
-F "limit=$limit_value;type=application/json"

If you pass an argument and application function doesn’t have it then it’s simply ignored. If an argument has a default value then you can omit it from the request input. Both of these features make it easy to update application code without breaking existing clients.

Uploading and downloading files

Application functions can receive files as current request inputs and return a file as a current request output. This makes it easy to build applications which process input files or produce an output file. File sizes of up to 5 TB are supported. The file input type is represented by the File class in Tensorlake SDK with the following interface:

class File:
    content: bytes  # Raw bytes of the file
    content_type: str  # MIME content type of the file

When an argument has a File type hint, Tensorlake SDK doesn’t attempt to deserialize the input from JSON and instead passes a File object with original request input binary content and content type. When the return type hint is File, Tensorlake SDK doesn’t JSON serialize the returned File object. It instead sets the request output content type to the File.content_type and the HTTP response body to the raw bytes in File.content.

from tensorlake.applications import function, application, File

@application()
@function()
def process_file(input: File) -> File:
    print(
        "Got file content type:",
        input.content_type,
        "size:",
         len(input.content),
         "bytes"
    )
    return File(
        # HTTP response body is the raw bytes of the input file.
        content=input.content,
        # HTTP response content type is the same as input file content type.
        content_type=input.content_type
    )

File.content field holds the raw bytes of the file uploaded to the application endpoint. At the moment, the SDK doesn’t support lazy loading large files, so the entire file is loaded into memory when the function is called. To pass a local file at /foo/bar/file_name.txt path to an application, you need to use a multipart/form-data HTTP request or just use Python SDK.

input_value='@/foo/bar/file_name.txt'
curl \
https://api.tensorlake.ai/applications/process_file \
-H "Authorization: Bearer $TENSORLAKE_API_KEY" \
-H "Accept: application/json" \
-F "input=$input_value"

Functions

Functions are the building blocks of applications. They are Python functions decorated with the @function decorator. Tensorlake functions can call other Tensorlake functions. The function call blocks until the called function returns its output to the calling function. For example, a simple application function which calls another function to process its input:

from tensorlake.applications import application, function

# Define an application function which is an HTTP entry point for the application.
@application()
@function()
def greet(name: str) -> str:
    if name.startswith("A"):
        return "Hello, A-name!"
    else:
        # Call another function to perform a specific processing of non-A names.
        return process_non_a_name(name) + " from greet!"

@function()
def process_non_a_name(name: str) -> str:
    return "A" + name[1:]

Every Tensorlake function call:

is executed in its own function container,
supports durable execution,
can run in parallel with other function calls,
can be retried independently if it fails,
has its own resource limits (CPU, memory, disk, GPU, timeout),
has its logs available in Tensorlake logging tools,
has its execution timeline available in Tensorlake tracing tools,
can report progress updates to extend its timeout,
can share state with other function calls of the same application request.

Every Python function decorated with @function becomes a Tensorlake function and thus gets all these capabilities automatically.

Configuring Tensorlake functions

The @function decorator allows you to set the following attributes:

description - A description of the function’s purpose and behavior.
cpu - The number of CPUs available to the function. The default is 1.0 CPU. See CPU.
memory - The memory GB available to the function. The default is 1.0 GB. See Memory.
ephemeral_disk - The ephemeral /tmp disk space available to the function in GB. The default is 2.0 GB. See Ephemeral Disk.
gpu - The GPU model available to the function. The default is None (no GPU). Please contact [email protected] to enable GPU support.
timeout - The timeout for the function in seconds. The default is 5 minutes. See Timeouts.
image - The image to use for the function container. A basic Debian based image by default. See Images.
secrets - The secrets available to the function in its environment variables. No secrets by default. See Secrets.
retries - Retry policy for the function. No retries by default if function failed. See Retries.
region - The region where the function will be deployed. Either us-east-1 or eu-west-1. The default is any of the regions.

The following code snippet shows an example of all the function attributes set to custom values.

from tensorlake.applications import function, Image, Retries

@function(
    # Use Ubuntu as a base image instead of the default Debian
    image=Image(base_image="ubuntu:latest"),
    # Make my_secret available to the function as an environment variable
    secrets=["my_secret"],
    # Description of the function in the workflow
    description="Measures the string using its length",
    # Retry the function twice if it fails
    retries=Retries(max_retries=2),
    # Function fails if it was running for more than 30 seconds and didn't report any progress
    timeout=30,
    # 2 CPUs are available to the function
    cpu=2,
    # 4 GB of memory is available to the function
    memory=4,
    # 2 GB of ephemeral /tmp disk space is available to the function
    ephemeral_disk=2,
    # Run the function in a container with GPU support
    gpu="H100",
    # Run the function in us-east-1 region only
    region="us-east-1",
)
def string_length(s: str) -> int:
    return len(s)

Function inputs and output

Tensorlake functions are not exposed as HTTP entry points unlike application functions. Because of this Tensorlake functions have minimal limitations on their signatures. Arguments and return value don’t require any type hints but have to be picklable. Most Python objects are picklable, except special cases like Processes, Threads, database connections, etc. If a Tensorlake function argument or return value is a Tensorlake SDK File object, then it bypasses pickling and is passed as-is to and from Tensorlake functions.

Application functions and Tensorlake Functions

Every application function decorated with @application() decorator is also a Tensorlake function. This is why every application function needs to be decorated with @function() decorator in addition to @application(). As application functions are HTTP entry points into Tensorlake applications, they have some differences compared to regular Tensorlake functions. Application functions:

Require JSON serializable type hints for all arguments and the return value.
Don’t support / and * in function arguments.
Don’t support *args and **kwargs.
Ignore function call arguments that are not defined in the function signature. This simplifies code migrations, i.e. if HTTP client sends extra arguments that the application function doesn’t take anymore after its code update.

Application functions can be called from regular Tensorlake functions. The call is executed in the current application request without creating a new one. So it behaves like a regular Tensorlake function call inside an application.

Classes

Sometimes a function needs one-time initialization, like loading a large model into memory. This is achieved by defining a class using the @cls decorator. Classes use their __init__(self) constructor to run any initialization code once on function container startup. The constructor can not have any arguments other than self. Any number of class methods can be decorated with @function.

from large_model import load_large_model
from tensorlake.applications import application, cls, function, run_remote_application

@cls()
class MyCompute:
    def __init__(self):
        # Run initialization code once on function container startup
        self.model = load_large_model()

    @application()
    @function(cpu=4, memory=16)
    def run(self, data: str) -> int:
        return self.model.run(data)

if __name__ == "__main__":
    run_remote_application("MyCompute.run", data="some input data")

Timeouts

When a function runs longer than its timeout, it is terminated and marked as failed. The timeout in seconds is set using the timeout attribute. The default timeout is 300 (5 minutes). Minimum is 1, maximum is 172800 (48 hours). Progress updates can be sent by the function to extend the timeout. See Request Context.

from tensorlake.applications import function

# Set a 30 minute timeout for long-running agent tasks
@function(timeout=1800)
def deep_research(prompt: str) -> str:
    ...

Retries

When a function fails by raising an exception or timing out, it gets retried according to its retry policy. The default retry policy is to not retry the function call. You can specify a custom retry policy using the retries attribute. If you allow retries, it’s typically a best practice to ensure that the function is idempotent unless this is not required for your use case.

from tensorlake.applications import function, Retries

# Retry the function once if it failed
@function(retries=Retries(max_retries=1))
def my_function() -> int:
    raise Exception("Something went wrong")

You can set default retry policy for all the functions in the application decorator. See the Configuring Applications guide.

Request Context

Functions can use a request context to share state between function calls of the same request. The context has information about the current request and provides access to APIs for the current request. You can access the request context directly from the RequestContext class.

from tensorlake.applications import RequestContext, function

@function()
def my_function(data: str) -> int:
    ctx: RequestContext = RequestContext.get()
    print(f"Request ID: {ctx.request_id}")
    ...

Request ID

Each request has a unique identifier accessible via ctx.request_id. This is useful for logging and debugging.

from tensorlake.applications import RequestContext, function

@function()
def my_function(data: str) -> str:
    ctx = RequestContext.get()
    print(f"Processing request: {ctx.request_id}")
    return ctx.request_id

Request State

The state API allows you to set and get key-value pairs scoped per request. Each new request starts with an empty state. Values can be any picklable object.

Method	Description
`state.set(key: str, value: Any) -> None`	Set a key-value pair
`state.get(key: str, default: Any \| None = None) -> Any \| None`	Get a value by key, returns default if not found

from tensorlake.applications import RequestContext, function

@function()
def first_function(data: str) -> int:
    ctx = RequestContext.get()
    ctx.state.set("user_data", data)
    ctx.state.set("processed", True)
    return second_function()

@function()
def second_function() -> int:
    ctx = RequestContext.get()
    data = ctx.state.get("user_data")
    return len(data)

Streaming Progress Updates

The progress API allows you to stream execution progress from your functions. This is useful for monitoring long-running tasks and providing real-time feedback to users.

from tensorlake.applications import RequestContext, function

@function()
def process_items(items: list) -> dict:
    ctx = RequestContext.get()
    results = []

    for i, item in enumerate(items):
        ctx.progress.update(i + 1, len(items), f"Processing item {i + 1}")
        results.append(process(item))

    return {"results": results}

Key features:

Automatic timeout reset - Each progress update resets the function timeout, allowing long-running agents to run indefinitely
Real-time streaming - Stream updates to frontends via Server-Sent Events (SSE)
HTTP API access - Query progress updates programmatically for custom dashboards and monitoring

See the Streaming Progress guide for detailed API reference, frontend integration examples, and best practices.

Request Metrics

The metrics API allows you to record custom metrics for monitoring and debugging.

Method	Description
`metrics.timer(name: str, value: int \| float) -> None`	Record a duration metric (in seconds)
`metrics.counter(name: str, value: int = 1) -> None`	Increment a counter by the given value. Every counter starts at 0.

import time
from tensorlake.applications import RequestContext, function

@function()
def my_function(data: str) -> int:
    ctx = RequestContext.get()
    start_time = time.monotonic()
    result = len(data)
    
    # Record metrics
    ctx.metrics.timer("processing_time", time.monotonic() - start_time)
    ctx.metrics.counter("items_processed", result)
    
    return result

CPU

The number of CPUs available to the function is set using the cpu attribute. Minimum is 1.0, maximum is 8.0. The default is 1.0. This is usually sufficient for functions that only call external APIs and do simple data processing. Adding more CPUs is recommended for functions that do complex data processing or work with large datasets. If functions use large multi-gigabyte inputs or produce large multi-gigabyte outputs, then at least 3 CPUs are recommended. This results in the fastest download and upload speeds for the data.

from tensorlake.applications import function

# Allocate 4 CPUs for data processing
@function(cpu=4)
def process_data(data: list) -> dict:
    ...

Memory

GB memory available to the function is set using the memory attribute. Minimum is 1.0, maximum is 32.0. The default is 1.0. This is usually sufficient for functions that only call external APIs and do simple data processing. Adding more memory is recommended for functions that do complex data processing or work with large datasets. It’s recommended to set memory to at least 2x the size of the largest inputs and outputs of the function. This is because when the inputs/outputs are deserialized/serialized both serialized and deserialized representations are kept in memory.

from tensorlake.applications import function

# Allocate 8 GB memory for loading large models
@function(memory=8)
def run_model(data: str) -> str:
    ...

Ephemeral disk

Ephemeral disk space is a temporary storage space available to functions at /tmp path. It gets erased when its function container gets terminated. It’s optimal for storing temporary files that are not needed after the function execution is completed. Ephemeral disks are backed by fast SSD drives. Using other filesystem paths like /home/ubuntu for storing temporary files will result in slower performance. Temporary files created using Python modules like tempfile are stored in ephemeral disk space inside /tmp. GB of ephemeral disk space available to the function is set using ephemeral_disk attribute. Minimum is 2.0, maximum is 50.0. The default is 2.0 GB. This is usually sufficient for functions that only call external APIs and do simple data processing. If the function needs to temporarily store large files or datasets on disk, then the ephemeral_disk attribute should be increased accordingly.

from tensorlake.applications import function

# Allocate 20 GB disk for downloading and processing large files
@function(ephemeral_disk=20)
def process_video(url: str) -> str:
    # Download video to /tmp
    # Process and return results
    ...

Overview

Concepts

Data Workflows

Guides

Production

Open Source

Applications

Configuring Applications

Application inputs and output

Calling Applications

Uploading and downloading files

Functions

Configuring Tensorlake functions

Function inputs and output

Application functions and Tensorlake Functions

Classes

Timeouts

Retries

Request Context

Request ID

Request State

Streaming Progress Updates

Request Metrics

CPU

Memory

Ephemeral disk

Overview

Concepts

Data Workflows

Guides

Production

Open Source

​Applications

​Configuring Applications

​Application inputs and output

​Calling Applications

​Uploading and downloading files

​Functions

​Configuring Tensorlake functions

​Function inputs and output

​Application functions and Tensorlake Functions

​Classes

​Timeouts

​Retries

​Request Context

​Request ID

​Request State

​Streaming Progress Updates

​Request Metrics

​CPU

​Memory

​Ephemeral disk

Applications

Configuring Applications

Application inputs and output

Calling Applications

Uploading and downloading files

Functions

Configuring Tensorlake functions

Function inputs and output

Application functions and Tensorlake Functions

Classes

Timeouts

Retries

Request Context

Request ID

Request State

Streaming Progress Updates

Request Metrics

CPU

Memory

Ephemeral disk