Workflows are not Generally Available yet. Please contact us for early access.
Setup
- The Tensorlake package provides the SDK to build graphs and a CLI to deploy them.
- Get an API key from the Tensorlake Dashboard.
Your First Workflow
Workflows are defined by writing Python functions and their data-flow is defined by connecting them as graphs. This allows parallel execution of disjoint parts of the workflows, map-reduce, dynamic routing, etc.greeter_graph.py
Deploy a Graph
https://api.tensorlake.ai/v1/workflows/greeter
.
Invoke a Graph
Once you have deployed your graph, it’s available as an HTTP endpoint. You can either invoke it via HTTP or use the Python SDK. Making a request to your graph will return a Request ID. You can use this request ID to get the output of the graph.Get the Output of a Graph
Once the graph is complete, you can get the output of the graph of any function in your workflow. This is useful because often the intermediate stages of a workflow might have useful data that you want to use in your application, or you might want them for debugging purposes.None
value in this case.
Generally, you would want to check if the request is complete before getting the output.
Test a Graph Locally
These Graphs can run locally on your laptop, so you can test them before deploying them to the cloud.greeter_graph.py
Include Python Dependencies and Secrets
Most real world workflows will require python packages and access to secrets. For example, if you are building a workflow to extract personal information from Driving licenses using OpenAI’s structured output - you will need the OpenAI package, and your OpenAI API key.structured_extraction.py
OPENAI_API_KEY
secret to authenticate with OpenAI.
Building custom images allows you to install pretty much anything you want in your function. The Image
api allows running any commands, or other standard commands Docker allows.
The secrets
argument is used to specify the secrets that are available to the functions. These secrets have to be specified before the functions are deployed.
So lets deploy the secrets first, get your OpenAI API key and then run the following command:
Programming Guide
Learn more about customizing functions and compute resources for workflows.
Dynamic Routing
Learn how to route data between functions dynamically during request processing.
Map-Reduce and Parallel Processing
Learn how to use map-reduce and parallel processing to process requests faster.
Images
Guide to building custom images for functions.