cURL
Python
JavaScript
PHP
Go
Java
curl --request POST \
--url https://api.tensorlake.ai/documents/v1/datasets \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"description": "<string>",
"name": "<string>",
"settings": {
"chunkStrategy": null,
"deliverWebhook": "false",
"figureSummarization": true,
"figureSummarizationPrompt": "none",
"formDetectionMode": "vlm",
"jsonSchema": "<any>",
"modelProvider": null,
"structuredExtractionPrompt": "none",
"tableOutputMode": "json",
"tableParsingMode": "tsr",
"tableSummarization": true,
"tableSummarizationPrompt": "none"
}
}' Create an ingestion workflow for structured extraction or document parsing.
A dataset is a collection of settings that help with organizing documents from the same domain and enable focused
document intelligence.
The dataset’s name must be unique.
Your data is NOT sent to a third party service(OpenAI, Anthropic, etc), and uses our own models to parse the document.
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
settings. chunkStrategy
enum<string> | null
default: none
The chunking strategy determines how the document is chunked into smaller pieces.
This is only supported in markdown mode.
Available options:
page,
section,
fragment
Whether to deliver a webhook when the job is completed.
A webhook needs to be configured for this to work.
If a webhook is not configured, the job will still be processed but the webhook will not be delivered.
settings. figureSummarization
Whether to summarize the contents of the figures.
settings. figureSummarizationPrompt
string | null
default: none
The prompt to use for figure summarization.
If not provided, the default prompt will be used.
settings. formDetectionMode
Whether to summarize the contents of the tables.
Available options:
vlm,
object_detection
The JSON schema to guide structured data extraction from the file.
Encode the JSON schema as a string.
If provided, the pages argument will be ignored.
settings. modelProvider
enum<string> | null
default: tensorlake
The model provider to use for structured data extraction.
Specifying tensorlake will use a private model, which runs on our servers.
Available options:
tensorlake,
claude-3-5-sonnet-latest,
gpt-4o-mini
settings. structuredExtractionPrompt
string | null
default: none
Overide the prompt to customize structured extractions. Use this if you
want to extract data from a file using a different prompt than the one
we use to extract.
settings. tableOutputMode
enum<string>
default: markdown
The mode to use for table output - JSON, Markdown or HTML
JSON mode is great for structured tables.
Markdown mode is great for tables without merged cells.
HTML mode is great for tables with merged cells.
Available options:
json,
markdown,
html
settings. tableParsingMode
The mode to use for table parsing - Table Structure Understanding or VLM
Table Structure Understanding is the default mode. It's great for structured tables.
VLM is great for unstructured or semi-structured tables.
Available options:
tsr,
vlm
settings. tableSummarization
Whether to summarize the contents of the tables.
settings. tableSummarizationPrompt
string | null
default: none
The prompt to use for table summarization.
Create a new dataset. Reference the name to insert structured data from documents extracted by the Structured Extraction API automatically.
