Context API Actions

Actions are the named enrichment operations the Context API can perform. Each action is specified by name in the actions object of a POST /content/process request.

All requests must include "version": "context.api/v2".

Every text and image action accepts an optional model field that chooses the AI model running that action. Model selection is in beta. Embedding actions do not accept model. Models other than the default can return different results, and those results are not always correct. Therefore, it is important to test a model on your own content before you rely on it. See Model Selection (Beta) for the available models, the default behavior, and the related errors.

Available Actions

Action key	Input type	Description
`imageClassification`	Image	Classifies an image into one of the provided categories.
`imageDescription`	Image	Generates a natural language description of an image.
`imageEmbeddings`	Image	Generates a vector embedding for an image.
`imageMetadataGeneration`	Image	Generates structured metadata from an image.
`namedEntityRecognitionImage`	Image	Detects named entities in an image.
`namedEntityRecognitionText`	Text	Extracts named entities from text.
`textClassification`	Text	Classifies a text document into one of the provided categories.
`textEmbeddings`	Text	Generates a vector embedding for a text document.
`textMetadataGeneration`	Text	Generates structured metadata from text.
`textSummarization`	Text	Generates a summary of a text document.

imageClassification

Classifies the input image into one of the provided categories.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Max Input Image Size	5 MB
Supported Image Formats	JPEG, PNG
Maximum Total Pixels per Image	8000 × 8000
Image Scaling	Images are scaled down preserving aspect ratio to meet size limits.
Recommended Max Pixels (Larger Side)	Less than 1568 pixels
Minimum Edge Size	Images under 200 pixels on any edge may degrade performance.

Parameters:

Field	Type	Required	Description
`classes`	`string[]`	Yes	At least two distinct, non-empty classification categories
`model`	`string`	No	AI model that runs this action. Must be one of the 5 vision-capable models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "imageClassification": {
      "classes": ["City", "Nature", "Indoor", "Portrait"]
    }
  }
}

Result field: imageClassification.result — string with the matched class label

imageDescription

Analyzes an image and generates a natural language description of its contents.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Accepted Image Formats	JPEG, PNG
Max Input Image Size	5 MB
Max Image Dimensions	8000 × 8000 pixels (scaled down preserving aspect ratio if exceeded)
Recommended Dimensions	Less than 1568 pixels on the longer side
Minimum Effective Size	Avoid images with any edge < 200 pixels (may degrade performance)

Parameters:

Field	Type	Required	Default	Description
`maxWordCount`	`integer`	No	`200`	Maximum number of words in the description. Must be > 0.
`model`	`string`	No	—	AI model that runs this action. Must be one of the 5 vision-capable models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	—	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "imageDescription": {
      "maxWordCount": 100
    }
  }
}

Result field: imageDescription.result — string with the generated description

imageEmbeddings

Converts an image into a high-dimensional vector representation. Use for visual similarity search and clustering.

Requirements: Input must be an image file.

Constraints:

Property	Value
Languages Supported	English
Max Input Image Size	15 MB
Supported Image Formats	PNG, JPEG
Output Vector Size	1,024

Parameters: None.

note

Image embeddings use path-based input only. The documentId input format is not supported for this action.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.jpg" }],
  "actions": {
    "imageEmbeddings": {}
  }
}

Result field: imageEmbeddings.result — flat float[] array (v2). See Embeddings for format details.

imageMetadataGeneration

Generates structured metadata from an image, optionally guided by example metadata templates.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Max Input Image Size	5 MB
Supported Image Formats	JPEG, PNG
Max Total Pixels per Image	8000 px × 8000 px
Image Scaling	Images are scaled down (preserving aspect ratio) to fit within size limits
Recommended Image Size	Less than 1568 pixels on the larger side
Minimum Image Size Warning	Images under 200 pixels on any edge may degrade performance

Parameters:

Field	Type	Required	Description
`kSimilarMetadata`	`object[]`	Yes	At least one example metadata object to guide generation. Each item is a free-form JSON object with representative keys and values.
`model`	`string`	No	AI model that runs this action. Must be one of the 5 vision-capable models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "imageMetadataGeneration": {
      "kSimilarMetadata": [
        {
          "location": "New York City",
          "keywords": "urban|city|skyline",
          "summary": "Urban street photography"
        }
      ]
    }
  }
}

Result field: imageMetadata.result — JSON object with generated metadata fields.

namedEntityRecognitionImage

Detects named entities (people, organizations, locations) in an image.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Max Input Image Size	5 MB
Supported Image Formats	JPEG, PNG
Maximum Total Pixels per Image	8000 × 8000
Image Scaling	Images are scaled down preserving aspect ratio to meet size limits.
Recommended Max Pixels (Larger Side)	Less than 1568 pixels
Minimum Edge Size	Images under 200 pixels on any edge may degrade performance.

Parameters:

Field	Type	Required	Description
`model`	`string`	No	AI model that runs this action. Must be one of the 5 vision-capable models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "namedEntityRecognitionImage": {}
  }
}

Result field: namedEntityImage.result — dictionary mapping entity type names to arrays of detected values (for example, { "organizations": ["Hyland"], "locations": ["Times Square"] })

namedEntityRecognitionText

Extracts named entities from document text.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Description
`model`	`string`	No	AI model that runs this action. Must be one of the 9 models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../document.pdf" }],
  "actions": {
    "namedEntityRecognitionText": {
      "instructions": {
        "focus": "Extract company names and contract dates"
      }
    }
  }
}

Result field: namedEntityText.result — dictionary mapping entity type names to arrays of detected values (for example, { "organizations": ["Hyland"], "dates_times": ["2025-01-01"] })

textClassification

Classifies a text document into one of the provided categories.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Description
`classes`	`string[]`	Yes	At least two distinct, non-empty classification categories
`model`	`string`	No	AI model that runs this action. Must be one of the 9 models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../document.pdf" }],
  "actions": {
    "textClassification": {
      "classes": ["Report", "Contract", "Invoice", "Other"]
    }
  }
}

Result field: textClassification.result — string with the matched class label

textEmbeddings

Generates a vector embedding for a text document.

Requirements: Input must be a text-based document.

Parameters: None

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../document.pdf" }],
  "actions": {
    "textEmbeddings": {}
  }
}

Result field: textEmbeddings.result — flat float[] array (v2). See Embeddings for format details.

textMetadataGeneration

Generates structured metadata from a text document, optionally guided by example metadata templates.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Description
`kSimilarMetadata`	`object[]`	Yes	At least one example metadata object to guide generation
`model`	`string`	No	AI model that runs this action. Must be one of the 9 models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../report.pdf" }],
  "actions": {
    "textMetadataGeneration": {
      "kSimilarMetadata": [
        {
          "document_type": "Financial Report",
          "author": "John Smith",
          "date": "2025-01"
        }
      ]
    }
  }
}

Result field: textMetadata.result — JSON object with generated metadata fields

textSummarization

Generates a summary of a text document.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Default	Description
`maxWordCount`	`integer`	No	`200`	Maximum word count of the summary. Must be > 0.
`model`	`string`	No	—	AI model that runs this action. Must be one of the 9 models listed in Model Selection (Beta). When omitted, the action runs on its default model.
`instructions`	`object`	No	—	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../report.pdf" }],
  "actions": {
    "textSummarization": {
      "maxWordCount": 150,
      "instructions": {
        "focus": "Key financial metrics and risks",
        "style": "professional"
      }
    }
  }
}

Result field: textSummary.result — string with the generated summary

Available Actions​

imageClassification​

imageDescription​

imageEmbeddings​

imageMetadataGeneration​

namedEntityRecognitionImage​

namedEntityRecognitionText​

textClassification​

textEmbeddings​

textMetadataGeneration​

textSummarization​

Available Actions

imageClassification

imageDescription

imageEmbeddings

imageMetadataGeneration

namedEntityRecognitionImage

namedEntityRecognitionText

textClassification

textEmbeddings

textMetadataGeneration

textSummarization