Context API Actions

Actions are the named enrichment operations the Context API can perform. Each action is specified by name in the actions object of a POST /content/process request.

All requests must include "version": "context.api/v2".

---

Available Actions

Action key	Input type	Description
imageClassification	Image	Classifies an image into one of the provided categories.
imageDescription	Image	Generates a natural language description of an image.
imageEmbeddings	Image	Generates a vector embedding for an image.
imageMetadataGeneration	Image	Generates structured metadata from an image.
namedEntityRecognitionImage	Image	Detects named entities in an image.
namedEntityRecognitionText	Text	Extracts named entities from text.
textClassification	Text	Classifies a text document into one of the provided categories.
textEmbeddings	Text	Generates a vector embedding for a text document.
textMetadataGeneration	Text	Generates structured metadata from text.
textSummarization	Text	Generates a summary of a text document.

---

imageClassification

Classifies the input image into one of the provided categories.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Max Input Image Size	5 MB
Supported Image Formats	JPEG, PNG
Maximum Total Pixels per Image	8000 × 8000
Image Scaling	Images are scaled down preserving aspect ratio to meet size limits.
Recommended Max Pixels (Larger Side)	Less than 1568 pixels
Minimum Edge Size	Images under 200 pixels on any edge may degrade performance.

Parameters:

Field	Type	Required	Description
classes	string[]	Yes	At least two distinct, non-empty classification categories
instructions	object	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "imageClassification": {
      "classes": ["City", "Nature", "Indoor", "Portrait"]
    }
  }
}

Result field: imageClassification.result — string with the matched class label

---

imageDescription

Analyzes an image and generates a natural language description of its contents.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Accepted Image Formats	JPEG, PNG
Max Input Image Size	5 MB
Max Image Dimensions	8000 × 8000 pixels (scaled down preserving aspect ratio if exceeded)
Recommended Dimensions	Less than 1568 pixels on the longer side
Minimum Effective Size	Avoid images with any edge < 200 pixels (may degrade performance)

Parameters:

Field	Type	Required	Default	Description
maxWordCount	integer	No	200	Maximum number of words in the description. Must be > 0.
instructions	object	No	—	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "imageDescription": {
      "maxWordCount": 100
    }
  }
}

Result field: imageDescription.result — string with the generated description

---

imageEmbeddings

Converts an image into a high-dimensional vector representation. Use for visual similarity search and clustering.

Requirements: Input must be an image file.

Constraints:

Property	Value
Languages Supported	English
Max Input Image Size	25 MB
Supported Image Formats	PNG, JPEG
Maximum Total Pixels per Image	2048 × 2048 × 3
Aspect Ratio (Width/Height)	Minimum: 0.25, Maximum: 4
Output Vector Size	1,024

Parameters: None.

note

Image embeddings use path-based input only. The documentId input format is not supported for this action.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.jpg" }],
  "actions": {
    "imageEmbeddings": {}
  }
}

Result field: imageEmbeddings.result — flat float[] array (v2). See Embeddings for format details.

---

imageMetadataGeneration

Generates structured metadata from an image, optionally guided by example metadata templates.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Max Input Image Size	5 MB
Supported Image Formats	JPEG, PNG
Max Total Pixels per Image	8000 px × 8000 px
Image Scaling	Images are scaled down (preserving aspect ratio) to fit within size limits
Recommended Image Size	Less than 1568 pixels on the larger side
Minimum Image Size Warning	Images under 200 pixels on any edge may degrade performance

Parameters:

Field	Type	Required	Description
kSimilarMetadata	object[]	Yes	At least one example metadata object to guide generation. Each item is a free-form JSON object with representative keys and values.
instructions	object	No	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "imageMetadataGeneration": {
      "kSimilarMetadata": [
        {
          "location": "New York City",
          "keywords": "urban|city|skyline",
          "summary": "Urban street photography"
        }
      ]
    }
  }
}

Result field: imageMetadata.result — JSON object with generated metadata fields.

---

namedEntityRecognitionImage

Detects named entities (people, organizations, locations) in an image.

Requirements: Input must be an image file.

Constraints:

Property	Value
Multilingual Support	Yes
Max Input Image Size	5 MB
Supported Image Formats	JPEG, PNG
Maximum Total Pixels per Image	8000 × 8000
Image Scaling	Images are scaled down preserving aspect ratio to meet size limits.
Recommended Max Pixels (Larger Side)	Less than 1568 pixels
Minimum Edge Size	Images under 200 pixels on any edge may degrade performance.

Parameters:

Field	Type	Required	Description
instructions	object	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../photo.png" }],
  "actions": {
    "namedEntityRecognitionImage": {}
  }
}

Result field: namedEntityImage.result — dictionary mapping entity type names to arrays of detected values (for example, { "organizations": ["Hyland"], "locations": ["Times Square"] })

---

namedEntityRecognitionText

Extracts named entities from document text.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Description
instructions	object	No	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../document.pdf" }],
  "actions": {
    "namedEntityRecognitionText": {
      "instructions": {
        "focus": "Extract company names and contract dates"
      }
    }
  }
}

Result field: namedEntityText.result — dictionary mapping entity type names to arrays of detected values (for example, { "organizations": ["Hyland"], "dates_times": ["2025-01-01"] })

---

textClassification

Classifies a text document into one of the provided categories.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Description
classes	string[]	Yes	At least two distinct, non-empty classification categories
instructions	object	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../document.pdf" }],
  "actions": {
    "textClassification": {
      "classes": ["Report", "Contract", "Invoice", "Other"]
    }
  }
}

Result field: textClassification.result — string with the matched class label

---

textEmbeddings

Generates a vector embedding for a text document.

Requirements: Input must be a text-based document.

Parameters: None

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../document.pdf" }],
  "actions": {
    "textEmbeddings": {}
  }
}

Result field: textEmbeddings.result — flat float[] array (v2). See Embeddings for format details.

---

textMetadataGeneration

Generates structured metadata from a text document, optionally guided by example metadata templates.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Description
kSimilarMetadata	object[]	Yes	At least one example metadata object to guide generation
instructions	object	No	JSON object providing additional context or guidance to the AI model

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../report.pdf" }],
  "actions": {
    "textMetadataGeneration": {
      "kSimilarMetadata": [
        {
          "document_type": "Financial Report",
          "author": "John Smith",
          "date": "2025-01"
        }
      ]
    }
  }
}

Result field: textMetadata.result — JSON object with generated metadata fields

---

textSummarization

Generates a summary of a text document.

Requirements: Input must be a text-based document.

Constraints:

Property	Value
Maximum input character	800K

Parameters:

Field	Type	Required	Default	Description
maxWordCount	integer	No	200	Maximum word count of the summary. Must be > 0.
instructions	object	No	—	JSON object providing additional context or guidance to the AI model.

Request example:

{
  "version": "context.api/v2",
  "objectKeys": [{ "path": "testing/.../report.pdf" }],
  "actions": {
    "textSummarization": {
      "maxWordCount": 150,
      "instructions": {
        "focus": "Key financial metrics and risks",
        "style": "professional"
      }
    }
  }
}

Result field: textSummary.result — string with the generated summary