Skip to main content

Examples and Instructions for Context API v2

This guide provides comprehensive examples for using the Context API v2 with detailed explanations of action-specific instructions.

Overview

The Context API v2 introduces structured actions where:

  • actions is an object (not an array)
  • Action names are keys in camelCase format
  • Each action's value is an object containing its configuration parameters

This allows for:

  • Independent configuration per action
  • Type-safe parameter passing
  • Clear validation messages
  • Better extensibility

Configuration Parameters vs Instructions

Each action has configuration parameters that control its behavior (e.g., maxWordCount, classes, kSimilarMetadata).

Additionally, actions that support an optional instructions field for custom AI guidance:

  • textSummarization - Guide summarization style and focus
  • imageDescription - Control description format and detail level
  • namedEntityRecognitionText - Specify entity types and focus areas
  • namedEntityRecognitionImage - Guide OCR and entity extraction
  • textClassification - Provide context for classification decisions
  • imageClassification - Specify classification focus areas
  • textMetadataGeneration - Guide metadata extraction strategy
  • imageMetadataGeneration - Control metadata identification approach

The instructions field accepts a custom object to provide additional context or guidance to the AI model.

Action Categories

Text Processing Actions

Text Summarization

Generate concise summaries of text documents with optional AI guidance.

Action Name: textSummarization

Configuration Parameters:

  • maxWordCount (optional): Maximum number of words in the summary
  • instructions (optional): Custom instructions object for summarization guidance

Example - Basic:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/report.pdf"
}
],
"actions": {
"textSummarization": {}
}
}

Example - With Word Limit:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/long-report.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 150
}
}
}

Example - Financial Report with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/quarterly-earnings.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 200,
"instructions": {
"focus": "Highlight key financial metrics, revenue, and outlook",
"style": "professional",
"emphasize": "financial_performance|growth_trends"
}
}
}
}

Example - Legal Document with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract-terms.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 250,
"instructions": {
"focus": "Extract key obligations, terms, conditions, and deadlines",
"style": "structured",
"highlightRisks": true,
"context": "legal contract analysis"
}
}
}
}

Example - Medical Document with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/patient-report.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 200,
"instructions": {
"focus": "Summarize diagnosis, treatment plan, and follow-up instructions",
"style": "clinical",
"emphasize": "critical_findings|treatment_recommendations",
"context": "medical report summary"
}
}
}
}

Output:

{
"id": "abc123",
"results": [
{
"objectKey": "documents/report.pdf",
"textSummary": {
"isSuccess": true,
"result": "This report analyzes quarterly performance...",
"error": null
}
}
]
}

Text Metadata Generation

Extract structured metadata from text using AI-powered analysis with example-based learning.

Action Name: textMetadataGeneration

Configuration Parameters:

  • kSimilarMetadata (required): Array of metadata example objects showing desired metadata structure
  • instructions (optional): Custom instructions object for AI guidance

Metadata Format Guidelines:

  • Use : to separate category from field name (e.g., "document:type")
  • Use | to separate possible values (e.g., "Invoice|Receipt|Contract")
  • Provide representative examples of expected values
  • Multiple metadata objects help the AI understand variations

Example - Document Classification:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:type": "Contract|Agreement|NDA|MOU",
"document:category": "Legal|Financial|Employment|Corporate",
"party:count": "2|3|Multiple",
"keywords:tags": "confidentiality|partnership|employment|services"
}
]
}
}
}

Example - Meeting Minutes Metadata:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/meeting-notes.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"meeting:type": "Board Meeting|Team Meeting|Client Call|Review",
"attendees:count": "2-5|6-10|11+",
"has:action_items": "yes|no",
"has:decisions": "yes|no",
"topics:list": "budget|strategy|project_update|personnel",
"urgency:level": "high|medium|low"
}
]
}
}
}

Example - Technical Documentation:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/api-spec.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:type": "API Documentation|User Manual|Technical Spec|Guide",
"technology:stack": "REST|GraphQL|gRPC|SOAP",
"programming:language": "Python|JavaScript|Java|C#|Go",
"version:type": "v1|v2|v3|beta|alpha",
"complexity:level": "beginner|intermediate|advanced",
"has:code_examples": "yes|no",
"keywords:tags": "authentication|endpoints|webhooks|integration"
}
]
}
}
}

Example - With Custom Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/legal-agreement.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:type": "Contract|Agreement|NDA|MOU",
"document:category": "Legal|Financial|Employment|Corporate",
"keywords:tags": "confidentiality|partnership|employment|services"
}
],
"instructions": {
"focus": "Extract formal legal terminology",
"detailLevel": "high"
}
}
}
}

Output:

{
"id": "def456",
"results": [
{
"objectKey": "documents/contract.pdf",
"textMetadata": {
"isSuccess": true,
"result": {
"document:type": "NDA",
"document:category": "Legal",
"party:count": "2",
"keywords:tags": "confidentiality"
},
"error": null
}
}
]
}

Text Classification

Classify documents into predefined categories.

Action Name: textClassification

Configuration Parameters:

  • classes (required): Array of at least 2 distinct classification categories
  • instructions (optional): Custom instructions object for AI guidance

Example - Document Type Classification:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/invoice-2024.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"invoice",
"receipt",
"contract",
"report",
"correspondence",
"other"
]
}
}
}

Example - Sentiment Classification:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/customer-feedback.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"positive",
"negative",
"neutral"
]
}
}
}

Example - Industry Classification:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/business-plan.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"technology",
"healthcare",
"finance",
"retail",
"manufacturing",
"services"
]
}
}
}

Example - With Custom Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/legal-contract.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"invoice",
"contract",
"receipt",
"report",
"other"
],
"instructions": {
"context": "legal and financial documents",
"priority": "accuracy over speed"
}
}
}
}

Output:

{
"id": "ghi789",
"results": [
{
"objectKey": "documents/invoice-2024.pdf",
"textClassification": {
"isSuccess": true,
"result": "invoice",
"error": null
}
}
]
}

Text Embeddings

Generate vector embeddings for semantic search and similarity matching.

Action Name: textEmbeddings

Configuration Parameters: None required

⚠️ Breaking Change in v2: The output format for text embeddings has changed from nested array List<List<float>> to flat array List<float>. See Breaking Changes to the Embeddings Format for migration details.

Example:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/article.pdf"
}
],
"actions": {
"textEmbeddings": {}
}
}

Output (v2 - Flat Array):

{
"id": "jkl012",
"results": [
{
"objectKey": "documents/article.pdf",
"textEmbeddings": {
"isSuccess": true,
"result": [-0.021741372, 0.005967313, -0.02682978, 0.01471012],
"error": null
}
}
]
}

Named Entity Recognition (Text)

Extract named entities (people, organizations, locations, etc.) from text documents.

Action Name: namedEntityRecognitionText

Configuration Parameters:

  • instructions (optional): Custom instructions object for entity extraction guidance

Example - Basic:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/news-article.pdf"
}
],
"actions": {
"namedEntityRecognitionText": {}
}
}

Example - Legal Document with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract.pdf"
}
],
"actions": {
"namedEntityRecognitionText": {
"instructions": {
"focus": "Extract party names and signatory individuals",
"entityPriority": "PERSON,ORGANIZATION",
"includeRoles": true,
"context": "legal document with multiple parties"
}
}
}
}

Example - Medical Document with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/medical-report.pdf"
}
],
"actions": {
"namedEntityRecognitionText": {
"instructions": {
"focus": "Extract patient names, doctor names, hospital names, and medication names",
"customEntityTypes": "PERSON,ORGANIZATION,MEDICATION",
"context": "medical report with patient and provider information"
}
}
}
}

Example - Business Email with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/business-correspondence.txt"
}
],
"actions": {
"namedEntityRecognitionText": {
"instructions": {
"focus": "Extract sender, recipient, and mentioned companies",
"prioritizeEmailAddresses": true,
"context": "business communication with contact information"
}
}
}
}

Output:

{
"id": "mno345",
"results": [
{
"objectKey": "documents/news-article.pdf",
"namedEntityText": {
"isSuccess": true,
"result": {
"PERSON": ["John Smith", "Jane Doe"],
"ORGANIZATION": ["Microsoft", "Google"],
"LOCATION": ["New York", "California"]
},
"error": null
}
}
]
}

Image Processing Actions

Image Description

Generate natural language descriptions of images using computer vision and AI.

Action Name: imageDescription

Configuration Parameters:

  • maxWordCount (optional): Maximum number of words in the description
  • instructions (optional): Custom instructions object for description guidance

Example - Basic:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-photo.jpg"
}
],
"actions": {
"imageDescription": {}
}
}

Example - With Word Limit:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/landscape.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 50
}
}
}

Example - Accessibility Description with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/webpage-screenshot.png"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 100,
"instructions": {
"focus": "Provide alt-text suitable for accessibility",
"detailLevel": "comprehensive",
"includeLayout": true,
"context": "web accessibility for visually impaired users"
}
}
}
}

Example - Product Catalog with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-item.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 75,
"instructions": {
"focus": "Emphasize product features, materials, and dimensions",
"includeBrandName": true,
"context": "e-commerce product listing"
}
}
}
}

Example - Medical Imaging with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/xray-scan.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 150,
"instructions": {
"focus": "Technical medical imaging description for clinical use",
"includeTechnicalTerms": true,
"detailLevel": "high",
"context": "medical imaging documentation"
}
}
}
}

Output:

{
"id": "pqr678",
"results": [
{
"objectKey": "images/product-photo.jpg",
"imageDescription": {
"isSuccess": true,
"result": "A modern smartphone with a sleek black design...",
"error": null
}
}
]
}

Image Metadata Generation

Extract structured metadata from images using AI with example-based learning.

Action Name: imageMetadataGeneration

Configuration Parameters:

  • kSimilarMetadata (required): Array of metadata example objects
  • instructions (optional): Custom instructions object for AI guidance

Example - Product Images:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/clothing-item.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"product:type": "T-Shirt|Dress|Pants|Shoes|Jacket|Hat",
"product:color": "Red|Blue|Green|Black|White|Yellow|Multi-color",
"product:gender": "Male|Female|Unisex",
"product:season": "Summer|Winter|Fall|Spring|All-Season",
"style:type": "Casual|Formal|Sports|Business|Vintage",
"keywords:tags": "fashion|clothing|apparel|style"
}
]
}
}
}

Example - Vehicle Inspection:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/vehicle-inspection.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"vehicle:type": "Sedan|SUV|Truck|Van|Motorcycle|Bus",
"vehicle:color": "White|Black|Silver|Red|Blue|Other",
"vehicle:condition": "Excellent|Good|Fair|Poor|Damaged",
"damage:visible": "yes|no",
"damage:location": "Front|Rear|Side|Multiple|None",
"damage:severity": "Minor|Moderate|Severe|Total",
"environment:location": "Street|Parking Lot|Driveway|Highway|Indoor"
}
]
}
}
}

Example - Real Estate Photos:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/property-listing.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"property:type": "House|Apartment|Condo|Townhouse|Commercial",
"room:type": "Living Room|Bedroom|Kitchen|Bathroom|Office|Exterior",
"style:architecture": "Modern|Traditional|Contemporary|Victorian|Colonial",
"condition:overall": "New|Excellent|Good|Needs Work|Renovation",
"features:notable": "Fireplace|Pool|Garden|Balcony|Hardwood Floors",
"lighting:quality": "Bright|Natural|Dim|Well-lit",
"furnishing:status": "Furnished|Partially Furnished|Unfurnished|Staged"
}
]
}
}
}

Example - With Custom Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-photo.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"product:type": "T-Shirt|Dress|Pants|Shoes|Jacket|Hat",
"product:color": "Red|Blue|Green|Black|White|Yellow|Multi-color",
"style:type": "Casual|Formal|Sports|Business|Vintage",
"keywords:tags": "fashion|clothing|apparel|style"
}
],
"instructions": {
"focus": "Emphasize brand and quality indicators",
"detailLevel": "high"
}
}
}
}

Output:

{
"id": "stu901",
"results": [
{
"objectKey": "images/clothing-item.jpg",
"imageMetadata": {
"isSuccess": true,
"result": {
"product:type": "T-Shirt",
"product:color": "Blue",
"product:gender": "Unisex",
"product:season": "Summer",
"style:type": "Casual",
"keywords:tags": "fashion"
},
"error": null
}
}
]
}

Image Classification

Classify images into predefined categories.

Action Name: imageClassification

Configuration Parameters:

  • classes (required): Array of at least 2 distinct classification categories
  • instructions (optional): Custom instructions object for AI guidance

Example - Vehicle Damage Assessment:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/car-photo.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"damaged_vehicle",
"undamaged_vehicle",
"not_a_vehicle"
]
}
}
}

Example - Document Type from Image:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/scanned-doc.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"passport",
"drivers_license",
"id_card",
"birth_certificate",
"other_document"
]
}
}
}

Example - Scene Classification:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/photo.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"indoor",
"outdoor",
"nature",
"urban",
"portrait"
]
}
}
}

Example - With Custom Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/vehicle-damage.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"damaged_vehicle",
"undamaged_vehicle",
"not_a_vehicle"
],
"instructions": {
"focus": "Pay attention to minor scratches and dents",
"context": "insurance claim assessment"
}
}
}
}

Output:

{
"id": "vwx234",
"results": [
{
"objectKey": "images/car-photo.jpg",
"imageClassification": {
"isSuccess": true,
"result": "damaged_vehicle",
"error": null
}
}
]
}

Image Embeddings

Generate vector embeddings for image similarity search.

Action Name: imageEmbeddings

Configuration Parameters: None required

⚠️ Breaking Change in v2: The output format for image embeddings has changed from nested array List<List<float>> to flat array List<float>. See Breaking Changes to the Embeddings Format for migration details.

Example:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product.jpg"
}
],
"actions": {
"imageEmbeddings": {}
}
}

Output (v2 - Flat Array):

{
"id": "yz567",
"results": [
{
"objectKey": "images/product.jpg",
"imageEmbeddings": {
"isSuccess": true,
"result": [0.234, -0.567, 0.891, -0.123, 0.456],
"error": null
}
}
]
}

Named Entity Recognition (Image)

Extract text entities from images using Optical Character Recognition (OCR) combined with Named Entity Recognition.

Action Name: namedEntityRecognitionImage

Configuration Parameters:

  • instructions (optional): Custom instructions object for OCR and entity extraction guidance

Example - Basic:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/business-card.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {}
}
}

Example - Invoice Document with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/invoice-scan.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {
"instructions": {
"focus": "Extract company names, invoice numbers, and monetary amounts",
"prioritize": "ORGANIZATION,MONEY,DATE",
"context": "financial document processing"
}
}
}
}

Example - Identification Document with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/drivers-license.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {
"instructions": {
"focus": "Extract name, address, license number, and expiration date",
"prioritize": "PERSON,ADDRESS,ID_NUMBER,DATE",
"handleMultipleFormats": true,
"context": "identity verification"
}
}
}
}

Example - Receipt with Instructions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/receipt-photo.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {
"instructions": {
"focus": "Extract vendor name, total amount, date, and item names",
"prioritize": "ORGANIZATION,MONEY,DATE,PRODUCT",
"handleMisalignment": true,
"context": "expense report processing"
}
}
}
}

Output:

{
"id": "abc890",
"results": [
{
"objectKey": "images/business-card.jpg",
"namedEntityImage": {
"isSuccess": true,
"result": {
"PERSON": ["John Doe"],
"ORGANIZATION": ["Acme Corp"],
"EMAIL": ["john@acme.com"],
"PHONE": ["555-1234"]
},
"error": null
}
}
]
}

Complex Multi-Action Examples

Example 1: Complete Document Analysis

Process a document with multiple text actions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/quarterly-report.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 200
},
"textClassification": {
"classes": [
"financial_report",
"business_plan",
"market_analysis",
"internal_memo"
]
},
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"report:type": "Quarterly|Annual|Monthly|Special",
"report:period": "Q1|Q2|Q3|Q4|FY",
"department:name": "Finance|Sales|Operations|Marketing",
"has:financials": "yes|no",
"keywords:tags": "revenue|expenses|growth|performance"
}
]
},
"namedEntityRecognitionText": {},
"textEmbeddings": {}
}
}

Example 2: Comprehensive Image Processing

Analyze product images with multiple actions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-photos/item-001.jpg"
},
{
"path": "images/product-photos/item-002.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 75
},
"imageClassification": {
"classes": [
"electronics",
"clothing",
"furniture",
"food",
"toys",
"other"
]
},
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"product:category": "Electronics|Clothing|Home|Sports|Beauty",
"product:condition": "New|Used|Refurbished",
"image:quality": "High|Medium|Low",
"background:type": "White|Studio|Lifestyle|Transparent",
"has:watermark": "yes|no"
}
]
},
"imageEmbeddings": {}
}
}

Example 3: Mixed Document Types

Process different document types with tailored actions:

{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract.pdf"
},
{
"path": "documents/invoice.pdf"
},
{
"path": "documents/report.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"contract",
"invoice",
"report",
"letter",
"form"
]
},
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:category": "Legal|Financial|Administrative|Technical",
"urgency:level": "high|medium|low",
"requires:signature": "yes|no",
"keywords:tags": "important|review|action_required|reference"
}
]
},
"namedEntityRecognitionText": {},
"textEmbeddings": {}
}
}

See Also