Skip to main content

Quickstart Guide

This guide will help you get started with creating and using different types of agents in our platform.

Prerequisites​

  • Valid authentication token
  • Access to the API endpoints
  • Appropriate permissions (Consumer for GET, Solution Builder for POST)

Creating Your First Agent​

Available Tools

Currently, only these tools are supported out of the box:

  • multiply: Basic calculator multiplication function
  • semantic_search: Document search using semantic similarity
  • mcp: Document management tools (Local Setup Only)
  • structured_output: Force JSON output.

Tool agents can perform specific tasks using predefined tools. Here are common examples:

Basic Calculator Agent​

{
    "name": "Calculator",
    "description": "Math assistant",
    "agentType": "tool",
    "config": {
        "tools": [
            {
                "tool_type": "function",
                "name": "multiply",
                "description": "Multiplies two numbers",
                "func_name": "multiply"
            }
        ],
        "llm_model_id": "anthropic.claude-3-haiku-20240307-v1:0",
        "system_prompt": "You are a helpful assistant with access to various tools.",
        "inference_config": {
            "max_tokens": 4000
        }
    }
}

Semantic Search Agent​

{
    "name": "DocumentSearcher",
    "description": "Document search assistant",
    "agentType": "tool",
    "config": {
        "tools": [
            {
                "tool_type": "rag",
                "name": "semantic_search", 
                "description": "Searches for Information on a document RAG Database using Semantic Search",
                "func_name": "semantic_search",
                "filter_value": {}
            }
        ],
        "llm_model_id": "anthropic.claude-3-haiku-20240307-v1:0",
        "system_prompt": "You are a helpful assistant with access to various tools.",
        "inference_config": {
            "max_tokens": 4000
        }
    }
}

Structured Output Agent​

Experimental Feature

Structured Output on Tool Agent is currently an experimental feature. The API and functionality may change in future releases.

{
    "name": "PersonExtractor",
    "description": "Extracts structured information about people",
    "agentType": "tool",
    "config": {
        "tools": [
            {
                "tool_type": "structured_output",
                "name": "structured_output", 
                "description": "Extracts structured information about a person from text",
                "output_schema": {
                    "type": "object",
                    "properties": {
                        "name": {
                            "type": "string",
                            "description": "Full name of the person"
                        },
                        "age": {
                            "type": "integer",
                            "description": "Age of the person"
                        },
                        "occupation": {
                            "type": "string",
                            "description": "Person's job or profession"
                        }
                    },
                    "required": ["name"]
                }
            }
        ],
        "llm_model_id": "anthropic.claude-3-sonnet-20240229-v1:0",
        "system_prompt": "You are a helpful assistant specialized in extracting structured information about people from text.",
        "inference_config": {
            "max_tokens": 4000,
            "temperature": 0.1
        }
    }
}

Example usage:

POST /v1/agents/{agent_id}/versions/{version_id-or-latest}/invoke
{
    "messages": [
        {
            "role": "user",
            "content": "John Doe is a 35-year-old software engineer" 
        }
    ]
}

Expected response:

{
    "name": "John Doe",
    "age": 35,
    "occupation": "software engineer"
}

To use with any tool agent just pass in the message dict.

POST /v1/agents/{agent_id}/versions/{version_id-or-latest}/invoke
{
    "messages": [
        {
            "role": "user",
            "content": "What's 3x5?" 
            
        }
    ]
}

Streaming Responses​

Our platform supports streaming responses for all agent types (Tool Agents and RAG Agents), allowing you to receive data incrementally as it's generated. This is particularly useful for long-running operations or when you want to display results to users in real-time.

Unified Streaming Endpoint​

  • All Agent Types: POST /v1/agents/{agent_id}/versions/{version_id-or-latest}/invoke-stream

The system automatically detects the agent type and routes to the appropriate backend service with the correct media type and response format.

Understanding the Stream​

When you invoke the streaming endpoint, the server will automatically detect your agent type and send back a sequence of data chunks with the appropriate format.

  • For Tool Agent streams, the system uses text/plain content type. Each line in the response body is a self-contained JSON object, followed by a newline character.
  • For RAG Agent streams, the system uses text/event-stream content type. Each line in the response body is a self-contained JSON object, followed by a newline character.

In both cases, the stream is terminated by a special marker: [DONE] (though the exact format of the [DONE] marker might vary, sometimes appearing within the last JSON chunk or as a separate line).

Processing Streamed Data​

To process the stream, your client should:

  1. Open a connection to the streaming endpoint.
  2. Read the response line by line.
  3. If a line is [DONE] (or indicates the end of the stream in its JSON content), close the connection.
  4. Otherwise, parse the line as a JSON object.
  5. Each JSON object (chunk) typically follows this structure: 
    {
        "choices": [
            {
                "delta": {
                    // Contains either 'content' or 'tool_calls' or both (though usually one at a time per chunk)
                },
                "index": 0 
                // ... other fields like 'finish_reason' might appear in some chunks
            }
        ],
        "id": "streaming-response-id", // Example ID
        "model": "model-name",         // Example model
        // ... other metadata
    }
  6. Extracting Content: The delta object may contain a content field with a segment of the text response. You should append these segments together to form the complete message. 
    // Example chunk with content
    {
        "choices": [{"delta": {"content": "The result of 3x5 is "}, "index": 0}], 
        // ...
    }
  7. Extracting Tool Calls: The delta object may also contain tool_calls. Tool call information, especially arguments, might be split across multiple chunks. You need to aggregate them. 
    // Example chunk starting a tool call
    {
        "choices": [{"delta": {"tool_calls": [{"id": "tool_123", "type": "function", "function": {"name": "multiply", "arguments": "{\\"a\\":"}}]}, "index": 0}],
        // ...
    }
    // Example chunk continuing the tool call arguments
    {
        "choices": [{"delta": {"tool_calls": [{"id": "tool_123", "function": {"arguments": "3, \\"b\\":"}}]}, "index": 0}],
        // ...
    }
    // Example chunk finishing the tool call arguments
    {
        "choices": [{"delta": {"tool_calls": [{"id": "tool_123", "function": {"arguments": "5}"}}]}, "index": 0}],
        // ...
    }
    You would aggregate these to get the full tool_calls object: 
    {
        "id": "tool_123",
        "type": "function",
        "function": {
            "name": "multiply",
            "arguments": "{\\"a\\": 3, \\"b\\": 5}" 
        }
    }

Example: Streaming with a Tool Agent (using cURL)​

Let's say you have a Tool Agent (like the Calculator example) with ID your-tool-agent-id and using the most recent version of config.

Request:

curl -N -X POST "http://your-api-base-url/v1/agents/your-tool-agent-id/versions/latest/invoke-stream" \\
-H "Authorization: Bearer your-jwt-token" \\
-H "Content-Type: application/json" \\
-d '{
    "messages": [
        {
            "role": "user",
            "content": "What is 3x5?"
        }
    ]
}'

Expected Response Stream (raw text/plain lines):

{"choices": [{"delta": {"role": "assistant", "content": ""}, "index": 0}], "id": "...", "model": "..."}
{"choices": [{"delta": {"content": "Okay, I can help with that. "}, "index": 0}], "id": "...", "model": "..."}
{"choices": [{"delta": {"tool_calls": [{"id": "call_abc123", "type": "function", "function": {"name": "multiply", "arguments": "{\\"a\\":"}}]}, "index": 0}], "id": "...", "model": "..."}
{"choices": [{"delta": {"tool_calls": [{"id": "call_abc123", "function": {"arguments": " 3, \\"b\\":"}}]}, "index": 0}], "id": "...", "model": "..."}
{"choices": [{"delta": {"tool_calls": [{"id": "call_abc123", "function": {"arguments": " 5}"}}]}, "index": 0}], "id": "...", "model": "..."}
{"choices": [{"delta": {}, "index": 0, "finish_reason": "tool_calls"}], "id": "...", "model": "..."} 
[DONE]

(Note: The exact chunking and content can vary. Some chunks might only establish the role, others might have empty content, and tool calls can be split. The finish_reason chunk indicates why the generation stopped, e.g. tool_calls or stop.)

Processing the above stream would yield:

  • Aggregated Content: "Okay, I can help with that. "
  • Aggregated Tool Call: 
    {
        "id": "call_abc123",
        "type": "function",
        "function": {
            "name": "multiply",
            "arguments": "{\\"a\\": 3, \\"b\\": 5}"
        }
    }
    Your application would then execute this tool call and potentially send the result back to the agent in a subsequent (non-streaming or streaming) call.

Example: Streaming with a RAG Agent (using cURL)​

For a RAG Agent with ID your-rag-agent-id and the most recent version of the config.

Request:

curl -N -X POST "http://your-api-base-url/v1/agents/your-rag-agent-id/versions/latest/invoke-stream" \\
-H "Authorization: Bearer your-jwt-token" \\
-H "Content-Type: application/json" \\
-d '{
    "messages": [
        {
            "role": "user",
            "content": "What is our vacation policy?"
        }
    ],
    "filterValue": {} 
}'

Expected Response Stream (raw text/plain lines):

{\"choices\": [{\"delta\": {\"role\": \"assistant\", \"content\": \"\"}, \"index\": 0}], \"id\": \"...\", \"model\": \"...\"}
{\"choices\": [{\"delta\": {\"content\": \"Our vacation policy states that...\"}, \"index\": 0}], \"id\": \"...\", \"model\": \"...\"}
{\"choices\": [{\"delta\": {\"content\": \" employees are entitled to X days...\"}, \"index\": 0}], \"id\": \"...\", \"model\": \"...\"}
{\"choices\": [{\"delta\": {}, \"index\": 0, \"finish_reason\": \"stop\"}], \"id\": \"...\", \"model\": \"...\"} 
[DONE]

(Note: The exact chunking and content can vary. The [DONE] marker might be part of the last JSON or a separate line.)

Processing the above stream would yield:

  • Aggregated Content: "Our vacation policy states that... employees are entitled to X days..."
tip

When using streaming, ensure your client correctly handles line endings and JSON parsing for each chunk. Remember that tools like Swagger UI may not display streaming responses correctly; cURL, Postman (with appropriate settings), or custom client code are better choices.

The unified /invoke-stream endpoint automatically detects your agent type and applies the correct media type and response format, so you don't need to worry about which specific endpoint to call.