Data Models for Tracing

A trace represents the overall process of tracking and visualizing the execution flow of your LLM application. Each observed function creates a span, and many spans together make up a trace.

• Trace: Complete execution flow containing multiple spans representing an LLM request’s full lifecycle.

• Span: Individual units of work (LLM calls, tool executions, retrievals) that compose a trace.

• Thread: Logical grouping of traces sharing execution context for organizing related operations, this will 99.9% be a conversation.

• End User: Human user interacting with the trace, which is usually also the consumer of the LLM application.

Trace

A trace contains multiple spans, and at the same time can belong to an end user and/or thread:

type Trace = {
    uuid: string;
    name?: string;
    input?: any;
    output?: any;
    startTime: string; // ISO datetime
    endTime: string; // ISO datetime
    environment?: 'production' | 'development' | 'staging' | 'testing';
    metadata?: Record<string, any>;
    tags?: string[];
 
    // Span collections
    baseSpans?: BaseSpan[];
    llmSpans?: LlmSpan[];
    retrieverSpans?: RetrieverSpan[];
    toolSpans?: ToolSpan[];
    agentSpans?: AgentSpan[];
 
    // Relationships
    threadId?: string;
    userId?: string;
 
    // Evals
    metricCollection?: string;
    llmTestCase?: LLMTestCase;
}

You can run online evals by providing BOTH the name of the metricCollection you’ve created on Confident AI and llmTestCase.

Span

All baseSpans types in a Trace are defaulted to custom spans, with the loosest type strictness:

type BaseSpan = {
    uuid: string;
    name: string;
    input?: string | Record<string, any> | any[];
    output?: string | Record<string, any> | any[];
    error?: string;
    status?: 'SUCCESS' | 'ERRORED'; // defaults to 'SUCCESS'
    startTime: string; // ISO datetime
    endTime: string; // ISO datetime
    parentUuid?: string;
    metadata?: Record<string, any>;
 
    // Evals
    metricCollection?: string;
    llmTestCase?: LLMTestCase;
}

You can run online evals by providing BOTH the name of the metricCollection you’ve created on Confident AI and llmTestCase.

LLM span

If you provide spans through the llmSpans of a Trace, your span will be created as an LLM span type, and will need an additional model argument:

type LlmSpan = BaseSpan & {
  model: string;
  input: Record<string, string> | string;
  output: Record<string, string> | string;
  costPerInputToken?: number;
  costPerOutputToken?: number;
  inputTokenCount?: number;
  outputTokenCount?: number;
}

The model argument allows you to automate the process of cost and latency tracking, which you can learn more about here.

Retriever span

If you provide spans through the retrieverSpans of a Trace, your span will be created as an retriever span type, and will need an additional embedder argument:

type RetrieverSpan = BaseSpan & {
  embedder: string;
  input: string;
  output: string[];
  topK?: number;
  chunkSize?: number;
}

Tool span

If you provide spans through the toolSpans of a Trace, your span will be created as an tool span type:

type ToolSpan = BaseSpan & {
  description?: string;
}

A tool span does not require additional mandatory arguments because it automatically takes the input and output of the BaseSpan as tool input parameters and outputs.

Agent span

If you provide spans through the agentSpans of a Trace, your span will be created as an agent span type:

type AgentSpan = BaseSpan & {
  availableTools: string[];
  agentHandoffs: string[];
}

A agent span is similar to the tool span where the it automatically takes the input and output of the BaseSpan as I/O if not set explicitly.