Custom

All this information are useful for tracking the usage of the API.

{
  endUser?: {
    id: string
    name?: string
  }
  sessionId?: string
  tags?: string[]
}

The maximum number of tokens to generate. Mind that in case the model generates more tokens than this value, the response will be truncated.

The array of messages to use as context for the model.

{
  role: 'system',
  content: string
}

{
  role: 'user',
  type: 'text'
  content: string
}

{
  role: 'user',
  type: 'image'
  content: Array<
  | {
      type: 'url'
      url: string
      detail?: 'auto' | 'low' | 'high'
    }
  | {
      type: 'base64'
      source: string
      media_type: 'image/jpeg' | 'image/png' | 'image/gif' | 'image/webp'
      detail?: 'auto' | 'low' | 'high'
    }
>
}

{
  role: 'assistant',
  type: 'text'
  content: string
}

{
  role: 'assistant',
  type: 'image'
  content: Array<
  | {
      type: 'url'
      url: string
      detail?: 'auto' | 'low' | 'high'
    }
  | {
      type: 'base64'
      source: string
      media_type: 'image/jpeg' | 'image/png' | 'image/gif' | 'image/webp'
      detail?: 'auto' | 'low' | 'high'
    }
>
}

{
  role: "tool_result"
  tools: Array<{
    type: 'function'
    data: {
      result: string
      name: string
    }
  }>
  is_error?: boolean
}

The model to use for generating the text.

The provider to use for generating the text. It can be either anthropic or openai or a custom provider.

The temperature value to use for generating the text.

The top_p value to use for generating the text.

The array of tools to be eventually used by the model.

{
  type: 'function'
  data: {
    name: string
    description?: string
    properties?: Record<string, unknown>
    required?: Array<string>
  }
}

The choice of the tool to be used by the model.

Behavior: If the type is auto, the model will choose the tool automatically.

{
  type: 'auto'
}

Behavior: If the type is tool, the model will use the tool with the given name.

{
  type: 'tool'
  name: string
}

Behavior: If the type is required, the model will be forced to use one of the available tools

{
  type: 'required'
}

Behavior: If the type is none, the model will not use any tool.

{
  type: 'none'
}

The array of strings that will cause the model to stop generating tokens.

If true, the model will generate the text in a streaming fashion.

The unique identifier of the response.

The model used to generate the response.

The response message generated by the model.

{
  role: 'assistant',
  type: 'text'
  content: string
}

{
  role: "tool_use",
  tools: Array<{
    type: 'function'
    data: {
      arguments: string
      name: string
    }
  }>
}

The reason why the model stopped generating tokens.

• max_tokens: The model reached the maximum token limit.

• stop_sequence: The model incountered one of the provided stop_sequence tokens.

• stop: The normal stop condition, the model reached the end of the response.

The stop_sequence token that caused the model to stop generating tokens. In case the stop_reason is not stop_sequence, this field will be undefined.

The usage object contains the number of tokens used in the input and output.

{
  input_tokens?: number
  output_tokens?: number
}

The logprobs object contains the log probabilities of the tokens generated by the model.

Array<{
  token: string
  logprob: number
  bytes?: Array<number>
  top_logprobs?: Array<{ token: string; logprob: number; bytes?: Array<number> }>
}>

The start time of the request in milliseconds.

The end time of the request in milliseconds.

The error object in case the request fails.