Input

The maximum number of tokens to generate. Mind that in case the model generates more tokens than this value, the response will be truncated.

The array of messages to use as context for the model.

{
  role: 'system',
  content: string
}

{
  role: 'user',
  type: 'text'
  content: string
}

{
  role: 'user',
  type: 'image'
  content: Array<
  | {
      type: 'url'
      url: string
      /**
       * @Anthropic Not supported
       */
      detail?: 'auto' | 'low' | 'high'
    }
  | {
      type: 'base64'
      source: string
      /**
       * @OpenAI Not supported
       */
      media_type: 'image/jpeg' | 'image/png' | 'image/gif' | 'image/webp'
      /**
       * @Anthropic Not supported
       */
      detail?: 'auto' | 'low' | 'high'
    }
>
}

{
  role: 'assistant',
  type: 'text'
  content: string
}

{
  role: 'assistant',
  type: 'image'
  content: Array<
  | {
      type: 'url'
      url: string
      /**
       * @Anthropic Not supported
       */
      detail?: 'auto' | 'low' | 'high'
    }
  | {
      type: 'base64'
      source: string
      /**
       * @OpenAI Not supported
       */
      media_type: 'image/jpeg' | 'image/png' | 'image/gif' | 'image/webp'
      /**
       * @Anthropic Not supported
       */
      detail?: 'auto' | 'low' | 'high'
    }
>
}

{
  role: "tool_result"
  tools: Array<{
    type: 'function'
    data: {
      result: string
      name: string
    }
  }>
  is_error?: boolean
}

The model to use for generating the text. Available models are:

All this information are useful for tracking the usage of the API.

⚠️ Only if you set the passIdToUnderlyingLLM to true, the endUser.id will be passed to the underlying provider. Otherwise none of the information will be passed to the provider.

{
  endUser?: {
    id: string
    name?: string
    passIdToUnderlyingLLM?: boolean
  }
  sessionId?: string
  tags?: string[]
}

Controls the randomness of the generated text. Lower values make the model more deterministic and repetitive, while higher values make the model more creative and unpredictable.

Controls the diversity of the generated text. Lower values make the model more repetitive, while higher values make the model more creative and unpredictable.

⚠️ This parameter is mutually exclusive with temperature.

The array of tools to be eventually used by the model.

{
  type: 'function'
  data: {
    name: string
    description?: string
    properties?: Record<string, unknown>
    required?: Array<string>
  }
}

The choice of the tool to be used by the model.

Behavior: If the type is auto, the model will choose the tool automatically.

{
  type: 'auto'
}

Behavior: If the type is tool, the model will use the tool with the given name.

{
  type: 'tool'
  name: string
}

Behavior: If the type is required, the model will be forced to use one of the available tools

{
  type: 'required'
}

Behavior: If the type is none, the model will not use any tool.

{
  type: 'none'
}

The array of strings that will cause the model to stop generating tokens.

If true, the model will generate the text in a streaming fashion.

The following options are specific to the OpenAI models. This means that they will be ignored if you use another provider.

For more info about those options check OpenAI docs

{
  presence_penalty?: number
  frequency_penalty?: number
  response_format?: {
    type: "text"
  } | {
    type: "json_object"
  } | {
    type: "json_schema"
    json_schema: {
      description?: string
      name: string
      schema: Record<string, unknown>
      strict?: boolean
    }
  }
  service_tier?: 'auto' | 'default'
  parallel_tool_calls?: boolean
  logprobs?: boolean
  top_logprobs?: number
  seed?: number
  stream_options?: {
    include_usage?: boolean
  }
  logit_bias?: Record<string, number>
}

The following options are specific to the Anthropic models. This means that they will be ignored if you use another provider.

For more info about those options check Anthropic docs

{
  top_k?: number
}