Why use the generate mode?
You can use thegenerate
mode to generate responses from different LLM providers using the same API. This mode is ideal for scenarios where you want to generate responses from different LLM providers without storing the messages.
Unified API
Same API for all supported LLM providers. Switch between providers without
changing a single line of code. Our API abstracts provider differences,
ensuring uniform input and output formats across all providers.
No vendor lock-in
Easily switch between various providers without vendor lock-in. The unified
interface abstracts away differences between provider APIs, allowing seamless
changes between various LLMs.
Prerequisites
Make sure to install the SDK of the providers you want to use:Generate without storing
When creating a new message, you can use thecreate
method to generate a response from the LLM provider. By default the storing option is enabled, which means the message request and response will be stored (Store mode). To disable storing the message, you can set the store.disabled
option to true
.