Skip to main content

Overview

While working with Inference class, you can also generate JSON output from the model inference. This is useful for example when you need to process the response in a structured way or when you want to store the elements of the response in a database.

Example

In this example we will use OpenAI tool calling, in which model will generate a JSON containing arguments for a function call. This way we can make the model generate a JSON object with specific structure of parameters.