Skip to main content

​
Overview

While working with Inference class, you can also generate JSON output from the model inference. This is useful for example when you need to process the response in a structured way or when you want to store the elements of the response in a database. Inference supports explicit response shaping through responseFormat, plus tools and toolChoice for tool calling.

​
Example

In this example we will use OpenAI JSON mode, which guarantees that the response will be in a JSON format. It does not guarantee compliance with a specific schema (for some providers including OpenAI). We can try to work around it by providing an example of the expected JSON output in the prompt.
NOTE: Some model providers allow to specify a JSON schema for model to follow via schema parameter of response_format. OpenAI does not support this feature in JSON mode (only in JSON Schema mode). 
<?php
require 'examples/boot.php';

use Cognesy\Messages\Messages;
use Cognesy\Polyglot\Inference\Data\ResponseFormat;
use Cognesy\Polyglot\Inference\Inference;

$data = Inference::using('openai')
    ->with(
        messages: Messages::fromString('What is capital of France? \
           Respond with JSON data containing name", population and year of founding. \
           Example: {"name": "Berlin", "population": 3700000, "founded": 1237}'),
        responseFormat: ResponseFormat::jsonObject(),
        options: ['max_tokens' => 64],
    )
    ->asJsonData();

echo "USER: What is capital of France\n";
echo "ASSISTANT:\n";
dump($data);

assert(is_array($data), 'Response should be an array');
assert(isset($data['name']), 'Response should have "name" field');
assert(strpos($data['name'], 'Paris') !== false, 'City name should be Paris');
assert(isset($data['population']), 'Response should have "population" field');
?>