Testing

The package provides dedicated testing fakes for all four facades, allowing you to mock LLM responses and make assertions about how your code interacts with the services. No real API calls are made when a fake is active, which makes tests fast, deterministic, and free of external dependencies.

StructuredOutput::fake()

The StructuredOutputFake intercepts all extraction calls and returns predefined responses. It records every call so you can assert against the response model class, messages, connection, and model that were used.

Basic Usage

use Cognesy\Instructor\Laravel\Facades\StructuredOutput;
use App\ResponseModels\PersonData;
use Tests\TestCase;

class PersonExtractionTest extends TestCase
{
    public function test_extracts_person_data(): void
    {
        // Arrange -- setup the fake with expected responses
        $fake = StructuredOutput::fake([
            PersonData::class => new PersonData(
                name: 'John Smith',
                age: 30,
                email: 'john@example.com',
            ),
        ]);

        // Act -- your code calls StructuredOutput
        $person = StructuredOutput::with(
            messages: 'John Smith is 30 years old',
            responseModel: PersonData::class,
        )->get();

        // Assert -- verify the result
        $this->assertEquals('John Smith', $person->name);
        $this->assertEquals(30, $person->age);

        // Assert that extraction was performed
        $fake->assertExtracted(PersonData::class);
    }
}
// @doctest id="cf84"

Response Mapping

Map response model classes to their fake responses. Each class returns its corresponding value when extracted.

$fake = StructuredOutput::fake([
    PersonData::class => new PersonData(name: 'John', age: 30),
    AddressData::class => new AddressData(city: 'New York'),
    OrderData::class => new OrderData(total: 99.99),
]);

// Each class returns its mapped response
$person = StructuredOutput::with(..., responseModel: PersonData::class)->get();
$address = StructuredOutput::with(..., responseModel: AddressData::class)->get();
// @doctest id="f950"

If you request a response model that has no mapping, the fake throws a RuntimeException with a helpful message telling you which class needs a fake response.

Response Sequences

Return different responses for sequential calls to the same response model class.

$fake = StructuredOutput::fake();

$fake->respondWithSequence(PersonData::class, [
    new PersonData(name: 'First Person', age: 25),
    new PersonData(name: 'Second Person', age: 30),
    new PersonData(name: 'Third Person', age: 35),
]);

// First call
$first = StructuredOutput::with(...)->get();  // First Person

// Second call
$second = StructuredOutput::with(...)->get(); // Second Person

// Third call
$third = StructuredOutput::with(...)->get();  // Third Person
// @doctest id="1164"

Available Assertions

$fake = StructuredOutput::fake([...]);

// Run your code...

// Assert extraction was called for a class
$fake->assertExtracted(PersonData::class);

// Assert extraction count
$fake->assertExtractedTimes(PersonData::class, 1);
$fake->assertExtractedTimes(PersonData::class, 3);

// Assert no extractions were performed
$fake->assertNothingExtracted();

// Assert messages contained specific text
$fake->assertExtractedWith(PersonData::class, 'John Smith');

// Assert configured connection was used
$fake->assertUsedConnection('anthropic');

// Assert model was used
$fake->assertUsedModel('gpt-4o');
// @doctest id="db33"

Accessing Recorded Calls

Inspect all recorded extraction calls for custom assertions.

$fake = StructuredOutput::fake([...]);

// Run your code...

// Get all recorded extractions
$recorded = $fake->recorded();

foreach ($recorded as $extraction) {
    echo "Class: " . $extraction['class'];
    echo "Messages: " . json_encode($extraction['messages']);
    echo "Model: " . $extraction['model'];
    echo "Connection: " . $extraction['connection'];
}
// @doctest id="becd"

Inference::fake()

The InferenceFake intercepts raw inference calls and returns responses based on pattern matching against the input messages.

Basic Usage

use Cognesy\Instructor\Laravel\Facades\Inference;
use Cognesy\Messages\Messages;
use Tests\TestCase;

final class InferenceFakeTest extends TestCase
{
    public function test_calls_inference(): void
    {
        // Arrange
        $fake = Inference::fake([
            'What is 2+2?' => 'The answer is 4.',
            'default' => 'I don\'t know.',
        ]);

        // Act
        $response = Inference::with(
            messages: Messages::fromString('What is 2+2?'),
        )->get();

        // Assert
        $this->assertEquals('The answer is 4.', $response);
        $fake->assertCalled();
        $fake->assertCalledWith('What is 2+2?');
    }
}
// @doctest id="01d3"

Pattern Matching

Responses are matched by checking whether the input message contains the pattern string. The first matching pattern wins. If no pattern matches, the default key is used as a fallback; if no default exists, an empty string is returned.

$fake = Inference::fake([
    'capital' => 'Paris is the capital of France.',
    'weather' => 'The weather is sunny.',
    'default' => 'I don\'t understand.',
]);

// Matches 'capital' (input contains the word)
$response1 = Inference::with(messages: Messages::fromString('What is the capital of France?'))->get();

// Matches 'weather'
$response2 = Inference::with(messages: Messages::fromString('How is the weather today?'))->get();

// No match, uses 'default'
$response3 = Inference::with(messages: Messages::fromString('Random question'))->get();
// @doctest id="17d0"

Response Sequences

Queue ordered responses that are returned regardless of input content.

$fake = Inference::fake();

$fake->respondWithSequence([
    'First response',
    'Second response',
    'Third response',
]);

// Returns responses in order
$first = Inference::with(...)->get();  // "First response"
$second = Inference::with(...)->get(); // "Second response"
// @doctest id="6b50"

Available Assertions

$fake = Inference::fake([...]);

// Assert inference was called
$fake->assertCalled();

// Assert call count
$fake->assertCalledTimes(3);

// Assert never called
$fake->assertNotCalled();

// Assert called with specific message text
$fake->assertCalledWith('What is the capital');

// Assert configured connection was used
$fake->assertUsedConnection('groq');

// Assert model was used
$fake->assertUsedModel('llama-3.3-70b');

// Assert called with specific tools
$fake->assertCalledWithTools(['search', 'calculate']);
// @doctest id="391b"

Embeddings::fake()

The EmbeddingsFake intercepts embedding requests and returns predefined or randomly generated vectors.

Basic Usage

use Cognesy\Instructor\Laravel\Facades\Embeddings;
use Tests\TestCase;

final class EmbeddingsFakeTest extends TestCase
{
    public function test_generates_embeddings(): void
    {
        // Arrange
        $fake = Embeddings::fake([
            'hello' => [0.1, 0.2, 0.3, 0.4, 0.5],
        ]);

        // Act
        $embedding = Embeddings::withInputs('hello world')->first();

        // Assert
        $this->assertIsArray($embedding);
        $fake->assertCalled();
        $fake->assertCalledWith('hello world');
    }
}
// @doctest id="4139"

Default Embeddings

If no pattern matches, a random normalized embedding vector is generated automatically. This is useful when you need an embedding but do not care about its exact values.

$fake = Embeddings::fake();

// Returns random 1536-dimensional embedding (matching OpenAI's default dimensions)
$embedding = Embeddings::withInputs('anything')->first();

$this->assertCount(1536, $embedding);
// @doctest id="eb6e"

Custom Dimensions

Match the dimensionality of your production embedding model.

$fake = Embeddings::fake()
    ->withDimensions(768); // Use 768 dimensions

$embedding = Embeddings::withInputs('test')->first();
$this->assertCount(768, $embedding);
// @doctest id="e85f"

Available Assertions

$fake = Embeddings::fake([...]);

// Assert embeddings were called
$fake->assertCalled();

// Assert call count
$fake->assertCalledTimes(2);

// Assert never called
$fake->assertNotCalled();

// Assert called with specific input
$fake->assertCalledWith('hello world');

// Assert configured connection was used
$fake->assertUsedConnection('openai');

// Assert model was used
$fake->assertUsedModel('text-embedding-3-large');
// @doctest id="c230"

AgentCtrl::fake()

The AgentCtrlFake intercepts code agent executions and returns predefined responses without launching any CLI processes.

Basic Usage

use Cognesy\Instructor\Laravel\Facades\AgentCtrl;
use Tests\TestCase;

final class AgentCtrlFakeTest extends TestCase
{
    public function test_generates_code(): void
    {
        // Arrange -- setup fake with expected responses
        $fake = AgentCtrl::fake([
            'Generated migration file: 2024_01_01_create_users_table.php',
        ]);

        // Act -- your code calls AgentCtrl
        $result = AgentCtrl::claudeCode()
            ->execute('Generate a users table migration');

        // Assert
        $this->assertEquals(0, $result->exitCode);
        $this->assertStringContainsString('migration', $result->text());

        $fake->assertExecuted();
        $fake->assertExecutedWith('migration');
    }
}
// @doctest id="89e0"

Response Sequences

Return different responses for sequential calls. If more calls are made than responses provided, the last response is repeated.

$fake = AgentCtrl::fake([
    'First response',
    'Second response',
    'Third response',
]);

$first = AgentCtrl::claudeCode()->execute('First');   // "First response"
$second = AgentCtrl::claudeCode()->execute('Second'); // "Second response"
$third = AgentCtrl::claudeCode()->execute('Third');   // "Third response"

$fake->assertExecutedTimes(3);
// @doctest id="b712"

Custom Responses

Create detailed fake responses with specific metadata using the AgentCtrlFake::response() factory method.

use Cognesy\AgentCtrl\Enum\AgentType;
use Cognesy\Instructor\Laravel\Testing\AgentCtrlFake;

$customResponse = AgentCtrlFake::response(
    text: 'Generated code output',
    exitCode: 0,
    agentType: AgentType::ClaudeCode,
    cost: 0.05,
);

$fake = AgentCtrl::fake([$customResponse]);

$response = AgentCtrl::claudeCode()->execute('Test');

expect($response->cost)->toBe(0.05);
expect($response->agentType)->toBe(AgentType::ClaudeCode);
// @doctest id="1842"

Fake Tool Calls

Simulate agent tool usage in your tests.

use Cognesy\Instructor\Laravel\Testing\AgentCtrlFake;

$responseWithTools = AgentCtrlFake::response(
    text: 'Created file',
    toolCalls: [
        AgentCtrlFake::toolCall(
            tool: 'write_file',
            input: ['path' => 'app/Models/User.php'],
            output: 'File created successfully',
        ),
        AgentCtrlFake::toolCall(
            tool: 'run_tests',
            input: ['path' => 'tests/'],
            output: 'All tests passed',
        ),
    ],
);

$fake = AgentCtrl::fake([$responseWithTools]);

$response = AgentCtrl::claudeCode()->execute('...');

expect($response->toolCalls)->toHaveCount(2);
expect($response->toolCalls[0]->tool)->toBe('write_file');
// @doctest id="efa5"

Available Assertions

$fake = AgentCtrl::fake([...]);

// Run your code...

// Assert execution occurred
$fake->assertExecuted();
$fake->assertNotExecuted();
$fake->assertExecutedTimes(3);

// Assert prompt content
$fake->assertExecutedWith('Generate a migration');

// Assert agent type
$fake->assertAgentType(AgentType::ClaudeCode);
$fake->assertUsedClaudeCode();
$fake->assertUsedCodex();
$fake->assertUsedOpenCode();

// Assert streaming was used
$fake->assertStreaming();

// Access recorded executions for custom assertions
$executions = $fake->getExecutions();
foreach ($executions as $exec) {
    echo $exec['prompt'];
    echo $exec['agentType']->name;
    echo $exec['model'];
    echo $exec['timeout'];
    echo $exec['directory'];
    echo $exec['streaming'] ? 'yes' : 'no';
}

// Reset fake state between test scenarios
$fake->reset();
// @doctest id="1cf3"

Testing Agent Services

use Cognesy\Instructor\Laravel\Facades\AgentCtrl;
use Tests\TestCase;

class CodeGeneratorService
{
    public function generateMigration(array $schema): string
    {
        $response = AgentCtrl::claudeCode()
            ->inDirectory(database_path('migrations'))
            ->execute("Generate migration for: " . json_encode($schema));

        if (!$response->isSuccess()) {
            throw new \RuntimeException('Code generation failed');
        }

        return $response->text();
    }
}

final class CodeGeneratorServiceTest extends TestCase
{
    public function test_generates_migration(): void
    {
        $fake = AgentCtrl::fake([
            'Migration created successfully',
        ]);

        $service = app(CodeGeneratorService::class);
        $result = $service->generateMigration(['table' => 'users']);

        $this->assertStringContainsString('Migration', $result);
        $fake->assertUsedClaudeCode();
        $fake->assertExecutedWith('users');
    }
}
// @doctest id="a251"

HTTP Client Faking

Since the package routes all HTTP traffic through Laravel’s HTTP client (Illuminate\Http\Client\Factory), you can also use Http::fake() to intercept requests at the HTTP transport level. This approach is lower-level than facade fakes and is useful when you need to test specific HTTP request/response shapes.

use Illuminate\Support\Facades\Http;
use Tests\TestCase;

final class HttpFakeTest extends TestCase
{
    public function test_with_http_fake(): void
    {
        Http::fake([
            'api.openai.com/*' => Http::response([
                'choices' => [
                    [
                        'message' => [
                            'content' => '{"name":"John","age":30}',
                        ],
                    ],
                ],
            ]),
        ]);

        // Your StructuredOutput calls will use the fake HTTP response
        $person = StructuredOutput::with(...)->get();

        Http::assertSent(function ($request) {
            return $request->url() === 'https://api.openai.com/v1/chat/completions';
        });
    }
}
// @doctest id="8867"

This works because the LaravelDriver HTTP transport uses the same Illuminate\Http\Client\Factory instance that Http::fake() instruments. Make sure the instructor.http.driver config is set to 'laravel' (the default).

Testing Services

When testing services that use Instructor through dependency injection, the facade fake automatically replaces the container binding. The container will resolve the fake instance for both facade calls and injected dependencies.

use Cognesy\Instructor\StructuredOutput;
use Tests\TestCase;

class PersonExtractor
{
    public function __construct(
        private StructuredOutput $structuredOutput,
    ) {}

    public function extract(string $text): PersonData
    {
        return $this->structuredOutput
            ->with(messages: $text, responseModel: PersonData::class)
            ->get();
    }
}

final class PersonExtractorTest extends TestCase
{
    public function test_extracts_person(): void
    {
        $fake = StructuredOutput::fake([
            PersonData::class => new PersonData(name: 'John', age: 30),
        ]);

        // The container will resolve the fake
        $extractor = app(PersonExtractor::class);
        $person = $extractor->extract('Some text');

        $this->assertEquals('John', $person->name);
    }
}
// @doctest id="abc3"

Best Practices

1. Always Setup Fakes First

Call fake() before any code that might trigger an extraction. Setting up a fake after the fact has no effect on calls that already happened.

use Tests\TestCase;

final class StructuredOutputBestPracticesTest extends TestCase
{
    public function test_example(): void
    {
        // FIRST: Setup fake
        $fake = StructuredOutput::fake([...]);

        // THEN: Run your code
        $result = $this->service->process();

        // FINALLY: Assert
        $fake->assertExtracted(...);
    }
}
// @doctest id="015c"

2. Use Realistic Test Data

Realistic fake responses help catch bugs that only surface with production-like data, such as edge cases in string formatting or numeric precision.

// Good -- realistic data
$fake = StructuredOutput::fake([
    InvoiceData::class => new InvoiceData(
        invoiceNumber: 'INV-2024-001',
        amount: 1234.56,
        dueDate: '2024-12-31',
    ),
]);

// Avoid -- placeholder data
$fake = StructuredOutput::fake([
    InvoiceData::class => new InvoiceData(
        invoiceNumber: 'test',
        amount: 0,
        dueDate: '',
    ),
]);
// @doctest id="14c3"

3. Test Edge Cases

Verify that your code handles empty collections, null optional fields, and other boundary conditions correctly.

use Tests\TestCase;

final class StructuredOutputEdgeCaseTest extends TestCase
{
    public function test_handles_empty_response(): void
    {
        $fake = StructuredOutput::fake([
            ItemList::class => new ItemList(items: []),
        ]);

        $result = $this->service->getItems();

        $this->assertEmpty($result->items);
    }

    public function test_handles_null_optional_fields(): void
    {
        $fake = StructuredOutput::fake([
            PersonData::class => new PersonData(
                name: 'John',
                age: 30,
                email: null, // Optional field
            ),
        ]);

        $person = $this->service->getPerson();

        $this->assertNull($person->email);
    }
}
// @doctest id="0867"

4. Verify Connection and Model Usage

Assert that your code routes requests to the correct provider and model, especially when different code paths use different connections.

use Tests\TestCase;

final class StructuredOutputConnectionTest extends TestCase
{
    public function test_uses_correct_model(): void
    {
        $fake = StructuredOutput::fake([...]);

        $this->service->processWithClaude();

        $fake->assertUsedConnection('anthropic');
        $fake->assertUsedModel('claude-3-5-sonnet-20241022');
    }
}
// @doctest id="ca18"

Native Agent Testing

Resolve Cognesy\Instructor\Laravel\Testing\NativeAgentTesting from the container when you want native-agent runtime tests without app-local glue. It can:

register a FakeAgentDriver capability
swap session storage to in-memory mode
replace broadcasting with a recording transport
replace telemetry export with a recording exporter

Example:

use Cognesy\Agents\Drivers\Testing\FakeAgentDriver;
use Cognesy\Instructor\Laravel\Testing\NativeAgentTesting;

$testing = app(NativeAgentTesting::class);

$driver = $testing->fakeDriver(FakeAgentDriver::fromResponses('done'));
$sessions = $testing->fakeSessions();
$broadcasts = $testing->fakeBroadcasts();
$telemetry = $testing->captureTelemetry();
// @doctest id="1065"

Those helpers are container-aware, so the native runtime bindings exposed by packages/laravel immediately use the swapped testing surfaces.

Packages

Instructor

Polyglot

Agents

Agent-ctrl

Telemetry

Sandbox

HTTP Client

Laravel

Symfony

Xprompt

​Testing

​StructuredOutput::fake()

​Basic Usage

​Response Mapping

​Response Sequences

​Available Assertions

​Accessing Recorded Calls

​Inference::fake()

​Basic Usage

​Pattern Matching

​Response Sequences

​Available Assertions

​Embeddings::fake()

​Basic Usage

​Default Embeddings

​Custom Dimensions

​Available Assertions

​AgentCtrl::fake()

​Basic Usage

​Response Sequences

​Custom Responses

​Fake Tool Calls

​Available Assertions

​Testing Agent Services

​HTTP Client Faking

​Testing Services

​Best Practices

​1. Always Setup Fakes First

​2. Use Realistic Test Data

​3. Test Edge Cases

​4. Verify Connection and Model Usage

​Native Agent Testing

Testing

StructuredOutput::fake()

Basic Usage

Response Mapping

Response Sequences

Available Assertions

Accessing Recorded Calls

Inference::fake()

Basic Usage

Pattern Matching

Response Sequences

Available Assertions

Embeddings::fake()

Basic Usage

Default Embeddings

Custom Dimensions

Available Assertions

AgentCtrl::fake()

Basic Usage

Response Sequences

Custom Responses

Fake Tool Calls

Available Assertions

Testing Agent Services

HTTP Client Faking

Testing Services

Best Practices

1. Always Setup Fakes First

2. Use Realistic Test Data

3. Test Edge Cases

4. Verify Connection and Model Usage

Native Agent Testing