Advanced Usage

This guide covers advanced patterns and features for power users who need fine-grained control over extraction behavior, streaming, validation, and multi-provider workflows.

Streaming

Streaming lets you receive partial results as the LLM generates them, rather than waiting for the entire response. This is essential for long-running extractions where you want to show progress, or for real-time UIs that display data as it becomes available.

use Cognesy\Instructor\Laravel\Facades\StructuredOutput;

$stream = StructuredOutput::with(
    messages: 'Extract detailed company information from this long document...',
    responseModel: CompanyData::class,
)->withStreaming()->stream();

// Handle partial updates
foreach ($stream->partials() as $partial) {
    echo "Company: " . ($partial->name ?? 'Loading...') . "\n";
    echo "Industry: " . ($partial->industry ?? 'Loading...') . "\n";
    echo "---\n";
}

// Get final complete result
$company = $stream->finalValue();
// @doctest id="574e"

Streaming with `partials()`

Each partial is a partially populated instance of your response model. Properties that have not been received yet will be null or their default value. This is useful for broadcasting live updates via WebSockets.

$stream = StructuredOutput::with(
    messages: 'Extract data...',
    responseModel: MyModel::class,
)
->withStreaming()
->stream();

foreach ($stream->partials() as $partial) {
    broadcast(new PartialUpdateEvent($partial));
}

$result = $stream->finalValue();
// @doctest id="e481"

Streaming Sequences

When extracting an array of items, the sequence() method yields the growing collection as each new item is completed.

$stream = StructuredOutput::with(
    messages: 'Extract all products from this catalog...',
    responseModel: [
        'type' => 'array',
        'items' => ProductData::class,
    ],
)
->withStreaming()
->stream();

foreach ($stream->sequence() as $items) {
    echo "Found: {$items->last()->name}\n";
}
// @doctest id="4a02"

Validation and Retries

Automatic Validation

Response models are automatically validated after deserialization. When validation fails, the package sends the error messages back to the LLM with a retry prompt, asking it to correct the response. This loop continues up to max_retries times.

use Symfony\Component\Validator\Constraints as Assert;

final class UserData
{
    public function __construct(
        #[Assert\NotBlank]
        #[Assert\Length(min: 2, max: 100)]
        public readonly string $name,

        #[Assert\Email]
        public readonly string $email,

        #[Assert\Range(min: 18, max: 120)]
        public readonly int $age,
    ) {}
}

// Extraction will retry if validation fails
$user = StructuredOutput::with(
    messages: 'Extract user from: john doe, email: invalid, age: 5',
    responseModel: UserData::class,
    maxRetries: 3,
)->get();
// @doctest id="d8ad"

Custom Validators

Implement the CanValidateObject contract for domain-specific validation logic that cannot be expressed with declarative attributes. The validate method must return a ValidationResult instance.

use Cognesy\Instructor\Validation\Contracts\CanValidateObject;
use Cognesy\Instructor\Validation\ValidationResult;

class BusinessRulesValidator implements CanValidateObject
{
    public function validate(object $dataObject): ValidationResult
    {
        if ($dataObject instanceof OrderData) {
            if ($dataObject->total < $dataObject->minimumOrderValue) {
                return ValidationResult::fieldError(
                    field: 'total',
                    value: $dataObject->total,
                    message: "Order total must be at least {$dataObject->minimumOrderValue}",
                );
            }
        }

        return ValidationResult::valid();
    }
}
// @doctest id="12d2"

Custom validators are registered on the StructuredOutputRuntime, not on the facade directly:

use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$runtime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withValidator(new BusinessRulesValidator());

$order = StructuredOutput::withRuntime($runtime)->with(
    messages: 'Extract order...',
    responseModel: OrderData::class,
)->get();
// @doctest id="a5a8"

Custom Retry Prompt

Customize the message sent to the LLM when validation fails. The {errors} placeholder is replaced with the actual error messages.

$result = StructuredOutput::with(
    messages: 'Extract data...',
    responseModel: MyModel::class,
    maxRetries: 3,
    retryPrompt: 'The extraction failed validation. Errors: {errors}. Please correct and try again.',
)->get();
// @doctest id="d7b2"

Data Transformation

Apply transformations to extracted data after deserialization. Transformers run after validation, so they can normalize, enrich, or restructure the data before it reaches your application code.

use Cognesy\Instructor\Transformation\Contracts\CanTransformData;

class NormalizePhoneNumbers implements CanTransformData
{
    public function transform(mixed $data): mixed
    {
        if ($data instanceof ContactData) {
            $data->phone = $this->normalize($data->phone);
        }
        return $data;
    }

    private function normalize(string $phone): string
    {
        return preg_replace('/[^0-9+]/', '', $phone);
    }
}
// @doctest id="251a"

Custom transformers are registered on the StructuredOutputRuntime, not on the facade directly:

use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$runtime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withTransformer(new NormalizePhoneNumbers());

$contact = StructuredOutput::withRuntime($runtime)->with(
    messages: 'Contact: John, phone: (555) 123-4567',
    responseModel: ContactData::class,
)->get();

// $contact->phone === '+15551234567'
// @doctest id="1a12"

Output Modes

Different LLMs support different output modes. The output mode controls the mechanism used to extract structured data from the model’s response. You can set the default mode in config/instructor.php or override it per-request via the runtime.

use Cognesy\Instructor\Enums\OutputMode;
use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$jsonSchemaRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::JsonSchema);

$toolsRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::Tools);

$jsonRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::Json);

$mdJsonRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::MdJson);

// JSON Schema mode (recommended for OpenAI)
$result = StructuredOutput::withRuntime($jsonSchemaRuntime)
    ->with(...)
    ->get();

// Tool/Function calling mode
$result = StructuredOutput::withRuntime($toolsRuntime)
    ->with(...)
    ->get();

// Simple JSON mode
$result = StructuredOutput::withRuntime($jsonRuntime)
    ->with(...)
    ->get();

// Markdown JSON (for Gemini)
$result = StructuredOutput::withRuntime($mdJsonRuntime)
    ->with(...)
    ->get();
// @doctest id="46da"

Few-Shot Learning

Providing input/output examples significantly improves extraction quality, especially for ambiguous or domain-specific data. Each example pairs an input string with a fully populated response model instance.

$person = StructuredOutput::with(
    messages: 'Extract: Jane Doe, 25 years old, jane@example.com',
    responseModel: PersonData::class,
    examples: [
        [
            'input' => 'John Smith is 30 years old and works at john@company.com',
            'output' => new PersonData(
                name: 'John Smith',
                age: 30,
                email: 'john@company.com',
            ),
        ],
        [
            'input' => 'Mary Johnson, age 45',
            'output' => new PersonData(
                name: 'Mary Johnson',
                age: 45,
                email: null,
            ),
        ],
    ],
)->get();
// @doctest id="9b9c"

System Prompts

System prompts set the overall behavior and domain context for the LLM. They are especially valuable when extracting specialized data.

$medical = StructuredOutput::with(
    messages: $patientNotes,
    responseModel: MedicalRecord::class,
    system: <<<'PROMPT'
        You are a medical records extraction specialist.
        Extract structured data from clinical notes.
        Use standard medical terminology.
        If information is unclear, mark as null rather than guessing.
        PROMPT,
)->get();
// @doctest id="3066"

Tool Descriptions

Customize how the response model is described to the LLM in the tool/function calling interface. This is particularly useful when the auto-generated name or description is not descriptive enough for the model to understand the task.

$result = StructuredOutput::with(
    messages: 'Extract invoice details...',
    responseModel: InvoiceData::class,
    toolName: 'extract_invoice',
    toolDescription: 'Extracts structured invoice data including line items, totals, and payment terms.',
)->get();
// @doctest id="bf9b"

Multiple Providers

Switch between providers based on the task at hand. Different providers offer different trade-offs in speed, accuracy, cost, and privacy.

class AIService
{
    // Fast, cheap extraction for simple tasks
    public function quickExtract(string $text, string $model): mixed
    {
        return StructuredOutput::connection('groq')
            ->with(messages: $text, responseModel: $model)
            ->get();
    }

    // High-quality extraction for complex tasks
    public function precisionExtract(string $text, string $model): mixed
    {
        return StructuredOutput::connection('anthropic')
            ->withModel('claude-3-opus-20240229')
            ->with(messages: $text, responseModel: $model)
            ->get();
    }

    // Local extraction for sensitive data
    public function privateExtract(string $text, string $model): mixed
    {
        return StructuredOutput::connection('ollama')
            ->with(messages: $text, responseModel: $model)
            ->get();
    }
}
// @doctest id="96e4"

Cached Context (Prompt Caching)

For repeated extractions with the same system prompt, examples, or large context, use withCachedContext() to signal that the context should be cached by providers that support prompt caching (e.g., Anthropic, OpenAI). This can significantly reduce latency and cost for subsequent calls.

$result = StructuredOutput::withCachedContext(
    system: 'You are a legal document analyzer...',
    examples: $examples,
)->with(
    messages: $newDocument,
    responseModel: LegalAnalysis::class,
)->get();
// @doctest id="5bc9"

Caching Strategies

Response Caching

Cache extraction results for identical inputs to avoid redundant API calls.

use Illuminate\Support\Facades\Cache;

class CachedExtractor
{
    public function extract(string $text, string $responseModel): mixed
    {
        $cacheKey = 'extract:' . md5($text . $responseModel);

        return Cache::remember($cacheKey, 3600, function () use ($text, $responseModel) {
            return StructuredOutput::with(
                messages: $text,
                responseModel: $responseModel,
            )->get();
        });
    }
}
// @doctest id="8d4a"

Semantic Caching

Use embeddings to find cached results for semantically similar (but not identical) inputs.

use Cognesy\Instructor\Laravel\Facades\Embeddings;

class SemanticCache
{
    public function extractWithCache(string $text, string $responseModel): mixed
    {
        // Generate embedding for input
        $embedding = Embeddings::withInputs($text)->first();

        // Check for similar cached results
        $cached = $this->findSimilar($embedding);
        if ($cached) {
            return $cached;
        }

        // Extract and cache
        $result = StructuredOutput::with(
            messages: $text,
            responseModel: $responseModel,
        )->get();

        $this->store($embedding, $result);

        return $result;
    }
}
// @doctest id="6428"

Batch Processing

Process multiple items efficiently, either synchronously or via queued jobs for large batches.

use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Bus;

class BatchExtractor
{
    public function extractBatch(Collection $documents): Collection
    {
        return $documents->map(function ($document) {
            return StructuredOutput::with(
                messages: $document->content,
                responseModel: DocumentData::class,
            )->get();
        });
    }

    // Or with queued jobs for large batches
    public function extractBatchAsync(Collection $documents): void
    {
        $jobs = $documents->map(fn ($doc) => new ExtractDocumentJob($doc));

        Bus::batch($jobs)
            ->name('Document Extraction')
            ->dispatch();
    }
}
// @doctest id="e559"

Error Handling

Graceful Degradation

Wrap extraction calls in try-catch blocks to handle API failures without crashing your application.

use Cognesy\Instructor\Laravel\Facades\StructuredOutput;

class ResilientExtractor
{
    public function extract(string $text): ?PersonData
    {
        try {
            return StructuredOutput::with(
                messages: $text,
                responseModel: PersonData::class,
            )->get();
        } catch (\Throwable $e) {
            Log::warning('Extraction failed', [
                'error' => $e->getMessage(),
                'text' => substr($text, 0, 100),
            ]);

            return null;
        }
    }
}
// @doctest id="ef56"

Fallback Providers

Automatically try alternative providers when the primary one fails. This pattern provides resilience against provider outages and rate limits.

class FallbackExtractor
{
    private array $providers = ['openai', 'anthropic', 'groq'];

    public function extract(string $text, string $model): mixed
    {
        foreach ($this->providers as $provider) {
            try {
                return StructuredOutput::connection($provider)
                    ->with(messages: $text, responseModel: $model)
                    ->get();
            } catch (\Throwable $e) {
                Log::warning("Provider {$provider} failed", [
                    'error' => $e->getMessage(),
                ]);
                continue;
            }
        }

        throw new RuntimeException('All providers failed');
    }
}
// @doctest id="dafa"

Performance Optimization

Reduce Token Usage

// Be concise in system prompts
$result = StructuredOutput::with(
    messages: $text,
    responseModel: MyModel::class,
    system: 'Extract data. Be concise.', // Short system prompt
)->get();

// Use smaller models for simple extractions
$result = StructuredOutput::withModel('gpt-4o-mini')
    ->with(messages: $text, responseModel: SimpleModel::class)
    ->get();
// @doctest id="5c65"

Parallel Extraction

Use Laravel’s concurrency features to run multiple extractions simultaneously.

use Illuminate\Support\Facades\Concurrency;

$results = Concurrency::run([
    fn () => StructuredOutput::with(messages: $text1, responseModel: Model::class)->get(),
    fn () => StructuredOutput::with(messages: $text2, responseModel: Model::class)->get(),
    fn () => StructuredOutput::with(messages: $text3, responseModel: Model::class)->get(),
]);
// @doctest id="b205"

Packages

Instructor

Polyglot

Agents

Agent-ctrl

Telemetry

Sandbox

HTTP Client

Laravel

Symfony

Xprompt

Advanced

Advanced Usage

Streaming

Streaming with `partials()`

Streaming Sequences

Validation and Retries

Automatic Validation

Custom Validators

Custom Retry Prompt

Data Transformation

Output Modes

Few-Shot Learning

System Prompts

Tool Descriptions

Multiple Providers

Cached Context (Prompt Caching)

Caching Strategies

Response Caching

Semantic Caching

Batch Processing

Error Handling

Graceful Degradation

Fallback Providers

Performance Optimization

Reduce Token Usage

Parallel Extraction

​Advanced Usage

​Streaming

​Streaming with partials()

​Streaming Sequences

​Validation and Retries

​Automatic Validation

​Custom Validators

​Custom Retry Prompt

​Data Transformation

​Output Modes

​Few-Shot Learning

​System Prompts

​Tool Descriptions

​Multiple Providers

​Cached Context (Prompt Caching)

​Caching Strategies

​Response Caching

​Semantic Caching

​Batch Processing

​Error Handling

​Graceful Degradation

​Fallback Providers

​Performance Optimization

​Reduce Token Usage

​Parallel Extraction

Advanced Usage

Streaming

Streaming with `partials()`

Streaming Sequences

Validation and Retries

Automatic Validation

Custom Validators

Custom Retry Prompt

Data Transformation

Output Modes

Few-Shot Learning

System Prompts

Tool Descriptions

Multiple Providers

Cached Context (Prompt Caching)

Caching Strategies

Response Caching

Semantic Caching

Batch Processing

Error Handling

Graceful Degradation

Fallback Providers

Performance Optimization

Reduce Token Usage

Parallel Extraction