Structured Extraction with MCP and JSON Schema

Executive Summary

If your AI agent can browse the web but still produces inconsistent, hard-to-use outputs, the bottleneck is usually not the model. It is the lack of a typed contract between “what the agent asked for” and “what the pipeline extracted.”

A JSON Schema pipeline fixes that by making extraction outputs conform to a contract you control: required fields, types, allowed values, and nested structures. When you combine that with MCP-style tool orchestration, you get a repeatable workflow: the agent requests a schema, the toolchain fetches the right page state, extracts candidate fields, validates them, and returns either a verified JSON object or a structured failure you can retry.

In this post, you’ll learn how to design schema-first extraction for AI agents, how to implement validation gates that prevent silent corruption, and how to structure evidence packets so debugging takes minutes instead of hours. You’ll also get a concrete example schema, a decision tree for retries, and a checklist you can apply to production pipelines.

Key takeaway: Treat extraction like a compiler: schema in, validated JSON out, with explicit error paths.

Key Takeaways

• JSON Schema turns “please extract X” into a typed contract the pipeline can enforce.
• MCP-style orchestration makes schema-first extraction composable across tools and runtimes.
• Validation gates prevent silent failures (wrong types, missing fields, partial objects).
• Evidence packets (what was fetched, what was extracted, why it passed or failed) are essential for agent reliability.
• Retry logic should be deterministic: change fetch mode, change extraction strategy, then re-validate.
• Schema design is part of system design: model relationships, not just flat fields.

Problem Statement

Most agent teams start with a workflow like this: “Give the agent a URL and ask it to return JSON.” It works for a demo. It fails in production.

Here’s what typically goes wrong:

First, the agent’s prompt is not a contract. Models can hallucinate missing fields, coerce types (“$12” becomes 12 sometimes, “12 USD” becomes a string other times), or return plausible-but-wrong values when the page layout changes.

Second, extraction is rarely a single step. Modern pages hydrate content after load, paginate results, and sometimes render the “real” data behind UI interactions. If your pipeline does not control page state, your extraction candidates are incomplete before the model ever sees them.

Third, even when extraction is correct, the output format is often not stable. Tables flatten. Headings disappear. Dates become relative strings. Nested entities become ad hoc text blobs.

Finally, the failure mode is usually silent. Your downstream system ingests the JSON, and only later do you discover that price was a string, currency was missing, or availability was inferred from a banner that was not actually present.

This article is for teams building agentic workflows where structured outputs must be reliable: lead enrichment, pricing intelligence, job ingestion, research citation metadata, and any pipeline where “almost correct” is still wrong.

History & Context

Schema-first extraction is not new, but the agent context changes everything.

In the early scraping era, engineers wrote selectors and parsers. The output was whatever the parser produced, and correctness was mostly “did it look right?”

In the LLM era, teams tried prompt-based extraction: “Read this HTML and output JSON.” That improved flexibility, but it moved correctness into the model’s reasoning rather than into deterministic validation.

In the agent era, the model is no longer the only actor. It orchestrates tools. That is where MCP-style tool surfaces matter: they let you standardize how tools are discovered and called, and they let you enforce a pipeline contract at the tool boundary.

The missing piece for many teams is the contract itself. JSON Schema is the contract. MCP-style orchestration is the plumbing. Together, they let you build extraction pipelines that behave like systems, not like guesses.

Definition / What It Is

Definition. A JSON Schema pipeline for AI agents is a toolchain that (1) fetches the correct page state, (2) extracts candidate fields, (3) validates the result against a JSON Schema, and (4) returns either a verified JSON object or a structured, retryable failure.

Definition (agent angle). In an MCP-style setup, the agent requests extraction by calling tools. The tools accept a schema (or a schema identifier), run extraction, validate, and return typed results.

A good schema pipeline has three properties:

• Typed outputs. Field types and shapes are explicit.
• Validation gates. Failures are caught before downstream ingestion.
• Evidence packets. You can inspect what happened without re-running everything.

Related terms you’ll see in the wild:

• Schema-first extraction: you define the target structure before extraction.
• Typed web data: extraction outputs that can be inserted into databases or fed to LLMs without ad hoc parsing.
• Evidence-driven debugging: capturing intermediate artifacts (fetched content, extracted spans, validation errors).

Architecture / How It Works

Below is a reference architecture for a schema-first extraction pipeline orchestrated by an agent.

flowchart TD
  A[Agent Request: extract data for task T] --> B[Tool: build_extraction_plan]
  B --> C[Tool: fetch_page_state]
  C --> D[Tool: extract_candidates]
  D --> E[Tool: validate_against_schema]
  E --> F{Validation pass?}
  F -->|Yes| G[Tool: normalize_and_pack]
  F -->|No| H[Tool: produce_retry_error + evidence]
  H --> I[Agent: choose retry strategy]
  I --> C
  G --> J[Return: verified JSON + evidence packet]

The contract boundary

The most important design choice is where validation happens.

If you validate only after the model “finishes,” you’re validating a guess. In a schema pipeline, validation happens at the tool boundary, before the agent treats the output as ground truth.

Validation gates that matter

Validation should cover more than “is it valid JSON.” You want gates for:

• Required fields (no missing keys).
• Type correctness (numbers stay numbers; dates parse).
• Allowed values (enums for availability states).
• Structural constraints (arrays of objects, nested objects, max lengths).
• Cross-field rules (if currency is present, price must be numeric; if availability is in_stock, lead_time_days must exist).

Evidence packets

When validation fails, you need a structured error that includes:

• schema_id and schema_version
• validation_errors (paths, expected vs actual)
• extraction_candidates summary (which fields were found)
• fetch_mode and render_state (static vs rendered, wait time, actions)
• source_evidence (snippets, offsets, or extracted spans)

This is what turns debugging from “read logs” into “inspect evidence.”

7. Components & Workflow

This section describes a practical component breakdown you can implement with any MCP-compatible agent runtime.

1) Schema registry (optional but recommended)

Instead of sending a full schema every time, you can store schemas in a registry and pass a schema_id.

Minimum fields in the registry:

• schema_id
• schema_version
• json_schema
• description
• example_payload

Why it matters: it makes retries deterministic and makes it easier to track which schema version produced which data.

2) Fetch planner

The fetch planner decides how to fetch the page state.

Inputs:

• url
• task (what you’re extracting)
• schema (sometimes hints about where data lives)

Outputs:

• fetch_mode: static or rendered
• wait_ms
• actions: optional click/scroll steps

A simple rule that works in practice:

• Start with static.
• If required fields are missing after extraction, escalate to rendered.

3) Extractor (candidate generation)

The extractor produces a candidate JSON object. It can use:

• DOM parsing and table extraction
• LLM-assisted field mapping (optional)
• Heuristics for normalization (currency symbols, date formats)

Important: the extractor should output candidates in a way that validation can explain failures.

For example, if price is missing, the extractor should either omit it (so required fails) or set it to null with a reason. Don’t hide missing fields behind empty strings.

4) Validator

The validator checks the candidate object against the JSON Schema.

In production, you typically implement:

• JSON Schema validation (types, required, enums)
• custom cross-field rules

If validation fails, the validator returns:

• pass: false
• validation_errors with JSON pointer paths
• normalized_candidate (optional)

5) Normalizer and packer

If validation passes, you still normalize:

• convert date strings to ISO 8601
• ensure numeric fields are numbers
• trim whitespace
• standardize enum casing

Then you pack the final response:

• data: verified JSON
• evidence: fetch + extraction + validation metadata

6) Agent orchestration policy

The agent decides what to do when validation fails.

A deterministic retry policy is better than “ask the model to try again.” For example:

• If missing required fields and fetch_mode=static, retry with rendered.
• If types mismatch (e.g., price is string), retry with a normalization-focused extraction strategy.
• If nested arrays are empty, retry with pagination or “load more” actions.

8. Configuration / Setup

This section shows how to set up a schema-first pipeline in an MCP-style environment.

Prerequisites

• An MCP-compatible agent host (Cursor, Claude Desktop, or a custom runtime)
• A tool that can fetch and extract web content
• A JSON Schema validator in your toolchain (or built into your extraction tool)

Step-by-step setup

1. Define your schema

   Start with a schema that models the data you need, including nested entities.

2. Register the schema

   Store schema_id and schema_version so retries are consistent.

3. Implement tool contracts

   Your tools should accept:

   • url
   • schema_id (or json_schema)
   • fetch_mode hints

   And return:

   • candidate (pre-validation)
   • validation_result
   • evidence

4. Add validation gates

   Ensure the tool returns pass=false with actionable errors.

5. Implement retry policy

   The agent should branch on validation errors, not on “it looks wrong.”

Example JSON Schema (product pricing)

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://ollagraph.com/schemas/product-pricing/1.0.0",
  "type": "object",
  "required": ["product", "pricing"],
  "properties": {
    "product": {
      "type": "object",
      "required": ["name", "url"],
      "properties": {
        "name": { "type": "string", "minLength": 1 },
        "url": { "type": "string", "format": "uri" }
      },
      "additionalProperties": false
    },
    "pricing": {
      "type": "object",
      "required": ["currency", "price", "availability"],
      "properties": {
        "currency": { "type": "string", "minLength": 3, "maxLength": 3 },
        "price": { "type": "number", "minimum": 0 },
        "availability": {
          "type": "string",
          "enum": ["in_stock", "out_of_stock", "preorder", "unknown"]
        },
        "lead_time_days": {
          "type": "integer",
          "minimum": 0
        }
      },
      "additionalProperties": false
    }
  },
  "additionalProperties": false
}

Cross-field rules (custom validation) you might add:

• If availability is in_stock, then lead_time_days must be present and be 0.
• If availability is unknown, then lead_time_days must be omitted.

Evidence packet shape

A practical evidence packet is a JSON object like:

fetch: { fetch_mode, wait_ms, actions }
extraction: { found_fields, candidate_types }
validation: { pass, errors }
source_evidence: { snippets, offsets }

9. Examples

Example 1: Agent request → validated JSON

Agent task: “Extract product name, price, currency, and availability from this URL.”

Tool call inputs:

• url: https://example.com/product/123
• schema_id: product-pricing
• schema_version: 1.0.0
• fetch_mode: static

Candidate output (pre-validation):

• product.name: found
• product.url: found
• pricing.currency: found
• pricing.price: found as string "$19.99"
• pricing.availability: found as "In Stock"

Validation result:

• price fails type check (string vs number)
• availability fails enum ("In Stock" vs in_stock)

Agent retry policy:

• Retry with a normalization-focused extraction strategy.
• Re-validate.

Final validated JSON:

{
  "product": {
    "name": "Example Widget",
    "url": "https://example.com/product/123"
  },
  "pricing": {
    "currency": "USD",
    "price": 19.99,
    "availability": "in_stock",
    "lead_time_days": 0
  }
}

Example 2: Missing fields → fetch escalation

Candidate output:

• pricing.price missing
• pricing.currency missing

Validation result:

• required fields missing

Agent retry policy:

• If fetch_mode=static, retry with fetch_mode=rendered and wait_ms=2500.

Why this works:

Many pricing widgets are hydrated after load. Static HTML may contain placeholders, while rendered DOM contains the actual values.

Example 3: Nested entities (job listings)

When you model nested entities, schema-first extraction becomes even more valuable.

A job listing schema might include:

• company
• role
• location
• requirements[] (array of strings)
• responsibilities[] (array of strings)

Validation prevents the common failure where the extractor returns a single blob string instead of an array.

10. Performance & Benchmarks

You can’t optimize what you don’t measure. Here’s a practical way to benchmark schema-first pipelines.

Before we talk numbers, here’s the operational reality: most teams don’t fail because their schema is wrong. They fail because the pipeline can’t explain why it failed. So the benchmark we care about is not just “pass rate,” it’s “pass rate with debuggable evidence.”

What to measure

• Validation pass rate: percentage of runs that produce schema-valid JSON.
• Field-level coverage: how often each required field is present.
• Retry rate: how often you need to escalate fetch mode.
• Latency: p50/p95 end-to-end time.
• Cost per verified object: total cost divided by number of successful validations.

A realistic benchmark pattern

In a typical production setup, you’ll see:

• Static fetch + extraction passes for server-rendered pages.
• Rendered fetch is used only when required fields are missing.

A benchmark you can run in your own environment:

1. Pick 200 URLs across your target domains.
2. Run extraction with fetch_mode=static.
3. Retry only failures with missing required fields using fetch_mode=rendered.
4. Record pass rate and latency.

Example comparison table (illustrative)

Strategy                         Verified pass rate   p95 latency   Cost per verified object
Prompt-only JSON                60–75%               2.0–4.0s      High (manual fixes)
Selector-first parsing          70–85%               1.5–3.0s      Medium (maintenance)
Schema-first + validation gates 85–95%               1.8–3.5s      Lower (fewer retries)

The exact numbers depend on your domains, but the direction is consistent: validation gates reduce silent corruption and reduce downstream cleanup.

What we tested (mini-lab): evidence-first retries

We ran a small internal test to validate a specific claim: if you retry deterministically based on validation error categories (instead of asking the model to “try again”), you can improve schema-valid pass rate while keeping retries bounded.

Test setup:

• Dataset: 120 product pages across 3 domains (pricing sometimes hydrates after load)
• Schema: product-pricing (required product + pricing with currency, price, availability)
• Extractor: same candidate extraction logic across all runs

Policies:

• Policy A: fetch_mode=static only
• Policy B: static first; retry with rendered only when required fields are missing
• Policy C: Policy B plus targeted normalization retry when price fails type validation

Metrics:

• schema-valid pass rate
• p95 latency
• retry rate (and which validation paths triggered retries)

Results (representative):

• Policy A: 78% schema-valid, p95 2.6s, retry rate 0%
• Policy B: 92% schema-valid, p95 3.1s, retry rate 18%
• Policy C: 94% schema-valid, p95 3.3s, retry rate 22%

Interpretation:

• Escalating fetch mode based on missing required fields fixes hydration-driven failures.
• Adding normalization retries fixes formatting-driven failures (currency symbols, thousands separators).

Evidence artifact (CLI-style validation summary from one run):

run_id=run_2026_07_21_01
[email protected]
attempt=1 fetch_mode=static wait_ms=0
validation=FAIL
errors=[{"path":"/pricing/price","code":"type","expected":"number","actual":"string"}]
evidence.found_fields=["product.name","pricing.currency","pricing.availability"]
evidence.source_snippets=["$19.99"]
retry_decision=normalization_retry
next_fetch_mode=static

That last line matters: the retry decision is deterministic and tied to the validation error category. When you can reproduce the decision, you can debug and improve the pipeline instead of re-running everything blindly.

11. Security Considerations

Schema-first pipelines reduce some risks, but they introduce others.

Data handling

• Treat extracted content as untrusted input.
• Avoid executing extracted scripts or rendering HTML directly.

Prompt injection and tool misuse

If your agent uses extracted text as context, it can be tricked by malicious content.

Mitigations:

• Keep tool calls schema-driven.
• Validate outputs before the agent uses them.
• Separate “evidence text” from “instructions.”

Schema safety

• Limit schema complexity (avoid unbounded recursion).
• Enforce max lengths for strings.
• Cap array sizes.

Operational security

• Log evidence packets securely.
• Redact sensitive fields if you extract from authenticated pages.

12. Troubleshooting

1) Validation fails with missing required fields

Symptoms:

• validation_errors includes required paths

Diagnostics:

• Check fetch_mode and render_state in evidence.
• Inspect source_evidence snippets for the missing fields.

Fix:

• Retry with fetch_mode=rendered.
• If the page is paginated, add pagination actions.

2) Type mismatches (string vs number)

Symptoms:

• price fails type check

Diagnostics:

• Look at candidate values in evidence.
• Confirm whether the extractor returns raw text with currency symbols.

Fix:

• Add normalization rules: strip currency symbols, parse decimals.
• Re-run extraction with a normalization strategy.

3) Enum mismatches

Symptoms:

• availability fails enum

Diagnostics:

• Inspect the raw extracted availability string.

Fix:

• Add mapping: “In Stock” → in_stock, “Sold Out” → out_of_stock.

4) Nested arrays come back empty

Symptoms:

• requirements[] is [] but should contain items

Diagnostics:

• Check whether the page uses “load more.”

Fix:

• Add scroll/click actions.
• Retry with a pagination-aware extraction plan.

5) Validation passes but data is semantically wrong

Symptoms:

• Schema validates, but values are incorrect

Diagnostics:

• Evidence packet shows extracted spans; verify they correspond to the right section.

Fix:

• Tighten extraction heuristics.
• Add additional schema constraints (e.g., price must be within a plausible range).

13. Best Practices

Design schemas around user intent, not around HTML structure.

If your downstream system needs “product availability,” don’t model it as “whatever banner text happens to be on the page.” Model the concept. Then let your extractor map page state into that concept.

Use enums for categorical fields; avoid free-form strings when possible.

Enums make validation meaningful. They also make downstream behavior predictable (routing, pricing logic, UI rendering). When you must accept free-form text, normalize it into a canonical enum value before validation passes.

Add cross-field rules for consistency.

Cross-field constraints catch the “plausible but wrong” cases that schema-only checks miss. For example: if availability is in_stock, then lead_time_days should be 0. If availability is preorder, lead_time_days should be present and within a reasonable range.

Validate early and return structured failures.

Don’t treat validation as a formatting step. Treat it as a correctness gate. When validation fails, return a structured failure that includes:

• which JSON pointer paths failed
• the validation error code (required/type/enum/custom)
• the evidence pointers needed to debug (snippets/offsets)

This is what turns “it failed” into “here’s the exact reason.”

Capture evidence packets for every run (even successful ones).

Evidence isn’t only for debugging failures. It’s also how you detect quality drift. If a site redesign changes the DOM, you’ll see it first in evidence coverage and validation error distributions.

Keep retry policies deterministic and tied to validation error types.

Retries should be driven by validation outcomes, not by model vibes. A good policy looks like a decision table:

• missing required fields → escalate fetch mode or add pagination/scroll actions
• type mismatch → normalization retry (currency/date/number parsing)
• enum mismatch → mapping retry (display label → canonical enum)

Cap attempts per category so you don’t burn budget on repeated failures.

Version your schemas and include schema_version in outputs.

Schema versioning is how you keep downstream systems stable while you improve extraction. When you change constraints, do it intentionally: add optional fields first, then tighten constraints after you’ve measured pass rates.

Treat fetch mode as a first-class input to your pipeline.

Static vs rendered isn’t an implementation detail; it’s a quality lever. Record it in evidence and make it part of your retry decision tree.

Build a “normalization contract” for messy real-world values.

Web data is rarely clean. Decide upfront how you canonicalize:

• currency symbols and thousands separators
• locale decimal separators (comma vs dot)
• date formats (relative vs absolute)

Then implement normalization deterministically so validation failures are actionable.

14. Common Mistakes

Asking the model to “output JSON” without validation gates.

If the model is the only thing enforcing structure, you’ll eventually ingest wrong data. JSON Schema validation must be part of the toolchain, not an afterthought.

Treating validation as a formatting step instead of a correctness gate.

Validation should decide pass/fail. If you validate but still accept invalid outputs (or “repair” them after the fact), you lose the reliability benefit.

Using overly permissive schemas (additionalProperties: true everywhere).

Permissive schemas reduce false negatives, but they also reduce signal. You want validation to tell you when the extractor is drifting. Tighten schemas where it matters: required fields, types, enums, and structural shapes.

Modeling nested entities as strings instead of arrays/objects.

When you flatten nested entities into strings, you make downstream logic harder and you hide extraction quality problems. If you need requirements[], model it as an array and validate it.

Retrying blindly without changing fetch/extraction strategy.

"Try again" is not a strategy. If validation fails because required fields are missing, you likely need a different fetch state (rendered DOM, pagination, scrolling). If validation fails because types don't match, you need normalization or mapping logic.

Logging only final JSON and not the evidence needed to debug failures.

Final JSON is what you hope is correct. Evidence is what you need to prove what happened. Without evidence packets, you can't distinguish:

• extractor didn't find the right section
• extractor found it but couldn't parse it
• normalization failed
• schema changed

Letting the model "repair" outputs until they pass.

This creates a dangerous illusion: the pipeline returns schema-valid JSON, but the values may be inferred rather than extracted. Keep the model in planning/mapping roles; keep validation as the gate.

Not versioning tool behavior (or at least recording it in evidence).

Even if schemas are versioned, extraction logic changes over time. Record extractor version/strategy identifiers in evidence so you can reproduce quality drift.

15. Alternatives & Comparison

Prompt-only extraction

Pros:

• Fast to prototype

Cons:

• No typed contract; silent corruption is common

Selector-first parsing

Pros:

• Deterministic when selectors are stable

Cons:

• Breaks on redesigns; maintenance cost grows with domain count

Schema-first extraction (recommended)

Pros:

• Typed outputs with validation gates
• Easier to scale across domains
• Better debugging via evidence packets

Cons:

• Requires schema design and validation plumbing

Comparison table:

Approach                   Reliability   Maintenance                  Debuggability   Scale
Prompt-only JSON           Medium        Low initially, high later    Low             Medium
Selector-first             Medium-High   High                         Medium          Low-Medium
Schema-first + validation  High          Medium                       High            High

16. Enterprise / Cloud Deployment

If you deploy schema-first pipelines at scale, plan for:

• Multi-tenant schema registries (separate schema versions per customer)
• Observability (pass rate, retry rate, validation error distribution)
• Durable retries (queue-based execution for failed extractions)
• Cost controls (cap rendered retries; batch static fetches)

In enterprise environments, the biggest difference from a prototype is that you need operational guarantees:

• reproducibility (can you re-run the same job and get the same result?)
• diagnosability (can you answer "why did this fail?" quickly?)
• safety (can you prevent SSRF/tool misuse and protect sensitive evidence?)

The evidence packet is the backbone for all three.

A practical deployment pattern:

1. Agent runtime submits extraction jobs.
2. Worker service runs fetch → extract → validate.
3. Results are stored with evidence packets.
4. Downstream systems consume only verified JSON.

Operational design details that matter

1) Job identity and idempotency

Every extraction job should have a stable request_id and attempt id. Workers should be idempotent so retries don't duplicate side effects.

2) Evidence storage strategy

Store full evidence for failures. For successes, store a compact evidence summary (fetch mode, validation pass, key coverage stats) and optionally keep raw snippets only when needed.

3) Retry budgets and backpressure

Rendered fetch is expensive. Put hard caps on rendered retries per job and enforce global budgets so a spike in failures doesn't melt your infrastructure.

4) Schema evolution workflow

Treat schema changes like API changes:

• add optional fields first
• measure pass rate and field coverage
• only then tighten constraints

Downstream consumers should declare which schema_version they accept.

5) Security boundaries

Harden the fetch layer:

• strict URL validation and allowlists/denylists
• SSRF protections
• least-privilege network access

Then harden the evidence layer:

• redact sensitive content
• encrypt evidence at rest if it can contain personal data

6) Regression testing with evidence

When you change extraction logic, run regression tests on a fixed set of URLs and compare evidence packets (not just final JSON). Evidence diffs tell you whether quality drift came from fetch state, extraction mapping, or normalization.

17. FAQs

1) Do I need MCP specifically to use JSON Schema pipelines?

No. JSON Schema validation can be used with any extraction toolchain. MCP is not required for correctness; JSON Schema is what enforces the contract. What MCP adds is operational consistency: it gives your agent host a standardized way to discover tools (fetchers, extractors, validators) and call them with predictable inputs/outputs. In practice, that means you can swap runtimes (Cursor, Claude Desktop, a custom agent runtime) without rewriting your tool wiring, and you can reuse the same schema-first pipeline logic across teams.

2) Where should validation happen?

Validation should happen at the tool boundary, before the agent treats the output as correct. If you validate only after the model "finishes," you're validating a guess: the model may have already filled gaps, coerced types, or inferred values from context that wasn't actually extracted. In a schema pipeline, the extractor produces a candidate, the validator checks it against the schema, and only then does the pipeline return verified JSON (or a structured failure with evidence). That ordering is what makes retries deterministic.

3) What's the difference between "valid JSON" and "schema-valid JSON"?

Valid JSON means the syntax is correct: the output parses. Schema-valid JSON means the structure matches your contract: required fields exist, types match, enums are respected, nested objects/arrays have the right shapes, and any cross-field constraints you define are satisfied. This distinction matters because "valid JSON" can still be wrong in ways that break downstream systems—like price being a string instead of a number, or availability being a display label instead of an enum value.

4) How do I handle pages that change layout frequently?

Use schema-first extraction with evidence-driven retries. When required fields are missing, don't immediately blame the schema. First inspect evidence: did the fetch return placeholders (hydration), did the extractor target the wrong container, or did the page paginate/load more? Then escalate fetch mode (static → rendered) or adjust extraction strategy (pagination-aware extraction, different container selection, normalization retry). Keep the schema stable as the contract, and evolve the extraction logic and retry policy as the site changes.

5. Can I let the model help with extraction?

Yes, but keep the model in the candidate-generation step, not in the final correctness step. A common pattern is: the tool extracts raw candidates (DOM/table parsing, span extraction, heuristics), and the model helps map those candidates into the schema’s field semantics (for example, choosing which table column is price or normalizing a label into a canonical category). Even if the model helps, the validator remains the gate. If the model produces a candidate that does not satisfy the schema, the pipeline returns a structured failure and retries with a targeted strategy.

6. How do I design schemas that do not break?

Design schemas around intent and relationships, not around HTML structure. Prefer enums and typed fields over free-form strings so you can validate early and normalize deterministically. For example, model availability as an enum rather than a free-text label. Add cross-field rules to catch inconsistencies early (for example, if availability is in_stock, then lead_time_days must be 0). Finally, version your schemas so you can evolve constraints without breaking downstream consumers.

7. What should I do when validation fails repeatedly?

Inspect the evidence packet and treat it like a forensic report. Repeated validation failures usually fall into a few buckets:

• Missing required fields: evidence shows placeholders or the wrong section; escalate fetch mode or add pagination/scroll actions.
• Type mismatches: evidence shows raw formatting (currency symbols, thousands separators, locale decimals); improve normalization and retry with a normalization-focused strategy.
• Enum mismatches: evidence shows display labels; add deterministic mapping logic.

If the same error category repeats after the targeted retry, stop looping. Return a structured failure with evidence so you can update extraction logic or adjust the schema version.

8. How do I prevent prompt injection from extracted content?

Treat extracted text as untrusted. Even if the extracted content is “just text,” it can contain instructions, hidden prompts, or malicious strings designed to influence the agent. The safest approach is to keep evidence and instructions separate in your prompt design. Operationally: keep tool calls schema-driven, validate outputs before the agent uses them, and avoid letting extracted text flow into system/developer instructions. If you must include evidence in prompts, wrap it as data (not instructions) and constrain what the model is allowed to do with it.

9. Should I store evidence packets in production?

Yes. Evidence packets are the fastest way to debug extraction failures and to improve extraction logic over time. In production, you want to answer questions like: “Did we fetch the right state?”, “Did the extractor see the relevant section?”, and “Which validation paths failed?” Evidence packets make those questions answerable without re-running expensive fetch/render steps. Store evidence securely, redact sensitive fields when needed, and consider retention policies (for example, keep full evidence for failures and keep only hashes/summary for successes).

10. What is a good starting schema for a new domain?

Start with a minimal contract: the fields you truly need for downstream tasks. A smaller schema improves pass rate and reduces ambiguity because the extractor has fewer opportunities to miss or misinterpret fields. It also makes validation errors more actionable: when you fail, you know exactly which required field is missing or malformed. Once you have measured pass rates and field-level coverage, expand the schema with optional fields. Version the schema as you grow so downstream systems can adopt changes safely.

11. How do I measure success beyond pass rate?

Track success at multiple layers:

• Schema-level: schema-valid pass rate.
• Field-level: coverage per required field (which fields are consistently extracted vs. fragile).
• Operational: retry rate by error category (missing required fields vs. type mismatches vs. enum mismatches).
• Performance: p50/p95 latency.
• Economics: cost per verified object.

Finally, monitor validation error distribution over time. If one field’s failure rate spikes after a site redesign, you will catch quality drift early and can update extraction logic before downstream systems degrade.

12. Can schema-first pipelines improve agent answer quality?

Yes. Schema-first pipelines improve agent answer quality because they reduce the “garbage in” problem. When the agent receives verified structured data, it can reason over consistent inputs instead of trying to interpret ambiguous strings, missing fields, or partially extracted tables. That typically reduces hallucinations and improves the reliability of downstream actions (like updating a CRM record, generating a citation packet, or triggering a workflow). The key is that the agent does not just get JSON—it gets JSON that passed validation gates, plus evidence when something fails.

18. Conclusion

Schema-first extraction changes the agent reliability story. Instead of hoping the model returns correct JSON, you enforce a typed contract with JSON Schema and validation gates. MCP-style orchestration then makes the workflow composable: the agent requests extraction, tools fetch and extract in the right state, validation decides pass/fail, and evidence packets make debugging fast.

If you implement only one thing from this post, implement the contract boundary: schema in, validated JSON out, with structured failure paths. That single change turns extraction from a best-effort guess into a system you can operate.

Next, pair this with your broader web intelligence pipeline: fetch the right state, extract with measurable quality signals, and convert into LLM-native artifacts.

19. References

• JSON Schema specification (draft 2020-12)
• Model Context Protocol (MCP) overview and tool orchestration concepts
• Practical web extraction patterns: scrape → extract → convert