Skip to main content
POST /v1/web/extract turns pages into JSON in the shape you choose. Describe the shape with a JSON Schema, a plain-language prompt, or both, and an LLM reads each page into it. Point it at one URL, a list, or every page under a path. Use it when you want specific facts, not the whole page: a product’s price and stock, a company’s funding round, every listing in a directory.

When to use it

  • You need typed fields, not prose. Give a schema and get exactly those keys back.
  • You want the same fields from many pages, or from every page under a path.
  • You need to check the result: each field can come back with a confidence and the passage it came from.
  • You want many pages combined into one list without duplicates, one row per real thing.

Example

Get a product’s details from one page, with confidence and sources on.
There is one entry in results per page. With showConfidence, each has a fields map giving, for every field, how sure we are and the passage it came from. With showSources, the pages that were actually read are listed in sources.

Every page under a path

A URL ending in /* means every page we find under that path, not a literal address. Each one is read and its results are added. You can mix plain URLs and paths in the same urls list.
One call reads at most 20 pages after paths are expanded, so it costs at most 100 credits. Use showSources to see which pages were read. For more pages, run several calls or crawl first and extract from the pages you pick.

Options

string[]
required
The pages to read, up to 10 entries. Each must be http or https. A trailing /* means every page under that path.
object
A JSON Schema for the shape you want back. Optional if you give a prompt. With both, the schema fixes the field names and types and the prompt says what to look for.
string
What to extract, in plain language, up to 2000 characters. Use it with a schema or on its own.
Search the web with your prompt and read up to 5 more pages, to fill fields your URLs do not cover. Needs a prompt.
boolean
default:"false"
Return the list of pages that were actually read, after paths and web search are expanded.
boolean
default:"false"
For each field, return a confidence from 0 to 1 and the passage the value came from.
boolean
default:"false"
Combine the results from every page into one list without duplicates, one row per thing, each with the pages it came from. Without it you get a separate result per page.
boolean
default:"false"
Keep headings, lists and tables when reading the page. Good for listing and catalogue pages.
number
Use a recent copy of each page if it is younger than this many milliseconds. Leave it out or set 0 to always read fresh. At most 7 days.
object
Read each page as if from another place: country is "US" or "DE", languages lists preferred languages in order. For sites that change prices or content by region. Any other country is refused with a 400 before any page is read.
object
Extra headers sent to the site for each page, for example the cookie a shop uses to pick your store, so prices and stock match what you see.

Response fields

object[]
One result per page, each with url, data (what was extracted, or null when nothing matched) and error (null on success). Includes fields when showConfidence is on.
object
For each field name, its confidence (0 to 1) and evidence (the passage it came from). Only with showConfidence.
string[]
The pages actually read. Only with showSources.
object[]
The combined list, one row per thing. Each row has data (all its fields from every page) and sources (every page that added to it). Only with mergeEntities.

Cost

5 credits per page that returns data. Pages that give nothing are free. A /* path or enableWebSearch can add pages, so the total depends on how many produce results, up to 20 pages per call. See Credits.

Search

Find the pages to extract from.

Crawl

Read a whole site, then extract from the pages you need.

Formats

The structured format, for data a page already publishes.

API reference

The full schema, with a live playground.