Cookbook

# Extract structured data in bulk

Run a batch that turns free text into JSON that fits your schema, then read the answers and the errors.

[All cookbooks](https://gridlinegpu.com/docs/cookbooks)

- Structured outputs
- Batch inference

## What you'll build

Turn order emails into JSON records and read any request errors.

The script sends six order emails to `gemma4-26b-a4b-it`, Google's Gemma 4, with a JSON schema as the `response_format`. The schema is enforced while the model writes, so each finished answer is JSON with the fields you asked for. It then reads the output file and the error file.

The same pattern fits invoices, contracts, reviews and logs: any text you want as rows.

### How it works

1. Emails and schema
2. Extraction batch
3. JSON and errors

Emails and a JSON schema go into a batch. Read the finished JSON records and check the error file for failed requests.

## What you need

Comfort with

Python, JSON and terminal basics

Models

`gemma4-26b-a4b-it` (see [Models](https://gridlinegpu.com/docs/models))

Jobs and cost

It creates one batch. Fine-tuning and batch jobs are paid for from your prepaid credits.

Time

The script waits for its jobs to finish. A time estimate is not available.

You need a Gridline token and credits: the [quickstart](https://gridlinegpu.com/docs#access) shows how to get both.

What a credit is worth, the credit packs and each model's rates are on the [pricing page](https://gridlinegpu.com/pricing).

## Run the complete recipe

Run these commands in your terminal. The download contains every step below, in order. Running it creates the paid jobs described above.

```
python -m pip install -U openai
export GRIDLINE_TOKEN="gla_..."
curl -O https://gridlinegpu.com/cookbooks/structured-extraction/extract_structured.py
python extract_structured.py
```

The whole script is one file: [`extract_structured.py`](https://gridlinegpu.com/cookbooks/structured-extraction/extract_structured.py). The rest of this page is that file, a step at a time.

To understand or change the script, follow the walkthrough below. Read each step in order; later steps use the files and variables from earlier ones.

## Set up

The script starts with the OpenAI client, pointed at Gridline's base URL with your token as the key, and the names the steps below use.

```
"""Pull structured data out of free text in bulk: every answer is JSON that fits your schema.

  python -m pip install -U openai
  export GRIDLINE_TOKEN="gla_..."
  python extract_structured.py

It creates one batch, charged to your prepaid credits.
"""

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gridlinegpu.com/openai/v1",
    api_key=os.environ["GRIDLINE_TOKEN"],
    timeout=60,
)

MODEL = "gemma4-26b-a4b-it"
```

## 1. Describe the answer you want

Describe the fields you need before asking the model to extract them.

The answer's shape is a `json_schema` response format. Batches take a subset of JSON Schema: types, `properties`, `required`, `additionalProperties`, `items`, `enum`, `const`, `anyOf`, and counts such as `maxItems` and `maxLength`. `pattern`, `format` and numeric bounds are refused.

```
# The schema is enforced while the model writes, so a finished answer always parses.

ORDER = {
    "type": "json_schema",
    "json_schema": {
        "name": "order_email",
        "schema": {
            "type": "object",
            "properties": {
                "customer": {"type": "string"},
                "order_id": {"type": "string"},
                "items": {
                    "type": "array",
                    "maxItems": 10,
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {"type": "string"},
                            "quantity": {"type": "integer"},
                        },
                        "required": ["name", "quantity"],
                        "additionalProperties": False,
                    },
                },
                "wants": {
                    "type": "string",
                    "enum": ["refund", "replacement", "update", "other"],
                },
            },
            "required": ["customer", "order_id", "items", "wants"],
            "additionalProperties": False,
        },
    },
}
SYSTEM = (
    "Extract the order from the email as JSON: the customer's name, the order id, each item "
    "with its quantity, and what the customer wants (refund, replacement, update or other)."
)
```

**What you should see** `ORDER` holds the answer schema in memory; this step prints nothing.

## 2. Write one request per document

Give each email its own request id so you can match answers back to documents.

One line per document. `custom_id` is yours and must be unique in the file. Set `max_tokens` on every structured request.

```
# Replace EMAILS with your own text. custom_id is how you match answers back to documents.

EMAILS = [
    "Hi, this is Dana Whitfield. Order A-10492 had two desk lamps and one extension lead. "
    "One lamp came broken, so please send a new one.",
    "Order B-77310: three packs of printer paper. They were never delivered and I would like "
    "my money back. Regards, Tom Okafor",
    "Hello. I ordered a standing desk and two monitor arms last week (C-20018). When will "
    "they ship? Priya Raman",
    "From: Luis Ortega. My order D-55120 was 12 notebooks and 4 boxes of pens. The pens are "
    "the wrong colour. Can you swap them?",
    "I am Mei Tanaka and my order number is E-31007. One office chair. The gas lift fails "
    "and the seat sinks. I want a refund.",
    "Sam Keller here, order F-88421, five whiteboard markers and one eraser. Thank you, "
    "everything arrived fine.",
]

with open("requests.jsonl", "w") as out:
    for n, email in enumerate(EMAILS):
        body = {
            "model": MODEL,
            "messages": [
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": email},
            ],
            "max_tokens": 300,
            "temperature": 0,
            "response_format": ORDER,
        }
        line = {
            "custom_id": f"email-{n}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": body,
        }
        out.write(json.dumps(line) + "\n")
```

**What you should see** The script writes `requests.jsonl`; no batch has started yet.

## 3. Run the batch

Send the request file as a batch and wait for its results.

Upload the file with `purpose="batch"`, create the batch and poll until it ends.

```
batch_input = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_input.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(batch.id, batch.status)
while batch.status not in ("completed", "failed", "expired", "cancelled"):
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
if batch.status != "completed":
    raise SystemExit(f"{batch.id} ended {batch.status}: {batch.errors}")
```

**What you should see** The batch id and initial status print, then the script waits for completion.

## 4. Read the answers

Check that an answer finished before parsing it as JSON.

Results come back in any order. An answer that stops at `max_tokens` has `finish_reason` `length`; it is returned and billed, and its JSON may be cut short, so check before you parse.

```
# Results come back in any order. An answer cut off at max_tokens may be half a JSON object,
# so check finish_reason before you parse.

orders = {}
if batch.output_file_id:
    for line in client.files.content(batch.output_file_id).text.splitlines():
        row = json.loads(line)
        choice = row["response"]["body"]["choices"][0]
        if choice["finish_reason"] == "stop":
            orders[row["custom_id"]] = json.loads(choice["message"]["content"])
        else:
            print(row["custom_id"], "was cut off: raise max_tokens and send it again")

for n in range(len(EMAILS)):
    print(f"email-{n}", orders.get(f"email-{n}"))
```

**What you should see** Each email id prints with its parsed record, or None if no complete record was read; cut-off answers get a message.

## 5. Read the errors

Read individual failures even when the rest of the batch completed.

A request that fails on its own goes to the error file with a code, such as `context_length_exceeded` or `structured_output_invalid`. The rest of the batch still completes.

```
# A request that failed on its own is in the error file. The rest of the batch still completed.

if batch.error_file_id:
    for line in client.files.content(batch.error_file_id).text.splitlines():
        row = json.loads(line)
        print(row["custom_id"], "failed:", row["error"]["code"], row["error"]["message"])
```

**What you should see** Failed requests print their id, error code and message; with no error file, nothing more prints.

## Next steps

Start by changing the sample data, then use these guides to go further.

- Use your own documents: replace `EMAILS`, the instruction and the schema.
- [Structured outputs](https://gridlinegpu.com/docs/batch-api#structured) has the rules for schemas, and [Models](https://gridlinegpu.com/docs/models) says which models take them.
- Train a small model to write this JSON for less: [Distill a large model into a small one](https://gridlinegpu.com/docs/cookbooks/distill-a-large-model).

### More recipes

---

Source: https://gridlinegpu.com/docs/cookbooks/structured-extraction
