Skip to content
Cookbook

Extract structured data in bulk

Run a batch that turns free text into JSON that fits your schema, then read the answers and the errors.

All cookbooks
  • Structured outputs
  • Batch inference

What you'll build

Turn order emails into JSON records and read any request errors.

The script sends six order emails to gemma4-26b-a4b-it, Google's Gemma 4, with a JSON schema as the response_format. The schema is enforced while the model writes, so each finished answer is JSON with the fields you asked for. It then reads the output file and the error file.

The same pattern fits invoices, contracts, reviews and logs: any text you want as rows.

How it works

Emails and a JSON schema go into a batch. Read the finished JSON records and check the error file for failed requests.

What you need

Comfort with
Python, JSON and terminal basics
Models
gemma4-26b-a4b-it (see Models)
Jobs and cost
It creates one batch. Fine-tuning and batch jobs are paid for from your prepaid credits.
Time
The script waits for its jobs to finish. A time estimate is not available.

You need a Gridline token and credits: the quickstart shows how to get both.

What a credit is worth, the credit packs and each model's rates are on the pricing page.

Run the complete recipe

Run these commands in your terminal. The download contains every step below, in order. Running it creates the paid jobs described above.

python -m pip install -U openai
export GRIDLINE_TOKEN="gla_..."
curl -O https://gridlinegpu.com/cookbooks/structured-extraction/extract_structured.py
python extract_structured.py

The whole script is one file: extract_structured.py. The rest of this page is that file, a step at a time.

To understand or change the script, follow the walkthrough below. Read each step in order; later steps use the files and variables from earlier ones.

Set up

The script starts with the OpenAI client, pointed at Gridline's base URL with your token as the key, and the names the steps below use.

"""Pull structured data out of free text in bulk: every answer is JSON that fits your schema.

  python -m pip install -U openai
  export GRIDLINE_TOKEN="gla_..."
  python extract_structured.py

It creates one batch, charged to your prepaid credits.
"""

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gridlinegpu.com/openai/v1",
    api_key=os.environ["GRIDLINE_TOKEN"],
    timeout=60,
)

MODEL = "gemma4-26b-a4b-it"

1. Describe the answer you want

Describe the fields you need before asking the model to extract them.

The answer's shape is a json_schema response format. Batches take a subset of JSON Schema: types, properties, required, additionalProperties, items, enum, const, anyOf, and counts such as maxItems and maxLength. pattern, format and numeric bounds are refused.

# The schema is enforced while the model writes, so a finished answer always parses.

ORDER = {
    "type": "json_schema",
    "json_schema": {
        "name": "order_email",
        "schema": {
            "type": "object",
            "properties": {
                "customer": {"type": "string"},
                "order_id": {"type": "string"},
                "items": {
                    "type": "array",
                    "maxItems": 10,
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {"type": "string"},
                            "quantity": {"type": "integer"},
                        },
                        "required": ["name", "quantity"],
                        "additionalProperties": False,
                    },
                },
                "wants": {
                    "type": "string",
                    "enum": ["refund", "replacement", "update", "other"],
                },
            },
            "required": ["customer", "order_id", "items", "wants"],
            "additionalProperties": False,
        },
    },
}
SYSTEM = (
    "Extract the order from the email as JSON: the customer's name, the order id, each item "
    "with its quantity, and what the customer wants (refund, replacement, update or other)."
)

What you should see ORDER holds the answer schema in memory; this step prints nothing.

2. Write one request per document

Give each email its own request id so you can match answers back to documents.

One line per document. custom_id is yours and must be unique in the file. Set max_tokens on every structured request.

# Replace EMAILS with your own text. custom_id is how you match answers back to documents.

EMAILS = [
    "Hi, this is Dana Whitfield. Order A-10492 had two desk lamps and one extension lead. "
    "One lamp came broken, so please send a new one.",
    "Order B-77310: three packs of printer paper. They were never delivered and I would like "
    "my money back. Regards, Tom Okafor",
    "Hello. I ordered a standing desk and two monitor arms last week (C-20018). When will "
    "they ship? Priya Raman",
    "From: Luis Ortega. My order D-55120 was 12 notebooks and 4 boxes of pens. The pens are "
    "the wrong colour. Can you swap them?",
    "I am Mei Tanaka and my order number is E-31007. One office chair. The gas lift fails "
    "and the seat sinks. I want a refund.",
    "Sam Keller here, order F-88421, five whiteboard markers and one eraser. Thank you, "
    "everything arrived fine.",
]

with open("requests.jsonl", "w") as out:
    for n, email in enumerate(EMAILS):
        body = {
            "model": MODEL,
            "messages": [
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": email},
            ],
            "max_tokens": 300,
            "temperature": 0,
            "response_format": ORDER,
        }
        line = {
            "custom_id": f"email-{n}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": body,
        }
        out.write(json.dumps(line) + "\n")

What you should see The script writes requests.jsonl; no batch has started yet.

3. Run the batch

Send the request file as a batch and wait for its results.

Upload the file with purpose="batch", create the batch and poll until it ends.

batch_input = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_input.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(batch.id, batch.status)
while batch.status not in ("completed", "failed", "expired", "cancelled"):
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
if batch.status != "completed":
    raise SystemExit(f"{batch.id} ended {batch.status}: {batch.errors}")

What you should see The batch id and initial status print, then the script waits for completion.

4. Read the answers

Check that an answer finished before parsing it as JSON.

Results come back in any order. An answer that stops at max_tokens has finish_reason length; it is returned and billed, and its JSON may be cut short, so check before you parse.

# Results come back in any order. An answer cut off at max_tokens may be half a JSON object,
# so check finish_reason before you parse.

orders = {}
if batch.output_file_id:
    for line in client.files.content(batch.output_file_id).text.splitlines():
        row = json.loads(line)
        choice = row["response"]["body"]["choices"][0]
        if choice["finish_reason"] == "stop":
            orders[row["custom_id"]] = json.loads(choice["message"]["content"])
        else:
            print(row["custom_id"], "was cut off: raise max_tokens and send it again")

for n in range(len(EMAILS)):
    print(f"email-{n}", orders.get(f"email-{n}"))

What you should see Each email id prints with its parsed record, or None if no complete record was read; cut-off answers get a message.

5. Read the errors

Read individual failures even when the rest of the batch completed.

A request that fails on its own goes to the error file with a code, such as context_length_exceeded or structured_output_invalid. The rest of the batch still completes.

# A request that failed on its own is in the error file. The rest of the batch still completed.

if batch.error_file_id:
    for line in client.files.content(batch.error_file_id).text.splitlines():
        row = json.loads(line)
        print(row["custom_id"], "failed:", row["error"]["code"], row["error"]["message"])

What you should see Failed requests print their id, error code and message; with no error file, nothing more prints.

Next steps

Start by changing the sample data, then use these guides to go further.

More recipes

Contact us

Whether you have a question about fine-tuning or batch, need help with a job, or want to share feedback, send us a note. We’ll make sure it reaches the right person.

New messageGridline / contact
0/5,000