- Fine-tuning
- Batch inference
What you'll build
Build a support-ticket classifier and compare it with its base model.
The script trains qwen3.5-9b on 240 labelled tickets and tests it on 60 more, written in wordings the training set does not contain. It runs the test on the fine-tuned model and on the base model in one batch, and prints how many each got right.
The tickets are synthetic, so the script runs as it is. The same steps fit any task with a short answer and examples of it: routing, tagging, moderation, yes-or-no checks.
How it works
What you need
- Comfort with
- Python and terminal basics
- Models
qwen3.5-9b(see Models)- Jobs and cost
- It creates one fine-tuning job and one batch. Fine-tuning and batch jobs are paid for from your prepaid credits.
- Time
- The script waits for its jobs to finish. A time estimate is not available.
You need a Gridline token and credits: the quickstart shows how to get both.
What a credit is worth, the credit packs and each model's rates are on the pricing page.
Run the complete recipe
Run these commands in your terminal. The download contains every step below, in order. Running it creates the paid jobs described above.
python -m pip install -U openai
export GRIDLINE_TOKEN="gla_..."
curl -O https://gridlinegpu.com/cookbooks/fine-tune-a-classifier/fine_tune_classifier.py
python fine_tune_classifier.pyThe whole script is one file: fine_tune_classifier.py. The rest of this page is that file, a step at a time.
To understand or change the script, follow the walkthrough below. Read each step in order; later steps use the files and variables from earlier ones.
Set up
The script starts with the OpenAI client, pointed at Gridline's base URL with your token as the key, and the names the steps below use.
"""Fine-tune a small model to route support tickets, then measure it against its base.
python -m pip install -U openai
export GRIDLINE_TOKEN="gla_..."
python fine_tune_classifier.py
It creates one fine-tuning job and one batch, both charged to your prepaid credits.
"""
import json
import os
import random
import time
from openai import OpenAI
client = OpenAI(
base_url="https://api.gridlinegpu.com/openai/v1",
api_key=os.environ["GRIDLINE_TOKEN"],
timeout=60,
)
BASE = "qwen3.5-9b"
SYSTEM = (
"Route the support ticket to one queue: billing, shipping, returns, account, "
"technical or sales. Answer with the queue name only."
)1. Make the data
Keep test tickets separate to check answers the model did not learn during training.
Each training row is one conversation in OpenAI's chat format: the instruction, the ticket, and the queue as the assistant's answer. A training file needs at least 10 rows. The script writes train.jsonl and test.jsonl; the test rows stay on your machine.
# Synthetic tickets, so the script runs as it is. Replace TICKETS with your own labelled rows.
TICKETS = {
"billing": [
"I was charged twice for my {plan} plan this month.",
"My invoice says {amount} but the {plan} plan costs less than that.",
"Please refund the {amount} you took on {day}. I cancelled before the renewal.",
"Why did the price of the {plan} plan go up without any notice?",
"I paid for order {order} and it never arrived. I want my money back.",
"I need a VAT invoice for the {amount} payment on {day}.",
"My card was declined, but {amount} still left my bank account.",
"You billed me for a full year. I chose monthly on the {plan} plan.",
],
"shipping": [
"Order {order} was due on {day} and it still has not arrived.",
"The tracking page for order {order} has not changed in a week.",
"Can you send order {order} to a different address? I moved.",
"My {product} shows as delivered but nothing is at my door.",
"How long does delivery of the {product} take to Canada?",
"The courier left a card for order {order}. How do I book a new delivery?",
"I paid for express delivery of my {product} and it came by standard post.",
"Only one of the two boxes for order {order} has turned up.",
],
"returns": [
"The {product} arrived with a cracked screen. I want to send it back.",
"How do I return the {product}? It is the wrong size.",
"I sent my {product} back on {day} and have heard nothing since.",
"You sent a {product} that I did not order. What do I do with it?",
"Can I swap my {product} for the newer model?",
"The {product} stopped working after two days. I want a replacement.",
"Where do I print the return label for order {order}?",
"Is the {product} still inside the 30 day return window? I bought it on {day}.",
],
"account": [
"I cannot sign in. The password reset email never comes.",
"Please change the email address on my account.",
"How do I turn on two-step sign-in?",
"I want to close my account and have my data deleted.",
"Somebody signed in to my account from another country on {day}.",
"I cannot sign in to pay my invoice. It says my account is locked.",
"How do I add a colleague to our {plan} account?",
"My name is spelled wrong on my profile and I cannot edit it.",
],
"technical": [
"The app closes every time I open the {product} settings page.",
"My {product} will not connect to wifi since the last update.",
"Exports from the dashboard fail with error 500.",
"The {product} battery runs out within an hour of a full charge.",
"Sync between my phone and my {product} has stopped working.",
"I get a blank white page after I sign in on Safari.",
"The API returns a timeout on every request since {day}.",
"The firmware update for my {product} stops at 80 percent.",
],
"sales": [
"Do you offer a discount for schools on the {plan} plan?",
"What is the difference between the {plan} plan and the one above it?",
"We need 200 units of the {product}. Can you send a quote?",
"Is there a free trial of the {plan} plan?",
"Does the {product} work with an iPhone before I buy one?",
"Can I pay for the {plan} plan by bank transfer and not by card?",
"When will the {product} be back in stock?",
"I would like a demo for my team before we choose a plan.",
],
}
FILLERS = {
"plan": ["Starter", "Team", "Pro", "Business"],
"product": ["headphones", "smart speaker", "fitness band", "tablet", "router", "camera"],
"amount": ["$19", "$49", "$120", "$245", "$980"],
"day": ["Monday", "3 March", "last Friday", "the 14th", "1 June"],
}
OPENERS = ["", "", "Hi, ", "Hello, ", "Good morning. "]
CLOSERS = ["", "", " Thanks.", " Please help.", " This is urgent."]
rng = random.Random(7)
def tickets(templates, count):
made = set()
while len(made) < count:
text = rng.choice(templates).format(
order=rng.randint(10000, 99999),
**{name: rng.choice(values) for name, values in FILLERS.items()},
)
made.add(rng.choice(OPENERS) + text + rng.choice(CLOSERS))
return sorted(made)
# The last two wordings of each queue never appear in training: the test is on unseen phrasings.
train, test = [], []
for queue, templates in TICKETS.items():
train += [(ticket, queue) for ticket in tickets(templates[:6], 40)]
test += [(ticket, queue) for ticket in tickets(templates[6:], 10)]
rng.shuffle(train)
def chat_row(ticket, queue):
return {
"messages": [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": ticket},
{"role": "assistant", "content": queue},
]
}
for name, part in (("train.jsonl", train), ("test.jsonl", test)):
with open(name, "w") as out:
for ticket, queue in part:
out.write(json.dumps(chat_row(ticket, queue)) + "\n")
print(f"{len(train)} training rows, {len(test)} held out for the test")What you should see The script writes train.jsonl and test.jsonl, then prints the training and held-out row counts.
2. Start the fine-tune
Upload your examples to start training and see the job's maximum charge.
Upload the file with purpose="fine-tune" and create the job. Creating a job starts it, and the response carries its quote: max_charge_usd is the most the job can cost. n_epochs is how many times the job reads the training file; the default is 1.
# Creating the job starts it. The response carries the quote: the most the job can cost.
training = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune")
job = client.fine_tuning.jobs.create(
model=BASE,
training_file=training.id,
hyperparameters={"n_epochs": 3},
)
print(job.id, "costs at most $", job.gridline["quote"]["per_token"]["max_charge_usd"])What you should see A job id and its maximum charge print when the job is created.
3. Wait for the model
Wait for training to finish before using your new model.
Poll until the job ends. A job that succeeds reports its eval loss before and after training in gridline.result, measured on training rows it held out. Lower after than before means the model fits your examples better. fine_tuned_model is your model's id.
while job.status not in ("succeeded", "failed", "cancelled"):
time.sleep(30)
job = client.fine_tuning.jobs.retrieve(job.id)
if job.status != "succeeded":
raise SystemExit(f"{job.id} ended {job.status}: {job.error}")
result = job.gridline["result"]
print("eval loss", result["eval_loss_before"], "->", result["eval_loss"])
print("your model:", job.fine_tuned_model)What you should see On success, the script prints eval loss before and after training, then your ft: model id.
4. Run the held-out tickets on both models
Give both models the same held-out tickets so their answers can be compared.
A batch can mix a base model with fine-tunes of it, so both run in one batch. custom_id records which model and which ticket each answer belongs to.
# One batch can mix a base model with fine-tunes of it, so both run on the same GPU.
MODELS = {"base": BASE, "tuned": job.fine_tuned_model}
with open("requests.jsonl", "w") as out:
for arm, model in MODELS.items():
for n, (ticket, _) in enumerate(test):
body = {
"model": model,
"messages": chat_row(ticket, "")["messages"][:2],
"max_tokens": 8,
"temperature": 0,
}
line = {
"custom_id": f"{arm}-{n}",
"method": "POST",
"url": "/v1/chat/completions",
"body": body,
}
out.write(json.dumps(line) + "\n")
batch_input = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=batch_input.id,
endpoint="/v1/chat/completions",
completion_window="24h",
)
while batch.status not in ("completed", "failed", "expired", "cancelled"):
time.sleep(30)
batch = client.batches.retrieve(batch.id)
if batch.status != "completed":
raise SystemExit(f"{batch.id} ended {batch.status}: {batch.errors}")What you should see This step waits for the batch without printing; scoring starts after it completes.
5. Score both
Count exact queue matches to see how each model did on the test set.
Results come back in any order, one line per request. The script matches them by custom_id and counts the answers that are exactly the expected queue.
answers = {}
for line in client.files.content(batch.output_file_id).text.splitlines():
row = json.loads(line)
answers[row["custom_id"]] = row["response"]["body"]["choices"][0]["message"]["content"]
for arm in MODELS:
right = sum(
answers.get(f"{arm}-{n}", "").strip().lower() == queue
for n, (_, queue) in enumerate(test)
)
print(f"{arm}: {right} of {len(test)} right")What you should see A base line and a tuned line show how many tickets each routed correctly.
Next steps
Start by changing the sample data, then use these guides to go further.
- Use your own rows: replace
TICKETSwith your labelled examples, and keep some of them out of training for the test. - Try another base: change
BASE. Fine-tuning lists the bases and the settings you can change. - Grade the model with the Evals API, not your own loop: Evaluate a fine-tune.
- The model is yours. Your ft: model shows how to download it.