- Embeddings
- Batch inference
What you'll build
Find passages closest in meaning to your questions using text vectors.
The script embeds 16 help-centre passages and three questions with qwen3-embedding-0.6b in one batch, then ranks the passages for each question by cosine similarity.
Gridline embeds in batches, so this fits work done ahead of time: indexing a corpus, clustering, finding duplicates, matching records. To embed questions as they arrive, run the same model where they arrive.
How it works
What you need
- Comfort with
- Python and terminal basics
- Models
qwen3-embedding-0.6b(see Models)- Jobs and cost
- It creates one batch. Fine-tuning and batch jobs are paid for from your prepaid credits.
- Time
- The script waits for its jobs to finish. A time estimate is not available.
You need a Gridline token and credits: the quickstart shows how to get both.
What a credit is worth, the credit packs and each model's rates are on the pricing page.
Run the complete recipe
Run these commands in your terminal. The download contains every step below, in order. Running it creates the paid jobs described above.
python -m pip install -U openai
export GRIDLINE_TOKEN="gla_..."
curl -O https://gridlinegpu.com/cookbooks/embeddings-search/embed_search.py
python embed_search.pyThe whole script is one file: embed_search.py. The rest of this page is that file, a step at a time.
To understand or change the script, follow the walkthrough below. Read each step in order; later steps use the files and variables from earlier ones.
Set up
The script starts with the OpenAI client, pointed at Gridline's base URL with your token as the key, and the names the steps below use.
"""Embed a set of passages in one batch, then search them by meaning.
python -m pip install -U openai
export GRIDLINE_TOKEN="gla_..."
python embed_search.py
It creates one batch, charged to your prepaid credits.
"""
import json
import math
import os
import time
from openai import OpenAI
client = OpenAI(
base_url="https://api.gridlinegpu.com/openai/v1",
api_key=os.environ["GRIDLINE_TOKEN"],
timeout=60,
)
MODEL = "qwen3-embedding-0.6b"
# Qwen3-Embedding embeds exactly the text you send: questions get an instruction in front,
# in the form its model card gives, and passages go bare.
TASK = "Given a customer question, retrieve the help article that answers it"1. Gather the text
Gather passages and questions together for this search example.
qwen3-embedding-0.6b takes inputs of up to 8,192 tokens and returns 1,024 numbers for each, normalized. Split longer documents into passages first.
# Replace PASSAGES with your own: help articles, product descriptions, tickets, notes.
PASSAGES = [
"To reset your password, choose Forgot password on the sign-in page and follow the email.",
"Invoices are issued on the first day of each month and sent to the billing contact.",
"You can change your plan at any time. The new price applies from the next billing day.",
"Standard delivery takes three to five working days. Express delivery takes one.",
"Returns are free within 30 days. Print a label from the Orders page.",
"Two-step sign-in can be turned on under Settings, Security.",
"The mobile app needs iOS 16 or Android 12, or later.",
"Refunds go back to the original payment method within five working days.",
"Team plans include shared folders and up to 50 members.",
"To export your data, open Settings, Privacy, and choose Download my data.",
"Our support team answers email between 9:00 and 18:00 UTC on weekdays.",
"A damaged item can be replaced or refunded. Send a photo with your order number.",
"You can pay by card or, on annual plans, by bank transfer.",
"The API allows 600 requests a minute for each key.",
"Closing your account deletes your data after 30 days.",
"Orders can be sent to a new address until they leave the warehouse.",
]
QUESTIONS = [
"how do I get my money back",
"can I log in with a second factor",
"what phones does the app run on",
]What you should see PASSAGES and QUESTIONS hold the sample text; this step prints nothing.
2. Write the requests
Format questions for the embedding model while keeping passage text unchanged.
The model embeds exactly the text you send, so each question gets an instruction in front, in the form its model card gives, and each passage goes bare. A line's input is one string or a list of up to 2,048. Here one line carries the passages and another the questions; a large corpus takes many lines, up to 50,000 in a file.
# One line can carry up to 2,048 inputs, and one file up to 50,000 lines.
queries = [f"Instruct: {TASK}\nQuery:{question}" for question in QUESTIONS]
with open("requests.jsonl", "w") as out:
for custom_id, texts in (("passages", PASSAGES), ("questions", queries)):
line = {
"custom_id": custom_id,
"method": "POST",
"url": "/v1/embeddings",
"body": {"model": MODEL, "input": texts},
}
out.write(json.dumps(line) + "\n")What you should see requests.jsonl contains the passage and question requests; no batch has started yet.
3. Run the batch
Embed the text before comparing its meaning locally.
An embeddings batch names /v1/embeddings as its endpoint. It is charged for input tokens only.
batch_input = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=batch_input.id,
endpoint="/v1/embeddings",
completion_window="24h",
)
print(batch.id, batch.status)
while batch.status not in ("completed", "failed", "expired", "cancelled"):
time.sleep(30)
batch = client.batches.retrieve(batch.id)
if batch.status != "completed":
raise SystemExit(f"{batch.id} ended {batch.status}: {batch.errors}")What you should see The batch id and initial status print, then the script waits for completion.
4. Read the vectors
Match returned vectors to the text that produced them.
Each result's data holds one embedding per input, with the input's position in index.
# Each result holds one embedding per input, with the input's position in "index".
vectors = {}
for line in client.files.content(batch.output_file_id).text.splitlines():
row = json.loads(line)
data = sorted(row["response"]["body"]["data"], key=lambda item: item["index"])
vectors[row["custom_id"]] = [item["embedding"] for item in data]
print(len(vectors["passages"]), "passages,", len(vectors["passages"][0]), "dimensions each")What you should see The script prints the passage count and the number of dimensions per vector.
5. Search
Rank passages by cosine similarity to each question's vector.
The script compares vectors in plain Python. Past a few thousand passages, use numpy or a vector database.
# Cosine similarity in plain Python. Past a few thousand passages, use numpy or a vector store.
def cosine(a, b):
dot = sum(x * y for x, y in zip(a, b))
return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))
for question, vector in zip(QUESTIONS, vectors["questions"]):
scores = [cosine(vector, passage) for passage in vectors["passages"]]
best = sorted(range(len(PASSAGES)), key=lambda n: scores[n], reverse=True)[:2]
print(question)
for n in best:
print(f" {scores[n]:.3f} {PASSAGES[n]}")What you should see Each question prints with its closest passages and their cosine scores.
Next steps
Start by changing the sample data, then use these guides to go further.
- Use your own text: replace
PASSAGESandQUESTIONS. - Try
bge-m3: changeMODEL, and send the questions without the instruction. See Models.