Advanced

# Run your own script

Send your own Python code to one GPU as a finite job, and get your declared output files back, verified.

## When to use this

For LoRA or QLoRA on a supported base, and batch inference on the result, use the [fine-tuning and batch APIs](https://gridlinegpu.com/docs): Gridline runs the recipe and your model gets an `ft:` id. Run your own script when you need code, a model or a recipe those APIs do not cover. This path is unchanged and is not getting new features. An adapter it trains, including through the SDK's `gl.finetune.lora()`, comes back as files you download; it does not become an `ft:` model.

01 · Choose a path

## How do you want to start?

Let a coding agent set up the SDK, or start directly below. Every path submits the same `gridline.yaml`.

Let a coding agent set up the SDK for your project. It stops before a paid run.

```
Help me run my own Python code on a Gridline GPU as finite source runs: batch inference over my data, and a LoRA fine-tune. Inspect this repository first. Take me from zero to finished jobs with verified outputs on the self-serve path, and stop for my approval before every paid step.

Ask only for what the repo cannot establish: the model and its pinned commit, the input files, the outputs I need, and which code should run on the GPU. Do not guess them.

1. Access. I need a Gridline account with source access. If I have none: sign up at https://gridlinegpu.com/signup, verify the email, then add credits by card at https://gridlinegpu.com/get-started (I agree to the Gridline GPU job terms and pay myself). The first purchase opens the account at once, and its credits work on any job. Never try to work around access or credit checks.
2. Install in a Python 3.10+ environment: python -m pip install 'gridlinegpu[source-contract]>=0.3.2,<0.4'. Guide me through gridline setup --apply (browser approval; it saves a private, scoped token file). Never ask for, print, paste or commit the token or its file contents, and never put a token in YAML, code, logs or chat. Then run gridline access and continue only when it says approved. Check credit with GridlineClient().credits() (availableUsdMicros is in millionths of a dollar).
3. Write gridline.yaml (version: 2) beside the code. Use a curated runtime instead of dependency files: runtime: gridline/hf-torch-2.5@r3 for inference (CPython 3.11, CUDA 12.4, torch 2.5.1, transformers 4.46.3, accelerate, safetensors, tokenizers, datasets, pandas, pyarrow, numpy 1.26.4), or runtime: {curated: gridline/hf-finetune-2.5@r4} for LoRA or QLoRA fine-tuning (torch 2.5.1, transformers 4.46.3, peft 0.13.2, trl 0.12.1, bitsandbytes 0.44.1, plus the inference set and evaluate). Do not add python, requirements or a lockfile next to a curated runtime. Leave out budget and execution.deadline_seconds; routing refuses both.
4. Check every limit before submitting:
- Exactly one GPU (resources.gpu.count: 1).
- No network at run time: no pip, Hugging Face Hub, HTTP or localhost. Declare each model under models: with source hf://owner/repo, its full 40-character revision and its files list; the platform fetches them into models/<name> before the code starts. Load them with local_files_only=True.
- Inputs: each input file at most 512 MiB.
- Outputs: declare every file under outputs: with a path under outputs/ and its max_bytes. At most 128 MiB per file, 1 GiB per job and 32 outputs. Keep every output I need required (the default): an optional output may be left out of the results today.
- Memory: on the hardened isolation tier each process has an 8 GiB data-memory cap, and an allocation past it fails inside the code (MemoryError). Load weights straight to the GPU and stream data rather than holding it all in CPU memory.
5. Batch inference: a job script that reads the inputs, runs the model on the GPU in batches, and writes results under outputs/ as it goes. The gridlinegpu README (https://pypi.org/project/gridlinegpu/) has a complete first job and the fine-tune YAML. Fine-tune: declare the base model under models: with its pinned Hugging Face commit revision, load it straight to the GPU, train a LoRA (or QLoRA) adapter, and write metrics.json plus the adapter's adapter_model.safetensors and adapter_config.json as required outputs. Full fine-tunes that return merged model weights are not yet offered.
6. Cost. While a job runs, part of my credit is held for it. Minutes after the machine is removed, the job is charged the actual GPU cost it used and the rest is released. A machine that started is charged for its time whatever the outcome. Show me the YAML, the code, the expected run time and my available credit, then wait for my explicit yes before submitting. Stop before client.run() for my approval.
7. Submit once: job = GridlineClient().run("gridline.yaml"). If it raises SubmissionOutcomeUnknown, call client.resume(error.submission_id) and never run() the same job again. Then result = job.wait() (a local timeout never cancels the job), job.outputs("results") (length and SHA-256 checked) and job.report() for the settlement. On failure, read result.failure_code and its corrective action, and fix the cause before a new run. partner_not_admitted means access is not approved yet (back to step 1); credit_insufficient means too little available credit. Finish by showing me the changed files, the job ids, the outputs and what each job cost.
```

Submit a gridline.yaml from your code once your account has credits.

```
from gridlinegpu import GridlineClient, SubmissionOutcomeUnknown

client = GridlineClient()
try:
    job = client.run("gridline.yaml")
except SubmissionOutcomeUnknown as error:
    # Never run() again for the same submission.
    job = client.resume(error.submission_id)

# Polls status; a local timeout never cancels the job.
result = job.wait()
print(result.outcome, result.failure_code)
if result.succeeded:
    # Verified downloads under results/
    job.outputs("results")
print(job.report()["settlement"])
```

Set up once, then download and run the first example.

```
python -m pip install \
  'gridlinegpu[source-contract]>=0.3.2,<0.4'
gridline setup --apply
gridline access

# The batch inference example
mkdir first-job && cd first-job
for f in README.md gridline.yaml job.py \
         prompts.txt submit.py; do
  curl -fsSLO \
    https://gridlinegpu.com/examples/curated-beta/$f
done
python submit.py
```

02 · Get access and set up

## Sign up, add credits, install once

1. [Sign up](https://gridlinegpu.com/signup) and verify your email.
2. Add credits by card at [Get started](https://gridlinegpu.com/get-started) to start right away; your credits work on any job.
3. Install [gridlinegpu](https://pypi.org/project/gridlinegpu/) with the source extra (Python 3.10+), create your token once, and check your access:

```
python -m pip install 'gridlinegpu[source-contract]>=0.3.2,<0.4'
gridline setup --apply
gridline access
```

03 · One workload file

## Describe the job in gridline.yaml

Name a curated runtime, the source files to send, the command, each output with its size bound, and one GPU. Start from a complete example: batch inference on `gridline/hf-torch-2.5@r3`, or LoRA and QLoRA fine-tuning on `gridline/hf-finetune-2.5@r4`.

[Download gridline.yaml](https://gridlinegpu.com/examples/curated-beta/gridline.yaml)

04 · Costs

## Jobs draw on your credits

While a job runs, part of your credits is held for it; minutes after the machine is torn down, the hold settles at the provider's cost for the machine time plus 25%, and the rest is released. `job.report()` shows what was held, charged and released. See the [package guide](https://pypi.org/project/gridlinegpu/) for limits, failure codes and cancellation.

05 · Source runs

## Source runs

Source runs (YAML `version: 2` through `GridlineClient.run`) send your own Python code and inputs to one GPU as a finite job and return your declared output files. They are funded by credits you add by card.

Your first card purchase at [Get started](https://gridlinegpu.com/get-started) opens your account at once; your credits then work on any job with no further approval. When they run out, add more from the dashboard by card or USDC, or contact us. Until then, `run` is refused with `partner_not_admitted` before anything is submitted.

### Curated runtime

Name a curated runtime instead of declaring dependencies. Gridline offers `gridline/hf-torch-2.5@r3`: CPython 3.11 and CUDA 12.4 with PyTorch 2.5.1 and pinned Hugging Face libraries. For LoRA and QLoRA fine-tuning, use `gridline/hf-finetune-2.5@r4` (see [Fine-tuning](https://gridlinegpu.com/docs/own-script#fine-tuning)). The SDK README lists every pinned package.

### Limits

- Inputs: each local input file is at most 512 MiB.
- Outputs: each file is at most 128 MiB (8 MiB when `max_bytes` is omitted), at most 1 GiB in total, and at most 32 outputs per job.
- One GPU per job, and no network at run time (below).
- Outputs are required unless you mark them `required: false`. For now, an optional output can be left out of the results even when your code wrote it; the job still succeeds. Keep every output you need required.

### No network for your code

Source runs have no network access, only Unix domain sockets. No TCP or UDP, localhost included. So these fail: local HTTP servers (FastAPI, TGI), Ray, and any pip or Hugging Face download at run time. vLLM (offline `LLM()` included) and NCCL or torch.distributed (whatever the init method) are expected to fail. The error is typically `PermissionError: [Errno 1] Operation not permitted`, or a library's own connection or offline error. Declare models and inputs: they are fetched by the platform, not by your code, and staged before your code starts.

If the machine cannot enforce this isolation, your code does not start and the job ends with `isolation_unsupported`: “The allocated machine cannot enforce the workload's required isolation.” The job's credit hold is released and you are not charged for it. Submit a new run; contact support if it repeats.

### Isolation tier

Your code runs on one GPU in a single-tenant container, under one of two isolation tiers depending on the machine the provider allocates. Where the machine supports it, your code runs in its own kernel control group (`kernel_cgroup`), which bounds its total memory and processes. On other machines it runs as a separate, unprivileged user with per-process limits of 8 GiB of data memory and 4 GiB per written file, a memory check about once per second, and a process limit of at least 512 (`hardened_user_separation`). The tier your run used is shown in its status. Memory under `hardened_user_separation` is sampled, not kernel-bounded; a run that exceeds it ends with `workload_out_of_memory`: “The workload ran out of memory.” A single process that reaches its 8 GiB data-memory limit is not stopped: its allocation fails inside your code (a Python `MemoryError`), and the run usually ends with `workload_runtime_failed`. See [Security](https://gridlinegpu.com/security#source-run-isolation-tiers) for the full list of limits. If your account needs `kernel_cgroup` only, contact support; runs then start only on machines that support it, which can take longer.

### Pinned models

Gridline fetches each declared model at its pinned commit. If a pinned model file has changed or is missing when it is fetched for your run, the run stops and that attempt is used up; it is not retried. A machine was already started, so its time is charged to your credits. A used-up attempt costs at most about $0.50: up to roughly 20 minutes (15 minutes of startup plus one 5-minute stop cycle) at the $1.50 per hour price ceiling, disk included. Check that pinned models are still public, then run again.

### No GPU capacity

If the selected GPU is not available, no machine is started and that attempt is not charged. The status reads “The selected GPU was not available, so no machine was started and this attempt was not charged. Trying the next matching offer.” If no matching GPU is left and no machine was started for the job, it ends with `routing_attempts_exhausted`: “No matching GPU was available right now. No machine was started for this job, so these attempts were not charged. Try again later or widen the GPU choice.” A machine that did start is charged to your credits for the time it ran, whatever the outcome.

06 · Fine-tune

## Fine-tuning: LoRA and QLoRA

Fine-tuning in your own script is a source run on the curated runtime `gridline/hf-finetune-2.5@r4`. It trains a LoRA adapter, or a QLoRA adapter on a 4-bit base model, on one GPU and returns the adapter files. Access, credits, SDK version and limits are the same as for [source runs](https://gridlinegpu.com/docs/own-script#source-runs).

### What the runtime provides

CPython 3.11 and CUDA 12.4 with torch 2.5.1, transformers 4.46.3, peft 0.13.2, trl 0.12.1, bitsandbytes 0.44.1, accelerate 1.1.1 and datasets 3.1.0. Do not add `python`, `requirements` or a lockfile next to a curated runtime.

### Declare the base model

Your code has no network, so it cannot download weights. Name the base model under `models:` with a pinned Hugging Face revision (a full commit hash) and the files you need. Gridline fetches and verifies them into `models/base` before your code starts.

```
runtime: gridline/hf-finetune-2.5@r4
models:
  base:
    source: hf://HuggingFaceTB/SmolLM2-135M
    revision: 93efa2f097d58c2a74874c7e644dbc9b0cee75a2
    files: [config.json, model.safetensors, tokenizer.json, tokenizer_config.json]
outputs:
  lora_adapter:
    path: outputs/lora/adapter_model.safetensors
    max_bytes: 33554432
  lora_config:
    path: outputs/lora/adapter_config.json
    max_bytes: 65536
```

A complete job is the fine-tuning example (`gridline-models.yaml`): LoRA in bf16 and QLoRA in 4-bit on SmolLM2-135M, with a 1,000-record dataset and a held-out eval. Download every file into one empty folder, keeping the layout shown, and submit it with `client.run("gridline-models.yaml")`:

| File | Role |
| --- | --- |
| [`gridline-models.yaml`](https://gridlinegpu.com/examples/finetune-lora-smollm2/gridline-models.yaml) | The job: pinned base model, required adapter outputs, one GPU |
| [`train.py`](https://gridlinegpu.com/examples/finetune-lora-smollm2/train.py) | Runs on the GPU: eval, LoRA, QLoRA, eval again |
| [`model-pins.json`](https://gridlinegpu.com/examples/finetune-lora-smollm2/model-pins.json) | The base model's pinned revision and file sizes |
| [`data/train.jsonl`](https://gridlinegpu.com/examples/finetune-lora-smollm2/data/train.jsonl) | 1,000 synthetic training records |
| [`data/eval.jsonl`](https://gridlinegpu.com/examples/finetune-lora-smollm2/data/eval.jsonl) | 32 held-out records |
| [`make_data.py`](https://gridlinegpu.com/examples/finetune-lora-smollm2/make_data.py) | Regenerates both data files byte for byte |

### Outputs and limits

- Declare each adapter file as an output and keep it required (the default). For now, an optional output can be left out of the results.
- Each output file is at most 128 MiB, at most 1 GiB per job: enough for LoRA and QLoRA adapters, not for full model weights.
- One GPU per job, and no network at run time.
- Under `hardened_user_separation` each process has an 8 GiB CPU data-memory limit. Load the base model straight to the GPU (for example with `device_map` or 4-bit loading) rather than building it in CPU memory first.

### Cost

While a job runs, about $2.23 of your credits is held for each window of about an hour. Minutes after the machine is torn down, the hold settles at the provider's cost for the machine time plus 25%, and the rest is released. Our acceptance job, LoRA and QLoRA on a 135M-parameter model, used about $0.06 of machine time (about $0.08 with the markup).

### Not yet offered

Full fine-tunes that return merged model weights need larger outputs and are not yet offered.

---

Source: https://gridlinegpu.com/docs/own-script