Models

# Models

Open models, pinned to a full Hugging Face commit. Fine-tune the instruct models, and run batches on them or on your fine-tuned versions.

## Catalog

Use the id as `model` when you create a fine-tuning job, or as `body.model` in a batch. `GET /openai/v1/models` lists the models your account can use right now.

| Model id | Source | Commit | Licence | Batch endpoint | Fine-tuning |
| --- | --- | --- | --- | --- | --- |
| `qwen3.8-27b` | Qwen/Qwen3.8-27B-FP8 | `c3a462cc3d38cf351f3b26cbd763469eeb74ef81` | Apache-2.0 | `/v1/chat/completions` | No |
| `nemotron-3.5-lightning-30b-a3b` | nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 | `bee7596271d1495f6992ae224aefde4410e816b8` | OpenMDW-1.1 | `/v1/chat/completions` | No |
| `gemma4-26b-a4b-it` | google/gemma-4-26B-A4B-it | `4d7ae4984b7db7de8f8457170b3f1a419ee76d52` | Apache-2.0 | `/v1/chat/completions` | No |
| `qwen3.5-9b` | Qwen/Qwen3.5-9B | `c202236235762e1c871ad0ccb60c8ee5ba337b9a` | Apache-2.0 | `/v1/chat/completions` | LoRA (adapter download only; batches on its ft: models are coming) |
| `gpt-oss-20b` | openai/gpt-oss-20b | `6cee5e81ee83917806bbde320786a8fb61efebee` | Apache-2.0 | `/v1/chat/completions` | LoRA (adapter download only; batches on its ft: models are coming) |
| `qwen3-embedding-0.6b` | Qwen/Qwen3-Embedding-0.6B | `97b0c614be4d77ee51c0cef4e5f07c00f9eb65b3` | Apache-2.0 | `/v1/embeddings` | No |
| `qwen3-8b` | Qwen/Qwen3-8B | `b968826d9c46dd6066d109eabc6255188de91218` | Apache-2.0 | `/v1/chat/completions` | LoRA, QLoRA |
| `qwen2.5-0.5b-instruct` | Qwen/Qwen2.5-0.5B-Instruct | `7ae557604adf67be50417f59c2c2f167def9a775` | Apache-2.0 | `/v1/chat/completions` | LoRA, QLoRA |
| `qwen2.5-1.5b-instruct` | Qwen/Qwen2.5-1.5B-Instruct | `989aa7980e4cf806f80c7fef2b1adb7bc71aa306` | Apache-2.0 | `/v1/chat/completions` | LoRA, QLoRA |
| `qwen2.5-7b-instruct` | Qwen/Qwen2.5-7B-Instruct | `a09a35458c702b33eeacc393d103063234e8bc28` | Apache-2.0 | `/v1/chat/completions` | LoRA, QLoRA |
| `llama-3.1-8b-instruct` | RedHatAI/Llama-3.1-8B-Instruct | `83c927477af31b2c66de4c0210ae5fa04b28289b` | Llama 3.1 Community License | `/v1/chat/completions` | LoRA, QLoRA |
| `mistral-7b-instruct-v0.3` | mistralai/Mistral-7B-Instruct-v0.3 | `c170c708c41dac9275d15a8fff4eca08d52bab71` | Apache-2.0 | `/v1/chat/completions` | LoRA, QLoRA |
| `bge-m3` | BAAI/bge-m3 | `5617a9f61b028005a4858fdac845db406aefb181` | MIT | `/v1/embeddings` | No |

- The chat models take up to 8,192 tokens of context in a batch and write at most 1,024 tokens per request; `gpt-oss-20b` takes up to 16,384 and writes at most 4,096.
- `gpt-oss-20b` reasons before it answers. The reasoning is written at low effort, counts against `max_tokens` and is billed as completion tokens; the result holds the answer alone. If the reasoning uses the whole allowance the answer comes back empty, so leave `max_tokens` room for both. It does not take a JSON `response_format`.
- `gpt-oss-20b`, `qwen3.5-9b` and `qwen3.8-27b` take a system message only as the first message, and need at least one user message. `qwen3.5-9b` and `qwen3.8-27b` are served for text only.
- `nemotron-3.5-lightning-30b-a3b` is NVIDIA's own 4-bit (NVFP4) build of the model, under the OpenMDW-1.1 licence.
- `qwen3-embedding-0.6b` returns dense embeddings, 1,024 dimensions, normalized, for inputs of up to 8,192 tokens. It embeds exactly the text you send: for retrieval, put the instruction the model card describes in front of each query yourself and send documents bare.
- `bge-m3` returns dense embeddings only, 1,024 dimensions, for inputs of up to 8,192 tokens. It is served from a safetensors conversion of BAAI's commit above.
- `llama-3.1-8b-instruct` is Meta's Llama 3.1 8B Instruct, served from Red Hat's ungated copy of Meta's files (the same bytes). Built with Llama: its use is subject to the [Llama 3.1 Community License](https://www.llama.com/llama3_1/license/) and [Acceptable Use Policy](https://www.llama.com/llama3_1/use-policy/).
- `qwen3-8b` answers with thinking off in batches, the form its fine-tunes are trained in.
- A qwen3-8b fine-tune made before batch serving of qwen3-8b began kept its LoRA adapter for download and created no fine_tuned_model: that job's fine_tuned_model is null and gridline.fine_tuned_adapter names the adapter (servable false). A new job on the same files creates a fine-tuned model you can run batches on.

Gridline is an independent service. Qwen, Mistral, Llama, gpt-oss, Nemotron, Gemma and bge-m3 are open models from their respective developers; Gridline is not affiliated with or endorsed by them.

## Your fine-tuned models

A fine-tuning job that succeeds registers its adapter under an `ft:` id, for example:

```
ft:qwen2.5-7b-instruct:o3f9a1c2d4e5:7b2c9e41a0d3
```

`GET /openai/v1/models` lists your `ft:` models alongside the catalog. No other account sees them, and naming one from another account is answered `model_not_found`, as is an id that does not exist. The [fine-tuning reference](https://gridlinegpu.com/docs/fine-tuning#model) explains each part of the id, how to download the adapter, and how to delete the model. An account holds at most 100 fine-tuned models.

## How fine-tuned models are served

- A batch on an `ft:` model runs its base model at the exact commit the adapter was trained on, with the adapter loaded.
- One base model per batch, mixed with up to 8 `ft:` models of that base, within what the base has been proved to serve: on qwen2.5-0.5b-instruct, qwen2.5-1.5b-instruct, 8 of any rank; on qwen2.5-7b-instruct, mistral-7b-instruct-v0.3, llama-3.1-8b-instruct, 4 adapters of rank 64 or 8 of rank 32 or lower; on qwen3-8b, 4 of any rank.
- Only bases in the batch catalog can serve batches. A model fine-tuned on any other base stays downloadable but cannot be named in a batch.
- A QLoRA adapter is trained against a 4-bit quantized base and served on the bf16 base, so its outputs can differ slightly from what training measured.

## Not offered

- Qwen2.5-3B: its licence is not Apache-2.0.
- Models of 70B parameters and larger.
- Fine-tuning embedding models.

---

Source: https://gridlinegpu.com/docs/models