batch-infer-lora

Batch chat completions with your LoRA adapter

Runs an OpenAI batch file of /v1/chat/completions requests through vLLM with a LoRA adapter applied (for example the adapter.tar.gz a finetune-lora job produced) and writes one result line per request, like batch-infer-vllm. Default base model Qwen3-4B-Instruct-2507 (Apache-2.0); the adapter must have been trained on the same base.

Status: Available
GPU: NVIDIA A10G · 24 GB a10g-24gb-x1
Default runtime: 1 h maximum
Publisher: Unify Compute

What it reads and writes

Input `/mnt/in`	requests.jsonl: one OpenAI batch line per request with url /v1/chat/completions; the model field is ignored, the adapter applies to every line
Input `/mnt/adapter`	adapter.tar.gz: a PEFT LoRA adapter directory (adapter_config.json, adapter_model.safetensors), as finetune-lora writes it
Output `/mnt/out`	results.jsonl: one OpenAI batch result line per request, matched by custom_id, model = LORA_NAME

What you can change

A submit may set: GPU (SKU) (sku), Cloud placement (providers), Environment variables (spec.env), Maximum runtime (spec.max_runtime_seconds). The image and the declared inputs and outputs are fixed: a different image is a different template.

Environment defaults
`MODEL`	`Qwen/Qwen3-4B-Instruct-2507`
`MODEL_REVISION`	`cdbee75f17c01a7cc42f958dc650907174af0554`
`MAX_MODEL_LEN`	`8192`
`MAX_LORA_RANK`	`64`
`LORA_NAME`	`lora`

Run it

With a key in $UNIFY_KEY (see the quick start), give each input an https:// URL, public or presigned, that ends in the file name above:

curl -s https://api.compute.unifyai.us/compute/v1/jobs \
  -H "Authorization: Bearer $UNIFY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "template": "batch-infer-lora",
  "spec": {
    "inputs": [
      {
        "source": "https://YOUR-BUCKET.s3.amazonaws.com/requests.jsonl",
        "mount": "/mnt/in"
      },
      {
        "source": "https://YOUR-BUCKET.s3.amazonaws.com/adapter.tar.gz",
        "mount": "/mnt/adapter"
      }
    ]
  }
}'

The job is billed for the GPU time it uses at the SKU's rate, the same as a job you write yourself.

Early access

GPU job execution is turned on per account while it rolls out. Create an account, then ask us to enable it as the quick start describes.

Where the image comes from

Image	`ghcr.io/felixsphere/unify-compute-finetune:finetune-2026-10-04`
Digest	`sha256:6d426175dcd0e2443df6da9cf68e96c079abe5a8b80d9f488d09019536272ac1`
Licence	Apache-2.0
Source	ghcr.io/felixsphere/unify-compute-finetune, built by publish-image.yml run 37266527011 from main 150de552 (finetune/Dockerfile on vllm/vllm-openai:v0.30.0-cu129@sha256:a67f8f18…, Apache-2.0; adds TRL 1.14.1, PEFT 0.21.2, datasets 5.0.1, accelerate 1.15.0, jsonschema 4.26.0 and torchvision 0.29.0+cu130, removes torchaudio; linux/amd64 only; licences in THIRD-PARTY-LICENSES.md, Workload images)

The digest is what runs. A tag can be re-pushed; a digest cannot, so what was reviewed is what your job pulls.