All templates

batch-extract-sglang

Structured extraction with SGLang

Runs an OpenAI batch file of /v1/chat/completions (or /v1/completions) requests through SGLang with every answer constrained to one JSON Schema you supply, and writes one result line per request, in the same shape as batch-infer-sglang. Decoding is grammar-guided, so each answer parses and matches the schema unless it hits max_tokens first; the log reports how many parse. Default model Qwen3-4B-Instruct-2507 (Apache-2.0), downloaded from Hugging Face at start.

Status
Available
GPU
NVIDIA A10G · 24 GB a10g-24gb-x1
Default runtime
1 h maximum
Publisher
Unify Compute

What it reads and writes

Input /mnt/inDirectory containing requests.jsonl: one OpenAI batch line per request with url /v1/chat/completions or /v1/completions; the model field is replaced with MODEL and any response_format with the schema
Input /mnt/schemaschema.json: the JSON Schema every answer must match
Output /mnt/outresults.jsonl: one OpenAI batch result line per request, in input order, matched by custom_id; each answer's content is a JSON document matching schema.json

What you can change

A submit may set: GPU (SKU) (sku), Cloud placement (providers), Environment variables (spec.env), Maximum runtime (spec.max_runtime_seconds). The image and the declared inputs and outputs are fixed: a different image is a different template.

Environment defaults
MODELQwen/Qwen3-4B-Instruct-2507
MODEL_REVISIONcdbee75f17c01a7cc42f958dc650907174af0554
MAX_MODEL_LEN8192
CONCURRENCY64

Run it

With a key in $UNIFY_KEY (see the quick start), give each input an https:// URL, public or presigned, that ends in the file name above:

curl -s https://api.compute.unifyai.us/compute/v1/jobs \
  -H "Authorization: Bearer $UNIFY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "template": "batch-extract-sglang",
  "spec": {
    "inputs": [
      {
        "source": "https://YOUR-BUCKET.s3.amazonaws.com/requests.jsonl",
        "mount": "/mnt/in"
      },
      {
        "source": "https://YOUR-BUCKET.s3.amazonaws.com/schema.json",
        "mount": "/mnt/schema"
      }
    ]
  }
}'

The job is billed for the GPU time it uses at the SKU's rate, the same as a job you write yourself.

Early access

GPU job execution is turned on per account while it rolls out. Create an account, then ask us to enable it as the quick start describes.

Where the image comes from

Imagelmsysorg/sglang:v0.5.19-cu129-runtime
Digestsha256:fe08f6f52bea380415792849a703d92279c86320f8c39283bcd04043f8b63c41
LicenceApache-2.0
Sourcehttps://hub.docker.com/r/lmsysorg/sglang (tag v0.5.19-cu129-runtime, pushed 2026-09-04, the newest CUDA 12.9 build; multi-arch index read from registry-1.docker.io on 2026-11-06; LICENSE at github.com/sgl-project/sglang/blob/v0.5.19/LICENSE)

The digest is what runs. A tag can be re-pushed; a digest cannot, so what was reviewed is what your job pulls.