Gemma 4

Checkpoints	Inputs
google/gemma-4-E2B-it, -E4B-it, -12B-it, -31B-it	text, images

Any Gemma 4 chat model serves as it is and is read the same way as a Qwen3.5 chat model. Each question is asked after the state with its options labeled A, B, C and so on, and its answer is read from the probabilities of those labels at the first token of the reply. After Z come two-letter labels, AA, AB and so on, each one token, so a question takes up to 256 options.

Start a server

flide serve --model google/gemma-4-E2B-it

Send a request

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund and attached two photos.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}},
    "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"}
  }
}'

Images

Images go in images, as data URLs or plain base64, and the model sees them ahead of the text.

IMG=$(base64 -w0 photo.jpg)
curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d @- <<EOF
{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund. A photo is attached.",
  "images": ["data:image/jpeg;base64,$IMG"],
  "questions": {
    "cracked": {"type": "noul", "instructions": "Does the photo show a cracked screen?"},
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}}
  }
}
EOF

Images need torchvision, installed from the same place as torch.

The prompt styles and limits on the Qwen3.5 page apply to Gemma 4 as well, except --quantize fp8, which is for Qwen3.5 only.