AJev

Checkpoints	Inputs
andyzhang232/ajev-gemma4-12b-lora7	text

AJev is a LoRA adapter on google/gemma-4-12B-it. Flide reads the base checkpoint and runs the adapter beside its weights, as peft does, with no extra package.

Flide asks it with the release's own prompt. Each question is a chat prompt of its own with one user turn. The turn tells the model to pick exactly one option, gives the state, the question and the options lettered A., B. and so on, and asks for the letter of the best option, or its code when the question has more than 26 options. A choice lists each option's name and description. A noul lists Yes and then No, each with the request's description or, without one, "Yes, the statement holds." and "No, the statement does not hold.". A score lists its levels from lowest to highest. A state that is not a string is written as compact JSON, and a state longer than 16384 tokens keeps its first 16384 tokens and ends with …[truncated].

The answer is the softmax over the option letters at the first token of the reply. A letter is read through its token with and without a leading space, and the two logits combine as a log-sum-exp. The temperature comes from the temperatures entry of ajev_lm_config.json in the checkpoint, one value per question type. The state comes first in every prompt, so Flide computes it once and forks it into the questions. Thinking is off.

Speed

On one H200, one request at a time, Flide answers 2.6 to 9.3 times faster than AJev's own LMPredictor on transformers 5.17 with peft and torch 2.14.1. Each workload is 32 requests, each run once on both sides before timing, and the table gives the median latency of a request. MultiRC requests ask about 49 questions each.

Workload	Flide	Release	Speedup
Decision Index	27.9 ms	103.3 ms	3.7×
RACE	21.3 ms	91.5 ms	4.3×
MultiRC	226.8 ms	2108.7 ms	9.3×
QuALITY long	180.8 ms	469.5 ms	2.6×

Start a server

flide serve --model andyzhang232/ajev-gemma4-12b-lora7

Send a request

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund and attached two photos.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}},
    "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"}
  }
}'

Limits and options

A request takes choice, noul and score questions. A choice takes up to 255 options and a score up to 255 levels. The longest prompt is 262144 tokens, of which the state takes at most 16384, and a longer prompt is refused. A request with images is refused. The template refuses nothing else.

Option	What it does
--format template --format-arg spec=andyzhang232__ajev-gemma4-12b-lora7	reads a copy of the checkpoint kept in a directory Flide does not recognize