flide run

flide run answers a file of requests offline, without a server. Each line of the file is one request with state and questions, and every other key of the line is copied into its result, so ids and labels travel with the answers.

cat > requests.jsonl <<'EOF'
{"id": 1, "state": "The package arrived two weeks late and the box was crushed.", "questions": {"complaint": {"type": "noul", "instructions": "The customer is complaining"}}}
{"id": 2, "state": "Thanks, the replacement works perfectly.", "questions": {"complaint": {"type": "noul", "instructions": "The customer is complaining"}}}
EOF
flide run --model Qwen/Qwen3.5-0.8B --rows requests.jsonl --out answers

answers/results.jsonl holds one result per request, in the order of the file, with status and the response. answers/timing.json splits the time into planning, the GPU and finishing, and answers/environment.json records the GPU, the PyTorch version and the options of the run.

CPU workers render and tokenize the requests while the GPU answers earlier ones in packed micro-batches, so a large file keeps the GPU busy. Files ending in .gz are read as gzip.

Option	Default	What it does
--model M	required	a Hub repository id or a local directory
--rows FILE ...	required	one or more JSON-lines files
--out DIR	required	where the results go
--format F	from the checkpoint	the model family
--format-arg KEY=VALUE		an option of the family
--plan-arg KEY=VALUE		an option of every request, such as debias=3
--name N	flide	the model name written into each response
--every N	1	keeps every n-th row
--limit N	all	stops after this many rows
--budget N	65536	tokens per micro-batch
--workers N	CPU count minus 2	processes that render and tokenize
--batch-invariant	off	the same bits for a request whatever it is batched with
--quantize fp8	off	Qwen3.5 decoder weights in FP8
--device D	cuda:0	the GPU to run on