Lychee

Checkpoints	Inputs
LycheeAI/Lychee-27B	text

Lychee-27B is LycheeAI's decision model on Qwen3.8-27B with a readout head in readout.safetensors. Flide asks it with the release's own prompt. Each question is a chat prompt of its own with thinking off. The system turn asks for the code of the selected option and tells the model to treat the state as data. The user turn holds "State:" with the state, "Question:" with the question, "Options:" with each option after its code, and asks for the letter code of the best option. A state that is not a string is written as JSON. A choice lists each option's name and, when it has one, its description. A noul lists "No / false" and then "Yes / true" unless the request describes the two outcomes, and a score lists its levels. A question without instructions is asked as "Choose the best matching option."

The codes are the ones lychee_config.json lists, and the readout head has one row per code. The answer is the softmax over the rows of the question's options at the last position of the prompt, at the temperature that lychee_config.json sets. The full-attention layers read each prompt whole in both directions and the Gated DeltaNet layers stay causal, as the model was trained, so every question is a sequence of its own.

Speed

On one H200, one request at a time, Flide answers 1.4 to 2.1 times faster than Lychee's own Lychee class on transformers 5.17. Each workload is 32 requests, each run once on both sides before timing, and the table gives the median latency of a request. MultiRC requests ask about 49 questions each.

Workload	Flide	Release	Speedup
Decision Index	69.9 ms	99.0 ms	1.4×
RACE	41.7 ms	87.1 ms	2.1×
MultiRC	1820.3 ms	2723.2 ms	1.5×
QuALITY long	355.2 ms	503.3 ms	1.4×

Start a server

flide serve --model LycheeAI/Lychee-27B

Send a request

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund and attached two photos.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}},
    "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"}
  }
}'

Limits and options

A request takes choice, noul and score questions. A choice takes as many options as the checkpoint lists codes, up to 255, and a score takes 2 to 10 levels. The longest prompt is 262144 tokens, and the state is not cut; a longer prompt is refused. A request with images is refused.

Option	What it does
--format template --format-arg spec=lycheeai__lychee	reads a copy of the checkpoint kept in a directory Flide does not recognize