Add a model family

A model family tells Flide how its checkpoints are asked and read. Flide runs the backbone; the family turns a request into a plan, names the heads that read the hidden states, and turns the head outputs into answers. The core knows nothing about prompts, answer slots or calibration, so a new family needs no change to it.

Methods

A family subclasses flide.formats.Format, registers under a name and implements three methods. A fourth, follow_up, is for families that read in rounds.

Method	What it returns
load_heads(backbone)	heads by name; each maps hidden states [rows, hidden] to outputs [rows, ...]
plan(request, **options)	a Plan: the shared prefix tokens, one suffix per branch, and the reads of each branch
finish(request, plan, outputs, **options)	the response and a dict for the server's record
follow_up(request, plan, outputs, **options)	optional: a further Plan that depends on this plan's outputs, or None

Flide runs every plan that follow_up returns and asks again, until it returns None. finish then gets the last plan and its outputs, so a family carries what it needs from earlier rounds in that plan's data.

The class attributes max_choice_options (255 by default) and max_score_levels (10) say how large a choice and a score the family reads. A larger question gets a 422 before plan is called.

A Plan holds the state's tokens once as prefix and one token list per question in suffixes. Flide computes the prefix once and forks it into every suffix. reads says where each branch is read: Read("name") takes the hidden state at the branch's last token after the final norm and passes it through the head called name; positions and layer pick other tokens and layers. In finish, outputs[b][r] is what read r of branch b returned, one row per position.

A family in one file

This family answers noul questions by comparing the output head's logits for " yes" and " no" after a plain prompt. register makes it available to the script, and format="yesno" picks it.

import math

import transformers
from flide import Engine
from flide.api.answers import make_noul
from flide.api.schema import SystemOneRequest, SystemOneResponse, Usage
from flide.formats import Format, Plan, Read, register


class TokenLogits:
    def __init__(self, weight, ids):
        self.weight = weight[ids].float()

    def __call__(self, hidden):
        return hidden.float() @ self.weight.T


@register("yesno")
class YesNo(Format):
    def __init__(self, model_dir, tokenizer=None):
        self.tokenizer = tokenizer or transformers.AutoTokenizer.from_pretrained(model_dir)
        self.ids = [self.tokenizer.encode(w, add_special_tokens=False)[0] for w in (" yes", " no")]

    def load_heads(self, backbone):
        weight, _ = backbone.output_head()
        return {"yesno": TokenLogits(weight, self.ids)}

    def plan(self, request, **_):
        encode = lambda text: self.tokenizer.encode(text, add_special_tokens=False)
        prefix = encode(f"Text: {request.state}\n")
        suffixes = [encode(f"Statement: {q.instructions}\nIs the statement true? Answer:")
                    for q in request.questions.values()]
        return Plan(prefix, suffixes, (Read("yesno"),), data=list(request.questions))

    def finish(self, request, plan, outputs, **_):
        answers = {}
        for qid, out in zip(plan.data, outputs):
            yes, no = out[0][0]
            answers[qid] = make_noul(1 / (1 + math.exp(no - yes)))
        usage = Usage(input_tokens=plan.tokens, output_tokens=len(answers))
        return SystemOneResponse(model="", answers=answers, usage=usage), {}


engine = Engine("Qwen/Qwen3.5-0.8B", format="yesno")
request = SystemOneRequest.model_validate({
    "state": "The invoice was paid on March 3, two weeks after it was due.",
    "questions": {"late": {"type": "noul", "instructions": "The invoice was paid late."},
                  "unpaid": {"type": "noul", "instructions": "The invoice is still unpaid."}},
})
response, _ = engine.decide(request)
print(response.model_dump()["answers"])

A family as a package

A family that others install registers in the flide.formats entry point group of its own package, and Flide loads it at startup.

[project.entry-points."flide.formats"]
yesno = "yesno_package:YesNo"

A checkpoint can name its family in decision_config.json with format_id, and register("yesno", format_ids=("yesno-v1",)) claims that id. A family can also claim checkpoints by overriding the class method detect(model_dir, config, decision_config), and set backbones to the model types it runs on.

The families that come with Flide live in plugins/ of the Flide repository. OneJev reads answer slots, with optional debiasing and media. Open-Jev reads one scalar per candidate from a head file of its own. Intern-Decision reads every question from one prompt, and torchcast reads a wide choice in rounds with follow_up.