FAQ

The server stops at startup with a message about CUDA

Flide checks the GPU before it loads a model and says what is wrong in one line. The most common case is a driver that runs only CUDA 12 under a PyTorch built for CUDA 13; Installation shows the fix. A CPU build of PyTorch, or a machine with no NVIDIA driver loaded, gets its own message.

A request with images gets a 500 about torchvision

The image processors of Qwen3.5, Gemma 4 and OneJev need torchvision, and the detail of the 500 says how to install it. Install it from the same place as torch; a torchvision from another index replaces your torch with the build it was made for.

A chat model gives other numbers than with Flide 0.1.1

Since 0.1.2, chat models are asked in a new prompt style by default, the one that also takes images. Start the server with --format-arg style=flide to get the prompt of 0.1.1 back.

The first start takes a minute or more

On a GPU the package carries no compiled kernels for, the first start compiles them. The log says so, and reports its progress at every quarter of the CUDA graph capture. The kernels stay in the Triton cache, and later starts take seconds. Installation lists the GPUs that start without compiling and shows how to carry kernels to other machines.

Which family serves my checkpoint?

The server logs it when it loads, and GET /health reports it as format. A chat model that no family claims is read at the first token of its reply and logged as such. --format picks a family by name, and flide formats lists the installed ones.

A request gets a 422 about its length

A chat model or OneJev takes up to 32768 tokens for the state plus one question and 65536 for the whole request. Raise them with --max-branch-tokens and --max-request-tokens, within what the GPU's memory allows.

A request gets a 422 about its options

Each family reads up to its own number of options per choice and levels per score. A chat model reads up to 256 of either, Intern-Decision 62 options and 10 levels, torchcast 255 options and 10 levels. The page of each model in Supported models gives its limits.

Are the answers the same as running each question alone?

Yes, up to rounding. Each question reads the state exactly as it would alone, and Flide computes the state once and forks it. With --batch-invariant, a request also gets the same bits whatever other requests share its batch.