Prefill
Model typesThe first phase of inference, where the model processes the input prompt (all tokens) in one pass. It determines time-to-first-token: near-instant on a GPU, it can take several seconds on a CPU for long prompts.
The first phase of inference, where the model processes the input prompt (all tokens) in one pass. It determines time-to-first-token: near-instant on a GPU, it can take several seconds on a CPU for long prompts.