Skip to main content
← Back to glossary

Model distillation

Techniques

A technique to transfer the capabilities of a large model (the teacher) to a smaller one (the student), which imitates the teacher's behavior instead of repeating its training. The result is a smaller, faster model with a small quality drop: in an extreme case, distilling a 35-step bidirectional video generator into a 4-step causal student is what lets Cosmos3-H-Dreams run in real time.

This term in other languages