LoRA

Low-Rank Adaptation. An efficient fine-tuning technique that updates only a small fraction of the model weights instead of all parameters. LoRA enables fine-tuning a 7B model on a single consumer GPU, generating adaptation files of a few MB instead of gigabytes.