Publishing state-of-the-art research on efficient fine-tuning techniques.
We introduce a novel gradient accumulation technique that reduces VRAM requirements for 70B parameter models by 45%, making state-of-the-art LoRA fine-tuning viable on consumer hardware.
Explore our complete archive of research on parameter-efficient fine tuning, quantization, and synthetic data generation.
We believe in open science. The code for our experimental optimizations is fully available to the community.
View GitHubWe collaborate with leading universities and AI research labs to push the boundaries of accessible machine learning. University labs receive compute grants and prioritized access to Langtrain Studio.