SwiftKV’s introduction comes at a critical moment for enterprises embracing LLM technologies. With the growth of use cases, organizations need solutions that deliver both immediate performance gains and long-term scalability. By tackling the computational bottlenecks of inference directly, SwiftKV offers a new path forward, enabling enterprises to unlock the full potential of their LLM production deployments. We are excited to provide the SwiftKV innovation on the Llama models with the launch of Snowflake-Llama-3.3-70B and Snowflake-Llama-3.1-405B with inference at a fraction of the cost (75% and 68% lower cost, respectively). The Snowflake-derived Llama models are a game changer for enterprises navigating the challenges of scaling gen AI innovation in their organizations in an easy and cost-effective way.
Getting started: Run a your own SwiftKV training by following this quickstart.
Because SwiftKV is fully open source, you can also deploy it on your own with model checkpoints on Hugging Face and optimized inference on vLLM. You can learn more in our SwiftKV research blog post.
We are also making knowledge distillation pipelines via ArcticTraining Framework open source so you can build your own SwiftKV models for your enterprise or academic needs. The ArcticTraining Framework is a powerful post-training library for streamlining research and development. It is designed to facilitate research and prototype new ideas for post-training without getting overwhelmed by complex abstraction layers or generalizations. It offers a high-quality, user-friendly synthetic data generation pipeline and a scalable, adaptable training framework for algorithmic innovation, as well as an out-of–the-box recipe for training your own SwiftKV models.
As gen AI innovation continues to expand across industries and use cases, optimizations such as SwiftKV are critical in bringing AI to end users in a cost-effective and performant manner. Now available as open source, SwiftKV makes enterprise-grade gen AI faster and less expensive to run. Taking it a step further, we are also launching Llama models optimized with SwiftKV in Snowflake Cortex AI. With Snowflake-Llama-3.3-70B and Snowflake-Llama-3.1-405B models, customers are seeing up to 75% lower inference costs. We are helping them build gen AI solutions that are both cost effective and high performing.
