Breaking News

An Indian AI Startup Proves LLMs No Longer Need Expensive GPUs

by VARINDIA 2025-04-26

With GPU shortages and rising costs hindering global AI development, solutions like Kompact AI can democratise AI and could pave the way for a more sustainable and inclusive AI ecosystem.

As artificial intelligence continues to revolutionize industries worldwide, high computational costs have remained a key barrier to widespread adoption—especially when it comes to running large language models (LLMs). However, an Indian startup is challenging that status quo. Ziroh Labs, a trailblazer in AI innovation, is redefining how LLMs operate by eliminating the need for costly and often scarce GPUs, making powerful AI capabilities more accessible than ever.

Reimagining AI Infrastructure with Kompact AI

While India accelerates its efforts under the IndiaAI mission to develop a homegrown foundational model, Ziroh Labs is taking a bold, alternative approach. The startup has unveiled Kompact AI, a groundbreaking platform designed to run sophisticated LLMs efficiently on widely available CPUs. By doing so, Ziroh Labs is tackling two of the biggest challenges in AI deployment: high infrastructure costs and limited hardware availability.

Traditionally, LLMs have relied heavily on GPU-powered systems for both model training and inference. GPUs, though powerful, are expensive and frequently in short supply—especially as global demand for AI accelerates. Kompact AI disrupts this dependency by optimizing CPU performance for LLM workloads, enabling businesses and developers to run resource-intensive models without investing in specialized hardware.

Key Features of Kompact AI

CPU-Optimized Inference: Kompact AI enables seamless inference of LLMs on widely accessible CPUs, significantly lowering the barrier to entry for organizations that previously couldn’t afford GPU infrastructure.

Scalable Fine-Tuning: The platform will soon support fine-tuning models with up to 50 billion parameters, providing companies with the flexibility to customize AI models without sacrificing performance.

Cost-Efficiency: By reducing reliance on GPUs, Kompact AI allows businesses to cut operational costs while still leveraging state-of-the-art AI solutions.

Accessibility and Inclusivity: Staying true to its founding principle of democratizing AI, Ziroh Labs’ solution empowers startups, SMEs, and research institutions to adopt AI technologies without budget constraints.

India’s AI landscape is rapidly evolving, with initiatives like IndiaAI focused on creating indigenous AI models to drive innovation across sectors. Ziroh Labs’ CPU-based approach complements these national efforts by providing scalable AI infrastructure that can be deployed across diverse environments, from small enterprises to large organizations.

By prioritizing accessibility, Ziroh Labs is not only reducing the digital divide but also accelerating AI adoption in sectors that have traditionally struggled with high infrastructure costs, such as healthcare, education, and agriculture.