Together AI's ATLAS adaptive speculator delivers 400% inference speedup by learning from workloads in real-time
Enterprises expanding AI deployments are hitting an invisible performance wall. The culprit? Static speculators that can't keep up with shifting workloads.Speculators are smaller AI models that work alongside large language models during inference. They draft multiple tokens ahead, which the main model then verifies in parallel. This technique (called speculative decoding) has become essential for enterprises trying to reduce inference costs and latency. Instead of generating tokens one at a time, the system can accept multiple tokens at once, dramatically improving throughput.Together AI today announced research and a new system called ATLAS (AdapTive-LeArning Speculator System) that aims to help enterprises overcome the challenge of static speculators. The technique provides a self- [...]
Rating
Innovation
Pricing
Technology
Usability
We have discovered similar tools to what you are looking for. Check out our suggestions for similar AI tools.
Baseten takes on hyperscalers with new AI training platform that lets you own your model weights
Baseten, the AI infrastructure company recently valued at $2.15 billion, is making its most significant product pivot yet: a full-scale push into model training that could reshape how enterprises wean [...]
AI inference costs dropped up to 10x on Nvidia's Blackwell — but hardware is only half the equation
Lowering the cost of inference is typically a combination of hardware and software. A new analysis released Thursday by Nvidia details how four leading inference providers are reporting 4x to 10x redu [...]
Boston Dynamics announces production-ready version of Atlas robot at CES 2026
After years of testing its humanoid robot (and forcing it to dance), Boston Dynamics' Atlas is entering production. The robotics company says the final product version of the robot is being built [...]
Boston Dynamics unveils production-ready version of Atlas robot at CES 2026
Boston Dynamics' Atlas is finally entering production. After years of testing this humanoid robot (and forcing it to dance), the robotics company announced at CES 2026 that the final version of t [...]
The team behind continuous batching says your idle GPUs should be running inference, not sitting dark
Every GPU cluster has dead time. Training jobs finish, workloads shift and hardware sits dark while power and cooling costs keep running. For neocloud operators, those empty cycles are lost margin.The [...]
Inference is splitting in two — Nvidia’s $20B Groq bet explains its next act
Nvidia’s $20 billion strategic licensing deal with Groq represents one of the first clear moves in a four-front fight over the future AI stack. 2026 is when that fight becomes obvious to enterprise [...]
Google debuts AI chips with 4X performance boost, secures Anthropic megadeal worth billions
Google Cloud is introducing what it calls its most powerful artificial intelligence infrastructure to date, unveiling a seventh-generation Tensor Processing Unit and expanded Arm-based computing optio [...]
IndexCache, a new sparse attention optimizer, delivers 1.82x faster inference on long-context AI models
Processing 200,000 tokens through a large language model is expensive and slow: the longer the context, the faster the costs spiral. Researchers at Tsinghua University and Z.ai have built a technique [...]
Train-to-Test scaling explained: How to optimize your end-to-end AI compute budget for inference
The standard guidelines for building large language models (LLMs) optimize only for training costs and ignore inference costs. This poses a challenge for real-world applications that use inference-tim [...]