Tensordyne Revives Logarithmic Math In A Bid To Cut AI Power Use

Log Math? Really?

So, what is log-math? Invented in the mid-16th century by John Napier, log math enables a simple mathematical trick that allows multiplication to be rewritten as addition; log(ab) = log(a) + log(b). And that can reduce power use, simplify circuits, and potentially speed up inference processing. The idea is especially attractive in AI because neural networks spend a lot of time on matrix math, and matrix multiplication is one of the most hardware-intensive parts of modern accelerators.

By moving computations into log space, a chip can exploit the fact that additions are simpler than multipliers, while still preserving much of the structure of the original computation. The tradeoff is that log-domain systems usually need approximations, extra conversion steps, or careful numerical handling, so they are not a universal replacement for standard arithmetic. Luckily, LLMs and transformers in general are not one of those workloads.

The benefits look quite attractive. Nvidia itself had been researching the potential for logarithmic math, as presented in a Nvidia Chief Scientist Bill Dally talk at HotChips in 2023. It remains unknown whether Nvidia abandoned the idea or could someday adopt it. My guess is that Nvidia put log math on the back burner as the company worked on more important data-center-as-a-computer challenges and the integration of the Groq LPU.

Why has logarithmic computing not become mainstream previously? Probably because of the effort required versus the benefits provided was not attractive. Nvidia has done quite well by reducing the precision of its math while maintaining model fidelity.

The Napier Processor and System

The processor looks well designed, from the ground up, as an inference processor. Two tiers of fast memory (SRAM and HBM3E) combine with the log-based math cores in a systolic array and are scaled up with a fast interconnect (from HPE Juniper) to scale up to 72 chips. The fabric is accessed over PCIe, which does limit its performance compared to an NVLink fabric.

The processor itself delivers 2.1 PetaFlops at 8-bit precision, and 1TB chip to chip latency, and support 256MB of SRAM, the fast on-chip memory that is used by Groq and Cerebras. But unlike those two players, Tensordyne backs that up with 144 GB of HBM3E. Hopefully this two-tiered memory architecture is hidden from the model developer.

Tensordyne packages up nine Napier chips and one Intel Xeon up into a 1 rack unit tray that has an 8 TB NVMe SSD and two 200GB Ethernet ports. And the board is air-cooled, which will be a relief to data centers that eschew liquid cooling expenses.

Let’s Compare

Let’s look at how Tensordyne’s Napier compares to Nvidia and AWS/Cerebras. Note that all these claims have not yet been verified by a third party. For starters, Tensordyne claims to be 13 times faster and 17X more tokens/second/MegaWatt than the Nvidia GB300 NVL72 running DeepSeek-R1 inferencing.

Comparing to the next generation Rubin + Groq 3 (which should be in production at about the same time or earlier) running a two trillion Mixture of Experts model, a single rack of four pods delivers 1300 tokens per second versus 800 TPS from nine racks of Nvidia processors, and consumes 120kW versus 1.5 megawatts. The cost per million tokens is only $11 for the log-based Tensordyne system versus $150 the Nvidia/Groq racks.

How Could Tensordyne Impact The Market?

Log math could get attention from major hyperscalers but, more importantly, customers in the AI Cloud and Data Center spaces, which could welcome a lower-cost, air-cooled system that provides excellent value in inference processing. Tensordyne will need to go an extra mile in the sales process to demonstrate ease of adoption, versatility and breadth in model choices, and precision in inference results.