Inference chip company Positron AI executive: Inference now drives two-thirds of AI workloads, shifting deployment to modular data centers
Cameron McCaskill, head of sales and business development at Positron AI Inc., stated at the SC26 supercomputing conference that inference workloads now account for approximately two-thirds of the traffic running through AI infrastructures, compared to one-third for training. This represents a significant reversal from 2023, when the split was roughly 85% training and 15% inference. According to McCaskill, this shift is driving the economic decisions behind how companies build out their AI infrastructure.
To capture this market change, Positron AI designs its inference appliances around commodity memory rather than High Bandwidth Memory (HBM), which is scarce and expensive. The company’s first-generation Atlas server runs on programmable chips, with plans to tape out its own silicon early next year for the Titan server. Oracle Corp. is bringing about 340 Atlas servers live in its cloud this quarter, and a neocloud provider has also signed on for Atlas and committed to Titan.
The design choice to skip HBM keeps power consumption low enough for air-cooled rooms. A single Positron server draws about 4,500 watts, and a full rack of Titan servers fits within 40 kilowatts. This footprint suits sites where operators can find a few megawatts of cheap power without heavy regulatory hurdles. McCaskill noted a rise in demand from modular data center operators who have half-megawatt to 2.5-megawatt capacity and aim to go from land and power to monetizable tokens within 90 to 120 days.
Sources :SiliconANGLE