Enterprise AI Adoption Stalls As Inferencing Costs Confound Cloud Customers 16
According to market analyst firm Canalys, enterprise adoption of AI is slowing due to unpredictable and often high costs associated with model inferencing in the cloud. Despite strong growth in cloud infrastructure spending, businesses are increasingly scrutinizing cost-efficiency, with some opting for alternatives to public cloud providers as they grapple with volatile usage-based pricing models. The Register reports: [Canalys] published stats that show businesses spent $90.9 billion globally on infrastructure and platform-as-a-service with the likes of Microsoft, AWS and Google in calendar Q1, up 21 percent year-on-year, as the march of cloud adoption continues. Canalys says that growth came from enterprise users migrating more workloads to the cloud and exploring the use of generative AI, which relies heavily on cloud infrastructure.
Yet even as organizations move beyond development and trials to deployment of AI models, a lack of clarity over the ongoing recurring costs of inferencing services is becoming a concern. "Unlike training, which is a one-time investment, inference represents a recurring operational cost, making it a critical constraint on the path to AI commercialization," said Canalys senior director Rachel Brindley. "As AI transitions from research to large-scale deployment, enterprises are increasingly focused on the cost-efficiency of inference, comparing models, cloud platforms, and hardware architectures such as GPUs versus custom accelerators," she added.
Canalys researcher Yi Zhang said many AI services follow usage-based pricing models that charge on a per token or API call basis. This makes cost forecasting hard as the use of the services scale up. "When inference costs are volatile or excessively high, enterprises are forced to restrict usage, reduce model complexity, or limit deployment to high-value scenarios," Zhang said. "As a result, the broader potential of AI remains underutilized." [...] According to Canalys, cloud providers are aiming to improve inferencing efficiency via a modernized infrastructure built for AI, and reduce the cost of AI services. The report notes that AWS, Azure, and Google Cloud "continue to dominate the IaaS and PaaS market, accounting for 65 percent of customer spending worldwide."
"However, Microsoft and Google are slowly gaining ground on AWS, as its growth rate has slowed to 'only' 17 percent, down from 19 percent in the final quarter of 2024, while the two rivals have maintained growth rates of more than 30 percent."
Yet even as organizations move beyond development and trials to deployment of AI models, a lack of clarity over the ongoing recurring costs of inferencing services is becoming a concern. "Unlike training, which is a one-time investment, inference represents a recurring operational cost, making it a critical constraint on the path to AI commercialization," said Canalys senior director Rachel Brindley. "As AI transitions from research to large-scale deployment, enterprises are increasingly focused on the cost-efficiency of inference, comparing models, cloud platforms, and hardware architectures such as GPUs versus custom accelerators," she added.
Canalys researcher Yi Zhang said many AI services follow usage-based pricing models that charge on a per token or API call basis. This makes cost forecasting hard as the use of the services scale up. "When inference costs are volatile or excessively high, enterprises are forced to restrict usage, reduce model complexity, or limit deployment to high-value scenarios," Zhang said. "As a result, the broader potential of AI remains underutilized." [...] According to Canalys, cloud providers are aiming to improve inferencing efficiency via a modernized infrastructure built for AI, and reduce the cost of AI services. The report notes that AWS, Azure, and Google Cloud "continue to dominate the IaaS and PaaS market, accounting for 65 percent of customer spending worldwide."
"However, Microsoft and Google are slowly gaining ground on AWS, as its growth rate has slowed to 'only' 17 percent, down from 19 percent in the final quarter of 2024, while the two rivals have maintained growth rates of more than 30 percent."
20+% growth is stalling? (Score:2)
Re:20+% growth is stalling? (Score:5, Insightful)
For a get-rich-quick scheme based on lies? Yes, that is stalling.
Re: (Score:2)
When your head is firmly buried in the sand, it may look like that.
The volatile pricing of inference is correctly seen as hard to plan for, which simply means that inference providers will start offering more rate limited flat-fee options and other pricing models that are attractive to customers who need the predictability. This is nothing special (cloud computing has had similar issues and solutions for years) and definitely not evidence for "OMG AI IS HYPE YOU IDIOTS" with which you so consistently pollut
The cloud is a trap (Score:2)
Run away
AI needs local compute
Re:The cloud is a trap (Score:4, Insightful)
It depends.
It's kind of like people who insist that renting an apartment is worse financially than owning a home. It depends.
If you move every year or two, renting is a much better deal than buying. If you move to a new city and don't yet know what part of town you want to live in, renting is better. If you don't have the ability or desire to maintain your own place / grounds / HVAC, renting is better.
On the other hand, if you are stable, stay put, are raising a family, and are able and willing to maintain your own place, owning is definitely better.
Computer hardware has many of the same tradeoffs between owning (on prem) and renting (cloud).
Re: The cloud is a trap (Score:3)
Re: (Score:2)
The problem is local compute is absurdly front loaded cost wise. The big Nvidia cards that can run the larger models (And no a 5080 isn't even close) the costs can run in the tens of thousands for just one. If your needs are modest a Mac Studio MIGHT cover inference (256gb unified memory goes a long way, and most of the key libraries now support metal) but for serious work you gotta go Nvidia, and that aint cheap.
Re: (Score:2)
Had a long conversation [perplexity.ai] about Cerebras or Graphcore and the power saving plus software developments in power savings make for some impressive numbers.
Bottom Line
The combined effect of next-gen hardware (Cerebras, Graphcore, etc.) and modern software optimizations can realistically reduce AI’s energy use per task by 70–98% compared to just a few years ago.
This is transformative for both cost and sustainability—especially as AI adoption continues to surge.
The biggest wins will come from deploying both hardware and software advances together.
AI isn't cheap (Score:4, Interesting)
For all the hype about AI being cheaper than human labor, the truth isn't so clear. Human call center workers, for example, make maybe $10 per hour, and in that hour, they can take a dozen or more calls, depending on the complexity. AI equivalents, by contrast, often license their tools per use, per call, per command, etc. It's not at all clear that the cost/benefit ratio will swing in favor of AI in the near future.
Those "free" AI tools you get from the big providers? Those are loss leaders, getting people to think they are cheap and easy to use. As soon as you want to do real work with AI, the prices escalate.
Re:AI isn't cheap (Score:4, Interesting)
The real cost is in the middle men.
A crazy market has popped up selling the tools that turn inference providers into stuff that's useful- and they charge 1000% markups to businesses.
I have an... acquaintance that is in that like of "work".
Company has need A, he solves need A with inference provider B, paying the cost-per-Mtok, while company pays him a flat fee of $5,000-$10,000 a month (for $hundreds of dollars of inference)
It's pretty fucking insane.
Re: (Score:3)
The big providers charge dollars per millions of tokens.
You aren't wrong about people charging large markups, but the big providers are up there as well. The price for chatgpt audio tokens is around $80 per million tokens, which adds up to around $10 per hour for a call. That is just purely for the chatgpt audio model, but you need a lot more stuff to actually make a useful product. Then in many cases the AI can't actually solve the issues and you still need a human (more cost), and then the customer is frustrated (potential loss of business is more cost).
Re: (Score:3)
While it may be true that the big providers charge a few dollars per million tokens, a raw "inference" isn't generally a useful product for specific use cases. Businesses need There is a significant amount of refinement required to make a useful AI product, from "inferences." That's why SalesForce, for example, can change $500 for "packs of 100,000". https://coastalcloud.us/resour... [coastalcloud.us] And even at that rate, you can't just sign up for the SalesForce AI, and tell it to start selling your product. SalesForce i
Fraudulent nomenclature (Score:3)
Re: (Score:2)
The result of a statistical model is generally called inference... And realistically, the result of an LLM is no difference.
But problematically, we assign more meaning to the produced tokens than the model promises.
Bring your own GPU (Score:1)