Generative artificial intelligence may be the next big thing, but it’s also the next big energy hog — and that’s a problem for all those data centers training and running AI models: It’s not clear how they’re going to withstand all the heat produced by increasingly large and power-hungry chips.

That’s one trend we uncovered in this newsgathering. I also offer an opinion on venture capitalist Marc Andreessen (TL;DR: thumbs mostly down). And as always, we covered a wide range of news on generative AI, ahead of our Supercloud 4 virtual editorial event coming Monday and Tuesday, Oct. 24 and 25 — details below.

This and other news — including prospects, especially in their AI efforts, for the big cloud computing players that report quarterly earnings results next week, as well as the big question of where most AI workloads will reside — are big topics of discussion on theCUBE Pod, John Furrier’s and Dave Vellante’s weekly podcast, out now on YouTube. And don’t miss Vellante’s weekly Breaking Analysis, coming over the weekend.

Supercloud special report: Generative AI transforms every industry

Ahead of our free Supercloud 4 virtual editorial event coming Monday and Tuesday (more on that at the bottom), we ran two features, with several more coming in the next few days, to get you up to speed on the latest trends in generative AI:

AI model training rekindles interest in on-premises infrastructure: Perhaps surprisingly, the cloud isn’t and increasingly won’t be everything when it comes to generative AI even on the model training side, Paul Gillin’s reporting found. That could give an opening to infrastructure companies such as Hewlett Packard Enterprise and Dell Technologies.

How companies are scrambling to keep control of their private data from AI models: “You don’t have to be a sophisticated hacker anymore,” one researcher told David Strom. “All you have to do now is ask ChatGPT to develop an exploit or to write some code to extract your data or find a hole in your infrastructure and play the role of an ethical hacker.”

AI’s global warming problem

No, not that global warming, but the warming in data centers that are tasked with training and running the sudden explosion of power-hogging generative AI models.

When I stopped by the Open Compute Project’s OCP Global Summit this week in San Jose to see what’s up with new data center technologies, the one thing that really struck me was all the talk about, and the companies selling, liquid cooling technologies. There was everything from so-called “cold plate” technologies that draw heat off chips and servers to full-immersion systems that spray liquids directly on server components or dip whole servers in liquid, and then disperse that heat to water pipes and heat exchangers.

Now, you might think, as I did initially, that electronics and liquids really don’t mix well. But the liquid isn’t water, for better or worse a pretty good conductor, but so-called dielectric fluids, meaning they’re insulators that don’t conduct electricity. Most of them are various kinds of oils: ExxonMobil had a booth and was touting its new fluids, and even Castrol — you know, the motor oil folks — had a booth.

Nearly every speaker at the summit, from Google and Meta Platforms to Intel and Advanced Micro Devices, said they believe chips — in particular graphics processing units from the likes of Nvidia, which happen to be perfect for AI workloads of various kinds — and the servers they reside in will have to move to liquid cooling to keep data centers from melting down. Rolf Brink, OCP’s project lead for immersion cooling and CEO of Promersion, which advises companies on liquid cooling, noted that chips are only going to get denser and thus hotter.

Liquid cooling is just one of the changes in data center technology that generative AI will require. OCP also identified chiplets for specialized computation, optical networking and large memory pools, all things startups and others have jumped on. But liquid cooling is the biggest challenge.

“We have to move to liquid cooling — it’s going to be pervasive,” said Zane Ball, Intel’s corporate vice president and general manager of datacenter engineering and architecture, who noted that McKinsey predicts energy demand in the average data center will double by 2030. Added OCP Chief Technology Officer Bijan Nowroozi: “We’re getting at the end of things you can do with moving air with fans.”

Cold plate technology is becoming a commodity, Brink noted, but it’s not going to be enough. Problem is, immersion cooling is three to five years behind cold plate in technology and market maturity, and it has many challenges to overcome. That means gen AI could run into a problem that goes beyond promising your first-born child in return for GPUs — which, by the way, are the big contributor to that warming. Among those many challenges:

* There are no real standards, so each solution is boutique and thus expensive and not really suitable for rolling out widely. “It’s been very hard for these guys to get on the same page,” former OCP CTO Bill Carter told me.

* Existing data centers can’t be easily retrofitted for liquid cooling, so entirely new ones have to be built. Meta, for instance, is planning for AI-focused data centers by 2025 that could include liquid cooling, Alexis Black Bjorlin, Meta’s just-departed vice president of infrastructure hardware, said at another conference recently.

* The liquids currently are expensive, up to $1,000 a gallon by one expert’s estimate.

* Some of them are highly toxic, containing potentially cancer-causing “forever chemicals,” dangerous enough — especially when used in “two-phase” immersion cooling that involves the liquid turning into a gas, which can escape into the environment. 3M plans to stop producing one such product by 2025 following multibillion-dollar lawsuits.

Some folks believe all the activity around liquid cooling simply points up the failure of today’s chipmakers and AI software creators. Ampere Computing is one company that has aimed at lower-power but still powerful processors built for cloud computing, first with an Arm core and now with its own, that don’t require liquid cooling. “We chose to rearchitect the CPU” to scale up more efficiently, Chief Product Officer Jeff Wittich told me.

Other necessary solutions involve downsizing large language models by using domain-specific data, so not all of them have to be so huge and energy-intensive, and designing AI hardware and software systems together from the start to improve efficiency.

But for now, that ship full of power-hungry chips has sailed, so there’s a lot of work ahead to make liquid cooling work. Andy Bechtolsheim, Arista Networks’ founder, chairman and chief development officer, said at the summit that a combination of cooling technologies will be needed, but all of them need to be more standardized.

“One size can’t fit all these technologies,” said Brink. “But in a few years, every data center will have to deal with liquid cooling one way or another.” His advice to those building data centers: “Brace for change.”

And a couple of significant announcements at the OCP summit:

Ampere and other chipmakers form AI Platform Alliance

Arm unveils Total Design ecosystem for customized system-on-chip development

More on the generative AI boom

Using LLMs to clean up data for other LLMs: Refuel.ai debuts a large language model to label the data needed to train other LLMs

Seems a little speculative, but the black-box nature of LLMs raises legitimate concerns: ​SEC chair Gary Gensler warns unregulated AI could lead to financial crisis And speaking of black boxes: Stanford’s AI transparency index shows foundational models are shrouded in secrecy

Better not forget China in the AI race: Chinese AI startup Baichuan raises $300M from Alibaba, Tencent and Xiaomi and Chinese AI startup Zhipu raises $300M in funding

Nutanix co-founder Dheeraj Pandey jumps into an up-and-coming area of gen AI: Product and support CRM startup DevRev debuts customizable LLMs

Multimodal search is coming: Objective raises $13M to bring multimodal search capabilities to any website or app

Maybe some help on data center overheating? IBM debuts power-efficient NorthPole machine learning processor

The AI lawsuits mount: Universal Music Group sues AI startup Anthropic over song lyric copyright infringement

In other selected AI news:

Nvidia’s DGX Cloud supercomputer for generative AI launches on Oracle Cloud

Nvidia and Foxconn to collaborate on AI data factories and smarter self-driving cars

OpenAI integrates its latest image generator DALL-E 3 into ChatGPT app

Meta’s newest AI system can decode images from human brain activity

Meta’s Habitat 3.0 simulates real-world environments for intelligent AI robot training And coincidentally: Nvidia develops AI system for programming robots

IBM Consulting to help companies get started with generative AI on AWS cloud

OpenAI updates ChatGPT with Bing and DALL-E 3 integrations

Cyber beat

Late-breaking on Friday: Okta shares fall 11% after company says client files were accessed by hackers via its support system (from CNBC)

Stirrings in cyber: Crunchbase data reveals cybersecurity funding picking up pace, but still lagging 2022

An interesting look at a cyber hackathon: A new hackathon is helping to make elections more secure

Bots on the rampage: Barracuda report reveals 50% of all internet traffic is bot-driven

And phishing in QR codes: SlashNext report warns users of the hidden dangers of QR codes

And fake sites with malware in Google ads: KeePass users targeted: Attackers leverage Google Ads for deceptive campaign

GitGuardian debuts new way to manage software encryption keys and secrets

Around the enterprise and beyond

First, a brief commentary on one master of the universe’s latest missive: 

Marc Andreessen this week issued another manifesto — The Techno-Optimist Manifesto — but unlike the brilliant Why Software Is Eating the World 12 years ago, this one lands flat. Basically, the entrepreneur-turned-uber-venture capitalist at Andreessen Horowitz offers a full-throated defense of technology and its benefits, and damn the critics, especially those who advocate embedding a greater measure of trust, safety and accountability into various technologies.

Now, Andreessen, a clear thinker when he stays out of politics, makes some good points. Technology, defined broadly, is the key source of human advancement. It results in greater productivity, the only thing that can over the long haul raise standards of living. Most of us outside the socialist fringe know this, so we’re generally optimistic about the march of technology. And it’s true that some government moves to rein in big tech have been clumsy and counterproductive, though there’s some new thinking on how to do better.

The basic problem here, though, is that Andreessen conflates the views of a kneejerk anti-tech fringe, who are loud enough that they dominate the media landscape, with those of the majority of us who love new technologies (mostly) but have noticed that some aren’t always unalloyed good.

Who could be happy with social media exacerbating societal division and producing a world in which millions of people don’t believe basic facts such as the effectiveness of life-saving vaccines? Or surveillance technologies tracking our every move? Or crypto schemes that seem to make off with millions of dollars of people’s money every other day? Or AI that some reasonable people believe (granted, probably unreasonably) could mean the end of humanity? Should we really demonize the notion that there needs to be some better judgment on how new technologies get used and provide real guardrails to avoid hurting the people their backers profess to be “creating a better world” for?

Andreessen’s slamming of any notion of oversight or caution instead sounds like the whining of a group of elites who take umbrage at any criticism. I mean, these guys run the world, and haul in an outsized and still increasing piece of the economic pie, yet manage to complain about largely nonexistent “central planning” making their lives harder. I can’t put it better than Kara Swisher, who tweeted: “Just stop being so careless and sloppy and arrogant and endlessly exhausting in your need to be licked up and down all day long.”

Speaking of manifestos, here’s a more useful one from Sam Lessin, a sensible guide for seed investors today (email required to view it). The gist: Don’t aim too high, since a higher-interest-rate environment has changed everything.

On to other topics…

Gartner forecasters weigh in with a seemingly optimistic 8% rise, though off this year’s depressed PC spending, maybe it’s reasonable. The firm says gen AI hasn’t been a driver yet and doesn’t seem to think it will be a big factor in 2024, but it could be larger than expected on the data center side in coming years: Gartner: Worldwide IT spending will top $5 trillion in 2024, up 8% over 2023

Multicloud ftw, from an unexpected source: Microsoft launches Radius, an open source application platform for the cloud-native era (from TechCrunch)

Earnings roundup:

SAP tops third-quarter expectations thanks to cloud growth

TSMC posts quarterly profit and revenue declines but tops expectations

Chip equipment supplier ASML shares slide on mixed earnings forecast

The noose tightens: US places new restrictions on AI chip exports to China

Data mesh gets real: Big-data startup dbt Labs rolls out data mesh architecture and enhances semantic layer

More early predictions for 2024: Gartner’s top 10 strategic technology trends for 2024 highlight growing adoption of AI

Crypto shenanigans never seem to end: New York sues the developers of the Gemini Earn crypto lending service

Amazon doubles down on drones: Amazon expands drone deliveries with its pharmacy and

Amazon plans to expand drone deliveries into UK and Italy in late 2024

Layoff watch:

Stack Overflow lets go 28% of its workforce in cost reduction push

LinkedIn lays off 668 staffers to simplify organizational structure

Waymo Cuts Staff in New Round of Layoffs (from The Information) Google and Verily too: Google and Other Alphabet Units Laid Off Staff as Industry Resumes Cost Cuts

Elon flail whale watch: X starts testing Not A Bot, a subscription model that will charge users $1 a year And two premium tiers

What’s next

Supercloud 4! on Oct. 24-25. Our live virtual editorial event on how generative AI is transforming every industry has a great lineup of guests (with more soon to come), along with analysts John Furrier and Dave Vellante, so register now and join us:

Reggie Townsend, VP of data ethics, SAS

David Linthicum, chief cloud strategy officer, Deloitte

Ori Goshen, co-founder and co-CEO, AI21 Labs

Lior Gavish, co-founder and chief technology officer, Monte Carlo Data

June Yang, VP, Cloud AI and Industry Solutions, Google Cloud

Scott Likens, global AI and innovation technology leader, PricewaterhouseCoopers

Jose Pedro Almeida, chief AI strategist in healthcare

Jeff Boudreau, chief AI officer, Dell Technologies

Sridhar Ramaswamy, co-founder, Neeva

Andy Pernsteiner, CTO, VAST Data

Chetna Mahajan, CIO, Amplitude

David Glick, SVP, Enterprise Business Services, Walmart

Aaron Amendolia, deputy CIO, National Football League

Howie Xu, SVP, engineering and AI/ML, Palo Alto Networks

Vijay Mital, corporate VP, AI architecture and strategy, Microsoft

Jayesh Govindarajan, SVP, Salesforce AI

Warren Barkley, cloud AI leader, Google Cloud

Vikram Joshi, CTO, Compute.ai

Joel Inman, CEO, Compute.ai

Vince Kellen, CIO, UC San Diego

Alice Steinglass, EVP and general manager of Salesforce Platform

Krishna Rangasayee, founder and CEO, SiMa.AI

Arun Subramaniyan, VP, cloud and AI, Intel

Swatee Singh, financial services EVP, chief data and AI Officer, TIAA

And a big week is coming for tech earnings, which should give an especially good read into the state of cloud computing as all three of the major cloud providers will report results, not to mention IBM and Intel, which will provide a window into mainstream tech spending: Alphabet/Google, Microsoft, IBM, Meta, Intel, Amazon and more.

Image: Bing Image Creator; photos: Robert Hof