Ars in San Jose recap: Infrastructure, sustainability, AI, cocktails

Last week, Ars Technica Editor-in-Chief Ken Fisher and I made the westerly trek to sunny San Jose, California, to kick off an event titled “Beyond the Buzz: An Infrastructure Future with GenAI and What Comes Next,” hosted in partnership with IBM. It was awesome to get to stand up on stage and talk to a room packed full of interested Ars readers, and for everyone who was able to come, thank you for being there! (For everyone who wasn’t able to come, that’s okay—we’re doing another event next month in Washington, DC. I’ll have more info about that at the end of this piece.)

The San Jose event was hosted at the Computer History Museum, which, as venues go, was absolutely on-brand and appropriate—and Ars would like to extend its thanks to the folks at CHM for being so kind and accommodating to our gathering of geeks.

“Our lineup of speakers and topics today reflects the complexity and rapid evolution of the tech landscape we all operate in,” noted Fisher in his opening remarks on the program. “We will be discussing not only the promise of generative AI, but also the challenges it brings in terms of infrastructure demands, security vulnerabilities, and environmental impacts.”

The panels

To Ken’s point, our first panel was on the environmental impact of ever-expanding data centers (and, often concomitantly, the AI services they’re providing). We spoke with Jeff Ball, scholar-in-residence of the Steyer-Taylor Center for Energy Policy & Finance at Stanford University; Joanna Wong, solutions architect for AI & Storage at IBM; and Ars’ own Senior Science Editor Dr. John Timmer.

One of the main points from the panel that I hadn’t fully grokked before but that made absolute sense after having it explained was Ball’s contention that “not all power is created equally”—that is, when looking at cloud resources as a way to shift environmental costs to a third party, the actual physical location of those cloud resources can have a tremendous effect on carbon footprint. The cost of utilizing a data center in Iceland and a data center in China may be roughly similar, but there’s a significant chance that the data center in China will be using coal power, while the Icelandic data center is likely on geothermal.

IBM’s Joanna Wong also noted that infrastructure is often plagued by unknown failure points—that is, problems that aren’t critical enough to cause failure, but nonetheless consume extra compute (and thus energy). Wong said that we should always be on the lookout for these points of failure. While we can worry about the energy costs of new technologies, we must be mindful that we’re probably already wasting resources and harming performance by not understanding our failure points or even our bottlenecks.

We then shifted to the ever-evolving land of security vulnerabilities and AI-generated (or at least AI-audited) code. For this one, I was joined by Stephen Goldschmidt, Global Platform Security Architect at Box; Patrick Gould, director of the Cyber & Telecom Portfolio for the Defense Innovation Unit of the Department of Defense; and Ram Parasuraman, executive director for Data & Resiliency at IBM.

This has been a contentious topic before, and as recently as our Ars Frontiers virtual conference in 2023, security experts have expressed unease at the idea of AI-generated code, given most LLMs’ habit of wildly confabulating things at the drop of a hat. But per our panelists, the most appropriate role for generative AI in coding is likely going to be augmenting human coding rather than replacing it—with AI helping to spot vulnerability-inducing typos in code, pushing the metaphorical broom behind a human coder and cleaning up mistakes. We’re still a long way off from trusting fully AI-generated code in production (unless you’re crazy or careless), but AI-vetted code? That future is here. Parasuraman put it best: “The question of how to trust AI output will never go away. What will change is the ways we verify and monitor that output.”

Finally, our closing panel was on “playing the infrastructure long game”—that is, planning one’s infrastructure to anticipate unanticipated problems. With me was Ashwin Ballal, chief information officer at Freshworks; Karun Channa, director of Product AI at Roblox; and Pete Bray, Global Product executive at IBM. It’s difficult to answer the question, “How do you anticipate unanticipated problems,” but with panelists running the gamut from cloud-native to hybrid with a heavy on-prem data center presence, they gave it a shot.

Perhaps unsurprisingly, the answer is a combination of smart requirements gathering, resiliency, and flexibility. Getting your hands firmly around your requirements is the inevitable first step; if your requirements planning goes well, then building a resilient infrastructure flows from that. If your infrastructure is resilient—and, most importantly, if you have some emergency operational money held in reserve—you should have flexibility in your infrastructure to respond to unexpected demand spikes (or at least the ability to temporarily throw some money at the load until the problem goes away). It’s not rocket science—and heck, even at companies that are doing actual rocket science, good requirements planning always wins the day.