The AI revolution unfolds: Outlining AI infrastructure scaling and complexity challenges

The generative artificial intelligence surge is reshaping the future of infrastructure, and companies such as Anyscale Inc. are at the helm. Anyscale Ray has been designed to simplify the entire process of scaling AI infrastructure at the enterprise level.

"There's going to be a massive infrastructure build-out for AI with any major technology, and with any infrastructure build-out, there's a hardware piece and a software piece," said Robert Nishihara (pictured, left), co-founder of Anyscale. "On the hardware side, we all know how successful Nvidia is. But the software piece, there's a lot of work to do. There's a lot of complexity to rein in that's growing in AI, and that's a lot of what we're trying to do."

Nishihara and Keerti Melkote (right), chief executive officer of Anyscale, spoke with theCUBE Research's John Furrier at the Anyscale Ray Summit 2024 event, during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. They discussed Anyscale Ray as a flexible and scalable AI compute engine capable of managing diverse workloads across multiple platforms. (* Disclosure below.)

A core challenge facing AI adoption today is the growing complexity of the infrastructure required to support it. Companies face fragmentation across various tools, cloud platforms, accelerators and machine learning models. While empowering, this diversity also slows down AI development. Companies require unified solutions that streamline classical machine learning and gen AI workloads, according to Nishihara.

"A lot of the people we work with are the AI platform engineers, the infrastructure engineers, the people who provide AI capabilities throughout their company and enable all the other teams to move quickly," he said. "Those are some of the main Ray users. And these people have been supporting AI workloads for a long time. They went through a large migration to enable deep learning beyond just predictive models; now they're enabling generative AI."

Anyscale sees the opportunity to democratize AI development through its managed services. By offering a unified platform that simplifies AI infrastructure, it aims to empower enterprises that may not have the in-house expertise to build AI platforms from scratch. Many organizations lack the resources to create proprietary platforms, and Anyscale's managed services help bridge that gap, according to Melkote.

"The real enterprise challenge is how do you take that expertise and put it out there?" he said. "What we see as the commercialization opportunity is to take that, offer it to the managed service and wrap it around professional services so customers can take their business problem and come up with a real solution as opposed to just tinkering with tools."

As microservices and Kubernetes have become standardized in modern software development, there is a growing need for AI platforms to integrate seamlessly into these environments. Anyscale enables its managed Ray clusters to run on Kubernetes, aligning with the needs of both data scientists and DevOps teams, Nishihara added.

"One of the things we announced earlier in the keynote today is the ability to deploy Anyscale and bring Anyscale-managed and optimized Ray clusters to any Kubernetes cluster," he said. "If you have a Kubernetes cluster, we can run on it. That really caters to this platform persona."

Here's the complete video interview, part of SiliconANGLE's and theCUBE Research's coverage of the Anyscale Ray Summit 2024 event: