Databricks has for years offered a secure place where enterprises can host their data (and run analytics on that data). In 2023, it acquired the generative AI company MosaicML, with the idea of helping its customers create customized AI models, which they could run in the same cloud that's already hosting their data.

In March, Databricks rolled out one of the first fruits of its MosaicML buy: a new LLM called DBRX. With DBRX, Databricks can offer its some-12,000 customers a secure cloud where they can also expose their data to advanced AI models, which, the company argues, helps avoid the security risks of sending proprietary data out through an API to an AI model hosted by another company. In some regulated industries such as finance and healthcare, avoiding that security risk is a major selling point. This is part of the secret of Databricks's success.

DBRX, which is available as open source, isn't as capable as such state-of-the-art models as Google's Gemini or OpenAI's GPT-4 but, as Databricks CEO Ali Ghodsi said during a press gathering in March, many enterprises don't require gigantic models for the kinds of applications they're looking to carry out. A financial institution, for example, might be able to use DBRX to look for signs of fraud among its databases of numbers, and a healthcare organization could use the AI to look for patterns of disease across thousands of electronic patient records.

Many of today's LLMs expend too much energy to tackle simple problems, which both uses up compute power and slows delivery of an answer to a user. DBRX addresses this issue by using a "mixture-of-experts" design that divides up the model's brain into 16 specialized "experts." When a specific type of calculation is requested, a "router" inside the model knows which "expert" to call on. The whole DBRX model contains 132 billion parameters, but because of that division of labor, it uses only 36 billion parameters at any given time, Ghodsi explained. For businesses that want to use AI for day-to-day operations, this style of LLM architecture could lower the barrier to entry.