Generative AI: Databricks extends its functional coverage

Training, as a last resort

In this sense, Databricks presented Mosaic Mosaic AI Model Training. Two modules will soon be available: Pretaining and Fine Tuning.

The Pretraining module is used to modify a large part of the weights of a pretrained LLM, more commonly called a foundation model. Databricks supports three pre-training methods: supervised fine-tuning, continuous training, and conversation completion. The first technique aims to “teach” the model new tasks, modify its tone of response or strengthen its ability to follow tasks. The second aims to strengthen the knowledge of the LLM in a specific field from a minimum of 1.5 million documents or samples. The third is to improve your ability to answer questions in an expected format.

This involves reducing pretraining costs tenfold compared to an infrastructure set up by a customer. Databricks cautions that it cannot support datasets larger than 10 trillion tokens, due to the availability of compute instances. “GPUs are expensive, my CFO reminds me every week,” jokes Ali Ghodsi, co-founder and CEO of Databricks. The company claims that more than 200,000 AI models were trained using its platform last year.

“GPUs are expensive, my CFO reminds me every week.”

The fine tuning module promises a no-code interface to simplify the implementation of the LoRA (Low Rank Adaptation of Large Language Models) technique. This consists of modifying a small part of the weightings of an LLM with the aim of specializing it in a field. Here, Databricks intends to propose modifying “open weight” LLMs, including DBRX, several Mistral models, Llama 2 and LLama 3.

If the technique is not flawless, it allows companies to adapt models to their uses at a lower cost.

Databricks spokespersons emphasize, however, that the use of Mosaic AI Model Training is only necessary if the company has already tried a set of prompt engineering techniques, has implemented a RAG (Retrieval Augmented Generation) architecture, or is not satisfied with the speed or cost of LLM inference or wishes to obtain ownership of a “custom” model.

RAG and results control

If the publisher knows that some of its clients want this mastery, it knows that they will also want ways to control the results of LLMs at lower costs. In this sense, he presented Mosaic AI Agent Framework. It brings together several tools to “design, deploy and evaluate” RAG applications.

In its documentation, Databricks details all of the steps necessary for setting up this type of application as well as how a RAG architecture works. Thus, it is possible to ingest structured and unstructured data into Delta tables or volumes. Then, you have to extract the data from PDF files or images, metadata. The editor recommends “cutting” the documents into “selected pieces” that will help refine an LLM’s answer to a question. Then, these “chunks” must be vectorized using an embedding model, before storing them in a Delta table synchronized with the Databricks Vector Search module. It itself indexes and stores the data in a vector database accessible by the RAG application. This synchronization should make it possible to automatically update the index as an embedding is added to the Delta table.

“I am extremely optimistic about the interest of the RAG and I continue to be. I think this technique is here to stay[…]»

“I am extremely optimistic about the interest of the RAG and I continue to be. I think this technique is here to stay, for three reasons,” says Ali Ghodsi during a press briefing. “First, companies want to control their data security, access and role management. Second, you can regularly update your data in your database, which is not possible if you use the basic model. Third, you can avoid hallucinations,” he argues.

Furthermore, Databricks intends to offer what it calls Unity GenAI AI Tools, that is to say a way to record remote functions, SQL, Python, calls to other LLMs, and to allow an agent “gifted with reason”, in short an LLM trained with the “chain of thought” technique of using these tools. They will be recorded as assets controllable from the Unity catalog, the governance layer of the “Data Intelligence Platform”.

Once an application is deployed, the Mosaic AI Agent Evaluation function allows you to evaluate “the quality, latency and cost of an AI application”. Here, Databricks relies on MLFlow and more particularly on the “Evaluate” API which allows these evaluations to be performed.

Developers must first build a “ground truth” dataset before relying on quality, performance and cost metrics.

Regarding the quality of the answers, Databricks offers two ways of evaluating: one based on user feedback and the other using an “LLM as a Judge”.

In the first case, the questions, model responses and user opinions are collected in a Delta table in order to establish statistics. The second case makes it possible to automate this process by relying on an LLM responsible for classifying the quality of the answers or results.

To keep control over these outcomes, Databricks introduced Mosaic AI Gateway, an MLFLow-based API adjoining the Model Serving deployment service. This endpoint should be used to set call limits to models, manage permissions and access, track usage, and set up guardrails (filters).

It’s ‘too early’ to declare a winner in generative AI

If it explains a little more clearly the articulation between the different bricks of its platform, Databricks is at the same stage as its competitors, according to analysts.

Like AWS, Google, Microsoft, Oracle, Snowflake and others, the majority of announcements are in public or private preview.

“It’s a good thing to see these announcements,” says Dave Menninger, analyst at Ventana Research [propriété d’ISG], from Searchdatamanagement, a sister publication of MagIT. “These are certainly steps in the right direction, but businesses need features that are generally available, reliable and backed by guaranteed support.”

“Like Snowflake [lors de sa conférence annuelle], Databricks announces previews,” adds Kevin Petrie, analyst at BARC US. “The real test will be getting to general availability as quickly as possible.”

It would therefore be “too early” to declare a winner in the GenAI arms race, according to Kevin Petrie.

“As GenAI will ultimately be more of a function than a standalone initiative, the winners will be those who can help companies integrate it into their existing systems and applications.”

“As GenAI will ultimately be more of a function than a standalone initiative, the winners will be those who can help companies integrate it into their existing systems and applications,” he suggests. A wish shared by Snowflake and Databricks customers, according to comments collected over the last two weeks by LeMagIT.

Dave Menninger does not see any offers to combine machine learning and generative AI. If that’s not as clear as it could be, that’s one of the goals of features like Unity GenAI Tools and the AI/BI suite.

Ali Ghodsi, for his part, suggests that Databricks and its competitors have time. “I’m very excited about it. Generative artificial intelligence is of course the future, but we’re only on day one in a million. So it’s very early, I think.”

The priority for companies, according to him, lies in security, governance and data engineering.