Supercharge Your Research with Retrieval Augmented Generation (RAG) AI

One of the values of artificial intelligence is quickly sifting through vast amounts of information and distilling down the key points.

The Challenge

ChatGPT and Gemini AI models are trained on the vast array of information available on the internet, which is often too broad for the specific work required by scientists, engineers, attorneys, and journalists.

The Solution

Retrieval Augmented Generation (RAG), a specific AI tool designed for extracting key information from many or large documents, distilling down the information. This is the same AI tool used at the top of many Google searches. You can build your own custom RAG model using Google NotebookLM.

How RAG works

The RAG model converts each document or file type (e.g., PDF, transcript, website) into what is called a vector database. This effectively means you can interact with the documents in a natural and conversational way. This is a massive improvement over the old method of searching for keywords, where a search would fail if you couldn’t think of the exact word, words, or combination.

With RAG AI, keywords are less critical. Even better, RAG distills the knowledge from the relevant documents and writes a summary based on them. The best feature is that each document is referenced throughout the summary.

Testing RAG with Google NotebookLM

One of the tests we conducted involved uploading around 100 journal papers and documents into Google NotebookLM. Some of the documents were older non-searchable PDFs. Despite this, all of the documents were successfully loaded.

With RAG models, queries are best if related to the focus area of the documents.

The test involved finding relevant papers regarding a specific technical topic. Only a few papers discussed this topic, but the researcher had a specific paper in mind but couldn’t remember the file name or author details. The researcher typed in a sentence asking about the key parameters related to their inquiry. The model provided a nice summary and referenced several related papers. Google NotebookLM quickly highlighted the relevant paper sections, allowing the researcher to find the specific section they were looking for.

Using AI can significantly speed up time and improve the quality of work by helping researchers sift through large documents.