Will my dream tool for search come true?
First version of "GPT-Interrogation Assistant" in the making
I've been dreaming of a search tool for my own research for years. I'm looking for a tool that helps me analyze my research effectively and discover meaningful things.
Since 2004, I have hoped that Google Desktop would do it for me. However, it was killed after a few years. I hoped for the best with Copernic Search and X1 Search - but neither program evolved into an intelligent agent.
Now GPT models, such as GPT-3.5, are capable of a wide range of language-related tasks, including answering questions, providing explanations, generating text, translating languages, and more.
GPT can help me formulate search queries or provide guidance on how to analyze my own data, but I have to copy and paste it from the files on my HD. Wouldn’t it be great that all my files are indexed automatically and fed into GPT?
Let’s call it GPT-Interrogation Assistant. Here’s one of the tasks that I would throw at it. Say I want to find a connection between banjo player Gwennethand Pyrtyr and Joe Biden in files on my computer, and summarize them:
So why would this be helpful? Without the help of a LLM, I have to uncover all the files containing mentions of a world-famous banjo player Gwennethand Pyrtyr and Joe Biden in the same paragraph manually:
I need to perform a search within my data files
I need to look for files that include both "Gwennethand Pyrtyr" and "Joe Biden" within the same paragraph.
Once I have identified the relevant files, I need to examine the content of each paragraph to find the specific information I am looking for.
I have to create a summary of the paragraphs with the mentioned individuals.
My dream tool should be able to search through a lot of data quickly, so I can find the insights I'm looking for. Using this tool, I can assess my research quickly and accurately, gain valuable insights, and answer my questions. It could also help if forgot the exact name of something, because it’s great in language related search.
So where is that tool? How do I get it? I asked some NLP and AI students if they would be open for this idea, especially because there are already some indexing programs out there. Andriy Mulyar came to the rescue. I asked him about my idea and he answered “great idea” - give me a few days.
Mulyar is author of GPT4All, a free-to-use, locally running, privacy-aware chatbot. No GPU or internet required. It’s for download right now, for Windows, Mac and Ubuntu.
Mulyar dropped out of his Ph.D. program to start a company called Nomic, which focuses on building tools for humans to interact with large-scale datasets from sources like Twitter and generated images. He believes that the key to building effective AI systems lies in having access to larger and cleaner datasets. Before starting Nomic, he spent a year working on generative AI products for radiology at Rad AI.
If a guy like him says “great idea”, I am hopeful. Will keep you posted.
Below: instructions how you install a very early beta on a MacBook. With that, you can ask your own PDF’s questions.
Keep reading with a 7-day free trial
Subscribe to Digital Digging with Henk van Ess to keep reading this post and get 7 days of free access to the full post archives.