GigaOm Sonar for Vector Databasesv2.0

1. Executive Summary

Vector databases store large volumes of unstructured data, making that data highly queryable and available for contextual use in AI applications. By storing the unstructured data in numerical vector format, these databases can then index that data and perform similarity searches over it, relative to vector representations of other content, or natural language queries. Vector databases thus enable organizations to represent any data—including the enormous amount of otherwise “dark” data they often possess—as vector embeddings, making it available for question answering, semantic search, retrieval augmented generation (RAG), and other applications of generative AI.

This approach allows vector similarity search engines to ascertain similarities between data of any structure variation, from almost any source, utilizing any data model or schema, to find answers and draw conclusions that other databases can’t.

This technology serves as the backbone to robust applications of generative AI, as it helps organizations leverage “foundation” models and other large language models (LLMs) and contextualize them within an organization’s industry, business model, and data. This is particularly the case in the context of the RAG concept, making LLM responses more tailored, with far fewer “hallucinations.” As such, it’s the bridge between an organization’s data, documents, and policies on the one hand and natural language queries and responses on the other.

These natural language interactions are swiftly becoming the de facto means of interfacing with data assets, as well as drastically lowering the skills required to query and understand data for any specific business purpose. Thus, the burgeoning target audience for these vector database computing platforms includes all enterprise personas, from the C-suite to business end-users. Moreover, a vector database’s utility expands to almost any enterprise application via the access they afford organizations to their data.

Despite some prominent use cases, the adoption of vector databases and similarity search engines is still in its nascent stage. The capabilities found in these computing platforms are undergoing a rapid shift, and the heretofore specialized space is encountering consolidation with other more mainstream, general-purpose databases. Vector database technology is now incorporated into a number of different solutions for everything from process automation to streaming data platforms and relational databases. Consequently, there’s impressive diversity found among the vendors, from open source to closed source offerings and from pure-play vector database platforms to other types of databases that have been enhanced with new vector capabilities.

In fact, vector similarity search functionality may be in the process of being incorporated into virtually all database platforms for storing and retrieving unstructured data. Certainly, we expect to see a greater understanding of vector technology and expectations for it by vendors in this year’s report as compared to last year. This development is reflected in the increased availability of lexical and hybrid search in addition to vector search and accommodations for multimodal search across text, image, and even video and audio content. As the market continues to mature, the demand for these capabilities will almost certainly increase.

This is the second year that GigaOm has reported on the vector database space in the context of our Sonar reports. This report builds on our previous analysis and considers how the market has evolved over the last year.

This GigaOm Sonar report provides an overview of the market’s vendors and their available offerings, outlines the key characteristics that prospective buyers should consider when evaluating solutions, and equips IT decision-makers with the information they need to select the best solution for their business and use case requirements.