1931: Paramount film star Peggy Shannon as a telephone switchboard operator in a scene from the ... [+]
Software interconnects. When we use enterprise computer systems at their most effective level, we are typically connecting applications with other applications, other data services, other operating systems and application components. If you’ve ever noticed the Internet, you’ll know why interconnectivity matters a great deal when working with an app, whether on a desktop or via a smartphone.
Software apps themselves often use a now fairly industry-standard approach to interconnectivity known as an Application Programming Interface (API). It’s a way of creating a bond and channel (some would say glue) between applications in a prescribed way (defined according to an agreed syntax) that enables software engineers to connect all the plugs they need to - not completely unlike the stereotyped image of the old-fashioned female telephone operative plugging in cables at an exchange.
Real-time vector database company DataStax has been working to plug in all the right cables with regard to its approach to enable generative AI applications. The company’s new ‘Data API’ (formerly known as the JSON API, the acronym denoting JavaScript Object Notion, an open standard file format and data interchange format) this month. The technology is intended to serve as a one-stop API for Retrieval Augmented Generation (RAG) development, that provides all the data and a complete RAG stack for the production of generative AI apps with high relevancy and low latency.
What is Retrieval Augmented Generation (RAG)?
Key to building generative AI software application development projects that have an essential connection to outside sources of information for essential additional relevance, timeliness and accuracy, RAG extends the realm and scope of so-called Large Language Model (LLM) responses by connecting LLMs to external sources of validated, certified or at least recognized knowledge data. This approach to creating an AI supplement was initially coined, proposed and documented by Facebook parent Meta; in the world of unbiased non-hallucinating generative AI, RAG is lord and lady of the manor these days.
Also debuting with its Data API product is an updated developer experience for Astra DB, the company’s vector database for building production-level AI applications.
“Astra DB is ideal for JavaScript and Python developers, simplifying vector search and large-scale data management, putting the power of Apache Cassandra behind a user-friendly but powerful API,” said Ed Anuff, chief product officer, DataStax. “This release redefines how software engineers build generative AI applications, offering a streamlined interface that simplifies and accelerates the development process for AI engineers.”
Looking at how the product works in more detail, Anuff explains that the new vector Data API and experience make the petabyte-scale power of the open source Apache Cassandra database available to JavaScript and Python developers in a more intuitive experience for AI development. Putting ease of use at the fore, DataStax says that we can now enjoy 20% higher relevancy, 9x higher throughput and up to 74x faster response times than other vector databases by using the JVector search engine. In a bid to help streamline the development process, the technology now introduces a dashboard, more efficient data loading (because data ingestion is always a headache) sa well as data exploration tools and integration with other AI and machine learning (ML) frameworks.
An out-of-the-box AI API
“Software application developers can use the Data API for an out-of-the-box AI ecosystem that simplifies integrations with major generative AI partners like LangChain, OpenAI, Vercel, Google’s Vertex AI, AWS, Azure and major platforms while supporting the breadth of security and compliance standards,” confirmed Anuff and team. “ Any developer can now support advanced RAG techniques such as FLARE [Forward Looking Active REtrieval Augmented Generation]and ReAct that must synthesize multiple responses, while still hitting latency SLAs.
Looking ahead, we’re talking about generative AI with RAG that is capable of doing more and being more accurate. We’re also talking about these additional techniques layered onto RAG that make it more capable of knowing ‘when’ and ‘if’ to look for more information on a future-facing basis. So, in other words, we’re talking about making AI intelligence more intelligent.
From what we see happening at DataStax - and the user experience factor is nice enough, but let’s look rather deeper than that - this is the point where we start to take generative AI applications forward with the supporting vector databases that they rely on as lifeblood and making that eminently flexible in real-time data deployments. Astra DB experience can instantly query real-time data updates for both vector and non-vector data, so let’s remember where we came from too.
Will the AI API telephone exchange now work seamlessly (that word that technology vendors love to use) and deliver perfect connections every time with no crossed lines and crystal clear reception from start to finish? Only a fool would bet on that level of perfection, even with AI in the mix… hold on please caller.
