TOPSHOT - A reflection on the helmet of a mechanic staff shows the team during practice of change of ... [+]
Intelligence is proliferating. As the hurly-burly surrounding development of generative AI (gen-AI) with its use of open Large Language Model (LLM) technologies designed to create ever more human-like machine intelligence builds further, much of the focus is on us - the users - and what we’re supposed to be able to do with it. With Natural Language Processing (NLP) interfaces enabling us to speak and type questions of our new AI engines, everyone wants to know whose job is going to be replaced or radically uprooted next.
While the majority of peoples’ jobs will (arguably) stay relatively stable but experience some welcome machine acceleration and automation support, what really matters now (arguably) is how we are going to build these new functions.
Hype first, skills second?
As the cartoon reminds us, it’s first pants and then shoes. Remember early cloud computing when it was all services and software and not enough security? Remember open source when it was a cute hobbyist pursuit for geeks until it ran the world? We might need to remember the gen-AI hype-cycle that played out this year and think about how we align the software tools, data procedures and system processes required to make it happen effectively, securely and responsibly.
Real-time AI and data platform company DataStax has big plans for getting things done right in this arena. The company has now laid down a new JSON API for its Astra DB database-as-a-service built on open source Apache Cassandra. Just to clarify here, JavaScript Object Notation (JSON, pronounced like the name Jason) is a text-based human-readable data interchange format used to exchange data - and, an Application Programming Interface (API) is a glueing & connecting mechanism used to join or provide access channels between two or more applications, data services, software components or operating systems. As we build the vector databases to serve language models behind gen-AI, DataStax says that for JavaScript developers building AI applications, the fastest way to create AI applications with vector search capabilities is to use the new Astra DB JSON API.
Available via the open source data API gateway, Stargate, the new JSON API lets JavaScript developers use Astra DB as a vector database for their Large Language Model (LLM), AI assistant and real-time generative AI projects. It provides an integrated way to work with Astra DB as a document database and now has compatibility with Mongoose, a popular open source object data modeling library for MongoDB. This makes it simple for JavaScript developers – the largest community of developers in the world – to build generative AI applications with vector search, using the library they know and prefer.
“There is increasing demand for databases to wear multiple hats in serving the enterprise, involving both broader operational capability and better analytics,” said Carl Olofson, research vice president for data management software, IDC. “Such enhancements include easy and practical support for vector search in support of a range of generative AI functions - and the need to incorporate those functions in both operational and analytical contexts. The flexibility of Cassandra lends itself to such enhancements, and this new JSON API for Astra DB is right in line with the emerging needs of DataStax developers.”
Reduced ‘hallucinations’ via concurrency
The Astra DB vector database is designed for building real-world, production-level AI applications with real-time data. With simultaneous search and update on distributed data and streaming workloads, Astra DB is said to offer ultra-low latency with relevant results that eliminate redundancies. All of which enable us to talk about the tools and toolsets needed to build responsive, accurate production gen-AI applications.
When we do this well, we get good generative AI. When we do it really well, we get gen-AI that more clearly appears to ‘know’ what it is doing. How would we know? Because this is the type of AI that works with fewer ‘hallucinations’ i.e. sometimes known as confabulations, AI hallucinations are occurrences when an AI engine or model delivers a response, calculation, prediction or other interaction with what it thinks is a high-confidence rating, but is in fact not apparently making use of its training data appropriately - either because it suffers from ‘overfitting’ where its training dataset is too small, its is using low-quality outdated data, its data is littered with too many human idioms or the AI model itself has been maliciously attacked in some way.
DataStax says that there are fewer hallucinations in this instance because the AI can draw from real-time data updates and increase responsiveness with concurrent queries and updates.
View from the CEO
"Thinking about the practical real-world use of the new technology fabrics we are creating with generative AI, people often talk about job losses. If we start from the top - when a company gets more productive, it generally tends to hire more people - if you are not being productive, you look for efficiencies - but the general trend is to hire more when additional productivity happens [and that's what AI brings], not reduce the number of staff. I always say that the competition is never AI vs people - it’s people with AI against people without the benefit and support of AI,” said Chet Kapoor, chairman & CEO, DataStax.
Kapoor reminds us that every technology wave has worked through a period of adolescence where first there are ‘incremental’ use cases that focus on comparatively smaller efficiencies. He says that the real transformative use cases happen (for example, mobile apps started with games, news pages and entertainment, but now we have really useful online banking apps) when we have apps and services that actually impact a company’s balance sheet.
“Think client-server, the Internet, then mobile, then cloud and now gen-AI - these trends have all played out with one building on top of the last (apart from client-server, which was a real reinvention and refactoring after the mainframe), so each one has happened faster than the last, which is a lot of the reason why gen-AI has burgeoned so quickly inside this year,” said Kapoor.
In terms of how quickly an enterprise will be able to adopt gen-AI, for Kapoor, this factor is both governed and throttled by a) whether the outputs of the implementation are regulated (if a bank chatbot gives the wrong advice, the bank is liable) and b) how that adoption will impact a company’s profits and losses.
“Think about how brands are perceived at this level - if you become the first bank (or for that matter a player in any major industrial space) that can create a revenue-generating secured governed service - then that will be the next big thing that we are all waiting for. So in this space, we must consider the role of Retrieval-Augmented Generation (RAG)," explained Kapoor.
For clarification then, originating in a paper published by Facebook parent Meta, RAG can be described as an AI framework built to refine and improve Large Language Model (LLM) responses in terms of their consistency and quality by connecting the AI model to external sources of ratified knowledge data. This process works to add to, clarify and supplement an LLM's internal categorization, classification and subsequent representation of data.
Rounding out, CEO Kapoor says that technologists have had some hard times when they didn’t always get share of voice. But now, and with generative AI fuelling so much new interest, developers are put back at the forefront, which helps ensure the CIO and CTO get a spot at the boardroom table. Crucially though, the CIO will turn to the developer first rather than the data scientist to get things done now - and that developer will (most likely say) ‘if you want me to move fast’ get compliance to help me do what we need to do’ and that’s how this whole story is playing out now.
Back to Astra
Traditionally, in order for developers to use Astra DB they had to have familiarity with CQL, a powerful but sometimes intimidating programming language. But according to Ed Anuff, in his role as chief product officer at DataStax, with the introduction of the JSON API, the company is democratizing access to Astra DB’s capabilities, making it more intuitive and accessible for the more than 13 million global JavaScript developers.
"With Astra DB’s new vector search capabilities, virtually any developer can now use Astra DB to build powerful generative AI applications using real-time data," said Anuff.
With the JSON API, JavaScript developers are no longer required to have a deep understanding of the above-referenced Cassandra Query Language (CQL) to work with Astra DB and Cassandra. Instead, they can continue to write in the language they’re familiar with to quickly develop AI applications – a necessity in current modern business environments.
Future thoughts & developments
DataStax chief product officer Ed Anuff and DataStax VP of developer relations Patrick McFadin are jointly realistic about what’s happening right now in the generative AI space. Speaking to press and analysts in informal closed sessions this month, the pair noted just how much of a maelstrom it might seem at the higher-level, but crucially, both men agree that (it could take a year, it could take five years) this technology will fundamentally and absolutely change the way we compute and process data at the back-end, which will of course further revolutionize the way users use applications.
We said at the start that much of the focus is still on the users and the apps; people want to know which people’s jobs in which call centre are going to be rendered redundant. DataStax isn’t focused on that apparent hysteria, the company is working to put tools in the hands of JavaScript developers building AI applications with vector search capabilities via its Astra DB JSON API and enabling them to use proprietary data to power an LLM with retrieval augmented generation.
The technology proposition here perhaps being, if we enable software engineers to build powerful clever apps in a robust and secure way - all without confabulations & hallucinations - then we can more carefully repurpose people where their job is being automated and give them higher value roles.
That might be the view through rose-colored glasses, but it’s for sure that a more chaotic path to generative AI will cause more turbulence. Even so, fasten your seatbelts.
