Dataiku Defines Development Routes To User-Centric AI

AI is smart, but also dumb. We know that Artificial Intelligence is intelligently smart, the clue is in the name. Equally, most of us have a fairly rudimentary understanding of the philosophy of garbage-in garbage-out, so we know that an AI engine fed with Large Language Model (LLM) data that isn’t fit for purpose will (in the most basic terms) probably not know what it is talking about. When AI is clueless or misled, then we start getting so-called AI hallucinations where the machines tell us things that - from their perspective - appear to be plausible, or cast-in-stone truths that are wholly authentic.

A valuable lesson here is design thinking, or more specifically end user critical thinking i.e. considering how an AI service will be used by businesspeople and consumers and so constructing and feeding it with the ingredients it needs to suit the exact recipe.

As we try and break down the mystery surrounding AI development, let’s just think of an AI service’s language model as a tool. We need to keep this tool sharp, maintained and well-protected, but most of all we need to give the right person the right tool for the right job at the right time. For the software developers and data scientists building AI, that means being able to present the resulting information from AI in ways that are consumable and clear to users so they can validate the output critically to decide how to best use the information.

Promoting end user critical thinking

“Let’s start with putting the effort into thinking about how people actually use what we produce. When it comes to data science and AI, user interaction is all too often overlooked in the process, so we must first look at tactical changes developers can make to support and encourage the end user's critical reasoning,” explained Conor Jensen, global field chief data officer at Dataiku. “Most of the products we’re creating with LLMs are adapting an existing process or workflow that people already use. As data scientists, if we don’t understand what that process or workflow looks like, we’re not going to be able to design something that is effective for our users. It’s not enough to define the problem and the scope: We’ve got to start sitting next to our users (ideally literally) and discover how they are working so that we can better understand how they’ll use our products.”

Dataiku is known for its all-in-one data science platform used by both data scientists and business analysts. It allows users to create custom applications to automate data preparation, pipelines, statistical analysis, or model development. Jensen takes his experience of working in this environment to say that this type of user-centric engagement and interaction can help us understand which questions to ask in order to develop generative AI-based products that are effective.

Building on this, developers need to present the information delivered by the AI in a way that encourages the users to interact with, validate and understand it. As we begin to understand our users a little better and learn to present information to them in a way that is appropriately consumable, the argument here is that the software engineering team will begin to develop products that actually help people in their everyday flow of work. If we fail to promote end user critical thinking, the issues we’ll see with LLMs (and indeed the AI services we build from them) will be lack of adoption, extended development times, products that miss the mark entirely and potential misuse.

Creating more ‘consumable’ AI

The next question must therefore be, how can software application developers make sure that they are creating output that is more consumable and more understandable for end users?

“Firstly, developers should be looking to design LLM output so that the LLM is integrated with other models to present its results,” specified Jensen. “For example, we all know LLMs aren’t strong at supporting a precise prediction, like a demand forecast. However, you can utilize an LLM that calls on the output of another model which is focused on providing that prediction. Then the LLM can augment the prediction with additional information, explanations, or charts to help a user interpret and use the results. By combining LLMs with other data and other models, you’re not just getting a blind answer: you’re getting an answer with an explanation that you can go and validate to ensure it’s correct.”

This becomes a bit of a cultural change in the way that we can encourage people to consume information. He notes that ultimately, we want users to consume LLM output in an informed way, so we need to give them the resources to do that. We need to start looking at more ways to augment LLMs to interact with other models so that it is very clear where answers come from.

Secondly says Jensen, we need to encourage models to talk to each other in order to evaluate and validate their responses. The most common Jensen and team are seeing software developers do this is by setting up a modified version of a Generative Adversarial Network (GAN).

As explained here by TechTarget, “A GAN is a machine learning model in which two neural networks compete with each other by using deep learning methods to become more accurate in their predictions. GANs typically run unsupervised and use a cooperative zero-sum game framework to learn, where one person's gain equals another person's loss.” Google Bard has taken a step towards this with a second LLM validating the data and answers given and highlighting where the user can be more or less certain of its response.

Empowering users to investigate output

“So far, we’ve put a lot of the onus on software developers to make better, more consumable products with LLMs, but users also need to meet us halfway. We need users to question output rather than consuming it blindly… and so, over time, users should create guides for consuming LLM product output,” notes Dataiku’s Jensen. “One of the best ways we can encourage users to be more inquisitive is to make improvements to our user interfaces so that it’s more obvious which parts of an output should be investigated. To accomplish this, we’re going to have to involve User eXperience (UX) and User Interface (UI) people more in the generative AI product design process as, more often than not, data scientists aren’t experts when it comes to user journeys.”

As users become more used to investigating LLM output, they will learn what certain models are good at, this way they can begin to document and flag what to watch out for amid a ‘normal’ range of answers. This user feedback will help developers and LLMs improve future outputs. It’s also worth considering the various mediums when it comes to LLM output. The Dataiku team is working on this effort by incorporating more visuals in output, as people may engage with visual output differently and may assess it more critically than text rather than just assuming it’s true.

“By working together, developers and users have a unique opportunity in LLMs. Over time we’ll be able to put more faith in models and they will likely become more accurate but, for now, we need to put in the effort together to proactively guide their true potential,” clarified Jensen.

Usability for users

It seems almost tautological to suggest that we really should be thinking about how much user usability we put into our AI product and service design. After all, it’s the users who will be using these technologies; although a degree of machine-to-machine adoption is required (especially perhaps inside the Internet of Things), at the end of the way AI is for us humans.

Jensen’s call for end user critical thinking is arguably quite refreshing i.e. we’ve spent most of the last 18-months talking about AI-enriched software brains and then skipping straight to their purported benefits and results. We’ve spent far fewer column inches worrying about user touchpoints, user feedback and user validation. As with all well-balanced things, surely it’s time for some AI yin-yang to give us a new intelligence equilibrium.