MEMBER INSIGHT
ABC Assist: Designing an AI application for responsibility in the UX, not just the outputs
11th September 2024
The ABC’s Digital Product team developed ABC Assist, a powerful AI tool that helps users to quickly locate information across ABC’s document archives, improving research process for the public broadcaster’s employees.
This insight was originally published by the ABC and is republished with permission.
By Michael Collett, journalist and producer at the Australian Broadcasting Corporation
One of the most promising abilities of AI technology is its ability to make it quicker and easier to find the information you need to do your job.
For instance, rather than having to manually search through a large number of documents to find information that you know is in there but are struggling to collate, wouldn’t it give you a huge head start if you could just type a question like “What have we said about X?” and get an AI-generated summary back that then allows for quick and easy digging into the original documents?
Read more: PMA runs workshop on Responsible AI & Public Media
In this blog post, I’ll talk about how ABC Digital Product’s AI/ML team have used semantic search in combination with large language models to design one of these systems at the ABC with an eye to quality – not simply when it comes to leveraging data science to improve the AI-generated outputs, but also a UX that encourages responsible usage and human oversight of this powerful but fallible technology.
ABC Assist and retrieval-augmented generation
The information retrieval product that we have created, ABC Assist, is designed to be adaptable but was built in the first instance for a specific internal use case: to save time for members of the ABC Corporate Strategy team who told us about their difficulties quickly finding information in their digital archive of ABC documents, from annual reports to five-year plans and Senate Estimates transcripts to think pieces.
To create this product, we leveraged the huge amount of research and development that’s currently being carried out in the new field of retrieval-augmented generation, while building on this work and adapting it to our specific needs.
Our data scientist Ariel Kuperman will be providing a deeper dive into our process on this in the near future on this blog, but in a nutshell, retrieval-augmented generation introduces a step to the process of getting an answer to a question from a large language model.
When you ask a question to a large language model that doesn’t use retrieval-augmented generation, the answer you get back will be based on the information in the LLM’s training data.
There are a few key limitations with this:
- Large language models only have knowledge up to a certain point in time (at the time of writing, ChatGPT did not have knowledge about anything after October 2023);
- Large language models only have information that is in their training data, which doesn’t include the private or organisation-specific information that is needed to answer many types of questions;
- Large language models aren’t able to provide primary sources for information they got from their training data.
So how does retrieval-augmented generation get around these limitations?
It introduces a search step. This can involve searching the internet as a whole, or a specific website, or a user’s own documents or emails, but the intent is the same: instead of generating an answer from scratch, the LLM is asked to generate an answer using the information that was retrieved during the search process.
In the case of ABC Assist, we hooked it up to a large archive of ABC documents so that it could search for relevant information to use in its responses.
This process involves another machine learning innovation that’s less heralded than large language models but extremely useful: semantic search.
This allows ABC Assist to find passages of text that have similar semantic meanings to the user’s search phrase, even when they don’t actually include the same words.
For instance, using semantic search, a user’s query for information about “electric vehicles” will still turn up passages of text that only refer to “EVs”, making it much easier to search effectively without leaving relevant information on the table.
The UX challenge that we faced
Retrieval-augmented generation delivers great results. For many use cases, including ours, it’s the difference between an LLM being able to answer a question and not being able to.
But in our use case, the challenge isn’t simply giving users good information. The challenge is giving them information that they can know is good and that they can actually use.
Otherwise, what are our users meant to do with an answer they get?
It’s the classic problem with AI at the moment. Even if a response looks good on the surface, it can be hard to know if it’s accurate or not. Are you just meant to take an answer at face value and accept it as gospel?
For our use case, that was never an option. For one thing, the technology isn’t reliable enough for that.
ABC Assist is capable of giving great answers, which is a testament to the huge amount of effort that has gone into optimising the data science and the UX behind the search process (to make sure it has the right material to work with when answering a question) and generation process (to make sure it has the right instructions for creating high quality responses).
Nevertheless, the problems that can creep into ABC Assist’s responses go beyond the well-documented and unsolved problem of ‘hallucination’; for instance, a response might contain an unjustified conclusion, or include misleadingly conflated information, or incorporate non-ABC information from the LLM’s own training data, or be missing a key fact from the retrieved information.
It’s essential, then, that we make it as quick and easy as possible for users to get to the actual source material that can be trusted, so they can see for themselves if the information in an answer is accurate or not.
There’s another reason for this: our product was designed to assist users, not do their jobs for them. The promise of ABC Assist wasn’t that you could ask it a question and just unthinkingly copy and paste its response into an email, or a report, or a presentation.
As our users have told us repeatedly, the value of ABC Assist for them is as a search tool; in that sense, the answers that it provides are simply gateways to information, and users need to remain in control of how this information then gets used.
So providing users with not just the information they need, but the original context of that information and a way to dig deeper, is key.
How we designed with this UX challenge in mind
Listen toour podcast
Uncovering and exploring the biggest
issues facing public media
It was identified early during development that one of the keys to meeting the needs of our users was including citations in ABC Assist’s responses so that they could see where information came from.
We then made these citations clickable so that user’s could instantly see the precise passage of text from the corpus of documents that ABC Assist used when generating the answer.
Finally, as well as providing citations, we also allowed users to see all of the information that was retrieved during the search process, even information that wasn’t used in the AI-generated summary, so that they could bypass these summaries entirely if they wanted to go straight to the original documents instead.
In combination, these elements have given users a way of quickly verifying the information in ABC Assist’s responses and digging deeper should they need to.
“Not only is sticking to ABC information important for fulfilling ABC Assist’s role as a search tool, but faithfulness to the retrieved information – all of it produced by the ABC and of a high standard – is also our best defence against not just inaccuracy but other potential AI pitfalls like toxicity and bias.”
Users’ ability to dig further and narrow in on exactly what they’re after is also aided by their ability to ask follow-up questions.
It required a fair amount of backend work to facilitate this, but the ability of LLMs to parse language meant ABC Assist was ultimately able to support follow-up questions that were as natural as you might hear in an actual conversation: “Sorry, that’s not quite what I was after… is there anything about this in our Senate Estimates transcripts?”
Our emphasis on ABC Assist as a search-focussed tool that can be used responsibly also informed our evaluation strategy.
For many retrieval-augmented generation products, the goal is simply to provide as accurate an answer to a question as possible.
This might seem like it’s just stating the obvious – and it might seem counterintuitive to hear that this wasn’t the aim for ABC Assist, or at least not exactly.
Our concern was less about whether ABC Assist’s answers were accurate – though this is obviously important – and more about whether they faithfully reflected the ABC documents they were based on.
This meant instructing ABC Assist, through prompt engineering, not to answer questions using its underlying LLM training data, even when this data was relevant to the question at hand: “Using ONLY this context information, it is your job to respond to the user’s last message…”
Not only is sticking to ABC information important for fulfilling ABC Assist’s role as a search tool, but faithfulness to the retrieved information – all of it produced by the ABC and of a high standard – is also our best defence against not just inaccuracy but other potential AI pitfalls like toxicity and bias.
While ABC Assist is faithful to the information it retrieved from the ABC corpus to a high degree, the potential for it to provide information from its own LLM training data remains (unavoidably so, as the ability of an LLM to provide a natural language response to a query is contingent on it having access to this training data). As such, it was important that we communicate to users the potential for ABC Assist to provide non-ABC information.
Where to from here?
While feedback we’ve received from users suggests we’re on the right track with ABC Assist, there is much more we can do to improve the user experience.
Currently, we’re working on providing users with the ability to apply search filters so that they have more control over which documents ABC Assist looks at when answering questions – for instance, so users can get a summary of a specific document, or only get information from a specific time period.
When it comes to facilitating responsible usage of this AI tool, we’re also reconsidering how ABC Assist’s responses are presented to the user to place more emphasis on the original documents and to include more signposting of the current limitations of the AI-generated summaries.
Finally, we’re in the early stages of looking at how our work with ABC Assist can be applied to more use cases beyond the ABC Corporate Strategy team.
As we do this, we’ll be guided by the same concerns: not just to make ABC Assist’s outputs as good as they can be, but to make this useful but imperfect technology compatible with responsible workflows.
About the author
Michael Collett is a journalist and producer at the Australian Broadcasting Corporation
Related Posts
25th July 2024
PMA runs workshop on Responsible AI & Public Media
Our latest workshop focussed on how to…