Want advice from UK government website about tax 'n' stuff? Talk to the chatbot

From the department of "this will go well" comes confirmation UK government is trialling an experimental chatbot with 15,000 business users, who can use it to ask questions about tax and available support.

At present, the chatbot, which is a tool built using OpenAI's GPT-4 technology, has been linked to 30 gov.uk business pages. The theory is that users will be able to better navigate the information contained therein by inputting simple queries, such as "set up a business," and get "straightforward, personalized answers" rather than trawling through dozens of pages.

The latest pilot follows an earlier one in which more than two-thirds of users found the chatbot helpful. Less than 15 percent disagreed. However, the first trial also showed that more testing and development were needed.

And accuracy is important if users are going to rely on the results from a chatbot. Air Canada memorably came unstuck after its chatbot gave a customer incorrect information about a flight booking.

The Register asked the UK's tax authority, His Majesty's Revenue and Customs (HMRC), how it would react if a citizen filed their taxes incorrectly after receiving less than accurate advice from an experimental chatbot. A spokesperson told us: "it is likely that the chatbot will still provide some inaccurate and erroneous results."

The department then directed us to the Department of Science, Innovation and Technology (DSIT) for further comment. DSIT has yet to respond.

The chatbot project is experimental, although there are hopes it might be rolled out across the full UK government website, which is made up of 700,000 pages, if all goes well.

Science Secretary Peter Kyle said in a canned statement, "With all new technology, it takes time to get it right so we're taking it through extensive trials with thousands of real users before it is used more widely."

Lessons have already been learned, and guardrails implemented to help the gov.uk chatbot detect questions it should and should not answer, such as those related to sharing sensitive financial information or taking a political position. Tweaks were also implemented to improve answer accuracy and completeness, according to DSIT. The AI Safety Institute has also been consulted.

It's all heady stuff. However, the health warning that the chatbot might (and probably will) dispense inaccurate or incomplete answers is important. This is, after all, experimental, and the onus remains on the user to ensure the proper rules are followed. ®