UK Watchdog Considers Probe Into Home Office's Use of OpenAI Tools for Asylum Claims
The Independent Chief Inspector of Borders and Immigration is considering investigating the UK Home Office's use of GPT-4-based AI tools in asylum processing due to accuracy and transparency concerns.
According to WIRED, the Independent Chief Inspector of Borders and Immigration (ICIBI) is considering an investigation into the UK Home Office's use of generative AI tools to process asylum applications. The potential probe follows public questioning by lawmakers and researchers regarding the tools' accuracy and the government's secrecy surrounding their use.
The ICIBI monitors the Home Office's management of borders, immigration, and customs. It has asked the digital rights nonprofit Open Rights Group (ORG) to share more information on the tools as it draws up plans for topics to inspect next year. ORG's findings come from a six-month investigation involving 11 parliamentary questions, two ministerial letters, and 10 public records requests, which the Home Office has largely withheld details against.
Documents obtained by ORG reveal that the Home Office uses the Asylum Case Summarisation (ACS) and Asylum Policy Search (APS) tools, built on OpenAI's 2023 GPT-4 large language model and Microsoft Foundry, to summarize interview transcripts and country origin information. The Labour government began piloting these tools in late 2024 to clear a backlog of around 100,000 asylum cases. While the number of people waiting for an initial decision halved in the year through June, appeals against decisions have ballooned.
Researchers are concerned about the accuracy of these AI tools in high-stakes decisions determining whether to send someone back to a country where they fear persecution. ORG found that the percentage of ACS-generated summaries deemed not fit for use—often due to factual errors or misrepresentations—is significantly higher than the published 9 percent error rate from the pilot evaluation. There is no quality control for APS summaries. In July, a UK judge accused the Home Office of citing an "AI hallucinated" document to refuse an asylum claim.