ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

Taxbit Engineering Practice: Retrieval and Security Strategies for Building Reliable AI Assistants in Regulated Industries

The Taxbit team shares engineering practices for building the AI assistant Taxbit Assist in regulated industries, focusing on specific methods to ensure answer reliability and security through context chunking, code-bound identity, and continuous quality monitoring.

2026-11-07 00:49 GMT+8

Taxbit is a digital asset tax and accounting compliance platform founded in 2018, serving enterprise clients such as exchanges, payment services, and brokers. Its newly launched AI assistant, Taxbit Assist, is embedded in the product dashboard, helping compliance teams obtain reliable answers in regulated environments. This practice demonstrates how specific engineering measures can ensure the credibility and security of answers when building AI assistants in sensitive fields.

When building Taxbit Assist, the team faced four core constraints: answers must be grounded and citable; the assistant must possess role and organization awareness to avoid guiding users into workflows they do not have permission to access; there must be a strict distinction between product documentation and tax advice; and regional data residency requirements must be supported. To meet these requirements, the team selected Amazon Bedrock AgentCore as the underlying platform, leveraging its managed agent runtime, built-in memory capabilities, and session isolation features, thereby allowing them to focus primarily on retrieval and security work.

Improvements in retrieval quality were mainly due to two key engineering practices. The first is the context chunking strategy: the team adopted semantic chunking and added the title and lead paragraph of the document to which each chunk belongs before every chunk via custom transformation hooks, ensuring that each chunk carries information about its scope. This change proved to be the most impactful retrieval optimization, effectively preventing misleading results caused by chunks losing their context. The second is retrieval process optimization: at query time, the system first obtains a large number of candidate passages through vector search, then uses a reranking model for fine-grained ranking, and limits the number of chunks returned per source document, thereby balancing recall and precision.

Sign in / Sign up
ReadingResearchRadarInvestment framework

In terms of security and data protection, Taxbit has implemented a series of strict measures. The knowledge base does not contain any tenant-specific content, only public documentation, code-generated product references, and manually curated articles. Identity verification information does not pass through the model but is bound directly to tool calls by trusted services, preventing prompt injection attacks. Additionally, the system uses Amazon Bedrock Guardrails to de-identify personally identifiable information (PII) in input messages and employs geography-aware inference profiles to ensure data is processed within designated regions, meeting data residency requirements in the European Union and the United States.

To continuously guarantee answer quality, Taxbit deployed a multi-layered evaluation mechanism in its production environment. The system samples 10% of real-time sessions and uses an LLM-as-a-judge evaluator to score answers across six dimensions, including usefulness and coherence. Meanwhile, a foundation model classifier detects whether answers exhibit tax advice drift, with a false positive rate close to zero, significantly lower than the approximately 25% rate of traditional topic filters. Corrected content reviewed by humans is written into the curated prefix of the knowledge base and triggers incremental indexing jobs, allowing expert corrections to take effect within minutes.

Taxbit Assist has been in production in the US and EU markets since May 2026. Internal analysis shows that more than 55% of users ask another question within 24 hours of their first. The average cost per conversation is less than 20 cents, with prompt caching technology providing approximately three-quarters of input tokens from the cache, drastically reducing costs. This practice provides an engineering paradigm worth referencing for building reliable AI assistants in regulated industries, particularly in retrieval enhancement, security boundaries, and continuous quality monitoring.

Sources

  • aws.amazon.comAmazon Web Services↗
Continue researching →