Experimentation without regulation is coming to an end. Accounting firms and tax software vendors alike will now need to adjust accordingly.
In June, the IRS Office of Professional Responsibility released its first set of guidelines for tax professionals using generative AI. It was a measured, yet firm set of guidelines that revolved around a simple principle: While AI is a useful tool to improve efficiency, it can’t come at the cost of confidentiality, diligence, or care.
Perhaps the guidelines came as a reality check for firms that have raced to get ahead in the AI adoption game, but there was also a much more specific and technical warning. The IRS singled out a specific kind of pitfall unique to AI systems and their data capabilities, which should prompt every firm to ask tough questions to their AI vendors.
Risk flagged by IRS: cross-client data leakage
In its own words, the IRS warned that “client privacy can be compromised when data generated for one client is repurposed by the program to respond to an inquiry concerning another client, or data compiled for a particular issue is spilled over into an algorithm and combined with a related tax issue involving a different client.”
If you boil that messaging down to its simplest form, you’re left with a concerning scenario. Imagine a preparer asking an AI assistant about treating a rental loss, and the system surfaces figures from a different taxpayer’s return because it “learned” from that previous case. That’s no hypothetical; it’s a consequence of AI tools built with a shared pool of client information or memory.
That carries consequences for tax firms that go far beyond embarrassment. And a failure to adequately address the risk of cross-client spillover could simultaneously implicate Circular 230 confidentiality duties, Section 7216 rules on using and disclosing taxpayer info, and the Written Information Security Plan that the IRS now requires preparers to attest to at a PTIN renewal.
Why AI can leak client data between clients, explained
The major distinction here is whether a system is designed to remember or to reason in the moment. To that end, here are the three common design choices that create risk with regards to the former:
- Training or fine-tuning on customer data. If vendors feed the returns from a firm’s clients into a model to improve it, the information influences the model for the foreseeable future. Once that info is in the model’s weights, it can resurface in any response to any other user. It’s the most permanent form of leakage, and also the hardest to undo.
- Shared memory or storage. Some AI products use service interfaces that retain context across sessions and users. Those are called “stateful endpoints,” and they might be convenient, but they also put client data in stores that can be reached when other requests need their information.
- Loose context handling. Even if training and storage aren’t in the picture, a system can leak by loading the wrong records into a particular prompt, or if it fails to guard one client’s context from another during a singular request.
None of these are inherent to all AI systems. They’re architectural decisions that can be prevented from the outset or designed out of the picture. The IRS’ guidance essentially invites firms to ask vendors to show their work on how those decisions were made.
What trustworthy AI tax software does in practice
There are controls that separate defensible, responsible AI workflows from liabilities. They apply to those who haven’t yet used AI, are building AI into their practice, or are already using it through vendors, and they can be used as a checklist to evaluate any tools that use taxpayer data.
- No training on your data. Both the vendor and any underlying AI model providers should put it in writing that client data is never used to train, fine-tune, or make weight adjustments. That’s the No. 1 way to prevent permanent cross-client leakage.
- Stateless, non-retaining processing. API endpoints that receive requests should be non-retaining, so model providers don’t store the data involved. Any retention, if necessary, should be temporary and limited only to abuse and safety monitoring.
- Context segmentation on a per-request basis. Each AI request gets only the minimum data required to complete its given task. That data comes from a single client and every client’s context is kept separate from all others.
- Defense against prompt injections. When AI reads a document from an untrusted source, the system needs to guard against any instructions hidden in that document trying to steal or cross-contaminate data.
- Encryption and U.S. data residency. All traffic in transit must be encrypted, and all firms that handle U.S. taxpayer data need to verify where that data is processed, with offshore processing subject to even more strict scrutiny.
- Human review and transparency. The IRS explicitly says practitioners must review every AI-generated document before it either goes to a client or the IRS itself. Those same practitioners must be transparent with their clients about how AI is used. In short, AI can’t be a wholesale replacement for sound judgment.
- Independent attestation. SOC 2, Type II reports audited by a third party verify that controls are tested, rather than simply making claims.
We thought about all this at Truss
We set out to build the best possible tax workflow software, and AI sat at the center of it from the jump, so these were questions we had to answer before anything was shipped. Our product approach aligns directly with all the controls discussed above.
We do not train, fine-tune, or adjust model weights using customer data, and neither do our model providers. Requests that come in run through stateless, non-retaining endpoints with only temporary provider-side retention that is required for safety monitoring. Our AI assistant only gets the specific records needed to answer each request, with each client’s data segmented from every other client’s. That prevents the exact type of spillover the IRS warns against.
We control for prompt injection any time a document comes in from an untrusted source, encrypt all traffic in transit, process on US-based infrastructure, and back it all with SOC 2, Type II controls audited by an independent third party. The system does not “learn” anything about your firm during requests. It reasons over the case in front of it and then forgets what it saw when the request is complete.
We are candid about our AI Trust and Safety Policy and allow firms to review all specifics during procurement. That kind of transparency is what the IRS is requesting across the industry and we want to be torch-bearers.
Frequently asked questions
Does using AI for tax preparation violate client confidentiality? Not inherently. The IRS guidelines indicate that AI complies with confidentiality duties as long as the tools protect client data, review their own output, and are transparent with clients. Confidentiality risk is incurred only when tools are trained on or pool client data.
What is cross-client data leakage in AI tax software? Cross-client data leakage is one taxpayer’s information surfacing in a response about a different taxpayer. It occurs when the AI is passed context in a conversation from multiple tax payers, or when a system trains itself on customer data. It can be prevented by segmenting each client’s context on a per-request basis.
What should firms ask AI vendors about data privacy? First and foremost, ask if client data is ever used for training purposes. It’s also important to know whether processing is stateless (vs. the potentially harmful “stateful”), how per-client context gets isolated, where the data is processed, and whether traffic is encrypted and vendors hold SOC 2, Type II attestation.
Do I still need to review AI-generated tax work? Unequivocally, yes. The IRS explicitly dictates that practitioners have to review all AI-created documents before they are delivered to a client or submitted to the IRS. Human supervision is still a requirement in the profession.
The bottom line
We learned this week that the IRS is not telling tax professionals to avoid AI – far from it. It’s telling them they have an obligation to use it responsibly, and it’s pointing out the risks that separate responsible tools and reckless ones.
Data leakage is a design problem that comes with design solutions. Firms need to ask the right questions and choose the right tools in order to use AI for its efficiency without risking the trust of their clients.
Truss is the more-in-one tax workflow platform — helping accounting firms collect client info, manage workpapers, prep returns with AI support, and deliver everything in one place. Book a demo.