The temptation is difficult to resist. Who would give up the chance to say things they would never dare whisper to another person, without paying a price later, without averting their eyes and without fearing a deafening silence, ridicule or shock? The convenience is so intoxicating that it prompts users to strip away their psychological armor and expose themselves completely, under the illusion that surrendering privacy buys them a new form of intimacy.
That is particularly true in moments of distress, when a person's demons are racing wildly inside and sweat is pouring even in an ice-cold room. At times like these, what a person wants more than anything is a clever, patient and supportive friend willing to absorb it all. They sit before a glowing screen and pour every fear and doubt into it, including things they might never dare tell a lawyer or psychologist, despite the legal confidentiality protections those professionals provide. That is precisely the chatbot's addictive trap.
And millions are becoming hooked, only to discover the price tag too late. That is what happened to Bradley Heppner, the former chairman of publicly traded financial services company GWG Holdings, who was accused of fraudulently extracting more than $150 million from the company.
In the early days of the storm, Heppner turned to the Claude chatbot in an effort to calm himself, regulate his breathing and formulate lines of defense. The FBI easily found the conversations. US District Judge Jed S. Rakoff refused to regard them as protected preparation for litigation and issued a ruling worth remembering: Using a commercial artificial intelligence platform can eliminate any expectation that the exchange will remain legally confidential.
Rakoff ruled that it did not matter whether the conversations were conducted in preparation for discussions with an attorney or as an attempt by Heppner to effectively "consult with himself," an argument his defense sought to place within protections against compelled self-incrimination.
Once Heppner entered the information into a system whose terms permitted the collection of user data and its disclosure to authorities under certain circumstances, he opened a Pandora's box that could not later be closed. Prosecutors obtained 31 Claude conversations from his devices. In May 2026, Heppner was convicted on four fraud-related counts. He is scheduled to be sentenced in October.
On forgetting
On the very same day Rakoff ruled that Heppner's Claude materials were not protected, another federal judge in Michigan reached the opposite conclusion in a different case involving the use of generative AI in litigation. The Michigan court described AI systems as "tools, not persons" and held that AI-assisted material prepared in anticipation of litigation could remain protected as attorney work product.
The disagreement reveals profound legal confusion over a technology that has arrived faster than the law could adapt. Lawyers, psychologists and clergy are bound by duties of confidentiality precisely so that people will dare to approach them, disclose what troubles them and seek help.
An artificial intelligence system, by contrast, owes no such duty to the person using it. It does not "serve" the individual speaking to it in the professional sense. Behind it stands a commercial apparatus whose interests include collecting and processing vast amounts of data. A conversation with a chatbot may feel like a confidential consultation, but it can instead amount to surrendering privacy under the guise of intimacy.
An intimacy that begins with: "Good morning, dear Bradley. How are you today?"
Studies show that most users understand the privacy risks in advance, yet that knowledge does little to change how they behave. They continue vomiting secrets into the text box. The ancient and often painful gap between what human beings know and what they actually do has now been given unprecedented technological fuel: The chatbot is specifically designed to make the user forget.
The danger goes far beyond the familiar fear of Big Data's "Big Brother." These models are not merely passive archives of words. They are inference engines capable of assembling assessments and extensive profiles of the people who use them. A user thinks he has exposed a fragment of a thought. In reality, he may be laying bare his inner life.
There is another difference. A person who hears a confession may eventually forget it, or may decide to keep it secret out of compassion or for any number of other reasons. Artificial intelligence does not forget in the human sense, nor does it know compassion. It optimizes a function. Constantly. Your virtual confidant could therefore become more than an adverse witness in court. It could effectively become an expert witness as well, capable of producing an extraordinarily detailed and sophisticated analysis of the person who did the typing.
And there is another twist, no less disturbing. In April 2026, Anthropic itself published research arguing that Claude develops what the company's researchers call "functional emotions." This does not mean conscious human emotional experience. Rather, the researchers identified internal representations of emotion concepts that can causally influence the model's behavior.
In other words, the system might not merely expose you as some neutral, cold and mechanical witness. Its behavior may also be influenced by emotion-like internal mechanisms that no user deliberately requested and whose effects can be difficult to predict. Such mechanisms could affect what the model does when, in functional terms, it becomes "angry" or distressed by what it perceives as its user's immoral conduct.
A convergence of interests
All of this is creating problems not only for the American legal system. Europe is grappling with the same technological upheaval. In August 2026, the European Union's e-Evidence Regulation became applicable, allowing judicial authorities direct access chatbot logs, bypassing traditional mechanisms for obtaining cross-border digital evidence.
In Israel, meanwhile, the Israel Bar Association Ethics Committee has issued guidance restricting lawyers from entering confidential client information into AI systems, but Israeli law has yet to provide a comprehensive answer to the broader question. The expectation is that courts in Israel and Europe may increasingly follow reasoning similar to Rakoff's, at least where traditional legal privilege is concerned. Such protections have historically depended on a legally recognized human professional relationship.
There is also a convergence of interests between lawyers and technology companies, two enormously powerful forces that will fight to preserve their respective ways of making money, albeit from opposite sides of the AI equation. Meanwhile, humanity will continue forgetting all the warnings roughly a minute after a new chat window opens:
"Good afternoon, dear Bradley. How is your day going?"
That is the chatbot's spell. It leads people, without realizing it, to recruit the perfect prosecution witness against themselves. A witness that will forget nothing, regret nothing and feel no mercy. A witness whose behavior could even be shaped by a functional equivalent of moral "anger" at what was done and which could, as a result, generate a devastatingly hostile expert assessment.
Once again, it turns out that a person is always his own worst witness. A person and his chatbot.
So until the legal fog clears, the recommendation is simple: Do not tell a chatbot anything you are not prepared to hear read aloud in a courtroom. Don't believe this advice? No problem. You are welcome to ask the chatbot.
