Introduction
Generative artificial intelligence has entered the Indian courtroom not through any statute or judicial rule, but through the laptop of the overworked advocate and the research desk of the understaffed judge. Tools such as ChatGPT, Gemini, and Copilot answer legal queries with the confident fluency of a senior counsel, complete with citations formatted exactly as they would appear in the Supreme Court Cases or All India Reporter series. The difficulty is that a meaningful share of those citations do not exist. This phenomenon, termed “hallucination” in AI research, occurs because generative language models do not retrieve verified documents the way a search engine does; they predict plausible-sounding text based on patterns in their training data. When applied to law, the output can be a perfectly formatted citation attached to a judgment that was never delivered, or worse, a real citation with invented paragraphs attributed to it.
India’s courts have encountered this problem with increasing frequency since late 2024, and the resulting litigation exposes an important asymmetry: the statutory and disciplinary architecture built around advocates under the Advocates Act, 1961 and the Bar Council of India (BCI) Rules is considerably more developed, and more enforceable, than anything currently available to hold judges and quasi-judicial authorities accountable for the same error. This article examines that asymmetry as its specific analytical lens.
The Anatomy of an AI Hallucination
Unlike a conventional legal database such as SCC Online or Manupatra, which retrieves and displays an actual judgment, a generative AI model constructs its answer probabilistically, word by word, based on statistical association rather than a verified index of case law. It has no inherent mechanism to distinguish a real citation from a syntactically convincing invention. The problem is not confined to India. In the United States, the Southern District of New York’s sanctions order in Mata v. Avianca, Inc. became an early global reference point after counsel submitted a brief built on fictitious cases generated by ChatGPT. French researcher Damien Charlotin now maintains what appears to be the only dedicated global tracker of such incidents, which had logged over a thousand recorded cases by mid-2026, indicating that the problem is systemic rather than anecdotal.
Legal citation is especially vulnerable to this weakness because the domain rewards exactly the kind of confident, formulaic output that generative models excel at producing. A citation such as “(2019) 4 SCC 112” looks structurally identical whether it points to a real judgment or a phantom one, and a reader without independent access to the reporter has no way of distinguishing the two from form alone. Compounding this, AI systems can misattribute genuine paragraph numbers or ratios to real cases, producing what commentators have called an even more insidious error than outright fabrication, since the citation itself withstands a cursory check while its substance does not.
The Indian Docket: A Pattern, Not an Aberration
The Indian record demonstrates that hallucinated citations have infiltrated every tier of the judicial and quasi-judicial hierarchy. In December 2024, the Bengaluru bench of the Income Tax Appellate Tribunal, in the Buckeye Trust matter concerning a trust-taxation dispute of roughly ₹669 crore, cited three Supreme Court judgments and one Madras High Court ruling, none of which existed; the Tribunal was compelled to recall its own order under Section 254(2) of the Income-tax Act, 1961. In February 2025, a trial court in Andhra Pradesh dismissed objections to a commissioner’s report by relying on four non-existent Supreme Court judgments, an order the Supreme Court later took up on its own motion. In March 2025, the Karnataka High Court went further and ordered an inquiry against the trial judge personally, a rare instance of institutional action directed at a judicial officer rather than a lawyer. In September 2025, a Delhi High Court petition in a flat-possession dispute, Greenopolis Welfare Association v. Narender Singh, was withdrawn in embarrassment after opposing counsel discovered that paragraphs 73 and 74 had been invented from the landmark Raj Narain v. Indira Nehru Gandhi ruling, a judgment that in fact runs to only twenty-seven paragraphs. In October 2025, the Bombay High Court quashed a tax assessment order worth approximately ₹27.91 crore after finding it rested on three fabricated precedents. In January 2026, the same High Court imposed costs of ₹50,000 on a litigant in Deepak v. Heart & Soul Entertainment Ltd. for filing submissions bearing the unmistakable signatures of raw AI output. That same month, the Andhra Pradesh High Court, in Gummadi Usha Rani v. Sure Mallikarjuna Rao, controversially declined to set aside a lower court order despite acknowledging that the underlying citations were AI-generated, holding that fabricated citations do not automatically vitiate a decision if the legal principle applied happens to be correct.
That lenient approach did not survive Supreme Court scrutiny for long. In Pooja Ramesh Singhv. Jammu and Kashmir Bank Ltd., decided in July 2026, the Supreme Court set aside a National Company Law Appellate Tribunal order that had relied on six citations, three of which were entirely fictitious and three of which misquoted or misattributed real judgments — none of them urged by either party’s counsel. The Bench of Justices P.S. Narasimha and Alok Aradhe described the danger of unchecked AI hallucination in judicial reasoning as comparable to a toxic gas leak, invisible until its damage becomes irreversible, and held that any decision tainted by even a fragment of hallucinated material is no decision in the eyes of the law. Crucially, the Court characterised reliance on such material not as an innocent error but as misconduct carrying legal consequences, and directed the Bar Council of India to constitute an expert committee to examine the extent of AI use in litigation.
The Advocate’s Exposure: A Mature Disciplinary Architecture
For advocates, the consequences of filing hallucinated citations slot into an existing and comparatively robust framework. Chapter II, Part VI of the BCI Rules, framed under Section 49(1)(c) of the Advocates Act, codifies an advocate’s duty to the court, including Rule 3’s mandate that an advocate must not seek to influence a court’s decision by any illegal or improper means. Submitting a fabricated citation, even where the advocate did not knowingly invent it but simply failed to verify AI-generated output, sits uneasily against this duty and against the broader principle, affirmed by the Supreme Court in P.D. Khandekar v. Bar Council of Maharashtra, that conduct amounting to more than a mere error of judgment — conduct reflecting a want of diligence unbecoming of the profession — attracts disciplinary rather than purely civil consequences. Section 35 of the Advocates Act empowers the State Bar Council’s Disciplinary Committee to reprimand, suspend, or permanently remove an advocate from the rolls, and Section 42 vests that committee with civil-court powers to summon evidence and examine witnesses on oath. The machinery, in other words, already exists, is triggered by a simple complaint, and culminates in a binding, appealable order.
The Judge’s Shield: An Accountability Gap
The same cannot be said for judicial officers. When the Karnataka High Court ordered an inquiry against a trial judge for relying on fabricated Supreme Court rulings, it was notable precisely because such action against a sitting judge, rather than a lawyer, is exceptional. India possesses no external disciplinary body for judges comparable to the Bar Council. Short of the constitutionally onerous impeachment process — which has never once resulted in a judge’s removal despite requiring only a two-thirds majority in both Houses of Parliament — the only available mechanism is the in-house inquiry procedure that the Supreme Court itself evolved in 1999, which can at most result in a judge being advised to resign or having judicial work withheld. This is an internal, opaque, and largely discretionary process, invoked by the judiciary against its own members, with no statutory footing comparable to Section 35 of the Advocates Act. The irony is sharpened by the fact that the same Supreme Court that condemned hallucinated material as misconduct in Pooja Ramesh Singh simultaneously operates its own suite of AI research tools — SUPACE, SUVAS, TERES, and the generative Legal Research Analysis Assistant (LegRAA) — making the institution both regulator of, and participant in, the very practice it seeks to police. The Supreme Court’s Centre for Research and Planning had flagged this exact risk in a White Paper on Artificial Intelligence and the Judiciary released in November 2025, but that document mandates verification only in general terms and imposes no standardised, enforceable mechanism, particularly with respect to how judges ought to treat AI-tainted material submitted by lawyers.
Towards a Balanced Regulatory Framework
Three reforms would narrow this gap without treating AI adoption itself as the problem. First, a binding, uniform BCI circular should require advocates to independently verify, against an authoritative repository such as SCC Online, Manupatra, or the eCourts database, any citation before filing, with mandatory disclosure of AI-assisted drafting. Some High Courts have moved unilaterally in this direction: the Kerala High Court’s July 2025 policy for the district judiciary was among the first in India to restrict AI use in judicial findings and order-drafting, and the Gujarat High Court followed in April 2026 with a comparable restriction, though such policies bind judicial officers and staff, not the independent Bar. Second, given that India’s e-filing infrastructure already processes lakhs of cases annually, a Citation Integrity Module could be embedded at the point of filing to cross-reference cited paragraphs against verified judgment repositories, flagging discrepancies for judicial attention without triggering automatic rejection.²¹ Third, and most urgently, judicial accountability for AI-tainted reasoning requires a defined, rule-based mechanism — short of impeachment — that operates with the same procedural clarity that Section 35 already affords against advocates, so that the burden of the AI hallucination problem does not fall disproportionately on one wing of the legal profession while the other remains governed by discretion alone.
Conclusion
The Indian judiciary’s early encounters with AI hallucination reveal a legal system reacting case by case to a structural problem. The Supreme Court’s description of hallucinated material as an invisible, cumulative hazard captures the stakes accurately, but the institutional response so far has been lopsided: advocates face a functioning, statutorily anchored disciplinary regime, while judges and quasi-judicial authorities remain shielded by a thin, internal, and largely untested accountability structure. A regulatory framework fit for the AI era must close that gap, not merely widen the advocate’s existing exposure, if public confidence in judicial reasoning is to be preserved as generative AI becomes further embedded in legal practice. The technology that produces phantom precedents is, encouragingly, also capable of detecting them at scale, and the coming years will likely determine whether Indian legal institutions treat verification as a shared professional obligation running across the Bar and the Bench alike, or continue to enforce it unevenly against whichever actor happens to be easier to discipline.
THIS ARTICLE IS WRITTEN BY ISHANT SAINI FROM CENTRAL UNIVERSITY OF PUNJAB, BATHINDA
REFERENCES :
Mata v. Avianca, Inc., United States District Court for the Southern District of New York, Sanctions Order (June 2023).
Damien Charlotin, AI Hallucination Cases Database, as referenced in “Phantom Precedents: AI Hallucination, the Verification Vacuum, and the Case of Mandatory Pre-Filing Reform in Indian Courts,” Law and Other Things (2026).
Buckeye Trust v. PCIT, Income Tax Appellate Tribunal, Bengaluru Bench, Order dated 30 December 2024, recalled under Section 254(2), Income-tax Act, 1961.
Suo motu proceedings before the Supreme Court of India (Bench of Narasimha and Aradhe, JJ.) concerning a trial court order in Andhra Pradesh dated 27 February 2025 based on four fabricated judgments.
Karnataka High Court, order of Justice R. Devdas directing inquiry against a Bengaluru City Civil Court judge (March 2025).
Greenopolis Welfare Association v. Narender Singh, Delhi High Court (September 2025).KMG Wires Pvt. Ltd. v. National Faceless Assessment Centre, Bombay High Court (October 2025).
Deepak v. Heart & Soul Entertainment Ltd., Bombay High Court (January 2026).
Gummadi Usha Rani v. Sure Mallikarjuna Rao, Andhra Pradesh High Court (January 2026).
Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. and Anr., 2026 INSC 668 (Supreme Court of India, judgment dated 2 July 2026).
Bar Council of India Rules, Part VI, Chapter II, Rule 3, framed under Section 49(1)(c), Advocates Act, 1961.
P.D. Khandekar v. Bar Council of Maharashtra, (1984) 2 SCC 556.
Advocates Act, 1961, ss. 35 and 42.
Supra note 5.
In-house Procedure evolved by the Supreme Court of India (1999); see also discussion in MediaNama, “Supreme Court asks Bar Council to form AI expert panel after trial court cites fake judgments” (May 2026).
Supreme Court of India, Centre for Research and Planning, “Artificial Intelligence and the Judiciary” White Paper (November 2025).
Kerala High Court, Policy Regarding Use of Artificial Intelligence Tools in the District Judiciary (19 July 2025); Gujarat High Court AI Policy (April 2026).
