Introduction: The Rise of Generative AI in Legal Practice
As the technology advances the use of AI had become prevalent almost in every field of work, and law certainly wasn’t remain untouched. Working Professionals rely on AI from drafting to research for relevant case laws for their arguments in their cases. Before advancement of AI the research processes were heavily taxing and time taking for not only the young advocates but even for experienced professionals. Initially they had to go through a tower of resources to fetch a desired or a relevant case. In contrast, AI tools have made those researches just one effective prompt away making the relevant information available at your discretion, and how you interact and prompt with the AI to reach either a desired ‘generation of information’ or an essentially ‘authentic caselaw’. Thus, I argue that while generative AI accelerates legal research and is considerably helpful in the field of law, the threat of hallucinations makes human verification an absolute ethical mandate to protect professional competence and judicial integrity.
AI as a Legal Research Assistant : Efficiency and Opportunity
“AI is having a profound effect on the practice of law. Though AI is more likely to aid than replace attorneys in the near term, it is already being used to review contracts, find relevant documents in the discovery process, and conduct legal research. More recently, AI has begun to be used to help draft contracts, predict legal outcomes, and even recommend judicial decisions about sentencing or bail.” (Stepka 1, Law Bots: How AI Is Reshaping the Legal Profession), Stepka is addressing a similar issue at hand, that how lawyers, firms and even some Judicial benches have already augmented AI effortlessly in their lives as an assistance with the thick workload. Stepka has also mentioned throughout the article with various instances which prove how AI is faster and more accurate than humans to some extent and lawyers are leveraging this quality of AI to work faster and more by incorporating various versions of AI and machine learning, aside from this authentic legal research databases have begun using AI helpers to enhance the research experience such as ‘Westlaw Edge’ and ‘SCC Online® AI . Stepka has given these progressing productive lawyers and legal professional AI users the term, ‘Superhuman Lawyers’, (Stepka 2). “ Law firms are increasingly utilizing AI in fulfilling their day-to-day tasks, including electronic discovery and predictive coding, litigation analysis and predictive analysis, contract management, due diligence reviews, “wrongdoing” detection, legal research, and deception detection. Some service providers even offer full-blown AI-based legal assistants, such as Harvey, which has partnered with OpenAI “to build a custom-trained model for legal professionals,”20 and Jus Mundi’s AI Assistant, which offers services such as case summaries, information retrieval, focused legal analysis, case application simulation and case precedent extraction.” (Lejniece 181, Ch 10 Good Governance Principles for Law Firms Using Artificial Intelligence). In the Chapter 10 of Good Governance principles for law firms using Artificial Intelligence it is discussed by Lejniece, that how in the progressing world where the daily life is progressively getting faster essentially needs a tool to make the legal processes conveniently efficient for the developing world similar to the insights of Stepka both of these journals focus on the increasing requirement of generative AI as a helping tool and how almost every AI agency is working on developing a platform for legal professionals.
The Hallucination Drawback: Why Verification Is Essential
The use of AI is inevitable, though generative AI can help the budding lawyers or even the experienced lawyers differently either by helping with creative arguments or increasing the productivity, the main question arises is the ‘Validity’ of the claims. Sure using creative arguments may land you a win but what is the credibility of the arguments or the research content produced or generated by the generative AI. As this is humanely impossible to keep track of each and every case law and legible discussions, then how can you rely on AI without double checking or verifying the arguments, research material and case laws cited by a generative AI. Generative AI is already being used in journalism and creative writing sectors but law is far different, a world which already has a lot of un-verified or fake news circulating is the duty of media and the citizen both to verify but in the situation of laws and cases the ‘Duty of Care’ is expected of the respective advocates and lawyers to exercise. A fake or hallucinated precedent can either make or break the judgement in either party’s favour or against them, irrespective of the actual facts or case laws existing in a scenario of a hallucinated precedents or facts both.
Ethical Governance of AI in Legal Practice
Stepka, further in his journal article mentions a dilemma concerning if AI is actually ready to practice law, he not only questions the availability of the AI tools at the discretion of criminal defendants or not, should attorney’s disclose their use of AI, could the use of generative AI be considered as either invasive or crucial in discovery process of cases. Taking from the insights of Stepka’s Journal, the final question we arrive at is that, to what extent the generative AI can be trusted, to what extent its use would be lawful and more importantly to what extent do we need to verify the results it produces and if they are ethically legible to cite.
Lejniece recommends a list of precautions on how a law firm should use AI, (1)Legal & Ethics Standards: Law Firms Should Comply with Applicable Legal & Ethical Standards. (2) Client Confidentiality & Privacy: Law Firms Should Ensure that AI is Used in Line with Client Confidentiality & Privacy Standards. (3) Client Awareness & Consent: Law Firms Should Ensure Client Awareness of & Consent to the Use of AI in the Provision of Legal Services. (4) Human Oversight & Approval: Law Firms Should Ensure Human Over- sight & Approval Mechanisms for AI-Generated Work Product. (5) Transparency & Explainability: Law Firms Should Ensure that the AI they Use is Transparent and its Generated Work Explainable. (6) Ultimate Responsibility: Law Firms Bear Ultimate Responsibility for any AI-Generated Work Product. (7) AI Training: Law Firms Should Provide Mandatory AI Training & Commit to Continuous Learning. (8) Minimizing Bias and Maximizing Diversity: Law Firms Should Use AI in Accordance with the Principles of Non-Bias, Fairness & the Promotion of Diversity & Inclusivity. (9) Managerial & Human Resources Decisions: Law Firms Should Use AI in Management and Human Resources Contexts in Good Faith.( Lejniece 182). Through these basic principles of governance of AI would increase transparency, efficiency and more importantly accountability. Use of AI can be unlimited in-fact it can go completely undetected as-well essentially affecting the court’s efficiency in providing justiciable outcomes. As, because of an un-verified actions of a legal professional should not affect their clients negatively. Thus, verification and compliance with basic duty of care towards AI and ethical management is necessary. Stated by Lejniece, “Compliance with applicable laws and ethics standards is the backbone and the absolute minimum standard of any legal practice. If a law firm chooses to follow a ‘bare bones’ policy in its deployment of AI, the firm should ensure that its use of these technologies complies with any applicable legal and ethical standards governing the practice of law.” ( Lejniece 183), through this Lejniece means that to verify an unverified source of information or a source which could possibly be not true, is ‘bare bones’, and using of such source comes with a duty of care towards your research, work or client that the information used must be verified that is the least of the duties the professional carries while using AI to make the use essentially ethical and legible to use. A legal professional holds a major responsibility of using Bona-fide information to support their research, cases or arguments. ‘Bare Bones’ direct interpretation is that even if a law firm does the bare minimum when using AI, That minimum must still meet legal and ethical obligations, simply put it is their fiduciary duty owed to clients to verify the information and they must practice reasonable care when applying or using that information such as citing case laws in-front of hon’ble courts.
Judicial Recognition of AI Hallucinations: The Supreme Court’s Response
Supporting these regulations posed by both Stepka and Lejniece is supported by various precedents. Such as, Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd., Civil Appeal No. 11950 of 2025 (S.C. July 2, 2026). The Hon’ble Supreme Court of India gave directions in the context of use of generative AI. In the case of Pooja Ramesh Singh, a Tribunal relied on artificially generated essentially hallucinated precedent to support its judgement. Thus, to maintain the integrity of adjudication and its processes the judgement was set aside by the Supreme Court of India. While analysing issues common to such circumstance where the a reasonable care was not taken in a country with over five crore pending cases, use of Generative AI without verification is lethal for the Indian Judiciary. In a country where the current cases take years to sort and on top of that the process of revisitation of cases to set aside the pre- determined judgement merely on the basis of Professional misconduct, because the legal professionals did not verify the information and case laws they were citing to influence the judgement. This negligent behaviour to not comply with basic duty of care and a fiduciary duty to clients will eventually collapse the judicial system and a path of justice in itself. And what more could be more embarrassing than to have Judgements, courts and appellant courts basically the eco-system of justice and criminal management as the epicentre of misinformation and propogandist values. Reckless use of AI will destabilise the very core of justice and revoke people’s trust on the system of Indian Judiciary itself. The primary catalyst to that situation will be the negligent legal professionals who couldn’t take duty of care and perform their fiduciary duty just to pose themselves as efficient and fast while in-fact deeming themselves unreliable. The Bench of case law Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd., Civil Appeal No. 11950 of 2025 (S.C. July 2, 2026), Hon’ble Justice Pamidighantam Sri Narasimha addresses this serious issue in his judgement for the case, “At the same time, it is necessary to clarify the position of law regarding a certain trait in AI’s responses, a tendency to generate non- existent, fake, or hallucinated results when replying to a prompt. We are neither concerned with the cause nor with the process of resolving such hallucinations; it is for the engineers and scientists to deal with them. For us, i.e., for those in the province of adjudication and determination of disputes, this by-product of AI, i.e., the production of fake, non-existent, and hallucinated material and its utilisation as precedents in law, is like the release of methyl isocyanate in the province of law and justice: ‘invisible, insidious, and catastrophic by the time anyone notices. It not only contaminates but takes away the very lifeblood of judicial determination.’”, (para 6). The Hon’ble Justice not only referred to similar irreparable damage due to not verified Generative AI, The hon’ble justice also directed the Bar Council of India to make regulations on submitting fake or hallucinated material before the court as if they are precedent of the law. The Supreme Court directed them to take action while acknowledging the requirement of consequences and accountability of individual’s actions as mentioned in paragraphs eight and nine of the judgment addressing the seriousness of the problem essentially made to make our lives easier but ending up formulating ‘Trust Issues’ in a carefully regulated system.
Conclusion: Verification as a Indispensable Professional Duty
Generative AI is no longer an object of the future but has become a worry of the present. While it poses primarily essential benefits to professionals and students alike, it has taken an unreliable turn in the legal field for lawyers, advocates and even judiciary. Its ability to streamline research, assist in drafting, and improve efficiency has undoubtedly transformed the manner. In which legal professionals work. However the same technology that promises speed and convenience also possesses the capacity to generate inaccurate, fabricated, or hallucinated information. As also recognised by the Supreme Court of India in the case of Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd., The use of hallucinated precedents has consequences that extend far beyond an individual case, it undermines judicial integrity, Erodes public confidence in the legal system and compromises the very foundation of seasoned adjudication, in a country where the judicial backlog is massive. The hallucinating ability and non-performance of ethical duty and duty of care the judicial system will be at the risk of being choked by the improper, unethical and unregulated use of Generative AI for research purposes in the legal field. Therefore, the integration of Generative AI In the legal sphere must always be accompanied by rigorous human oversight, independent verification and unwavering adherence to ethical and professional standards following the precedent set in the above cited case law. Ultimately, the responsibility for every submission made before a court rests with the legal profession who chooses to rely upon the algorithm. In the age of artificial intelligence, the duty to verify is not merely a matter of best practice, it is an indispensable ethical and professional obligation that the legal professionals owe to their clients and the system of justice to maintain the due credibility. As Justice Must Not be Outsourced to an algorithm.
THIS ARTICLE IS WRITTEN BY ANUSHREE GARG
REFERENCES :
Stepka, Matthew. “Law Bots: How AI Is Reshaping the Legal Profession.” Business Law Today, 2022. JSTOR, https://www.jstor.org/stable/27196322. Accessed 15 July 2026.
Stepka, Matthew. “Law Bots: How AI Is Reshaping the Legal Profession.” Business Law Today, 2022. JSTOR, https://www.jstor.org/stable/27196322. Accessed 15 July 2026.
LEJNIECE, AIJA. “Good Governance Principles for Law Firms Using Artificial Intelligence.” Transforming Arbitration: Exploring the Impact of AI, Blockchain, Metaverse and Web3, edited by MAUD PIERS and SEAN MCCARTHY, Radboud University Press, 2025, pp. 179–216. JSTOR, http://www.jstor.org/stable/jj.28414751.14. Accessed 15 July 2026.
LEJNIECE, AIJA. “Good Governance Principles for Law Firms Using Artificial Intelligence.” Transforming Arbitration: Exploring the Impact of AI, Blockchain, Metaverse and Web3, edited by MAUD PIERS and SEAN MCCARTHY, Radboud University Press, 2025, pp. 179–216. JSTOR, http://www.jstor.org/stable/jj.28414751.14. Accessed 15 July 2026.
LEJNIECE, AIJA. “Good Governance Principles for Law Firms Using Artificial Intelligence.” Transforming Arbitration: Exploring the Impact of AI, Blockchain, Metaverse and Web3, edited by MAUD PIERS and SEAN MCCARTHY, Radboud University Press, 2025, pp. 179–216. JSTOR, http://www.jstor.org/stable/jj.28414751.14. Accessed 15 July 2026.
Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd., Civil Appeal No. 11950 of 2025 (S.C. July 2, 2026).
