The Emergence of AI in Legal Proceedings
Artificial Intelligence, with its rapid advancements, is increasingly becoming a part of legal workflows. However, a critical flaw has emerged: AI systems, in their quest to provide answers, may fabricate information. This phenomenon, known as AI hallucination, occurs because AI tends to prioritize confident responses over admitting uncertainty. Consequently, AI-generated citations and legal propositions may appear authoritative despite lacking any basis in reality.
Case Studies: AI’s Impact in Indian Courts
The ramifications of AI hallucination are no longer hypothetical in Indian courtrooms. There have been instances where petitions were dismissed or penalties confirmed based on non-existent citations. In a landmark decision in July, the Supreme Court of India overturned orders from the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) regarding Essel Infraprojects’ insolvency. The Court found that the tribunal had relied on judgments that were entirely fabricated [Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. (2026 INSC 668)].
Similarly, in September, the Supreme Court annulled a ₹425 crore customs penalty. The adjudicating officer’s reliance on fictitious citations was deemed critical to the decision, which had been previously upheld by the Gujarat High Court [Vijay Ghanshyam Gadiya v. Union of India (2026 INSC 947)].
Recommendations for AI Use in Legal Systems
The Supreme Court Advocates-on-Record Association (SCAORA), in its 2026 comments on the Draft Regulations for AI Usage in Courts, emphasized the ease with which AI can generate realistic but nonexistent citations. SCAORA recommended that the task of verifying citations should remain a human responsibility.
While AI’s role in legal research and drafting is inevitable, the critical step lies in ensuring the accuracy of AI-generated citations before they are submitted as legal documents. A citation that appears correct must be rigorously checked for authenticity. Verification must be an integral part of the legal workflow wherever AI contributes to research or drafting.
The Role of Grayscope Verify
Grayscope Verify offers a solution by providing a verification layer to ensure citation accuracy. Users can upload legal documents to receive reports that verify each citation against the Grayscope Library. This library is publicly accessible and constantly updated, encompassing Supreme Court judgments from 1950 onwards, with ongoing additions from High Courts and Tribunals.
The verification report serves as a transparent audit trail, detailing when citations were extracted and checked. Verified citations are linked to the Grayscope Library and the eCourts record, providing an independent source for verification. Each citation also includes the Court’s neutral citation, INSC number, and SCR reference, facilitating direct access through eCourts.
Importantly, each verified citation and the report itself are assigned cryptographic fingerprints, ensuring they are tamper-proof. Any alterations are detectable, maintaining the integrity of the legal document.
Conclusion
The Supreme Court’s rulings underscore the necessity of independent verification of AI-generated legal authorities. While AI can significantly aid legal work, the responsibility to ensure the accuracy of citations remains paramount. Legal professionals are encouraged to utilize tools like Grayscope Verify to ensure their submissions are both accurate and reliable.
Chirag Jhumkhawala, co-founder of Grayscope, emphasizes the importance of this verification step in maintaining the credibility of legal proceedings.
Disclaimer: The views expressed in this article are those of the author(s) and do not necessarily represent the views of Bar & Bench.
