Building a Multilingual Legal Risk Database and ClauseRisk Visual Analytics System for Contracts and Online Service Agreements

Documents containing binding legal terms and risk obligations remain largely unread, particularly in multilingual contexts where regulatory systems vary across jurisdictions. This paper presents a multilingual Retrieval-Augmented Generation (RAG) system designed to identify potentially risky clauses across Arabic, English and French legal, regulatory and contractual documents, instantiated on Algerian legislation as a fully specified reference framework. In addition, we introduce a visual analytics system tailored for legal domain experts, integrating the underlying datasets, model outputs and evaluation results to support interactive analysis and model interpretability. The system’s knowledge base is grounded in a trilingual corpus of 2535 sentences, comprising 315 fully annotated clauses whose cross-lingual legal equivalence was verified by a panel of ten legal experts. Performance evaluation was conducted on EVAL-EEM, an external corpus of 81 annotated passages extracted from 55 real-world sources. Because EVAL-EEM was used during development for decision-configuration tuning, it is characterised as a development corpus rather than a strictly held-out test set; consequently, both in-sample metrics and nested cross-validated estimates are reported. On the binary risk-identification task, the proposed system improves from an initial F1 score of 0.340 to 0.806 in the final in-sample evaluation and 0.792 under nested cross-validation. GPT-4o without retrieval achieves an F1 score of 0.775, showing no statistically significant difference from the complete RAG architecture (p=0.39), whereas supervised baselines, including a fine-tuned XLM-R, exhibit substantially lower performance on external documents. These empirical findings demonstrate the potential of statute-grounded multilingual LLM reasoning augmented by traceable retrieval evidence. However, because configuration selection was conducted on the evaluation corpus itself, these results should be interpreted as preliminary and specific to the evaluated knowledge base, dataset and legal visual analytics system. Further validation on larger, unexposed benchmarks, along with formal user studies, is necessary to establish cross-jurisdictional portability, empirical generalizability and system usability.