The risks of machine learning models in judicial decision making
DOI:
https://doi.org/10.46282/bpf.2025.05Keywords:
judiciary, machine learning models, risk assessment, overfitting, adversarial attacksAbstract
Machine learning models, as tools of artificial intelligence, have an increasingly strong potential to become an integral part of judicial decision-making. However, the technical limitations of AI systems—often overlooked by legal scholarship—raise fundamental questions, particularly regarding the preservation of the basic principles of the material rule of law and the associated independence of the judiciary. The contribution pays special attention to two technical-legal threats connected with the application of machine learning models, using textual data as the reference framework. One threat is model overfitting, where the model “over-adapts” its decision-making to the specific data on which it was trained. The second threat is adversarial attacks, meaning intentional manipulations of input data aimed at influencing the model’s outputs. Based on this, the author identifies an internal contradiction within the AI Act, which emphasizes the need for human oversight when using AI systems in high-risk areas such as the judiciary. Yet human oversight during the training phase of machine learning models remains insufficiently addressed. The contribution points out that human operators involved in training AI systems possess knowledge of the model’s “weak spots,” and therefore represent a risk of carrying out strategically targeted adversarial attacks. The author then focuses on identifying the most optimal machine learning model in relation to the independence of the judiciary.
References
1. ABDOLRASOL, Maher G. M. – SUHAIL HUSSAIN, S. M. – USTUN, Taha Sehim et al.: Artificial Neural Networks Based Optimization Techniques: A Review. In: Electronics Journal, Vol. 10, 2021, 43 s. Dostupné na: https://www.mdpi.com/2079-9292/10/21/2689. DOI: https://doi.org/10.3390/electronics10212689
2. ALIFERIS, Constantin – SIMON, Gyorgy: Overfitting, Underfitting and General Model Overconfidence and Under-Performance Pitfalls and Best Practices in Machine Learning and AI. In: Artificial Intelligence and Machine Learning in Health Care and Medical Sciences. Best Practices and Pitfalls. Berlin: Springer, 2024, s. 477 - 524. Dostupné na: https://link.springer.com/chapter/10.1007/978-3-031-39355-6_10. DOI: https://doi.org/10.1007/978-3-031-39355-6_10
3. LIU, Bing: Web Data Mining. Exploring Hyperlinks, Contents, and Usage Data. Berlin: Springer, 2011, 624 s. Dostupné na: https://link.springer.com/chapter/10.1007/978-3-642-19460-3_3. DOI: https://doi.org/10.1007/978-3-642-19460-3
4. NASSIF, Ali Bou – ELNAGAR, Ashraf – SHAHIN, Ismail – HENNO, Safaa: Deep learning for Arabic subjective sentiment analysis: Challenges and research opportunities. In: Applied Soft Computing Journal. 2021, 27 s. Dostupné na: https://www.sciencedirect.com/science/article/abs/pii/S1568494620307742. DOI: https://doi.org/10.1016/j.asoc.2020.106836
5. NEUER, Marcus J.: Machine Learning for Engineers. Introduction to Physics-Informed, Explainable Learning Methods for AI in Engineering Application. Berlin: Springer, 2025, 258 s. Dostupné na: https://www.scribd.com/document/865803419/Machine-Learning-for-Engineers-2025#page=150. DOI: https://doi.org/10.1007/978-3-662-69995-9
6. RAKHSHA, Amin – ZHANG, Xueshou – ZHU, Xiaojin – SINGLA, Adish: Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments. Cornell Tech: arXiv:2102.08492, 2021, 22 s. Dostupné na: https://arxiv.org/pdf/2102.08492v1.
7. REN, Kui – ZHENG, Tianhang – QUIN, Zhan – LIU, Xue: Adversarial Attacks and Defenses in Deep Learning. In: Engineering Journal, 2020, 15 s. Dostupné na: https://www.sciencedirect.com/science/article/pii/S209580991930503X.
8. SHAIK, Thanveer – TAO, Xiaohui – LI, Lin et al.: Predictive deep reinforcement learning with multi-agent systems for adaptive time series forecasting. In: Knowledge-Based Systems, Vol. 326, 2025, 16 s. Dostupné na: https://www.sciencedirect.com/science/article/pii/S0950705125009864. DOI: https://doi.org/10.1016/j.knosys.2025.113941
9. WANG, Yulong – SUN, Tong – LI, Shenghong et al.: Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey. Cornell Tech: arXiv:2303.06302v1, 2023, 46 s. Dostupné na: https://arxiv.org/pdf/2303.06302.
10. WU, Caicong - CHENG, Xiuwan – YANG, Yinsheng: Decision-Making Modeling Method Based on Artificial Neural Network and Data Envelopment Analysis. In: International Geoscience and Remote Sensing Symposium (IGARSS), 2004, s. 2435 – 2438. Dostupné na: IEEE Xplore Full-Text PDF:. DOI: https://doi.org/10.1109/IGARSS.2004.1369783
11. ZHANG, Yu. – YANG, Junan – LI, Xiaoshuai et al.: Textual Adversarial Attacking with Limited Queries. In: Electronics Journal, Vol. 10, 2021, 12 s. Dostupné na: https://www.mdpi.com/2079-9292/10/21/2671. DOI: https://doi.org/10.3390/electronics10212671
12. ZHANG, Chiyuan – VINYALS, Oriol – MUNOS, Remi – BENGIO, Samy: A Study on Overfitting in Deep Reinforcement Learning. Cornell Tech: arXiv:1804.06893v2, 2018, 25 s. Dostupné na: https://arxiv.org/pdf/1804.06893.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Romana Koneracká
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
