Machine Learning (ML) methods are boosting the applications of Artificial Intelligence (AI) in all human activities. Differently from ordinary computer software and applications, AI not only converts inputs into outputs, but can also change the surrounding environment, with the risk of creating harm for individuals, organizations and the environment. This is the reason why authorities, regulators and standard bodies around the world have begun to monitor the risks arising from the adoption of AI methods. For example, the European Union has introduced the Artificial Intelligence Act (AI Act), which puts forward a number of key compliance requirements to AI in terms of sustainability, accuracy, fairness and explainability, compulsory for high-risk applications. The United States Department of Commerce has introduced an AI risk management framework, based on a similar set of compliance requirements, to be voluntarily adopted by organizations that employ AI applications.  

The above developments require, to be practically implemented, the availability of a set of statistical metrics that can actually measure whether AI applications are compliant, as well as the risk of not being compliant. A consistent set of metrics for AI compliance and risk management does not exist yet. To fill the gap, Giudici and Raffinetti (2023, Finance Research Letters) introduced a model for the evaluation of the compliance of the AI applications in finance to the European AI Act. According to the Authors' perspective, such compliance can be translated into four "S.A.F.E." key principles: "S" for sustainability; "A" for accuracy; "F" for fairness; "E" for explainability. Their approach is based on the employment of Shapley-Lorenz values, a normalized variant of the well-known Shapley values which inherits from the latter its computational complexity.  

In this deliverable, the previous approach is extended in three main ways: a) we develop a model valid for all applications of AI, and not only for those in finance; b) we provide a model that is less computationally intensive, and easier to interpret; c) we develop a model that is scalable, and generalizable to all AI compliance requirements. In more detail, with reference to a) the paper considers different cases and a simulation study; with reference to b) we introduce metrics that are interrelated by means of a common mathematical root: the Lorenz curve, which is related to the Receiver Operating Characteristic curve. This allows the integration of all measures into an agnostic score that can be employed to assess the trustworthiness of any AI application. With reference to c), we develop a model that is based on ranks, and that appears as a "box", as it can be enriched by new metrics as new compliance requirements emerge, in particular for generative AI, whose risks are still under investigation. For these reasons, we call our approach a "Rank Graduation Box" (RGB).  

The deliverable is organized as follows: a methodology section introduces the theoretical framework of our proposal, and the description of the proposed RGB metrics; an application section presents some experimental scenarios, to assess the validity of the proposal; a conclusion section concludes the paper with some final remarks and comments.  

In summary, the key ingredients of the proposal are the Lorenz curve, the dual Lorenz curve and the concordance curve, which are statistical tools widely used to summarize the distribution of income and wealth. In the paper, we propose to extend these tools to provide a set of metrics that can assess the compliance of AI applications within a common unified framework.