70 pre-built scenarios

Test Case Library

Browse, filter, and select from curated test cases for evaluating AI systems across all dimensions.

Sign in

Access your evaluation projects.

or

Forgot your password?

Don't have an account? Create one

Create account

Start evaluating your AI systems.

or skip the password

Already have an account? Sign in

Reset password

Enter your email and we'll send you a reset link.

Back to sign in

Choose a new password

Enter and confirm your new password below.

Evaluation Builder

Define your system, select criteria, choose test cases, and run your evaluation.

1
System
2
Criteria
3
Test Cases
4
Evaluate
5
Results

Define Your System

Select Evaluation Criteria

These five dimensions define what "good" looks like for your AI system. We've pre-set the weights based on your system type - higher weight means that dimension counts more toward your overall score.

Hover the ⓘ icon next to each criterion to learn what it measures and when you might want to adjust its weight.

Select Test Cases

We've suggested test cases based on your system type. Select the ones you want to include, or add your own.

Run Evaluation

For each test case, paste the actual system output and score the result.

0 / 0 scored

Your Projects

+ New Evaluation
EU AI Act

EU AI Act Compliance Audit

Pre-configured evaluation template with all high-risk test cases mapped to EU AI Act obligations. Produces structured test evidence to support the testing and accuracy parts of your Articles 9-15 technical documentation.

Includes 54+ high-risk test cases across all 5 evaluation dimensions

Back to Dashboard

Compare Evaluations

Session Activity

Your wizard activity in this tab. This log stays in your browser and clears when you refresh or close the page. See our Privacy Policy for the site analytics we do collect.

Blog

Insights on AI evaluation

Practical guides, frameworks, and best practices for testing AI systems before production.

Views expressed in these articles reflect the EvalSpec team's experience and research. They do not constitute professional, legal, or regulatory advice.

← Back to all posts

EU AI Act Compliance Checklist for AI Deployments

Correction: this post has been updated. See what actually changed and what didn't for the current EU AI Act timeline.

The EU AI Act is the world's first comprehensive legal framework for artificial intelligence. Its provisions for high-risk AI systems apply from 2 December 2027 for Annex III stand-alone systems and 2 August 2028 for Annex I product-embedded systems. Article 50 transparency obligations, covering chatbot disclosure, synthetic content marking, and deepfake labelling, still apply from 2 August 2026. If you are deploying AI in Europe - or deploying AI that affects people in Europe - you need to understand what it requires and how to demonstrate compliance.

What the EU AI Act requires

The Act establishes mandatory requirements for AI systems based on their potential to cause harm. For high-risk systems, these requirements are substantial: documented risk management processes, technical robustness standards, human oversight mechanisms, transparency obligations, and ongoing monitoring after deployment.

The compliance burden is not just administrative. The Act requires that you can demonstrate, with evidence, that your system meets the technical standards. Saying "we tested it" is not sufficient. You need documented test procedures, results, and the ability to show that your testing methodology was appropriate for the risk level of your system.

Enforcement is handled by national market surveillance authorities, with penalties for non-compliance reaching up to 30 million euros or 6% of global annual turnover - whichever is higher. The Act also includes a right of redress for individuals affected by high-risk AI decisions.

Risk classification

The Act classifies AI systems into four risk tiers. Understanding where your system sits determines what compliance obligations apply.

  • Unacceptable risk - Systems that are prohibited outright. Includes social scoring systems, real-time biometric surveillance in public spaces, and systems that exploit psychological vulnerabilities.
  • High risk - Systems in regulated domains (healthcare, employment, credit, education, critical infrastructure, law enforcement) that must meet full compliance requirements including conformity assessments before deployment.
  • Limited risk - Systems with specific transparency obligations, such as chatbots that must disclose they are AI, or deepfake content that must be labelled.
  • Minimal risk - Systems with no mandatory requirements beyond good practice. Most AI applications fall here, but this does not mean testing is optional - it means the legal floor is lower.

If you are uncertain about your classification, err on the side of caution. The cost of treating a high-risk system as minimal risk is far higher than the cost of applying appropriate evaluation standards to a minimal-risk system.

What to test

For high-risk systems, the Act mandates testing across several specific areas. Each of these maps directly to evaluation activities you can structure and document.

  • Accuracy across intended use cases - Performance must be declared and demonstrated. You need test results showing accuracy across the range of inputs your system is expected to handle in production.
  • Robustness to errors and faults - Systems must be resilient to errors in inputs, unexpected conditions, and attempts to manipulate outputs. Adversarial testing is not optional.
  • Bias and fairness - Systems must be tested for discriminatory outputs across protected characteristics. This includes testing on demographically diverse test sets and documenting findings.
  • Human oversight mechanisms - You must be able to demonstrate that humans can intervene in, override, or shut down the system. The evaluation should include testing these mechanisms.
  • Data governance - Training and test data must be documented. Gaps, biases, and limitations in the data must be identified and addressed.
  • Transparency and explainability - Users and oversight authorities must be able to understand what the system does and why it produces the outputs it does.

How EvalSpec maps to Article requirements

EvalSpec's evaluation framework was designed with regulatory alignment in mind. The five dimensions map directly to the technical requirements in the Act's key Articles.

Article 9 (Risk Management System) requires documented risk identification, analysis, and mitigation. EvalSpec's Safety and Compliance dimension structures exactly this - forcing you to document identified risks, test for them explicitly, and record your mitigation approach with evidence.

Article 15 (Accuracy, Robustness, and Cybersecurity) sets technical performance standards. EvalSpec's Accuracy and Groundedness and Robustness and Edge Cases dimensions are designed to produce the documented test evidence Article 15 requires.

Article 13 (Transparency and Provision of Information) requires that systems are designed to be sufficiently transparent. EvalSpec's Alignment and Usefulness dimension includes transparency scoring - whether the system behaves in ways that are understandable and predictable to users.

Article 14 (Human Oversight) requires that human oversight is technically feasible and has been tested. EvalSpec includes human oversight as a structured evaluation checkpoint within the compliance dimension.

Using a structured evaluation tool does not replace legal review, but it produces the documented evidence base that legal review and regulatory inspection will look for.

Try EvalSpec free - evaluate your AI system across all 5 dimensions. No code required.

Start your free evaluation

Testing an AI system yourself?

Get the 70 test cases behind this framework, indexed by dimension and difficulty, or run a free evaluation with your own AI system.

The information in this article is provided for general guidance only. EvalSpec is a diagnostic evaluation tool and does not provide legal advice, regulatory certification, or compliance guarantees. For legal or regulatory questions, consult a qualified professional.

404

Page not found

The page you're looking for doesn't exist.

Back to home