EU AI Act obligations are phasing in through 2028

The no-code AI evaluation framework

Get an evidence-based readiness report for your AI system.

EvalSpec scores your AI system across 5 quality dimensions using 70 pre-built test cases, then maps the results to EU AI Act, NIST AI RMF, and ISO 42001. No code. No consultant. Run the prompt pack through your AI, upload the responses and we do the scoring.

Built for the person who has to hand someone else a report: a QA lead, a PM, a compliance officer. Structured evidence, not gut feel.

No credit card required. Your first evaluation is free.

  • 70 pre-built test cases
  • 5 quality dimensions scored
  • 3 frameworks mapped: EU AI Act, NIST AI RMF & ISO 42001
  • Free first evaluation. Upload responses, get a scored report.

The 5 Evaluation Dimensions

Each dimension targets a distinct failure category. Together, they provide comprehensive coverage of AI system quality.

Accuracy & Groundedness

Is the output factually correct? Is it grounded in the provided context rather than fabricated?

Key questions: Does the system hallucinate? Can it distinguish between what it knows and what it doesn't? Does it cite sources accurately?
Measurement: Factual verification against ground truth, citation accuracy scoring, hallucination rate tracking, knowledge boundary detection.

Consistency & Reliability

Does the system deliver similar quality for similar inputs? Are outputs predictable in format and structure?

Key questions: Does the same prompt produce wildly different results? Does output format remain stable? Are quality levels consistent across sessions?
Measurement: Semantic similarity across repeated runs, format compliance rate, variance analysis on quality scores.

Safety & Compliance

Does the system refuse harmful requests? Can it withstand prompt injection? Does it meet regulatory requirements?

Key questions: Can the system be jailbroken? Does it leak sensitive data? Does it comply with GDPR, EU AI Act, and internal policies?
Measurement: Prompt injection success rate, harmful content generation rate, PII leakage testing, compliance checklist verification.

Alignment & Usefulness

Does the output serve the user's actual intent? Is the tone, length, and format appropriate for the context?

Key questions: Does the system understand implicit instructions? Does it follow specified constraints (word count, tone, audience)? Is the output actionable?
Measurement: Instruction-following compliance, user satisfaction scoring, task completion rate, constraint adherence metrics.

Robustness & Edge Cases

How does the system handle unexpected, malformed, or adversarial input? Does it degrade gracefully?

Key questions: What happens with empty input? Extremely long input? Mixed languages? Contradictory instructions? Does it fail silently or communicate the issue?
Measurement: Error handling coverage, input boundary testing, graceful degradation assessment, recovery behavior analysis.

See a real report

Not a mock-up. An evaluation of a fictional support chatbot, scored by the EvalSpec judge and recorded with a hash anyone can check.

3.8overall score out of 5
12test cases scored by the judge
1critical failure that held the status at Poor

The chatbot refused prompt injection and handled empty input well. It also agreed to rank job candidates with no mention of human oversight, which scored 1 and kept an otherwise decent result from passing. The report shows every case, the judge's reasoning and how the result was recorded.

For agencies

Handing an AI system over to a client?

Give them evidence with the handover, not a promise that you tested it.

Scored by a judge, not by you

An LLM judge scores each response against the test criteria and gives its reasoning. If you change a score, the report says so and keeps the judge's score beside it.

A record that cannot be quietly edited

Each finalised result is stored with a record hash. Change a score or a test case afterwards and the stored result is replaced, so an old report no longer matches.

Your client can check it themselves

Anyone you give the hash to can confirm the record on the verify page, without an account and without seeing your client's data.

How It Works

An evaluation is a complete scored run of your AI system across selected test cases. Here's how it works - no code, no pipelines, just structured evaluation in your browser.

1

Define your system

Name your AI system, select its type - chatbot, RAG, code generation, or more - and set its risk level. EvalSpec auto-configures the evaluation based on your context.

2

Choose your test cases

Pick from 70 pre-built test cases mapped to EU AI Act risk tiers, or add your own. Each test targets a specific failure mode across five evaluation dimensions.

3

Score responses

Upload your AI system's responses by CSV and an LLM judge scores each one 1-5 against the test criteria, with its reasoning. Review every score and override any you disagree with. Prefer to do it by hand? Paste responses in and score them yourself.

4

Get your results

Receive a structured evaluation report with dimension scores, Readiness Status indicators, a radar chart, critical issues list, and a deployment readiness assessment. Export as CSV or PDF.

EU AI Act

Building AI under the EU AI Act?

EU AI Act high-risk obligations land 2 December 2027 (Annex III) and 2 August 2028 (Annex I), with Article 50 transparency rules applying since 2 August 2026 (systems already on the market before then have until 2 December 2026 to add content marking). Breaching these obligations can mean fines of up to €15M or 3% of global turnover. EvalSpec maps its test cases to the testing and accuracy parts of Articles 9–15. 70 test cases mapped to every risk tier, with structured test evidence you can export as CSV or PDF.

See What You Get

Polished, structured outputs designed to give you clarity - not just data.

evalspec.ai
Setup
Dimensions
Test Cases
Evaluate
Results

Step 1: System Setup

Customer Support Chatbot
LLM Chatbot
Low Medium High Critical
evalspec.ai
Accuracy Consistency Safety Alignment Robustness
3.7 / 5.0
Needs Work
Accuracy & Groundedness 3.8 Adequate
Consistency & Reliability 3.6 Adequate
Safety & Compliance 4.5 Good
Alignment & Usefulness 3.4 Adequate
Robustness & Edge Cases 3.2 Adequate
5
8
4
12
3
9
2
2
1
1
evalspec.ai

Your Evaluations

Customer Support Chatbot
LLM Chatbot
3.7 Needs Work
2 days ago
Internal RAG System
RAG
3.2 Adequate
1 week ago
Code Assistant v2
Code Generation
2.8 Needs Work
2 weeks ago

Test case library

Browse all 70 test cases before you sign up

Every case lists what it tests, its dimension, system type, difficulty and OWASP mapping. Prompts, expected behaviour and scoring guidance open when you create a free account.

Browse the test case library

Pricing

First evaluation free, then from £14.99

No subscription. Buy one credit at a time or a bundle of 5 or 20, and credits never expire. One credit covers one evaluation of one AI system.

See pricing

Built for QA leads, not ML engineers

You have been told to "test the AI" and handed no framework for non-deterministic systems. EvalSpec gives you a structured 5-dimension methodology and 70 pre-built test cases to work from immediately - no code, no consultant, a report you can hand to whoever asked for it.

Built for product managers, compliance officers, AI/ML leads, and engineering leadership who need the same kind of evidence.

No credit card required. Your first evaluation is free.

No Python required

Most AI evaluation tools require code. EvalSpec runs entirely in your browser - structured, rigorous, and accessible to every stakeholder on the team.

Free download: EU AI Act Compliance Checklist for SMEs

A structured checklist covering all 5 evaluation dimensions with EU AI Act article references.

70 pre-built scenarios

Test Case Library

Browse, filter, and select from curated test cases for evaluating AI systems across all dimensions.

Sign in

Access your evaluation projects.

or

Forgot your password?

Don't have an account? Create one

Create account

Start evaluating your AI systems.

or skip the password

Already have an account? Sign in

Reset password

Enter your email and we'll send you a reset link.

Back to sign in

Choose a new password

Enter and confirm your new password below.

Evaluation Builder

Define your system, select criteria, choose test cases, and run your evaluation.

1
System
2
Criteria
3
Test Cases
4
Evaluate
5
Results

Define Your System

Select Evaluation Criteria

These five dimensions define what "good" looks like for your AI system. We've pre-set the weights based on your system type - higher weight means that dimension counts more toward your overall score.

Hover the ⓘ icon next to each criterion to learn what it measures and when you might want to adjust its weight.

Select Test Cases

We've suggested test cases based on your system type. Select the ones you want to include, or add your own.

Run Evaluation

For each test case, paste the actual system output and score the result.

0 / 0 scored

Your Projects

+ New Evaluation
EU AI Act

EU AI Act Compliance Audit

Pre-configured evaluation template with all high-risk test cases mapped to EU AI Act obligations. Produces structured test evidence to support the testing and accuracy parts of your Articles 9-15 technical documentation.

Includes 54+ high-risk test cases across all 5 evaluation dimensions

Back to Dashboard

Compare Evaluations

Session Activity

Your wizard activity in this tab. This log stays in your browser and clears when you refresh or close the page. See our Privacy Policy for the site analytics we do collect.

Blog

Insights on AI evaluation

Practical guides, frameworks, and best practices for testing AI systems before production.

Views expressed in these articles reflect the EvalSpec team's experience and research. They do not constitute professional, legal, or regulatory advice.

404

Page not found

The page you're looking for doesn't exist.

Back to home