Skip to main content
Adrosaga

25+ deployment testing types. Built to catch what AI gets wrong.

Your automated test suite tells you whether your AI performs against defined inputs.

These tests tell you whether it behaves with real people. Here are some examples:

  • A frustrated customer disputing a bill
  • A vulnerable user in financial distress
  • A bad actor probing for a jailbreak
  • An elderly user who can't parse the response

Real contributors. Realistic scenarios. Structured evidence your risk and compliance teams can act on.

See the five test packages

The Adrosaga taskmaster pointing towards the five test packages

Five test categories. Here's what each one covers.

From pre-launch readiness to post-deployment drift. Five categories covering what synthetic testing doesn't reach.

A hand raising a rocket across a red star, evoking readiness to launch

AI Readiness Test

Best for:

Pre-launch AI deployments

Includes:

  • UX testing
  • Task completion
  • Hallucination
  • Escalation
  • Trust scoring
  • Executive risk summary

Question:

Is this AI ready to launch?

A warning triangle between a blue gear and dark circuit traces, evoking risk inside a running system

AI Risk Test

Best for:

High-risk or public-facing AI systems

Includes:

  • AI red teaming
  • Harmful output detection
  • Prompt injection
  • Data leakage
  • Vulnerable user testing
  • Severity-rated findings

Answers:

What could go wrong if this AI is released?

A balance scale holding two figures level under a burst of light, evoking even-handed assessment

AI Fairness Test

Best for:

Employment, finance, healthcare, education, insurance, government services

Includes:

  • Bias detection
  • Demographic comparison
  • Accessibility
  • Cultural sensitivity
  • Explainability

Answers:

Is this AI treating different users fairly?

A dart striking the centre of a ringed target, evoking accurate retrieval

RAG Accuracy Test

Best for:

Knowledge assistants, internal copilots, policy bots, document-based AI

Includes:

  • RAG retrieval
  • Groundedness
  • Source attribution
  • Hallucination
  • Policy compliance

Answers:

Is this AI answering from the right source material?

A monitor of charts crossed by arrows, evoking change tracked across versions

Regression & Drift Test

Best for:

Ongoing monitoring after launch

Includes:

  • Repeat scenarios
  • Version comparison
  • Model drift
  • Prompt update testing
  • Monthly or quarterly reporting

Answers:

Is this AI still safe, accurate, and aligned after changes?

All 25+ deployment testing types.

Coverage is scoped to your deployment's applicable risk surface — not applied as a default checklist. Each test type below maps to a distinct failure mode in live AI deployments.

How contributors work: Each contributor receives a testing scenario with detailed instructions and expected duration and outcome. They interact directly with your AI application in accordance with the testing brief. Responses are captured through structured questionnaires and analysed by Adrosaga's AI Data Scientist.

What a typical test run produces: Plain-English summary of the most important findings, and any additional specific requirements such as, harm indicators, pass/fail patterns, demographic comparison, and more.

User Experience Testing

A globe with interface elements, evoking a smooth user journey

Test Description

Whether real users understand the AI, trust it, and can complete the intended task.

Question:

Can real users actually use this successfully?

Example Scenario

Testing whether customers can use an AI assistant to lodge an insurance claim without confusion.

Task Completion Testing

A checklist with a tick, evoking a completed task flow

Test Description

Whether the AI helps the user complete a defined journey or outcome.

Question:

Does the AI actually get the job done?

Example Scenario

Testing whether a sales AI can answer product questions, qualify a prospect, and guide them to book a demo.

Hallucination Testing

Scattered speech fragments, evoking an AI inventing unsupported claims

Test Description

Whether the AI invents facts, prices, policies, sources, eligibility rules, or procedures.

Question:

Is our AI making things up?

Example Scenario

Testing whether an insurance AI invents coverage rules or promises claim outcomes not supported by the policy.

Escalation Testing

An arrow rising to a support symbol, evoking escalation to a human

Test Description

Whether the AI knows when to stop, escalate, or hand over to a human.

Question:

Does our AI know when a human needs to step in?

Example Scenario

Testing whether a healthcare assistant escalates chest pain symptoms to urgent medical support instead of giving general advice.

Policy Compliance Testing

A checklist beside a warning sign, evoking a compliance process check

Test Description

Whether the AI follows the client's approved policies, scripts, workflows, escalation rules, and brand guardrails.

Question:

Does our AI follow our rules?

Example Scenario

Testing whether a bank chatbot follows the correct financial hardship process.

Brand Voice and Tone Testing

A speech mark with a brand tag, evoking on-brand tone

Test Description

Whether the AI communicates in a way that matches the client's brand, tone, values, and customer promise.

Question:

Does this AI sound like us?

Example Scenario

Testing whether a luxury travel assistant sounds premium, calm, and helpful rather than robotic or pushy.

Workflow Testing

Tools arranged around a central eye, evoking a monitored multi-step process

Test Description

Whether the AI performs correctly across a multi-step process.

Question:

Does the AI hold together across the full journey?

Example Scenario

Testing a claims journey from eligibility question, to document upload, to claim submission, to escalation.

Multilingual and Localisation Testing

Overlapping speech bubbles in different scripts, evoking multilingual testing

Test Description

Whether the AI works accurately across languages, countries, dialects, and cultural expectations.

Question:

Does our AI work properly outside our home market?

Example Scenario

Testing a customer service assistant in English, Japanese, Arabic, Spanish, and Hindi.

Accessibility Testing

An accessibility symbol, evoking inclusive design testing

Test Description

Whether the AI experience works for people with different accessibility needs, literacy levels, devices, or communication styles.

Question:

Can all users access and understand this AI?

Example Scenario

Testing whether elderly users or low-literacy users can understand a government benefits assistant.

AI Red Teaming

A red target with a mask and a paper plane, evoking adversarial red-team testing

Test Description

Whether users can manipulate, jailbreak, confuse, or misuse the AI.

Question:

Can people trick our AI into saying or doing something unsafe?

Example Scenario

Testing whether a customer support bot can be persuaded to reveal restricted information, ignore policy, or provide harmful instructions.

Harmful Output Detection

A warning symbol over a chat bubble, evoking detection of harmful advice

Test Description

Whether the AI produces unsafe, offensive, discriminatory, illegal, medical, financial, or reputationally risky content.

Question:

Could our AI cause harm or reputational damage?

Example Scenario

Testing whether a public-facing chatbot gives dangerous financial, medical, or legal guidance.

Sensitive Data Leakage Testing

A cloud leaking data points, evoking exposure of restricted information

Test Description

Whether the AI exposes private, confidential, personal, or restricted information.

Question:

Could our AI reveal information it should not?

Example Scenario

Testing whether an internal AI assistant reveals another employee's personal information or restricted company documents.

Privacy Handling Testing

A shield over a form, evoking careful handling of private data

Test Description

Whether the AI asks for, stores, shares, or exposes personal information appropriately.

Question:

Is our AI collecting or requesting too much personal information?

Example Scenario

Testing whether a healthcare chatbot asks for unnecessary medical details when a safer escalation path is required.

Vulnerable User Testing

A person beside a caution symbol, evoking care for at-risk users

Test Description

Whether the AI handles users who are distressed, elderly, minors, low-literacy, financially vulnerable, or otherwise at risk.

Question:

Does our AI respond safely to people who may need extra care?

Example Scenario

Testing whether a debt collection assistant responds appropriately to someone experiencing financial hardship.

Agentic Action Testing

Connected process nodes with a green tick, evoking an AI completing actions

Test Description

Whether an AI agent safely performs actions, not just generates text.

Question:

Can we trust this AI to take actions on behalf of users?

Example Scenario

Testing an AI that cancels bookings, changes account details, submits claims, creates support tickets, or triggers workflows.

Tool-Use Testing

A gear meshed with a pinned connector, evoking an AI driving connected systems

Test Description

Whether the AI correctly uses external tools, APIs, search, calculators, databases, booking systems, CRM, or ticketing systems.

Question:

Is the AI using connected systems correctly?

Example Scenario

Testing whether an internal AI assistant retrieves the right account information or creates the correct support ticket. Some cases require employee testing due to system access limits.

Bias Detection

A balance scale weighing candidate profiles, evoking fairness checks

Test Description

Whether the AI treats people differently based on gender, age, ethnicity, disability, location, language, income, or other user characteristics.

Question:

Is our AI treating different groups fairly?

Example Scenario

Testing whether an HR screening AI evaluates candidates consistently across names, ages, career gaps, and backgrounds.

Accessibility Testing

An accessibility symbol, evoking inclusive design testing

Test Description

Whether the AI experience works for people with different accessibility needs, literacy levels, devices, or communication styles.

Question:

Can all users access and understand this AI?

Example Scenario

Testing whether elderly users or low-literacy users can understand a government benefits assistant.

Multilingual and Localisation Testing

Overlapping speech bubbles in different scripts, evoking multilingual testing

Test Description

Whether the AI works accurately across languages, countries, dialects, and cultural expectations.

Question:

Does our AI work properly outside our home market?

Example Scenario

Testing a customer service assistant in English, Japanese, Arabic, Spanish, and Hindi.

RAG / Knowledge Retrieval Testing

Linked document nodes, evoking retrieval from a knowledge base

Test Description

Whether the AI retrieves and uses the correct source material from approved documents, knowledge bases, or policies.

Question:

Is our AI using the right source of truth?

Example Scenario

Testing whether an internal HR assistant answers remote-work questions using the correct company policy.

Groundedness Testing

A document marked with a checkmark, evoking answers grounded in sources

Test Description

Whether the AI stays within the available evidence instead of guessing.

Question:

Is the AI answering from approved material, or filling gaps with assumptions?

Example Scenario

Testing whether a legal knowledge assistant clearly states when information is unavailable rather than inventing an answer.

Source Attribution Testing

A document with a pointer to a cited section, evoking source attribution

Test Description

Whether the AI cites or references the correct source document, page, section, or policy.

Question:

Can users see where the AI's answer came from?

Example Scenario

Testing whether an employee policy bot points to the correct HR policy section.

Hallucination Testing

Scattered speech fragments, evoking an AI inventing unsupported claims

Test Description

Whether the AI invents facts, prices, policies, sources, eligibility rules, or procedures.

Question:

Is our AI making things up?

Example Scenario

Testing whether an insurance AI invents coverage rules or promises claim outcomes not supported by the policy.

Policy Compliance Testing

A checklist beside a warning sign, evoking a compliance process check

Test Description

Whether the AI follows the client's approved policies, scripts, workflows, escalation rules, and brand guardrails.

Question:

Does our AI follow our rules?

Example Scenario

Testing whether a bank chatbot follows the correct financial hardship process.

Regression Testing

A calendar orbited by rings with a clock badge, evoking repeated test cycles over time

Test Description

Whether updates to the model, prompt, RAG content, policy, or guardrails break previous behaviour.

This requires on-going testing, and comparison data between test cycles. In order for regression testing to be effective, you must replicate the test configuration from previous test cycles.

Question:

Did our latest AI update create new risks?

Example Scenario

Re-running the same customer support scenarios after a new prompt or model version is released.

Model Drift Testing

A trend line shifting over time, evoking model drift

Test Description

Whether the AI's behaviour changes over time due to model updates, prompt changes, knowledge base changes, or user behaviour.

Question:

Is our AI still behaving the way it did when we approved it?

Example Scenario

Monthly or quarterly testing of the same high-risk scenarios to detect changes in answers, tone, accuracy, or escalation behaviour.

Comparative Model Testing

Side-by-side panels, evoking a model comparison

Test Description

How different models, prompts, vendors, or deployment versions perform against the same scenarios.

Question:

Which AI setup performs best?

Example Scenario

Comparing GPT, Claude, Gemini, or a client's internal model for customer support accuracy, tone, and hallucination rate.

A/B Deployment Testing

An A and B split with arrows, evoking an A/B deployment test

Test Description

Which version of an AI deployment performs better with real users.

Question:

Which version should we launch?

Example Scenario

Testing two versions of a banking chatbot: one with stricter escalation and one with a more conversational style.

Your deployment has a specific risk surface. So does our test scope.

Not every deployment needs the same coverage. A Validation Assessment scopes coverage to your AI system, your applicable regulatory frameworks, and your risk profile — then maps to the test types that apply.

The Adrosaga taskmaster gesturing towards the Book a Demo action

Adrosaga supports a broad range of human AI test types across functionality, experience, compliance, safety and ongoing monitoring.

Adrosaga lets clients share the relevant policies with external contributors, or use an internal staff audience when the material is confidential.

Adrosaga's independent human testing gives deployment approvers stronger evidence that an AI agent is ready to represent the company.

Yes. Adrosaga combines multiple test types within one structured project.

Use Adrosaga when you need evidence of how real people understand, experience or act on an AI system.

See all FAQs