Generate
Thousands of candidate items, written against tight design briefs and anchored to specific behaviors.
How we build it
A thousand simulations before anything is built.
AI has changed how an assessment can be designed, and it has not changed how one has to be proven. This page shows what we built with it and the evidence behind each piece, and it goes deeper than the rest of the site on purpose.
Before I became an I-O psychologist, I trained as an aerospace engineer in the nineties, just as computers got good enough to simulate the physical world and design stopped being build-it-and-see. Psychology is getting its wind tunnel now, and the simulation finds the design while people prove it.
Tariq Shaban, Co-Founder & Product & Innovation DirectorThe method
The same five steps build every assessment.
The same method built the behavioral framework, the personality suite, the judgment scenes and the reasoning test. The first four steps take a day each, and the fifth takes as long as it needs. Judgment’s nine-step build runs inside the same five-step method, and a full client build, including your experts’ time, takes four weeks.
Review and edit
Every item is checked for target, reading level and wording that could disadvantage a group, and it comes back edited or comes out.
Pseudo Factor Analysis
A structural read of the items themselves, showing which ones are likely to behave before anyone has answered one.
Pilot on Qarin twins
Every new item is answered by Qarin, digital twins of 662 working adults, before it reaches a candidate.
Trial on people
Every instrument, without exception, because the simulation finds the design and the trial proves it.
The technology
How the technology worksSix pieces of technology, each with a psychologist in charge.
Each one does the slow part of the work, and a named psychologist signs off what it produces before the next step starts.
- SJT Studio Scripts, casting, rendering and production of the judgment scenes, with a psychologist approving each step.
- Pseudo Factor Analysis Reads the structure of items from their wording before anyone has answered.
- Qarin digital twins Pilot new items on twins of real respondents before the human trial.
- AI keying panel Rates the response options to inform the scoring key; the experts’ agreement sets it.
- Hold-to-view engine The reasoning question stays on screen only while the candidate holds it there, with an interaction trace beside it.
- Integrity model A confidence score built from how the test was taken, never from the answers, raised as a flag for a person to review.
Checked against the real thing
Nothing in the pipeline is taken on trust.
Subject-matter experts’ judgments are checked for agreement before they become a scoring key, and every stage a model touches is checked against human data on the same content. The chart shows the example we publish: how much of the design blueprint Pseudo Factor Analysis recovers from the items alone, beside a conventional factor analysis of real responses to the same items.
Its job is to decide which items go to trial, never to replace the trial. At the factor level the three models combined recover the blueprint about as often as the conventional analysis does, and the trial on real people decides the rest.
Recovery scales with the model
Blueprint recovery, %The blueprint gradient, both lenses
Mean value across item pairs| Factor recovery | Facet recovery | |
|---|---|---|
| MiniLM-L6 22M | 70 | 55 |
| MiniLM-L12 33M | 69 | 54 |
| mpnet-base 110M | 74 | 66 |
| Aggregate 3 models | 79 | 70 |
| Empirical EFA n = 790 | 78 | 89 |
| Semantic similarity (mpnet) | Polychoric correlation | |
|---|---|---|
| Within facet | .45 | .55 |
| Within factor, cross-facet | .25 | .31 |
| Between factors | .18 | .11 |
Source: the Miraai Validation Study, 2026, blueprint recovery by embedding model, 790 completed forms. Read it on the Open Science Framework
Qarin digital twins
We pilot new items on twins of 662 working adults.
Pseudo Factor Analysis reads the structure of an item, but it cannot tell you how people will answer it. So before a new item reaches a candidate, it is answered by Qarin, digital twins of 662 working adults who took the personality assessment, each twin built from one person’s biodata and a written profile of their personality, answering items that person has never seen. 100 of the twins are released publicly on GitHub; the validation study used 790 completed forms.
We tested the Qarin twins the way we test everything else, by trying to break them. We hid deliberately bad items among good ones, items that were gendered, age-typed, double-barreled or wrongly keyed, and the Qarin pilot caught 33 of 36 of them. It does not replace the human trial, which still happens every time, and it means the people in that trial answer items that have already been through the tunnel.
662working adults, each with a Qarin twin built from their biodata and a written profile of their personality, checked against the same people’s real answers to 144 pairs
33 / 36planted defects caught by the Qarin pilot in the critical areas, with the weakest quarter of items flagged at AUC .85
Every timea human trial follows, and no Qarin result reaches a norm table or a report
On the same 144 pairs, the twins’ item means correlate .59 with the people’s, their item-total correlations .75, and factor reliability reproduces to within .05 (.89 against .89). Source: The Pilot Before the Pilot, Miraai technical report, 2026. Read it on the Open Science Framework · The code to run the twins, on GitHub
The behavioral framework
We define the behavior before we measure it.
A hiring test is judged by how well it predicts performance, yet in most organizations performance is one overall rating from one manager. Competency models were meant to fix that, and most are written in a two-day workshop. Ours was built from published evidence, 898 behavior statements from 140 published sources, and the structure came out of that material rather than out of a room of people agreeing with each other.
898behavior statements
140published sources
6 · 18 · 77clusters, competencies and observable behaviors
All seventy-seven behaviors, placed by how closely they relate. The six clusters were not assigned afterwards, they are where the behaviors grouped themselves. Drag, zoom, or click a behavior to see it and its source.
The evidence
Documented in full, for every assessment.
The old standards for documenting an assessment were never enough for anyone outside the publisher to check its claims, so we publish the work behind ours.
Personality
Six factors, twelve traits and twenty-four facets, calibrated on 790 completed forms from working adults. The validation study is public with its report, hypothesis outcomes, analysis log and pair statistics, more than two hundred pages covering reliability, stability, format equivalence, fairness, criterion evidence, faking resistance and norms.
Read itJudgment
Every scenario bank is documented from end to end, from where each scenario came from and who approved it to how the key was derived and what the fairness testing found. No stage opens until a named psychologist has signed the one before it, and the whole method, with the keying study against a panel of experts, is public.
Read itReasoning
Underneath the mechanic you hold is an adaptive test scored on one item response scale, so every score means the same thing whichever questions a person saw. Its hold-to-view design makes cheating harder, and its design is documented to the same standard as everything else on this page.
See itWhere AI is used
AI does the work, and a person makes every call.
Every place a model touches the work, what it does there and who decides, in one table. AI builds and checks the assessment. No AI model scores, ranks or rejects a candidate.
| Use | Where | What it does | Who decides |
|---|---|---|---|
| Generating content | Every build | Drafts and screens items, scenes and responses, far more than an assessment needs. | A named psychologist approves each stage, and the approval is kept. |
| Pseudo Factor Analysis | Every build | Reads item wording to predict structure before anyone has answered. | Its read decides which items go to trial, and a human trial follows every time. |
| Qarin digital twins | Piloting | Simulate responses to find items that fail or are unfair. | No twin result reaches a norm table or a report, and a human trial follows every time. |
| AI rater panel | Building Judgment keys | Informs the scoring key while the test is built. Experts’ agreement sets the final key. | Experts. |
| Scoring candidates | All Miraai assessments | Fixed scoring keys and item response models. In Judgment, a candidate’s score comes only from how they rate the responses to each scene, and answering out loud is not part of the test. No AI model scores, ranks or rejects a candidate. | People |
| Conversational guide | Guidance only | Explains results and answers only from the verified career library. | Hands anything else to a counselor. |
| Career fit score | Guidance only | A computed profile-match coefficient between the person’s profile and each occupation’s, not a trained model. | Every match shows the traits behind it, and a counselor holds the conversation. |
| Integrity checks | Reasoning and delivery | Identity check by facial validation, a model-based confidence score from how the test was taken, and optional AI video proctoring, a flag for a person to review, never a verdict. | A person, never the check. |
Where AI is used, and where it isn’tResearch in progress: scoring open answers
Open Research Program
Our research is open to collaboration.
We run joint studies with universities and practitioners, and share our instruments and data for academic work.
Built to the APA/AERA/NCME Standards and the SIOP Principles.
Every study below is on the Open Science Framework under its own DOI, with what it tested, who took part, what it found and what is public, so a reader can check a claim without asking us first. The same index is at research.html as plain text.
Research index
The Miraai Open Research Program 2026
- What it tested
- The program that holds every Miraai study, how the studies connect, and the standards each one is run to.
- Sample
- All studies
- What it found
- The protocols, reports, analysis logs and results of every study below are linked from one place, and de-identified data and analysis code are shared with approved researchers under a research agreement, except the Qarin digital twin code and 100 released profiles, which are public on GitHub.
- Public
- Program paper, links to every study, the code for the digital twins · doi:10.17605/OSF.IO/RH3M7
The Miraai Validation Study 2026
- What it tested
- The personality assessment in its pictured and text formats: structure, reliability, two-week stability, equivalence of the two formats, fairness, criterion relationships, resistance to AI faking, and, in an operational chapter of the report, how it performed with 15,069 real applicants.
- Sample
- 790 completed forms from working adults, 313 retested at two weeks, 152 who took both formats, and, in the report’s operational chapter, 15,069 applicants under real stakes
- What it found
- Factor reliability .82 to .93 (α) and .88 to .95 on the model matched to the format; two-week stability .77 to .88; the two formats equivalent (ΔCFI at most .001, no pair with format DIF); no pair beyond the negligible DIF boundary by gender, age or education; Conscientiousness .42 with organizational citizenship, Agreeableness .48 with citizenship toward individuals, Conscientiousness −.42 with counterproductive behavior; first- and second-language applicants within a tenth of a standard deviation on every factor.
- Public
- The full report, the hypothesis outcomes, the analysis log, the pair statistics and the codebook; de-identified data and analysis code for approved researchers under a research agreement · doi:10.17605/OSF.IO/U693G
The Pilot Before the Pilot 2026
- What it tested
- Whether Qarin digital twins, each built from one person’s biodata and a written profile of their personality, reproduce what a human pilot finds about new items, and which planted defects they catch. 662 twins were built, and 100 are released publicly on GitHub.
- Sample
- 662 working adults and their twins, 144 pairs, 116 people on two published inventories, 92 new items with 69 planted defects
- What it found
- Item means correlate .59 with the people’s, item-total correlations .75, factor reliability reproduces to within .05 (.89 against .89), group differences reproduce almost exactly, the weakest quarter of items is flagged at AUC .85, and the twins catch 33 of 36 planted defects in the critical areas.
- Public
- The report, the analysis log, the codebook, 100 Qarin profiles and the code to run them on GitHub · doi:10.17605/OSF.IO/PV67F
The AI Panel Against the Experts 2026
- What it tested
- How an AI rater panel compares with a panel of human experts in keying a situational judgment test, and whether the test keyed with the panel’s help is as sound and as fair as the one keyed by people.
- Sample
- Both panels rating the same 96 responses, and 9,433 real completions of a leadership judgment test rescored under every key
- What it found
- The panel’s ratings correlate .84 with the experts’. It rates harder (by 0.57 on a five-point scale) and its raters agree with one another too much, so the key is calibrated for both. The panel-keyed test is as internally consistent as the expert-keyed one, and its items behave the same for men and women, older and younger candidates, and across ethnic groups.
- Public
- The report as a PDF · doi:10.17605/OSF.IO/NJXF7
Behavioral, Model-Agnostic Integrity in Online Ability Testing 2026
- What it tested
- Whether watching how a reasoning test is taken, rather than what the answers contain, catches outside help: a field comparison of monitored and unmonitored sessions, and an experiment with a session confidence score.
- Sample
- More than 106,000 reasoning test sessions in the field, with and without monitoring, and an experiment with 372 participants
- What it found
- Unmonitored sessions scored higher than proctored ones, most on verbal and numerical reasoning. A confidence score built from how the test was taken, never from what was answered, tells helped sessions from honest ones (AUC .82), and a low score raises a flag for a person to review, never a verdict.
- Public
- The paper as a PDF; presented at SIOP 2026 · doi:10.17605/OSF.IO/CKNRA
Fast Tests, Expert Judgment 2026
- What it tested
- How video situational judgment tests are built at scale with industrial-organizational psychologists in charge: the nine-step pipeline, the standard applied at each step, what the psychologist decides, and the research the design rests on.
- Sample
- The pipeline itself, with its keying step measured against a panel of human experts and 9,433 candidates
- What it found
- Language models draft at every step and a psychologist makes every decision, written to an audit log with who made it and why. No step opens until the one before it is approved. An eighteen-scenario bank went from incident extraction to an approved draft key, with its faking review, for about twenty dollars in model calls.
- Public
- The paper, licensed for non-commercial reuse · doi:10.17605/OSF.IO/BQHRA
The Miraai Behavioral Framework 2026
- What it tested
- A behavioral framework built from published evidence rather than a workshop: the statements, the sources and the structure that came out of them.
- Sample
- 898 behavior statements from 140 published sources
- What it found
- Six clusters, eighteen competencies and seventy-seven behaviors, each traceable to its sources.
- Public
- The framework paper, licensed for non-commercial reuse · doi:10.17605/OSF.IO/NP63V
A validation claim is only as good as what you can check. Ours sit on the Open Science Framework and GitHub, with the protocols, the analysis logs and the results beside them. Most providers publish a brochure and hide the evidence that does not look favorable. We publish the whole study, not a summary. What we expected to find, what we found and how we analyzed it are all in the open.
Next step
Send us one job description.
We'll build the first scene for it and show you, so you can look at the real thing instead of reading about it.