Reperi AI
reperi.aiHow Reperi AI Hired a QA Analyst / Customer Experience Associate Through Experiential Assessment
Reperi AI is an AI-native workforce intelligence platform that helps organizations make smarter hiring and talent management decisions. Using advanced AI and multi-agent systems, Reperi discovers, evaluates, and identifies high-potential talent both inside and outside organizations, enabling companies to build proactive talent pipelines, improve succession planning, and make data-driven workforce decisions through behavioral, cognitive, and performance-based insights.
Reperi AI partnered with TeamCraft to run a two-step experiential assessment to hire a QA Analyst / Customer Experience Associate : a hybrid role requiring both technical rigor and customer-facing empathy.
The Challenge
Reperi AI was growing fast. Their platform was gaining traction, and the team needed someone who could operate at the intersection of product quality and customer success. This was not a standard hire.
The role demanded a rare hybrid profile:
- Technical enough to investigate bugs, run smoke tests, and work alongside the AI engineering team
- Customer-facing enough to guide users through onboarding, answer support questions, and ensure a smooth product experience
- Startup-ready and omfortable with ambiguity, remote work, and wearing multiple hats
- Detail-obsessed because in AI products, a small bug can mean a broken model output or a confused customer
Traditional hiring methods were not going to cut it. Resumes cannot show how someone actually writes a bug report. Interviews cannot simulate how someone behaves when a customer is stuck during onboarding. Reperi AI needed to see candidates in action before making an offer.
A Two-Step Experiential Assessment
TeamCraft designed a two-step assessment specifically for the QA Analyst / Customer Experience Associate role.
Step 1: Preliminary Aptitude Quiz
An AI-graded technical baseline covering manual testing, web testing, API testing, system testing, and QA fundamentals. This established a technical floor and filtered out candidates who lacked foundational QA knowledge without burning engineering time on unqualified applicants.
Step 2: Team Project Simulation
Once the technical baseline was established, shortlisted candidates moved into a team project simulation. They were grouped into small, simulated project teams and given structured tickets focused on testing workflows, bug reporting and documentation, onboarding flow validation, and customer support simulation. This evaluated practical QA execution, documentation quality, analytical thinking, customer communication, and overall product understanding all within a realistic, time-bound environment.
Tools & Stack Used: Exploratory Testing, Test Case Creation, Bug Reporting, Jira Documentation, Smoke Testing, Customer Support Simulation, UX Evaluation
The Process
TeamCraft's platform handled the entire workflow:
| Stage | What Happened |
|---|---|
| Candidate Applies | Reperi AI posted the role; TeamCraft automatically imported applicants and began pre-screening |
| Takes the Assessment | Each candidate completed the real-world, project-based assessment designed for the specific role |
| Joins a Team Project | Candidates were grouped into small, simulated project teams to mirror real workplace dynamics |
| AI + Human Evaluation | TeamCraft's AI engine and human evaluators scored performance across multiple dimensions |
| Selection | Top candidates were shortlisted with detailed reports on technical, collaborative, behavioral, and professional competencies |
The Results
Overall Assessment Metrics
| Metric | Result |
|---|---|
| Total Candidates | 7 |
| Assessed Candidates | 6 |
| Final Shortlists | 6 |
| Tickets Reviewed | 22 |
| Total Tickets Closed | 22 |
| Avg. Bug Reporting Quality | 64.6 |
| Avg. Technical Competency Score | 85.6 |
| Avg. Project Execution Score | 64.6 |
| Avg. Professional Behavioral Score | 77.8 |
Top Performer: Rank #1
| Metric | Score |
|---|---|
| Total Score | 89.53 (Very Good) |
| Bug Reporting Quality | 96 |
| Technical Competency Score | 95.43 |
| Professional Behavior Skills | 85 |
| Tickets Completed | 5 / 5 |
| Attempts | 5 / 5 |
Detailed Technical Breakdown
| Evaluation Dimension | Score | Weight | Contribution |
|---|---|---|---|
| QA Testing & Analytical Capability | 96 | 45% | 43.2 |
| Project Execution & Engagement | 73 | 25% | 18.25 |
| Documentation & Communication Quality | 95.4 | 20% | 19.08 |
| Professional Competency & Collaboration | 90 | 10% | 9.0 |
| Final Score | 89.53 |
Sub-Skill Performance
| Skill | Score |
|---|---|
| Exploratory Analysis | 94.4 |
| Test Coverage | 96.2 |
| Attention to Detail | 95.2 |
| Testing Accuracy | 95.2 |
| Documentation | 96.8 |
| Bug Detection | 94.8 |
Why This Hire Mattered
The top candidate did not just test well. They demonstrated the exact hybrid capabilities Reperi AI needed:
- Technical rigor: 96 in QA Testing & Analytical Capability, with near-perfect scores across all testing sub-skills
- Documentation excellence: 95.4 in Documentation & Communication Quality -- critical for a role that requires clear bug reports and clear customer guidance
- Collaboration under pressure: 90 in Professional Competency & Collaboration, proven in a simulated team environment
- Ownership mentality: 5/5 tickets completed, showing reliability and follow-through in a remote setting
The experiential format surfaced what traditional hiring could not: this candidate could sit between engineering and customers, translating technical issues into human solutions.
Business Impact
| Outcome | Detail |
|---|---|
| Speed to Quality Signal | Moved from role posting to data-driven shortlist in days, not weeks |
| Hybrid Role Validation | Proved the candidate could handle both QA and CX responsibilities before Day 1 |
| Reduced Hiring Risk | Objective, multi-dimensional scoring replaced gut-feel decisions with performance data |
| Remote-First Confidence | Assessment was designed for remote execution; the top performer thrived in that format |
| AI-Augmented Decision Making | Reperi AI used TeamCraft's AI + human evaluation to make its own talent decision, modeling the data-driven approach it sells to customers |
Why TeamCraft Worked for Reperi AI
| TeamCraft Capability | How It Helped |
|---|---|
| Role-Specific Assessment Design | Not a generic QA test: tailored for QA plus Customer Experience |
| Real-World Simulation | Candidates worked on actual ticket types they would see on the job |
| Multi-Dimensional Scoring | Technical, Collaborative, Behavioral, and Professional scores provided a complete candidate picture |
| AI + Human Evaluation | Objective data from AI grading, nuanced judgment from human evaluators |
| Detailed Reporting | Per-candidate scorecards with weighted contributions and sub-skill breakdowns |
| Remote-Native | Entire assessment run remotely, matching Reperi AI's distributed team model |
Frequently Asked Questions
Why did Reperi AI choose experiential assessment over traditional interviews? The role required a rare hybrid of QA technical skills and customer-facing empathy. Resumes and standard interviews cannot evaluate how someone actually writes a bug report or guides a confused customer. The hands-on simulation revealed real performance, not interview performance.
How were team dynamics evaluated in a remote setting? Candidates worked in simulated project teams, collaborating on structured tickets. Evaluators scored communication quality, documentation clarity, and responsiveness: all critical signals for a remote-first, startup environment.
Can other AI companies use this approach for hybrid roles? Yes. The two-step model: aptitude baseline followed by role-specific simulation: works for any hybrid role where technical depth and human skills both matter. It is especially effective for early-stage companies that need multi-capable hires.
What made the top candidate stand out? The top performer scored 96 in Bug Reporting Quality and 95.43 in Technical Competency, while also demonstrating strong professional behavior with an 85. They completed all 5 tickets and showed they could operate at the intersection of engineering and customer success.