What is QA Scoring?
QA Scoring provides data-driven insights into each client’s QA performance:- First-Pass Rate: Percentage of tasks that pass QA on the first review
- Average Cycles: How many QA review cycles a client typically needs
- QA Score: A multiplier used to adjust QA time estimates
Understanding QA Scores
Score Values
A QA score of 1.0 means tasks typically pass on the first QA review:| Score | First-Pass Rate | Meaning |
|---|---|---|
| 1.0 | ~100% | Work passes first time (ideal) |
| 1.5 | ~60% | Often needs one round of fixes |
| 2.0 | ~30% | Usually needs multiple QA cycles |
How It’s Calculated
The QA score is based on:- First-Pass Rate: What percentage of tasks pass QA on the first review
- Average Cycles: The mean number of review cycles needed before passing
- Sample Size: More completed QA reviews = higher confidence
Only tasks with completed QA reviews contribute to the score. In-progress tasks don’t affect calculations until QA is complete.
Confidence Levels
Like deliverability scores, QA scores have confidence levels:| Confidence | Reviews | Reliability |
|---|---|---|
| Low | Fewer than 5 | Limited data - use with caution |
| Medium | 5-15 | Reasonable confidence |
| High | 15+ | Reliable data for scheduling |
How QA Scores Affect Scheduling
T-Shirt Size Suggestions
When Alan suggests a t-shirt size for a development task, it considers the client’s QA score:- High QA scores (>1.5) may trigger a recommendation to bump up the size
- The reasoning explains if QA overhead influenced the suggestion
Subtask Time Estimates
When a quote is approved and converted to a task:- Subtasks are created with pre-calculated AI suggestions
- QA review subtasks get adjusted estimates based on QA score
- Higher QA scores = more time suggested for QA activities
Example
For a client with QA score of 1.8:- Standard QA allocation: 30 minutes
- Adjusted suggestion: ~54 minutes (30 x 1.8, rounded to 15 min)
- Reasoning: “Adjusted for high QA cycles (1.8x avg)“
Viewing QA Scores
QA scores are displayed in the Client Intelligence section of each client’s detail page.What You’ll See
- QA Score: The multiplier (e.g., 1.5)
- First-Pass Rate: Percentage (e.g., 60%)
- Average Cycles: Mean review cycles (e.g., 1.5)
- Confidence: Based on sample size
- Sample Size: Number of completed reviews
Interpreting the Data
- High First-Pass Rate
- Moderate QA Cycles
- High QA Cycles
Score near 1.0, first-pass rate above 80%What it means:
- This client’s work typically passes QA on first review
- Requirements are usually clear and well-understood
- Developer work is consistently high quality
- Standard QA time allocations are appropriate
- Consider this client’s positive pattern
QA Event Tracking
What’s Recorded
Each QA subtask completion records:- Task and Requirement: What was reviewed
- Cycle Number: Which review round (1st, 2nd, etc.)
- Pass/Fail: Whether QA passed or failed
- Developer: Who did the development work
- Reviewer: Who performed the QA review
- BugHerd Points: Number of issues found (if applicable)
Automatic Score Updates
After each QA completion:- The QA event is recorded
- Client’s QA score is recalculated
- Score history is updated (if change > 0.1)
Resetting QA Scores
When to Reset
Consider resetting when:- The client’s development processes have fundamentally changed
- New team members have significantly improved quality
- Historical data is no longer representative
How to Reset
Use the API endpoint or admin interface to reset:- Score returns to 1.0 (neutral)
- Sample size resets to 0
- Reset is logged in score history
Impact on AI Suggestions
QA scores influence AI in two ways:1. T-Shirt Size Suggestions
When generating size suggestions for quotes:- QA metrics are included in the AI prompt
- High QA scores may trigger recommendations to size up
- Reasoning explains if QA overhead was a factor
2. Subtask Time Suggestions
When tasks are created from approved quotes:- Each subtask gets a pre-calculated AI suggestion
- QA subtasks are adjusted by the QA score
- Suggestions appear in the schedule dialog
Related Documentation
Client Intelligence
Overview of all client scoring systems
Subtasks
Understanding subtask types and scheduling
T-Shirt Sizing
How time estimates work
Task Phases
QA phase in the task lifecycle