AI-assisted literature screening
A public health lab must screen 6,000 abstracts for a systematic review. AI ranks them by likely relevance; two people still screen, highest-ranked first.
Illustrative example. Figures are invented to show the method.
Outcome hypothesis
If we use AI relevance ranking in abstract screening, we expect screening hours to drop by 40% because reviewers see likely matches first and can stop earlier, provided recall stays at or above 95% against dual human screening.
What was measured
- Tool license: $1,200
- Setup: 10 hours
- Validation sample screening: 30 hours, counted as part of the cost
- Used on one review by both screeners
- Methods librarian set the stopping rule
- Screening hours: 220 to 130
- Recall on validation sample: 96%
- Two relevant studies found only by human screeners
Lens gates
| Lens | Result | Finding |
|---|---|---|
| Quality | Pass | Recall met the 95% threshold. |
| Capability and readiness | Pass | Methods librarian validated the protocol. |
| Equity and accessibility | Watch | License cost may keep smaller labs from using it. |
| Governance and risk | Pass | Method disclosed in the paper per journal policy. No unpublished data uploaded. |
| Evidence quality | Pass | Within-study validation gives strong attribution. |
What can be claimed
Task time reduced and capacity redeployed: the saved hours went to full-text review, which finished three weeks early.
That the method works for every review. Recall should be validated each time.
Evidence, safeguards, and capacity support wider use. Offer it through the library to other review teams, with the validation protocol required.
Library reviews recall results across the first five teams.
Validation is part of the cost. Leaving out the 30 validation hours would make the tool look cheaper than it is.