First-year advising assistant
An AI assistant on the student portal answers registration and policy questions, books advising appointments, and hands personal or high-stakes questions to an advisor.
Illustrative example. Figures are invented to show the method.
Outcome hypothesis
If we offer an AI advising assistant in first-year registration support, we expect more students to clear registration holds before the deadline because answers are available at any hour, provided answers are accurate and accessible, and personal questions reach an advisor.
What was measured
- Platform: $120,000 a year
- Knowledge base build: 400 hours
- Content upkeep: half of one position
- Advisor time reviewing escalations
- Access: all 4,200 first-year students
- Use: 46% used it at least once
- Quality: answer audit found 91% accurate, below the 97% threshold set for policy answers
- Holds cleared before deadline: 78% to 81%
- Three incidents of wrong deadline information
- Lower satisfaction among mobile and screen-reader users
Lens gates
| Lens | Result | Finding |
|---|---|---|
| Quality | Fail | 91% accuracy on policy answers, against a 97% threshold. The High band sets a minimum; this use case set a stricter bar for policy answers. |
| Capability and readiness | Watch | Advisors were not staffed for the escalation volume. |
| Equity and accessibility | Fail | Screen-reader users could not complete the booking flow. |
| Governance and risk | Watch | Incidents were logged, but who owns a wrong answer was unclear. |
| Evidence quality | Watch | One term, compared with last year’s cohort. A new registration system also launched. |
What can be claimed
A small observed rise in holds cleared on time.
That the assistant caused the rise. The new registration system is a competing explanation.
Quality and accessibility gates failed, and strong usage doesn't offset that. Limit the assistant to booking and official links until policy answers meet the threshold and booking passes an accessibility audit.
Re-audit answers and accessibility before next registration period.
Set thresholds before launch. Because 97% was agreed in advance, the decision to restrict was quick and uncontested.