Decision guideOne standard, any scale

The AI is approved. Is it working, and is it worth it?

Every approved use, from one person’s task to a campus platform, is judged the same way: the same six steps, the same three ratings, the same five checks, and the same decision rules. Only the scale changes, from one person to the whole institution.

DepthInformal to formal ProcessSix steps What to includeItemized by depth CriteriaStrong, adequate, weak DecideRatings to a decision By levelWhat changes

Go as light or as formal as the use needs

The questions never change: Cost, Use, Outcomes, and the five checks. What changes is how carefully you gather the evidence and how formally you record the decision.

Depth 15 minutes Quick self-check Answer the three questions from what you’ve noticed. Rate by judgment. Record: Nothing, or a note to selfFits: Trying a tool on your own tasks

Depth 22 weeks Light check Jot before-and-with-AI numbers for one task. Run the five checks. Record: A note or simple spreadsheetFits: Regular personal or small-team use

Depth 3One review cycle Standard review Baseline, comparator, a 3 to 5 metric scorecard, all five checks. Record: Registry entry and unit scorecardFits: Team and unit workflows

Depth 4A term or more Formal evaluation Phased or controlled design, subgroup results, full cost attribution, metric contract. Record: Registry, portfolio scorecard, governance decisionFits: Campus-wide or high-stakes uses

Move up a depth when more people rely on it, the stakes rise, money or contracts are involved, or the decision will inform a budget. The review band sets the minimum depth and how often to re-rate. Re-rate right away after a model change, a quality drop, an incident, or a cost rise.

The process: six steps, for every use

An individual might do this in a notebook over two weeks. A unit does it with a scorecard each review cycle. The steps are the same.

Step 1Name the useThe task, workflow, or service, and the outcome it should improve.
Step 2Set the baselineHow the work was done before, or a non-AI alternative.
Step 3Collect the evidenceCost, Use, and Outcomes, plus the five checks.
Step 4Rate itStrong, adequate, or weak on each, using the shared definitions below.
Step 5Decide and recordApply the decision rules. Record the decision, owner, and next review date.
Step 6Reflect and shareWhat did we learn? What would we do differently? What should change in how we measure or decide? Share it.

What to include: Cost, Use, and Outcomes

Each depth adds items to the one before. Choose a depth to see what to count.

ACost{{ ck.costCount }}

BUse{{ ck.utilCount }}

COutcomes{{ ck.impactCount }}

✓ include at this depth · ○ added at a deeper level. Detail: measurement domains · how to define each metric

When the data is thin

Most units can’t run a controlled study, and they don’t need to. Collect the least evidence that fits the review band, and say plainly what it can’t show.

No baseline? Use a proxy

Time 5 to 10 tasks by hand this week, before switching
Use last term’s figures from the same office
Compare with a similar team not using AI yet
Ask users to estimate, and label it self-reported

Minimum evidence by band

Low: before-and-after timing, plus a spot check of 10 to 20 outputs
Moderate: add a baseline from records and one outcome measure
High: add a comparison group or phased rollout, plus results by subgroup
Critical: add an impact assessment and alternatives considered

Common mistakes

Counting logins as value
Claiming freed hours as budget savings
Crediting AI for change caused by a new system or staffing
Leaving out review and correction time
Measuring once and never again

How strong is the evidence?

Say how strong your evidence is, and label every number by how you got it. The bar rises with the review band: Low can act on level 2, while High and Critical need level 4 or 5.

Level	What it means
1 · Anecdotal	User impressions or informal observation
2 · Measured	A documented baseline and an after measure
3 · Repeated	Measured over several periods or groups
4 · Comparative	A comparison group, or other explanations ruled out
5 · Causal	An experimental or quasi-experimental design

Label	Use when
Measured	Observed during the evaluation period
Estimated	Calculated from measured data, with the method stated
Projected	A forecast of future results, with assumptions stated
Unverified	Reported but not yet checked

Every evaluation reports its baseline, sample, period, method, other likely causes of change, and how certain the result is. Decide on measured results; treat projections as reasons to keep collecting.

The criteria: one rating scale for everyone

Rate each of the three on the same scale. What you count differs by level; what “strong” means does not.

	Strong	Adequate	Weak	Watch out
A · Cost	Lower cost per successful, checked result than before	About the same, with other gains	Higher than before, or unknown	Count review and fixing time. Count hours as capacity, and dollars as savings only when spending falls.
B · Use	Used in the intended workflow by most intended users, and sustained	Used by some, or in part of the workflow	Little or falling use, or access without use	Count use in real work, not logins or seats.
C · Outcomes	Clear gain on the primary outcome compared with the baseline	Some gain, or mixed across measures	No gain, or worse	Report observed change separately from what can be attributed to AI.
Five checks	Pass on all five	Watch on one or more	Fail on any one	Quality, readiness, equity, governance and risk, evidence. The bar rises with the review band.

Using 0 to 1 scores, as in the inventory? Strong is 0.70 or more, adequate 0.50 to 0.69, weak below 0.50.

From ratings to a decision

Set the ratings on the left. Rules are read top to bottom, and the first match is the decision.

Decision · rule {{ res.rule }} {{ res.decision }}

Record the decision, an owner, and the next review date. Scorecard and monitoring

What changes with scale: one person to the whole institution

The same three questions, measured at the right scale, rated and decided by the right person.

	Individual	Team	Unit	Institution
The use	One task	One workflow	A service, or several workflows	The portfolio or a campus platform
Cost counted	Your time, including checking and fixing	Shared tools, setup, review, coordination	Budget, staff time, support	Portfolio spend, infrastructure, governance, exit
Use measured	Which tasks, how often	Share of workflow runs using AI	Adoption by role; priority workflows covered	Reach (each person once), duplication, unapproved use
Outcomes measured	Time and quality on the task	End-to-end time, rework, quality	Service and mission outcomes, by subgroup	Strategic outcomes, equity, financial effect
Who rates	You	Team lead	Unit lead, with a data steward	AI governance lead or council
Who decides	You, within the approved use	Team lead	Unit lead for Low band; governance above	Governance lead, council, or executive sponsors, by band
Recorded in	Optional self-note	Registry entry	Registry and unit scorecard	Portfolio scorecard
Reported to	Counted in aggregate by tool and role	Unit	Institution	Budget and IT portfolio review

Individual guide →Unit guide →Full decision matrix →

Six rules that keep it consistent across campus

1Same definitionsStrong, adequate, and weak mean the same thing at every level and in every unit.

2Same baseline ruleAlways compare with how the work was done before, or a non-AI alternative.

3Same checksAll five, every time. Only the bar changes, set by the review band.

4Same decisionsFive options, each with an owner and a next review date.

5Roll up decisions, not scoresNever average ratings across uses. Report how many uses got each decision, and why.

6Re-rate when scope growsA use that spreads to the next level is rated again there, with that level’s measures.