← Back to the blog

Blog

AI in Projects: Match Review Depth to Risk

A qualitative study published on 16 October examines 26 specific AI incidents across five technical project environments. Its most useful finding is not a productivity promise. Not every AI output needs the same level of control.

Of the 26 incidents, 13 were classified as positive, seven as negative and six as near misses. Almost half, in other words, produced an adverse outcome or were only defused because a person intervened. How much review was needed also differed considerably from task to task. Drafting and documentation usually required limited correction. Scheduling, technical data analysis and sensitive information called for much stronger human involvement.

Uniform control wastes attention

The usual response is that every AI output must be checked. That is correct, but too imprecise for project governance. If a meeting summary and a budget forecast receive identical treatment, one ends up reviewed too heavily or the other too lightly.

I use AI in software projects every day. In my work, the depth of review depends on the consequence of an error, not on the tool. That leads to a simple pattern:

  • Low impact: Agenda drafts, wording and summaries call for a plausibility check and a clear working-status label.
  • Medium impact: Defect classification, effort estimates or schedule options require validated inputs, documented rules and samples verified against the original information.
  • High impact: Forecasts, budget decisions, go/no-go papers or supplier acceptance need a named owner, a second professional review and an entry in the decision log.

This is in line with the recent GPM argument that AI competence should reflect the role and the specific use case. The NIST AI Risk Management Framework takes a similar view, treating trustworthiness as a concern across design, use and evaluation instead of a one-time tool approval.

A practical pattern for the next steering meeting

Take every AI-generated item in the next steering pack and assign it one of three review levels. Each level gets an owner, a review step and evidence. For AI-supported UAT triage, that could mean versioning the classification rules, manually checking ten percent of assignments and documenting every priority change together with its source.

What project leaders should do now

Do not start with a generic AI policy. Start by classifying the five most common AI outputs in your project by the potential cost of a wrong decision, then define concrete controls. How deeply you review should depend on what a wrong decision would cost, not on how convincing an output sounds.

← Back to the blog