Your CI Is Green. Now What?
Modern engineering teams have no shortage of checks.
CI runs tests.
Linters catch code issues.
Security tools scan dependencies.
SAST checks source code.
Code review tools inspect pull requests.
All of that is useful.
But there is a gap that becomes more obvious as systems and engineering teams get larger:
A passing PR is not necessarily a low risk PR.
A small change in a heavily used service can matter more than a large change in an isolated utility. A change touching code that has caused incidents before deserves different attention from a change in code that has been stable for years.
Most tools are not trying to answer that question. They are answering narrower questions:
Does it compile?
Did the tests pass?
Are there security findings?
Does it follow the policy?
Those signals are important, but engineers still have to put them together and decide:
How risky is this change to ship?
Looking at the change in context
This is the problem we are working on with Tomosu.
Tomosu calculates a Production Risk Index by looking at a change across eight risk indexes rather than treating the diff in isolation.
The indexes cover areas such as fragility, drift, governance compliance, runtime signals, code volatility, deployment velocity, and escalation.
The point isn't to replace CI, testing, security scanning, or human review.
It is to add another layer of context.
Imagine two pull requests:
PR A
12 files changed
Tests passing
Low traffic service
PR B
8 files changed
Tests passing
Frequently deployed service
Historically involved in incidents
Touches critical dependencies
A traditional review workflow may give both PRs similar treatment because the visible checks are green.
The production risk is not necessarily similar.
Why this matters with AI generated code
This becomes even more relevant as AI coding tools generate more changes.
AI can make a diff look clean.
That doesn't tell you whether the code is being introduced into a fragile subsystem, whether it expands the blast radius of a service, or whether similar changes have caused operational problems before.
The reviewer needs context, not just another set of comments on the diff.
That's the direction we're exploring with Tomosu: use the signals already available across a software system and turn them into something closer to a risk signal before deployment.
It's still early, and there are plenty of hard problems around historical data, new repositories, noisy signals, and explaining why a change received a particular score.
But I think this is becoming a useful engineering question:
When everything is checked, how do we decide what actually deserves our attention?
You can try the Production Risk Index on a repository here:
