99.9% Isn’t Safe: Why Recall Value Is the Real Metric for Safety-Critical AI

Recall Value
Please share to show your support
Print Friendly, PDF & Email

Team | E-Journal Times Magazine | 7/10/2026

Imagine a security or medical AI system that’s right 99.9% of the time. That sounds close to perfect. In a safety-critical system, it can still be dangerous, and the reason why says a lot about how we should actually be judging these models.

The number that hides the danger

Accuracy just tells you the percentage of total predictions a model got right. That sounds like the obvious thing to optimize, until you notice that in most safety and security systems, the dangerous event is rare. Most transactions aren’t fraud. Most patients don’t have the disease being screened for. Most access attempts are legitimate.

When the dangerous case is rare, a model can rack up a very high accuracy score while still being almost useless at catching that rare case. In the extreme, a model that never once flags the dangerous event can still score above 99% accuracy, simply because it’s “correct” every time the danger wasn’t there.

This is exactly why recall (sometimes called sensitivity) is the metric that actually matters here. Recall value answers one specific question: of all the real dangerous cases that existed, how many did the model actually catch? It ignores how the model did on the easy, safe majority and puts the spotlight squarely on the cases you built the system to catch in the first place.

Recall Value Is the Real Metric

What recall measures actually, and how it’s used in training

Every prediction a model makes falls into one of four buckets, usually laid out in what’s called a confusion matrix:

  • True Positive: a real danger, correctly caught.
  • False Negative: a real danger, missed.
  • True Negative: a safe case, correctly cleared.
  • False Positive: a safe case, incorrectly flagged.

Recall is calculated from just two of those four buckets:

Recall = True Positives / (True Positives + False Negatives)

In plain terms: out of everyone who was actually a real danger, what fraction did the model catch? Notice that true negatives don’t appear in that formula at all. That’s the key difference from accuracy: it sums correct predictions across both classes and can look great purely because the “safe” class is easy and common.

In practice, here’s how recall gets used while a model is being built:

  • It’s calculated on validation data after every training run, tracked separately for each class rather than as one blended number.
  • If recall on the dangerous class is too low, engineers can push it up during training through class weighting (penalizing a missed dangerous case more heavily than a missed safe case) or by gathering more examples of the rare, dangerous class so the model sees it often enough to learn it well.
  • After training, recall can also be raised by adjusting the decision threshold. Most models output a probability, and a default cutoff of 0.5 decides which side a prediction falls on. Lowering that cutoff catches more real dangers, at the cost of also flagging more safe cases as false alarms.
  • That last point is really a trade-off, not a free win. Pushing recall up almost always pulls precision down (the percentage of flagged cases that turn out to be real). Deciding where to set that trade-off isn’t a modeling question so much as a business and safety question: how costly is a miss, compared to how costly is a false alarm?

A real example: when a hospital-wide AI missed most of its target

This isn’t a hypothetical concern. In 2021, researchers at the University of Michigan published a study in JAMA Internal Medicine testing a sepsis-prediction model built into Epic’s electronic health record system, used across hundreds of U.S. hospitals to warn clinicians when a patient might be developing sepsis, a life-threatening condition where early detection saves lives.

At the threshold the hospital was using, the model’s sensitivity (recall) was just 33%, meaning it missed roughly two out of every three real sepsis cases. Its negative predictive value was 95%, a number that looks reassuring on paper but is largely a byproduct of sepsis being uncommon in the overall patient population, not evidence the model was actually good at spotting it. The model’s own developer had reported stronger internal performance numbers before deployment. The real-world validation told a very different story.

That gap between “looks fine in aggregate” and “misses most real cases” is the entire argument for why recall, not accuracy or overall performance claims, has to be the headline number in any safety-critical system.

JAMA Internal Medicine paper
This figure from the actual JAMA Internal Medicine paper shows exactly why threshold choice matters: as the Epic Sepsis Model’s score threshold rises, sensitivity (recall) drops sharply from 100% toward near 0%, while specificity climbs the opposite way toward 100%, the classic precision-recall trade-off in action.

At the threshold hospitals actually used (around 6), sensitivity was already down near 33%, meaning most real sepsis cases were being missed, even as specificity and NPV looked comfortably high. The bottom histogram shows most hospitalizations scored low on the model, which is exactly why a threshold set to reduce false alarms ended up sacrificing so much real-world recall.

Recall Value: Why a single miss outweighs a false alarm

Every model like this makes two kinds of mistakes:

  • False negative: a real danger is missed and classified as safe. In medicine, that’s a disease that goes untreated. In security, that’s a threat that walks through the door labeled “clear.”
  • False positive: a safe case gets flagged for review. It costs someone time. It’s not nothing, but it’s rarely dangerous on its own.

These two mistakes are not equally bad, and treating them as if they were is one of the most common mistakes in how AI systems get evaluated. When a missed case can mean a patient dies, a breach goes undetected, or an injury happens, the model should be tuned, and judged to minimize false negatives specifically, even if that means accepting more false alarms in exchange. That’s not a nice-to-have design choice. It’s the whole point of calling a system “safety-critical.”

Even great recall doesn’t mean zero risk.

Here’s the part worth being honest about, rather than oversimplifying: even a model with excellent recall, say 99%, will still miss 1% of real cases. That sounds small in isolation, but the actual harm it causes depends on how many real cases exist in the first place, not on the total volume of data the system processes.

Also read, “Interpolation in Computer Vision is the mathematical process of estimating and filling in missing pixel values whenever an image is upscaled or downscaled,” at https://journals-times.com/2026/07/05/interpolation-in-computer-vision-what-actually-happens-when-you-resize-an-image/

Consider a hospital network that screens 5 million patients a year for a serious but uncommon condition affecting roughly 1 in 500 people. That’s about 10,000 real cases annually, a small slice of the 5 million total, but a real number of real people. A model with 99% recall on that condition will still miss around 1% of those 10,000 cases, roughly 100 people a year who have the condition and are told, in effect, that they don’t.

Run the same math on a national fraud-detection system processing billions of transactions, where only a few million of those turn out to be genuine fraud, and even a 99% recall model still lets tens of thousands of real fraudulent transactions through every year. The lesson isn’t “the model is bad.” It’s that no recall number, however impressive, is the same thing as “solved.”

That’s why recall should be treated as the number you’re optimizing toward, not a certificate of safety once it’s high enough. Real systems handle the remaining gap with layers built around the model, not inside it:

  • Human review for uncertain cases. Most models output a probability, not a flat yes/no. Predictions that land in an uncertain middle band, say between 30% and 70% confidence, are exactly the cases the model is least sure about, and exactly the cases worth routing to a person before any action is taken, rather than letting the model’s default threshold silently decide.
  • Deliberately trading precision for recall where it matters. If a missed case is far more costly than a false alarm, the system should be tuned that way on purpose, accepting more manual reviews in exchange for catching more real danger, rather than defaulting to whatever threshold produces the best-looking overall accuracy.
  • Re-validating on live, real-world data on a regular schedule, not just once at launch. A model’s recall on the data it was built and tested on is not a permanent guarantee: patient populations shift, fraud patterns evolve, camera and sensor conditions change. The gap between a vendor’s internal benchmark and real-world performance is often invisible until someone actually goes and checks, which is precisely the kind of independent, real-world re-validation that exposed the true performance of the sepsis prediction model in the first place.
Recall Value Is the Real Metric

Recall Value: The takeaway

A safety-critical AI system should never be sold, or trusted, on its accuracy score alone. Ask what its recall is on the specific dangerous case it exists to catch. Ask how that number was validated in the real world rather than just claimed internally. Ask what happens to the cases it’s still going to miss. That last question doesn’t have a clean answer from the model itself. It has to be answered by the humans and processes built around it.

Primary source:

Please share to show your support

Leave a Reply

Up ↑

Discover more from E-JOURNAL TIMES MAGAZINE

Subscribe now to keep reading and get access to the full archive.

Continue reading