The production line had just started up when the alarm sounded first.

Good products get kicked out by the system, while real defects slip right under the radar.

The customer stood nearby, the boss stared at the screen, and the engineer, holding the mouse, began to sweat in the palms of his hands.

Anyone who has ever performed visual defect detection should be familiar with this scene.

False positive—good products are wrongly classified as defective.

Omission of inspection—defective products are released as good ones.

It may sound like just two metrics, but when you get down to the shop floor, they translate into yield rates, rework, complaints—and the increasingly silent expressions on the engineers’ faces.

Many projects start off smoothly.

The sample image is clear, the defects are also obvious, and the algorithm’s bounding boxes are accurate.

The customer nodded, the project manager breathed a sigh of relief, and the engineer had even started thinking about what to eat for dinner.

As soon as it was put onto the actual production line, the plot immediately changed.

01 Defects are hard to detect—often, the difficulty lies in their “similarity.”

The most troublesome aspect of the site isn't that defects are completely invisible.

What’s even more troublesome is that defects and normal fluctuations look too much alike.

The surface texture of the good product is slightly heavier, triggering a system alarm.

The defect has a lighter color and weaker edges—system allows it to pass.

It's only then that you realize: the hardest sentence in defect detection is:

“Um, does this actually count as a defect?”

An industrial site is nothing like a laboratory.

The product has texture, the materials vary from batch to batch, lighting can cause color shifts, positioning can lead to misalignment, and reflections can even change appearance.

Dirt, scratches, dents, color variations, and burrs—sometimes they really do look like a family.

The human eye needs to spend an extra couple of seconds looking at it, so naturally, for a machine to make consistently reliable judgments every single time, the pressure is considerable.

So false positives and missed detections are unavoidable; in many cases, it’s not that the system is deliberately neglecting its job.

The scene itself already has too many gray areas.

02 With too many false positives, production will soon get restless.

False positive—meaning the product is fine, but the system insists there’s a problem.

At first, it sounded okay.

At least the defect wasn't let go.

But the production site doesn't calculate it that way.

With an increasing number of false positives, good products are being rejected, the yield rate drops, manual re-inspections pile up, and the production rhythm slows down.

The production manager will soon ask:

“Is this system actually helping, or is it just making things more complicated?”

The common causes of false positives are also very realistic.

The product itself has fluctuations, and the system is mistaking normal textures for anomalies.

The light source angle has changed, the product’s position is off, the lens is dusty, there’s background glare, and the image differs from what it looked like during debugging.

The standards are set too tightly; as soon as the threshold is tightened, the system immediately becomes overly sensitive and reacts to every little thing.

Even a bit of texture, a slight color variation, or a hint of reflection can all be mistaken for defects.

The engineer said, “If I loosen the threshold a bit, there’ll be fewer false alarms.”

The customer immediately replied, “Then won’t it leak?”

The air became instantly silent.

This isn't a difficult question to answer.

This is so real.

03 Once a leak occurs, the pot gets even hotter.

The thought of missed detections is even more chilling.

The product clearly has a defect, yet the system reports it as normal.

False positives can at most cause good products to suffer unjustly.

Once a missed inspection makes it to the later stages—or worse, ends up in the customer’s hands—the situation could become serious.

Why is there a leak?

The reasons aren't complicated, but they're all deeply painful.

Some defects inherently have low contrast.

Shallow scratches, minor dents, and tiny blemishes appear almost identical to the normal areas when photographed.

The human eye, relying on experience, can still sense that something “doesn’t quite feel right,” while a machine might only detect a slight change in grayscale.

Some defect forms exhibit significant variation.

Today is a dot; tomorrow is a line; the day after tomorrow is like a slight color difference.

The earlier samples weren't covered, the system hadn't seen them before, so it's perfectly normal to let them through.

Some defects are also in fixed locations.

Appearing at the edges, in hole positions, in reflective areas, and in textured regions, the detection difficulty varies significantly.

It looked pretty stable during the prototype stage, but only after mass production did we realize:

Being able to detect something and being able to detect it consistently over the long term are two very different things.

04 If you tighten the threshold, you’ll get false positives; if you loosen it, you’ll miss true positives.

This is precisely where defect detection becomes the most frustrating.

Make the system checks stricter—minor defects will be easier to catch.

But normal fluctuations can also easily be misinterpreted.

Loosen the system requirements a bit, and it’ll be smoother to release qualified products.

Even minor defects might slip by unnoticed.

The lower the probability of missed defects, the better.

Production hopes that the number of false detections is as low as possible.

The customer would ideally hope for neither of the two.

The engineer had only one thing on his mind:

“Everyone, calm down first and let’s clarify the standards.”

Which defects must be detected?

Which visual fluctuations are acceptable?

Under what circumstances is manual review required?

Which risks are better to overreport than to omit?

If these boundaries aren't clarified in advance, even if you tweak the parameters to the point of questioning your sanity, it'll still be hard to truly solve the problem.

What we fear most on-site isn't that the questions are difficult.

What I fear most is that right from the start of a project, everyone’s understanding of “defects” isn’t aligned.

05 Don't blame the algorithm every time something goes wrong.

As soon as many projects run into trouble, the first reaction is:

“Is it the algorithm that’s not working?”

Algorithms are certainly important.

But visual inspection is a systems engineering endeavor.

If any link—whether it’s the light source, lens, camera, mechanical positioning, sample quality, inspection standards, or on-site environment—is unstable, the blame could ultimately fall on the algorithm.

The defect is very shallow; the light isn't coming out.

No matter how powerful the algorithm is, it can only force its way through an unclear image.

The product position has been consistently off, and the detection area keeps drifting along with it.

Further model optimization is also a battle against systematic errors.

The sample consists of only a few dozen images, and the defect types are not yet comprehensive.

After going live, relying on the system to handle things on its own is basically just letting the on-site team pick up the slack for you.

So, if you want to reduce false positives and false negatives, don't just dive right into obsessing over parameters.

Let me start with a few basic questions:

Did the image capture the defect clearly?

Have you checked the fluctuation range of normal goods yet?

Are there enough defective samples?

Are the testing standards explained clearly?

Have you taken the on-site changes into account?

Many times, after fiddling around for ages, you realize the problem was hidden in the lighting, the setup, the sample, the standards—or in the boundary conditions that weren't clearly defined from the start.

06 For a good project, first lay out the pitfalls.

Defect detection cannot avoid false positives and missed detections, because industrial environments are constantly changing.

The product will change.

The environment will change.

Defects can change.

Customer standards may also change.

What the visual system needs to do is, under the premise of clear boundaries, keep errors within an acceptable range.

If it can be solved through imaging, don't force it onto the algorithm.

If you can rely on institutional stability, don't count on later compensation.

If it can be covered by samples, don't rely on luck after going live.

If it can be clearly explained through standards, don't leave engineers to guess on-site.

Next time you do defect detection, don't rush to ask:

“Can the algorithm recognize it?”

First ask:

“Can this defect produce a stable image?”

“Where is the boundary between normal and abnormal?”

“Which is more unacceptable—the false positive or the false negative?”

“Can this project demonstrate reliable performance, or is it ready for mass production?”

When visual projects go wrong, it’s often not because the technology simply can’t handle it.

But instead, I initially oversimplified the problem.

The on-site environment won't take the time to let you slowly learn through trial and error.

A reliable approach is to first lay all the challenges out on the table, then tackle them one by one.