Automated Optical Inspection using Computer Vision and Deep Learning
Vamsi Cheekurthi · 2026
Intelligent visual inspection for electronics manufacturing quality assurance
Developer: Vamsi Cheekurthi
Technology: Python · TensorFlow · OpenCV · Streamlit
Dataset: DeepPCB — 6 defect class annotated PCB image pairs
Printed Circuit Board quality assurance is a critical bottleneck in electronics manufacturing. Boards that ship with undetected defects cause:
- Field failures — costly recalls and warranty claims
- Safety risks — particularly in automotive, aerospace, and medical devices
- Production losses — rework, scrap, and assembly line delays
| Approach | Limitation |
|---|---|
| Manual inspection | Operator fatigue, inconsistency, ~70% detection rate |
| Rule-based AOI | Rigid templates; fails on new layouts or lighting variation |
| Simple thresholding | High false-positive rate; cannot classify defect type |
Design a system that:
- Automatically localizes defect regions on a PCB image
- Accurately classifies each defect into its specific type
- Delivers results in under 3 seconds with an exportable report
The system combines the interpretability of classical image processing with the representational power of a deep convolutional neural network.
Single PCB Image
│
▼
Defect Localization ← OpenCV adaptive thresholding + Canny edges
│
▼
ROI Extraction ← Contour analysis → 128×128 crops
│
▼
CNN Classification ← EfficientNetB0 → 6-class softmax
│
▼
Annotated Result ← Bounding boxes · CSV log · PNG export
Key Design Decision: Single-image inference — no reference template required. This makes the system deployable on manufacturing lines without the overhead of maintaining a paired-image database for each board variant.
The system is trained and evaluated on the DeepPCB dataset, a widely used benchmark in PCB visual inspection research.
| Property | Details |
|---|---|
| Image pairs | Template + test PCB image pairs |
| Defect classes | 6 distinct manufacturing defect types |
| Image format | Grayscale and color JPEG |
| Annotation | Class-level folder structure |
| Class | Visual Characteristic | Manufacturing Cause |
|---|---|---|
| Missing Hole | Absent drill hole | Incorrect drilling depth or position |
| Mouse Bite | Partial edge chamfer | PCB routing or mechanical damage |
| Open Circuit | Broken conductor trace | Etching defect or physical break |
| Short | Unintended trace connection | Over-etching, solder bridge |
| Spur | Copper protrusion from trace | Under-etching |
| Spurious Copper | Excess copper deposit | Resist failure during etching |
Dataset split: 70% training / 15% validation / 15% testing (random seed 42, stratified by class)
Before classification, potential defect regions must be located within the full PCB image.
For the labeled training dataset, each test image is aligned to its reference template:
- ORB Keypoint Detection — 5,000 keypoints extracted from both images
- Homography Estimation — RANSAC-robust perspective warp aligns test to template
- Absolute Difference —
|medianBlur(template) − medianBlur(aligned_test)| - Otsu's Thresholding — Automatically selected binary threshold from histogram
- Morphological Cleanup — Close (5×5) fills blob holes; Open (3×3) removes noise
Without a template, a two-channel localization approach is used:
| Channel | Method | Captures |
|---|---|---|
| Adaptive Threshold | Gaussian local window (31×31, C=8) | Local intensity anomalies — missing holes, spurs, excess copper |
| Canny Edges | Gradient-based (low=40, high=160) | Structural breaks — open circuits, shorts |
Both channels are merged with a bitwise OR and cleaned with the same morphological sequence, producing a binary defect mask.
Once the binary defect mask is produced, individual defect candidates are isolated:
cv2.findContours retrieves the external boundary of each white blob in the mask. Each contour is evaluated against quality filters:
| Filter | Threshold | Purpose |
|---|---|---|
| Minimum area | 80 px² | Reject sub-pixel noise |
| Maximum area | up to 50,000 px² | Reject whole-board artifacts |
| Aspect ratio | ≤ 3.5 | Reject elongated non-defect traces |
| Circularity (Missing Hole only) | ≥ 0.45 | Enforce round-hole morphology |
For each accepted contour:
- Centroid
(cx, cy)is computed from image moments - A 128 × 128 pixel patch is cropped, centred on the centroid
- The patch is resized (if needed) and normalized for CNN input
This 128×128 format matches the EfficientNetB0 training resolution and provides sufficient context around each defect without including irrelevant surrounding circuitry.
Input: 128 × 128 × 3 (RGB)
└─ EfficientNetB0 Backbone (pre-trained on ImageNet)
└─ GlobalAveragePooling2D
└─ BatchNormalization
└─ Dropout (0.4)
└─ Dense (256 units, ReLU)
└─ BatchNormalization
└─ Dropout (0.3)
└─ Dense (6 units, Softmax)
└─ Output: 6-class probability vector
| Property | EfficientNetB0 | ResNet50 | VGG16 |
|---|---|---|---|
| Parameters | ~4.2 M | ~23 M | ~138 M |
| Accuracy (ImageNet Top-1) | 77.1% | 76.0% | 71.3% |
| Inference time (CPU) | Fast | Moderate | Slow |
| Compound scaling | ✅ | ❌ | ❌ |
EfficientNetB0 uses compound scaling — simultaneously scaling network depth, width, and resolution — delivering higher accuracy at lower computational cost than alternatives.
| Phase | Description | Learning Rate | Backbone |
|---|---|---|---|
| Warm-Up | Head-only training, backbone frozen | 1 × 10⁻³ | All layers frozen |
| Fine-Tuning | Joint training, top 30 backbone layers unfrozen | 1 × 10⁻⁴ | Top 30 layers active |
Training uses Adam optimizer, categorical cross-entropy loss, early stopping (patience: 5/8 epochs), and aggressive data augmentation (rotation, flip, zoom, brightness, shear) to maximize generalization.
| Metric | Target | Achieved |
|---|---|---|
| Test Set Accuracy | ≥ 95% | Evaluated post-training |
| Inference Speed | ≤ 3 000 ms | ~200–400 ms (CPU) |
| False Positive Rate | Minimize | Reported per-class |
| False Negative Rate | Minimize | Reported per-class |
- Training curves — Accuracy and loss over epochs (warm-up + fine-tune phases)
- Confusion matrix — 6×6 heatmap of true vs. predicted classes
- Classification report — Per-class precision, recall, F1-score, and support
- Prediction evaluation — Ground-truth match rate on unseen test image pairs
A centroid-proximity matching criterion is used: a prediction is counted as correct if the predicted centroid falls within ±32 pixels of the annotated ground-truth centroid and the class label matches.
This evaluation reflects real-world AOI deployment conditions, where spatial localization accuracy matters as much as classification accuracy.
The web application provides a complete, end-to-end inspection workflow without requiring any command-line interaction from the operator.
| Section | Functionality |
|---|---|
| Image Upload | Drag-and-drop zone for JPG/PNG/BMP/TIF |
| Metric Dashboard | Defects found, type count, avg confidence, inference time |
| Performance Gate | Real-time ✅ / |
| 4-Panel Visual Output | Original → Mask → Heatmap → Annotated |
| Prediction Table | Per-defect class badge, confidence bar, centroid, bbox |
| Distribution Chart | Bar chart of defect class frequencies |
| Export Suite | PNG · CSV · Defect Mask · Evaluation Report TXT |
| Sidebar | Model path override, pipeline step indicator, class legend |
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Original │ │ Defect Mask │ │ Anomaly │ │ Annotated │
│ Input │ │ (Binary) │ │ Heatmap │ │ Output ✅ │
│ │ │ │ │ (Red tint) │ │ [bbox+label] │
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
Every inference run produces four immediately downloadable files, generated fully in-memory:
The original PCB image overlaid with color-coded bounding boxes and class labels. Each defect class has a unique color for instant visual identification (e.g., Red=Missing Hole, Blue=Short, Green=Open Circuit).
A structured spreadsheet with one row per detected defect:
timestamp, defect_class, confidence_%, cx, cy, bbox_x, bbox_y, bbox_w, bbox_h
2026-02-19T11:30:00, Mouse_bite, 94.72, 143, 97, 115, 69, 56, 56
2026-02-19T11:30:00, Spur, 88.15, 210, 185, 182, 157, 56, 56
...
inference_time_ms, 287.4
The binary segmentation output — white regions represent defect candidates, black regions represent clean PCB. Useful for QA audit trails and algorithm debugging.
A plain-text summary including:
- Timestamp and inference time
- Performance gate result (PASS / FAIL vs. ≤ 3 000 ms)
- Total defects found, unique class count, average confidence
- Full per-defect breakdown with coordinates
| Stage | Operation | Typical Duration |
|---|---|---|
| Image decode | cv2.imdecode from bytes |
~5–15 ms |
| Preprocessing | Grayscale + blur + threshold + Canny | ~15–30 ms |
| Morphological cleanup | Close + Open | ~5–10 ms |
| Contour detection | findContours + filtering |
~5–15 ms |
| CNN inference (per defect) | EfficientNetB0 forward pass | ~30–80 ms |
| Annotation | Rectangle + text drawing | ~5 ms |
| Total (typical) | ~200–400 ms |
Result: The system comfortably reaches the ≤ 3 000 ms per-image target on a standard CPU, with significant headroom.
| Layer | Technology | Role |
|---|---|---|
| Image Processing | OpenCV 4.8 | Alignment, thresholding, contour detection, annotation |
| Array Operations | NumPy | All pixel-level array manipulation |
| Deep Learning | TensorFlow / Keras 2.16 | EfficientNetB0 training and inference |
| Transfer Learning Base | EfficientNetB0 (ImageNet) | Pre-trained feature extractor |
| Evaluation Metrics | Scikit-learn | Accuracy, F1, confusion matrix, FP/FN rates |
| Visualization | Matplotlib, Seaborn | Training curves, confusion matrix heatmaps |
| Web Application | Streamlit 1.32 | Browser-based UI, file upload, download buttons |
| Image I/O | Pillow | PIL crop operations and PNG export |
| Dataset | DeepPCB | 6-class annotated PCB defect image pairs |
| Language | Python 3.12 | End-to-end implementation |
| Environment | Python venv | Isolated, reproducible dependency management |
| Enhancement | Description | Priority |
|---|---|---|
| Real-Time Camera Feed | Integrate st.camera_input for live manufacturing-line capture |
High |
| Edge Deployment | Export model to TFLite for Jetson Nano / Raspberry Pi | High |
| PDF Report Export | Generate formal QA PDFs using reportlab or fpdf2 |
Medium |
| Defect History Database | Log results to MongoDB/SQLite for trend analysis | Medium |
| Automated Alerts | SMS/Email notifications for critical defect types (Short, Open Circuit) | Medium |
| Multi-Board Batch Mode | Process entire folders of PCB images automatically | Low |
| Confidence Thresholding UI | Slider to filter detections below a user-set confidence level | Low |
A complete, production-ready automated optical inspection system that:
✅ Localizes defect regions from a single PCB image using adaptive thresholding and edge detection — no reference template required
✅ Classifies defects into six industry-standard categories using a fine-tuned EfficientNetB0 CNN trained on the DeepPCB dataset
✅ Presents results through a browser-based Streamlit interface with 4-panel visual output, a detailed prediction table, and real-time metric feedback
✅ Exports four downloadable artefacts per run — annotated PNG, CSV log, binary mask, and evaluation report
✅ Performs end-to-end inference in ~200–400 ms on CPU — well within the 3-second operational requirement
✅ Documents the full system architecture, algorithm parameters, and usage in comprehensive technical references
Thank You — Questions?
PCB Defect Detection & Classification System · Vamsi Cheekurthi · 2026