← AI Engineering
Deep Learning · Computer Vision · Lane Detection

Lane Detection: What a Straight Line Can't See

Comparing a hand-built classical pipeline (Canny edge detection + Hough transform) against a U-Net trained with transfer learning, on highway dashcam footage. The real story isn't just which one wins — it's a Canny pipeline that fails completely on reflective road studs instead of painted lines, and an evaluation metric that turned out to structurally favor one method's output format over the other's.

View on GitHub Dataset source (TuSimple)
Python OpenCV PyTorch U-Net ResNet34 Transfer Learning

The problem

Given a single dashcam frame from a highway driving video, locate the current lane's left and right boundaries. Two structurally different approaches were built and compared: a classical pipeline relying entirely on hand-written geometric rules (edge detection, straight line fitting), and a U-Net trained on thousands of labeled TuSimple frames to recognize lane patterns directly from data.

Diagnostic visualization showing the raw Canny edge map of a highway frame, with a clean continuous line on the left side of the road and no visible edge at all on the right side, where the lane marking is made of reflective road studs instead of painted line
The edge map that revealed the failure: a clean, continuous edge on the left (a painted line) versus nothing at all on the right, where the lane marking turned out to be Botts' Dots — reflective raised pavement markers, not a continuous painted edge.

Key results

60.4%
U-Net point accuracy (20px tolerance)
16.3%
Phase A point accuracy (same metric)
13.1%
U-Net video missing rate
3.03px
U-Net mean frame-to-frame jump

A structural failure, traced step by step

A first batch evaluation showed a striking asymmetry: the classical pipeline's left line was missing in 36% of frames, but the right line was missing in 82% — too large a gap to accept as "the right side is just harder." Rather than guessing which parameter to retune, a diagnostic function saved every intermediate pipeline step (edge map, ROI-masked edges, raw Hough segments color-coded by outcome) for several failing frames. The edge map above made the cause immediately visible: the right-side marking in these scenes is made of Botts' Dots, not a painted line — Canny found zero edge pixels there, and every step downstream had nothing to work with, as a direct structural consequence, not a threshold that happened to be mistuned.

Side by side comparison of the classical pipeline and U-Net predictions on the same Botts Dots frame, showing the classical pipeline finding almost nothing on the dotted side while U-Net correctly segments all visible lanes including the dotted one
Same frame, two methods. The classical pipeline (left) finds only the painted left line; U-Net (right) correctly segments every visible lane, including the one marked only by scattered reflectors — it doesn't need a continuous edge, having learned the pattern contextually from training data.

When the evaluation metric itself needed auditing

Converting the classical pipeline's fitted line into a mask (to compute IoU against the ground truth) produced a near-zero score — 0.016 — disproportionate to how correct the line actually looked overlaid on the image. Rather than accepting the number, the two masks were visualized directly, color-coded by source.

Overlay of ground truth mask in green and classical pipeline mask in red on a black background, showing the two lines running close and parallel with only a thin sliver of white overlap where they coincide
Ground truth (green) vs. the classical pipeline's fitted line (red), overlaid. The lines are genuinely close — but a rigid straight-line fit can't follow the road's slight curvature, and on an already-thin mask, even a small offset means almost no pixel overlap. This is a metric-format mismatch, not a positioning bug.

TuSimple's own benchmark avoids this exact problem by not using IoU at all — it checks, at each annotated height, whether the predicted x position falls within a pixel tolerance of the true one. Implementing that point-distance metric for both methods gave the fairest comparison in the project, penalizing neither method for its output format.

Results

Test set: label_data_0601.json, 410 frames, held out from training.

MetricPhase AU-Net
IoU (2-lane restricted ground truth)0.0280.320
TuSimple-style point accuracy (20px)16.3%60.4%
Video missing rate (8 clips, 160 frames)47.2%13.1%
Video mean frame-to-frame jump10.36px3.03px
U-Net wins on every metric that was made fair to compute — but the classical pipeline's weaknesses were never patched around after the Botts' Dots diagnosis. No threshold retuning can make Canny detect an edge that isn't there; the finding was treated as a structural limitation worth documenting, not a bug to fix. U-Net itself was trained on a lightweight subset with no data augmentation, so its numbers likely have headroom left on the table too.

Tools used

Python OpenCV PyTorch torchvision NumPy Google Colab