Recognizing crosswalk on aerial images

Question

I'm trying to find x/y coordinates of crosswalks on aerial images.

For example on this image:

...I'd like to extract two nodes in the center of the two yellow crosswalks.

I started playing around with OpenCV on Python. The first approach I tried was to isolate the yellow colored parts in the image. The results are not perfect though.

As the pattern of a crosswalk is quite unique, I think there should be a way to quite reliably detect the presence and the coordinates of a crosswalk. As I don't have much experience with image processing, can you give me some pointers in the right directions, with a small example? Maybe I should work with FFT, or create a classifier?

Here's the code I used for the screenshot above.

# -*- coding: utf-8 -*-
import cv2
import numpy as np

def nothing(x):
    pass

# Create a window
cv2.namedWindow('image')

# Create trackbars
cv2.createTrackbar('H-', 'image', 16, 180, nothing)
cv2.createTrackbar('H+', 'image', 25, 180, nothing)
cv2.createTrackbar('S-', 'image', 40, 255, nothing)
cv2.createTrackbar('S+', 'image', 150, 255, nothing)
cv2.createTrackbar('V-', 'image', 97, 255, nothing)
cv2.createTrackbar('V+', 'image', 172, 255, nothing)

while 1:
    # Choose image
    img = cv2.cvtColor(cv2.imread('img/1.jpg'), cv2.COLOR_BGR2HSV)

    # Get slider values
    h = cv2.getTrackbarPos('H-', 'image'), cv2.getTrackbarPos('H+', 'image')
    s = cv2.getTrackbarPos('S-', 'image'), cv2.getTrackbarPos('S+', 'image')
    v = cv2.getTrackbarPos('V-', 'image'), cv2.getTrackbarPos('V+', 'image')

    # Thresholds
    lower = np.array([h[0], s[0], v[0]], np.uint8)
    upper = np.array([h[1], s[1], v[1]], np.uint8)

    # Calculate mask using thresholds
    mask = cv2.inRange(img, lower, upper)

    # Combine original image and mask
    h, w = img.shape[:2]
    vis = np.zeros((h, w * 2), np.uint8)
    vis = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)
    vis[:h, :w] = cv2.cvtColor(img, cv2.COLOR_HSV2BGR)
    vis[:h, w:w * 2] = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)

    # Show image
    cv2.imshow('image', vis)
    k = cv2.waitKey(1) & 0xFF
    if k == 27:  # ESC
        break

cv2.destroyAllWindows()

Answer 1

This is not a simple segmentation task since there are multiple orientations, complex backgrounds, and other objects with similar color distribution. As mentioned in the comment Haar features are very descriptive for this pattern. Road feature detection and estimation Stephen Se and Michael Brady discusses that the crosswalk edges are interesting features they are usually straight lines with periodic intervals. One could use the hough transform to accumulate lines and then filter lines with the slope variables to look for target patterns. This Masters Thesis on Urban Artifacts Detection - Stefania Pedrazzi talks about the same feature: The main property of crosswalk stripes, also called zebra stripes, is bipolarity, i.e. the alternation of bright and dark areas. Bipolarity causes high gradient values at stripes' edges. For this reason, computing gradients and extracting lines will provide a good starting point. Then, it will only remain to iterate row- and column-wise through the image, checking the geometric structure of consecutive lines. Though this is for lateral viewpoints, one can still use the geometric features.