How To Get The Checkbox Position Or Region On The Image Using Opencv

My Goal is to detect the position off all checkbox using openCV and use this position to detect what checkbox is checked or not. could you give me some help please? Thank you. Ther

Solution 1:

Here's a possible solution. It involves creating a mask of the check boxes using morphology. We first create a vertical mask such that all elements above a certain height will survive the morphological operation. We repeat this step to obtain a horizontal mask, now looking for elements above a certain width. Joining both masks will yield a final mask that should contain the check boxes and some small noise. Finally, we can filter the noise by detecting contours on the final image and filtering blobs based on area. The algorithm is pretty straightforward, here are the steps:

1. Create a binary image of the input via Otsu's Thresholding
2. Generate a vertical mask using morphology
3. Generate an horizontal mask using morphology
4. Join both masks via a bitwise OR
5. Detect contours on the final mask
6. Filter contours below a certain threshold area

Let's check out the code:

# Imports:
import numpy as np
import cv2

# Set image path and image name:
path = "D://opencvImages//"
fileName = "74k9I.png"# Read the image in default mode:
inputImage = cv2.imread(path + fileName)

# Prepare a deep copy for results:
inputImageCopy = inputImage.copy()

# Convert BGR to Grayscale
grayImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)

# Threshold via Otsu:
_, binaryImage = cv2.threshold(grayImage, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)

The first bit produces the initial binary mask:

Let's apply morphology. I'm applying an erosion followed by a dilation. The Structuring Element (SE) is only a (rectangular) column of size 8 x 1. The dimensions will "erase" blobs on the image that are below the height set by the SE. This will create the vertical mask:

# Create Vertical Mask:opIterations = 1# Structuring Element dimensions:kernelCols = 1kernelRows = 8# Get the structuring element:morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelCols, kernelRows))
# Perform Erode and Dilate:verticalMask = cv2.morphologyEx(binaryImage, cv2.MORPH_ERODE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
verticalMask = cv2.morphologyEx(verticalMask, cv2.MORPH_DILATE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)

This is the vertical mask:

Alright, let's now repeat the procedure but this time we will switch the SE's dimensions to create a row of 1 x 8. This will isolate the wider elements, be sure to operate on the initial binary mask:

# Create Horizontal Mask:# Get the structuring element:morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelRows, kernelCols))
# Perform Erode and Dilate:horizontalMask = cv2.morphologyEx(binaryImage, cv2.MORPH_ERODE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
horizontalMask = cv2.morphologyEx(horizontalMask, cv2.MORPH_DILATE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)

This is the horizontal mask:

We probably can tune the filter by adjusting the SE's dimensions, but for now, let's roll with this result. Let's produce the final mask by ORing these two images:

# OR the masks to create the final checkbox mask:finalMask = cv2.bitwise_or(verticalMask, horizontalMask)

Which yields this image, the final check boxes mask:

Note that the check boxes are mostly undisturbed by the filtering pipeline, although some noise still remains. However, the check boxes are the larger elements of the image - they are also much larger than the remaining noise. Let's detect contours and filter them using a minimum area (and a maximum) threshold. We can also approximate the check boxes' contours to nice bounding rectangles:

# Get contours of the final mask:
contours, hierarchy = cv2.findContours(finalMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# Store the checkbox bounding rectangle here:
rectanglesList = []

# Look for the outer bounding boxes (no children):for _, c inenumerate(contours):

    # Get current blob area:
    currentArea = cv2.contourArea(c)

    # Set a min area threshold:
    minArea = 150
    maxArea = 200# Look for target contours:if minArea < currentArea < maxArea:

        # Approximate the contour to a polygon:
        contoursPoly = cv2.approxPolyDP(c, 3, True)
        # Get the polygon's bounding rectangle:
        boundRect = cv2.boundingRect(contoursPoly)

        # Store rectangles in list:
        rectanglesList.append(boundRect)

        # Get the dimensions of the bounding rect:
        rectX = boundRect[0]
        rectY = boundRect[1]
        rectWidth = boundRect[2]
        rectHeight = boundRect[3]

        # Set bounding rect:
        color = (0, 0, 255)
        cv2.rectangle( inputImageCopy, (int(rectX), int(rectY)),
                   (int(rectX + rectWidth), int(rectY + rectHeight)), color, 2 )

        cv2.imshow("Checkboxes", inputImageCopy)
        cv2.waitKey(0)

Note that additionally, I stored the bounding rectangles in the rectanglesList. This is the final image, with the target bounding rectangles drawn in red: