Filling holes in an image using OpenCV ( Python / C++ )

By | November 23, 2015
imfill : How to fill holes in a binary image

Figure 1. Left : Image of a nickel. Center : Thresholded and inverted. Right : Holes filled.

In this tutorial we will learn how to fill holes in a binary image. Consider the image on the left in Figure 1. Let’s say we want to find a binary mask that┬áseparates the coin from the background as shown in the right image. In this tutorial the circular region that contains the coin will also be referred to as the foreground.

Notice that the boundary of the coin is dark and distinct from its white background. So, we use simple image thresholding to separate the boundary from the background. In other words, we say pixels with intensities above a certain value ( threshold ) are the background and the rest are the foreground. The center image shows this thresholded image ( black represents background, and white represents foreground ). Unfortunately, even though the boundary has been nicely extracted ( it is solid white ), the interior of the coin has intensities similar to the background. Therefore, the thresholding operation cannot distinguish it from the background. How do we fill all pixels inside the circular boundary with white ?

MATLAB has a function called imfill that allows you to fill holes, and you can use it in the following way.

% MATLAB code for filling holes in a binary image. 
im = imfill(im,'holes');

imfill in OpenCV

There is no imfill function in OpenCV, but we can surely write one! The idea is rather simple. We know the pixel (0,0) is connected to the background. So we can extract the background, by simply doing a floodfill operation from pixel (0, 0). Pixels that are not affected by the floodfill operation are necessarily inside the boundary. The flood-filled image when inverted and combined with the thresholded image gives the foreground mask!

imfill opencv steps

Figure 2.

Steps for implementing imfill in OpenCV

Please refer to Figure 2. while reading the steps below.

  1. Read in the image.
  2. Threshold the input image to obtain a binary image.
  3. Flood fill from pixel (0, 0). Notice the difference between the outputs of step 2 and step 3 is that the background in step 3 is now white.
  4. Invert the flood filled image ( i.e. black becomes white and white becomes black ).
  5. Combine the thresholded image with the inverted flood filled image using bitwise OR operation to obtain the final foreground mask with holes filled in. The image in Step 4 has some black areas inside the boundary. By design the image in Step 2 has those holes filled in. So we combine the two to get the mask.

C++ and Python code for filling holes in a binary image

And here is how it is done in code
C++

 
#include "opencv2/opencv.hpp"

using namespace cv;

int main(int argc, char **argv)
{
    // Read image
    Mat im_in = imread("nickel.jpg", IMREAD_GRAYSCALE);

  
    // Threshold.
    // Set values equal to or above 220 to 0.
    // Set values below 220 to 255.
    Mat im_th;
    threshold(im_in, im_th, 220, 255, THRESH_BINARY_INV);
    
    // Floodfill from point (0, 0)
    Mat im_floodfill = im_th.clone();
    floodFill(im_floodfill, cv::Point(0,0), Scalar(255));
    
    // Invert floodfilled image
    Mat im_floodfill_inv;
    bitwise_not(im_floodfill, im_floodfill_inv);
    
    // Combine the two images to get the foreground.
    Mat im_out = (im_th | im_floodfill_inv);

    // Display images
    imshow("Thresholded Image", im_th);
    imshow("Floodfilled Image", im_floodfill);
    imshow("Inverted Floodfilled Image", im_floodfill_inv);
    imshow("Foreground", im_out);
    waitKey(0);
    
}

Python

import cv2;
import numpy as np;

# Read image
im_in = cv2.imread("nickel.jpg", cv2.IMREAD_GRAYSCALE);

# Threshold.
# Set values equal to or above 220 to 0.
# Set values below 220 to 255.

th, im_th = cv2.threshold(im_in, 220, 255, cv2.THRESH_BINARY_INV);

# Copy the thresholded image.
im_floodfill = im_th.copy()

# Mask used to flood filling.
# Notice the size needs to be 2 pixels than the image.
h, w = im_th.shape[:2]
mask = np.zeros((h+2, w+2), np.uint8)

# Floodfill from point (0, 0)
cv2.floodFill(im_floodfill, mask, (0,0), 255);

# Invert floodfilled image
im_floodfill_inv = cv2.bitwise_not(im_floodfill)

# Combine the two images to get the foreground.
im_out = im_th | im_floodfill_inv

# Display images.
cv2.imshow("Thresholded Image", im_th)
cv2.imshow("Floodfilled Image", im_floodfill)
cv2.imshow("Inverted Floodfilled Image", im_floodfill_inv)
cv2.imshow("Foreground", im_out)
cv2.waitKey(0)

Other techniques

There are other ways to solve the same problem. One way is to use morphological close operation. However, for morphological operations to work you will need to know the maximum size of the hole. Another way is to use findContours to find the contours and then fill it in using drawContours. I prefer the simplicity and speed of the technique described in this post.

Subscribe

If you liked this article, please subscribe to our newsletter to download all code and images used in this post. You will also receive a free Computer Vision Resource guide. In our newsletter we share OpenCV tutorials and examples written in C++/Python, and Computer Vision and Machine Learning algorithms and news.

Subscribe Now

Category: how-to Tags: ,

About Satya Mallick

I am an entrepreneur with a love for Computer Vision and Machine Learning with a dozen years of experience (and a Ph.D.) in the field. In 2007, right after finishing my Ph.D., I co-founded TAAZ Inc. with my advisor Dr. David Kriegman and Kevin Barnes. The scalability, and robustness of our computer vision and machine learning algorithms have been put to rigorous test by more than 100M users who have tried our products.

  • Paul Yan

    I am trying to implement this solution for OpenCV (3.1.0) for Unity. I’m stuck here with your C++ example: floodFill(im_floodfill, cv::Point(0,0), Scalar(255));

    According to the wrapper’s API (http://goo.gl/npfOQv), floodFill requires 4 arguments (not 3). The target MAT, a mask MAT, a seed point, and a scalar value. What should I use as the mask MAT here?

    • Paul Yan

      I noticed in the Python example, you are creating a mask that is 2 pixels wider and taller, but it’s not clear to me what’s conceptually happening here. Can you please expand on that a little?

  • amrosik

    Hello Mr. Mallick,

    I am wondering what would happen, if the picture of the coin had little mistakes, I mean little gaps in the contour. The floodfill would then penetrate the circle through these gaps, and fillup the inside of the coin. My question is, if there is a possibility to “extrapolate” a contour in order to form a >closed contour Although the vessels are all circular, unfortunately the camera is not distant enough, therefore there non-central vessels will be seen partly from the side. How could I compensate for this?
    One way of geting the information needed to detect the vessel would be to put the camera on a slider, and let it make images from 2 different perspectives. If from your point of view this method is robust, I will go for it.

    • stan

      I am think the solution would be to dilate and erode the image as needed to connect the gaps before you floodfil

  • Gaijin

    So i think the filling part worked for me, however the background turned yellow. Rebinarizing the image does not work. Anyone has an idea how this could happen?