Monday, November 7, 2011

Computing a disparity map in OpenCV

A disparity map contains information related to the distance of the objects of a scene from a viewpoint. In this example we will see how to compute a disparity map from a stereo pair and how to use the map to cut the objects far from the cameras.
The stereo pair is represented by two input images, these images are taken with two cameras separated by a distance and the disparity map is derived from the offset of the objects between them. There are various algorithm to compute a disparity map, the one implemented in OpenCV is the graph cut algorithm. To use it we have to call the function CreateStereoGCState() to initialize the data structure needed by the algorithm and use the function FindStereoCorrespondenceGC() to get the disparity map. Let's see the code:
def cut(disparity, image, threshold):
 for i in range(0, image.height):
  for j in range(0, image.width):
   # keep closer object
   if cv.GetReal2D(disparity,i,j) > threshold:
    cv.Set2D(disparity,i,j,cv.Get2D(image,i,j))

# loading the stereo pair
left  = cv.LoadImage('scene_l.bmp',cv.CV_LOAD_IMAGE_GRAYSCALE)
right = cv.LoadImage('scene_r.bmp',cv.CV_LOAD_IMAGE_GRAYSCALE)

disparity_left  = cv.CreateMat(left.height, left.width, cv.CV_16S)
disparity_right = cv.CreateMat(left.height, left.width, cv.CV_16S)

# data structure initialization
state = cv.CreateStereoGCState(16,2)
# running the graph-cut algorithm
cv.FindStereoCorrespondenceGC(left,right,
                          disparity_left,disparity_right,state)

disp_left_visual = cv.CreateMat(left.height, left.width, cv.CV_8U)
cv.ConvertScale( disparity_left, disp_left_visual, -16 );
cv.Save( "disparity.pgm", disp_left_visual ); # save the map

# cutting the object farthest of a threshold (120)
cut(disp_left_visual,left,120)

cv.NamedWindow('Disparity map', cv.CV_WINDOW_AUTOSIZE)
cv.ShowImage('Disparity map', disp_left_visual)
cv.WaitKey()
These are the two input image I used to test the program (respectively left and right):
Result using threshold = 100

Result using threshold = 120
Result using threshold = 180

5 comments:

  1. Thanks for the disparity map code.
    Do you know of any pythonese version of the full stereo calibration, etc. code from the Learning OpenCV book? I'm surprised that no one has posted this already. Please let me know if anyone has a version to share: samador@haverford.edu

    ReplyDelete
  2. Why do you scale the 16 bit signed disparity image with factor -20:

    cv.ConvertScale( disparity_left, disp_left_visual, -20 );

    I think you should scale it with -16 into an 8 bit, shouldn't you?

    With best regards,
    Marc

    ReplyDelete
  3. Hello Mark, thanks for your comment.
    It's a mistake. As you noticed, the correct scaling factor is -16.

    bests

    ReplyDelete
  4. very good tutorial for stereo vision with a powerful vision software like Opencv. I add a link of your article to my post about stereo vision systems uses in robotics, tutorials and resources

    ReplyDelete
  5. The results really good! but the programs take a lot of time to execute.
    GC is comparatively slower than BM, but results are much better.
    One could not expect such good disparity map from BM!
    Thanks for sharing

    ReplyDelete