Efficient Slanted PatchMatch for Realtime Spacetime Stereo

Harris Nover, Supreeth Achar, Dan B Goldman



We present ESPReSSo, the first real-time implementation of spacetime stereo, offering improved quality vs. existing real-time systems. ESPReSSo uses a local stereo reconstruction algorithm that precomputes subpixel-shifted binary descriptors, then iteratively samples those descriptors along slanted disparity plane hypotheses, applying an edge-aware filter for spatial cost aggregation. Plane hypotheses are shared across rectangular tiles, but every pixel gets a different winner, much as in PatchMatch Filter. This architecture performs very few descriptor computations but many cost aggregations, and we tune our choice of descriptor and filter accordingly: We propose a new 32-bit binary spacetime descriptor breve that combines the benefits of small spatial extent with robustness to scene motion, and the system aggregates costs using the permeability filter, a very efficient edge-aware filter. Our prototype system outputs 60 depth frames per second on a desktop GPU, using less than 11ms total computation per frame.