Designing Effective Inter-Pixel Information Flow for Natural Image Matting

Yagiz Aksoy, Tunc Ozan Aydin and Marc Pollefeys
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017
Spotlight presentation
Designing Effective Inter-Pixel Information Flow for Natural Image Matting

For an input image (a) and a trimap (b), we define several forms of information flow inside the image. We begin with color-mixture flow (c), then add direct channels of information flow from known to unknown regions (d), and let effective share of information inside the unknown region (e) to increase the matte quality in challenging regions. We finally add local information flow to get our spatially smooth result (f).

Abstract

We present a novel, purely affinity-based natural image matting algorithm. Our method relies on carefully defined pixel-to-pixel connections that enable effective use of information available in the image and the trimap. We control the information flow from the known-opacity regions into the unknown region, as well as within the unknown region itself, by utilizing multiple definitions of pixel affinities. This way we achieve significant improvements on matte quality near challenging regions of the foreground object. Amongst other forms of information flow, we introduce color-mixture flow, which builds upon local linear embedding and effectively encapsulates the relation between different pixel opacities. Our resulting novel linear system formulation can be solved in closed form, and is robust against several fundamental challenges in natural matting, such as holes and remote intricate structures. While our method is primarily designed as a standalone natural matting tool, we show that it can also be used for regularizing mattes obtained by various sampling-based methods. Our evaluation using the public alpha matting benchmark suggests a significant performance improvement over the state-of-the-art.

Pre-print

BibTeX

@INPROCEEDINGS{ifm,
author={Aksoy, Ya\u{g}{\i}z and Ayd{\i}n, Tun\c{c} and Pollefeys, Marc},
booktitle={Proc. CVPR},
title={Designing Effective Inter-Pixel Information Flow for Natural Image Matting},
year={2017},
}

Benchmark results

More results and experiments will be released on this page towards CVPR 2017.

Implementation

We can not release the original source code. However, a reimplementation of the method is available as a part of the affinity-based matting toolbox. While this reimplementation does not give the same results as the original implementation, we recommend its use in comparisons and extensions.