Research

My primary research goal is to enable visual artists to achieve their vision quickly and effectively. Specifically, I focus on colors and transparency in images and videos. The topic of my ongoing PhD is the extraction of partially transparent layers with the main target application being professional movie post-production.

Publications


Yağız Aksoy, Tunç Ozan Aydın and Marc Pollefeys
arXiv:1707.05055 [cs.CV], 2017
We present a novel, purely affinity-based natural image matting algorithm. Our method relies on carefully defined pixel-to-pixel connections that enable effective use of information available in the image. We control the information flow from the known-opacity regions into the unknown region, as well as within the unknown region itself, by utilizing multiple definitions of pixel affinities. Among other forms of information flow, we introduce color-mixture flow, which builds upon local linear embedding and effectively encapsulates the relation between different pixel opacities. Our resulting novel linear system formulation can be solved in closed-form and is robust against several fundamental challenges of natural matting such as holes and remote intricate structures. Our evaluation using the alpha matting benchmark suggests a significant performance improvement over the current methods. While our method is primarily designed as a standalone matting tool, we show that it can also be used for regularizing mattes obtained by sampling-based methods. We extend our formulation to layer color estimation and show that the use of multiple channels of flow increases the layer color quality. We also demonstrate our performance in green-screen keying and further analyze the characteristics of the affinities used in our method.
@ARTICLE{ifmExt,
author={Aksoy, Ya\u{g}{\i}z and Ayd{\i}n, Tun\c{c} Ozan and Pollefeys, Marc},
journal = {\tt arXiv:1707.05055 [cs.CV]},
title={Designing Effective Inter-Pixel Information Flow for Natural Image Matting},
year={2017},
}

Yağız Aksoy, Tunç Ozan Aydın and Marc Pollefeys
CVPR, 2017 (accepted as spotlight presentation)
We present a novel, purely affinity-based natural image matting algorithm. Our method relies on carefully defined pixel-to-pixel connections that enable effective use of information available in the image and the trimap. We control the information flow from the known-opacity regions into the unknown region, as well as within the unknown region itself, by utilizing multiple definitions of pixel affinities. This way we achieve significant improvements on matte quality near challenging regions of the foreground object. Among other forms of information flow, we introduce color-mixture flow, which builds upon local linear embedding and effectively encapsulates the relation between different pixel opacities. Our resulting novel linear system formulation can be solved in closed-form and is robust against several fundamental challenges in natural matting such as holes and remote intricate structures. While our method is primarily designed as a standalone natural matting tool, we show that it can also be used for regularizing mattes obtained by various sampling-based methods. Our evaluation using the public alpha matting benchmark suggests a significant performance improvement over the state-of-the-art.
@INPROCEEDINGS{ifm,
author={Aksoy, Ya\u{g}{\i}z and Ayd{\i}n, Tun\c{c} Ozan and Pollefeys, Marc},
booktitle={Proc. CVPR},
title={Designing Effective Inter-Pixel Information Flow for Natural Image Matting},
year={2017},
}

Yağız Aksoy, Tunç Ozan Aydın, Aljoša Smolić and Marc Pollefeys
ACM Transactions on Graphics, 2017
We present a new method for decomposing an image into a set of soft color segments, which are analogous to color layers with alpha channels that have been commonly utilized in modern image manipulation software. We show that the resulting decomposition serves as an effective intermediate image representation, which can be utilized for performing various, seemingly unrelated image manipulation tasks. We identify a set of requirements that soft color segmentation methods have to fulfill, and present an in-depth theoretical analysis of prior work. We propose an energy formulation for producing compact layers of homogeneous colors and a color refinement procedure, as well as a method for automatically estimating a statistical color model from an image. This results in a novel framework for automatic and high-quality soft color segmentation, which is efficient, parallelizable, and scalable. We show that our technique is superior in quality compared to previous methods through quantitative analysis as well as visually through an extensive set of examples. We demonstrate that our soft color segments can easily be exported to familiar image manipulation software packages and used to produce compelling results for numerous image manipulation applications without forcing the user to learn new tools and workflows.
@ARTICLE{scs,
author={Ya\u{g}{\i}z Aksoy and Tun\c{c} Ozan Ayd{\i}n and Aljo\v{s}a Smoli\'{c} and Marc Pollefeys},
title={Unmixing-Based Soft Color Segmentation for Image Manipulation},
journal={ACM Trans. Graph.},
year={2017},
pages = {19:1-19:19},
volume = {36},
number = {2}
}

Yağız Aksoy, Tunç Ozan Aydın, Marc Pollefeys and Aljoša Smolić
ACM Transactions on Graphics, 2016
Due to the widespread use of compositing in contemporary feature films, green-screen keying has become an essential part of post-production workflows. To comply with the ever-increasing quality requirements of the industry, specialized compositing artists spend countless hours using multiple commercial software tools, while eventually having to resort to manual painting because of the many shortcomings of these tools. Due to the sheer amount of manual labor involved in the process, new green-screen keying approaches that produce better keying results with less user interaction are welcome additions to the compositing artist's arsenal. We found that --- contrary to the common belief in the research community --- production-quality green-screen keying is still an unresolved problem with its unique challenges. In this paper, we propose a novel green-screen keying method utilizing a new energy minimization-based color unmixing algorithm. We present comprehensive comparisons with commercial software packages and relevant methods in literature, which show that the quality of our results is superior to any other currently available green-screen keying solution. Importantly, using the proposed method, these high-quality results can be generated using only one-tenth of the manual editing time that a professional compositing artist requires to process the same content having all previous state-of-the-art tools at his disposal.
@ARTICLE{keying,
author={Ya\u{g}{\i}z Aksoy and Tun\c{c} Ozan Ayd{\i}n and Marc Pollefeys and Aljo\v{s}a Smoli\'{c}},
title={Interactive High-Quality Green-Screen Keying via Color Unmixing},
journal={ACM Trans. Graph.},
year={2016},
volume = {35},
number = {5},
pages = {152:1--152:12},
}

Yağız Aksoy and A. Aydın Alatan
IEEE ICIP, 2014
Most of the mobile applications require efficient and precise computation of the device pose, and almost every mobile device has inertial sensors already equipped together with a camera. This fact makes sensor fusion quite attractive for increasing efficiency during pose tracking. However, the state-of-the-art fusion algorithms have a major shortcoming: lack of well-defined uncertainty introduced to the system during the prediction stage of the fusion filters. Such a drawback results in determining covariances heuristically, and hence, requirement for data-dependent tuning to achieve high performance or even convergence of these filters. In this paper, we propose an inertially-aided visual odometry system that requires neither heuristics nor parameter tuning; computation of the required uncertainties on all the estimated variables are obtained after minimum number of assumptions. Moreover, the proposed system simultaneously estimates the metric scale of the pose computed from a monocular image stream. The experimental results indicate that the proposed scale estimation outperforms the state-of-the-art methods, whereas the pose estimation step yields quite acceptable results in real-time on resource constrained systems.
@INPROCEEDINGS{yaksoyicip14a,
author={Aksoy, Ya\u{g}{\i}z and Alatan, A. Ayd{\i}n},
title={Uncertainty Modeling for Efficient Visual Odometry via Inertial Sensors on Mobile Devices},
booktitle={IEEE International Conference on Image Processing (ICIP)},
year={2014}}

Florian Angehrn, Oliver Wang, Yağız Aksoy, Markus Gross and Aljosa Smolic
IEEE ICIP, 2014
Free viewpoint video enables interactive viewpoint selection in real world scenes, which is attractive for many applications such as sports visualization. Multi-camera registration is one of the difficult tasks in such systems. We introduce the concept of a static high resolution master camera for improved long-term multiview alignment. All broadcast cameras are aligned to a common reference. Our approach builds on frame-to-frame alignment, extended into a recursive long-term estimation process, which is shown to be accurate, robust and stable over long sequences.
@INPROCEEDINGS{anghernicip14,
author={Florian Angehrn and Oliver Wang and Ya\u{g}{\i}z Aksoy and Markus Gross and Aljo\v{s}a Smoli\'{c}},
title={MasterCam FVV: Robust Registration of Multiview Sports Video to a Static High-Resolution Master Camera for Free Viewpoint Video},
booktitle={IEEE International Conference on Image Processing (ICIP)},
year={2014}}

Yağız Aksoy
Master's Thesis, Middle East Technical University, 2013
With the increase in the number and computational power of commercial mobile devices like smart phones and tablet computers, augmented reality applications are gaining more and more volume. In order to augment virtual objects effectively in real scenes, pose of the camera should be estimated with high precision and speed. Today, most of the mobile devices feature cameras and inertial measurement units which carry information on change in position and attitude of the camera. In this thesis, utilization of inertial sensors on mobile devices in aiding visual pose estimation is studied. Error characteristics of the inertial sensors on the utilized mobile device are analyzed. Gyroscope readings are utilized for aiding 2D feature tracking while accelerometer readings are used to help create a sparse 3D map of features later to be used for visual pose estimation. Metric velocity estimation is formulated using inertial readings and observations of a single 2D feature. Detailed formulations of uncertainties on all the estimated variables are provided. Finally, a novel, lightweight filter, which is capable of estimating the pose of the camera together with the metric scale, is proposed. The proposed filter runs without any heuristics needed for covariance propagation, which enables it to be used in different devices with different sensor characteristics without any modifications. Sensors with poor noise characteristics are successfully utilized to aid the visual pose estimation.
@MASTERSTHESIS{yaksoymetu13,
author = {Aksoy, Ya\u{g}{\i}z},
title = {Efficient Inertially Aided Visual Odometry towards Mobile Augmented Reality},
school = {Middle East Technical University},
year = {2013},
month = {August}}

Yağız Aksoy and A. Aydın Alatan
ECCV Workshops, 2012
Shadows are illuminated as a result of Rayleigh scattering phenomenon, which happens to be more effective for small wavelengths of light. We propose utilization of false color images for shadow detection, since the transformation eliminates high frequency blue component and introduces low frequency near-infrared channel. Effectiveness of the approach is tested by using several shadow-variant texture and color-related cues proposed in the literature. Performances of these cues in regular and false color images are compared and analyzed within a supervised system by using a support vector machine classifier.
@INPROCEEDINGS{yaksoyeccvw12,
author={Aksoy, Yagiz and Alatan, A. Aydin},
title={Utilization of False Color Images in Shadow Detection},
booktitle={European Conference on Computer Vision (ECCV) Workshops},
year={2012}}

Yağız Aksoy, Ozan Şener, A. Aydın Alatan and Kemal Uğur
IEEE ICIP, 2012
We propose a complete still image based 2D-3D mobile conversion system for touch screen use. The system consists of interactive segmentation followed by 3D rendering. The interactive segmentation is conducted dynamically by color Gaussian mixture model updates and dynamic-iterative graph-cut. A coloring gesture is used to guide the way and entertain the user during the process. Output of the image segmentation is then fed to the 3D rendering stage of the system. For rendering stage, two novel improvements are proposed to handle holes resulting from depth image based rendering process. These improvements are also expected to enhance the 3D perception. These two methods are subjectively tested and their results are presented.
@INPROCEEDINGS{yaksoyicip12,
author={Aksoy, Yagiz and Sener, Ozan and Alatan, A. Aydin and Ugur, Kemal},
title={Interactive 2D-3D Image Conversion for Mobile Devices},
booktitle={IEEE International Conference on Image Processing (ICIP)},
year={2012}}

Publications in Medicine


Impact of transrectal prostate needle biopsy on erectile function: Results of power Doppler ultrasonography of the prostate
Altug Tuncel, Ugur Toprak, Melih Balci, Ersin Koseoglu, Yagiz Aksoy, Alp Karademir and Ali Atan
The Kaohsiung Journal of Medical Sciences, 2014
We evaluated the impact of transrectal prostate needle biopsy (TPNB) on erectile function and on the prostate and bilateral neurovascular bundles using power Doppler ultrasonography imaging of the prostate. The study consisted of 42 patients who had undergone TPNB. Erectile function was evaluated prior to the biopsy, and in the 3rd month after the biopsy using the first five-item version of the International Index of Erectile Function (IIEF-5). Prior to and 3 months after the biopsy, the resistivity index of the prostate parenchyma and both neurovascular bundles was measured. The mean age of the men was 64.2 (47–78) years. Prior to TPNB, 10 (23.8%) patients did not have erectile dysfunction (ED) and 32 (76.2%) patients had ED. The mean IIEF-5 score was 20.8 (range: 2–25) prior to the biopsies, and the mean IIEF-5 score was 17.4 (range: 5–25; p < 0.001) after 3 months. For patients who were previously potent in the pre-biopsy period, the ED rate was 40% (n = 4/10) at the 3rd month evaluation. In these patients, all the resistivity index values were significantly decreased. Our results showed that TPNB may lead to an increased risk of ED. The presence of ED in men after TPNB might have an organic basis.
@ARTICLE{Tuncel2014,
title = "Impact of transrectal prostate needle biopsy on erectile function: Results of power Doppler ultrasonography of the prostate ",
author = "Altug Tuncel and Ugur Toprak and Melih Balci and Ersin Koseoglu and Yagiz Aksoy and Alp Karademir and Ali Atan",
journal = "The Kaohsiung Journal of Medical Sciences ",
volume = "30",
number = "4",
pages = "194 - 199",
year = "2014",}

Transrectal prostate needle biopsy has a negative effect on neurovascular tissue and associated with an increased risk of erectile dysfunction: results of power doppler imaging of the prostate
Altug Tuncel, Ugur Toprak, Melih Balci, Ersin Koseoglu, Yagiz Aksoy, Alp Karademir and Ali Atan
Congress of the European Society for Sexual Medicine (ESSM), 2012
@incollection{Tuncel2012,
year={2012},
booktitle={Congress of the European Society for Sexual Medicine},
title={Transrectal prostate needle biopsy has a negative effect on neurovascular tissue and associated with an increased risk of erectile dysfunction: results of power doppler imaging of the prostate},
author={Tuncel, A and Toprak, U. and Balci, M. and Koseoglu, E. and Aksoy, Y. and Karademir, A. and Atan, A.},}