Major venues

Yağız Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys and Wojciech Matusik
ECCV, 2018
Illumination is a critical element of photography and is essential for many computer vision tasks. Flash light is unique in the sense that it is a widely available tool for easily manipulating the scene illumination. We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefi t from having separate illuminations. Different than the typical use of crowdsourcing in generating computer vision datasets, we make use of the crowd to directly take the photographs that make up our dataset. As a result, our dataset covers a wide variety of scenes captured by many casual photographers. We detail the advantages and challenges of our approach to crowdsourcing as well as the computational effort to generate completely separate flash illuminations from the ambient light in an uncontrolled setup. We present a brief examination of illumination decomposition, a challenging and underconstrained problem in flash photography, to demonstrate the use of our dataset in a data-driven approach.
author={Ya\u{g}{\i}z Aksoy and Changil Kim and Petr Kellnhofer and Sylvain Paris and Mohamed Elgharib and Marc Pollefeys and Wojciech Matusik},
booktitle={Proc. ECCV},
title={A Dataset of Flash and Ambient Illumination Pairs from the Crowd},

Yağız Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys and Wojciech Matusik
ACM Transactions on Graphics (Proc. SIGGRAPH), 2018
Accurate representation of soft transitions between image regions is essential for high-quality image editing and compositing. Current techniques for generating such representations depend heavily on interaction by a skilled visual artist, as creating such accurate object selections is a tedious task. In this work, we introduce semantic soft segments, a set of layers that correspond to semantically meaningful regions in an image with accurate soft transitions between different objects. We approach this problem from a spectral segmentation angle and propose a graph structure that embeds texture and color features from the image as well as higher-level semantic information generated by a neural network. The soft segments are generated via eigendecomposition of the carefully constructed Laplacian matrix fully automatically. We demonstrate that otherwise complex image editing tasks can be done with little effort using semantic soft segments.
author={Ya\u{g}{\i}z Aksoy and Tae-Hyun Oh and Sylvain Paris and Marc Pollefeys and Wojciech Matusik},
title={Semantic Soft Segmentation},
journal={ACM Trans. Graph. (Proc. SIGGRAPH)},
pages = {72:1-72:13},
volume = {37},
number = {4}

Alexandre Kaspar, Geneviève Patterson, Changil Kim, Yağız Aksoy, Wojciech Matusik and Mohamed Elgharib
ACM CHI Conference on Human Factors in Computing Systems, 2018
In this work, we propose two ensemble methods leveraging a crowd workforce to improve video annotation, with a focus on video object segmentation. Their shared principle is that while individual candidate results may likely be insufficient, they often complement each other so that they can be combined into something better than any of the individual results - the very spirit of collaborative working. For one, we extend a standard polygon-drawing interface to allow workers to annotate negative space, and combine the work of multiple workers instead of relying on a single best one as commonly done in crowdsourced image segmentation. For the other, we present a method to combine multiple automatic propagation algorithms with the help of the crowd. Such combination requires an understanding of where the algorithms fail, which we gather using a novel coarse scribble video annotation task. We evaluate our ensemble methods, discuss our design choices for them, and make our web-based crowdsourcing tools and results publicly available.
author={Alexandre Kaspar and Genevi\`eve Patterson and Changil Kim and Ya\u{g}{\i}z Aksoy and Wojciech Matusik and Mohamed Elgharib},
title={Crowd-Guided Ensembles: How Can We Choreograph Crowd Workers for Video Segmentation?},
booktitle={Proc. ACM CHI},

Yağız Aksoy, Tunç Ozan Aydın and Marc Pollefeys
CVPR, 2017 (spotlight)
We present a novel, purely affinity-based natural image matting algorithm. Our method relies on carefully defined pixel-to-pixel connections that enable effective use of information available in the image and the trimap. We control the information flow from the known-opacity regions into the unknown region, as well as within the unknown region itself, by utilizing multiple definitions of pixel affinities. This way we achieve significant improvements on matte quality near challenging regions of the foreground object. Among other forms of information flow, we introduce color-mixture flow, which builds upon local linear embedding and effectively encapsulates the relation between different pixel opacities. Our resulting novel linear system formulation can be solved in closed-form and is robust against several fundamental challenges in natural matting such as holes and remote intricate structures. While our method is primarily designed as a standalone natural matting tool, we show that it can also be used for regularizing mattes obtained by various sampling-based methods. Our evaluation using the public alpha matting benchmark suggests a significant performance improvement over the state-of-the-art.
author={Aksoy, Ya\u{g}{\i}z and Ayd{\i}n, Tun\c{c} Ozan and Pollefeys, Marc},
booktitle={Proc. CVPR},
title={Designing Effective Inter-Pixel Information Flow for Natural Image Matting},

Yağız Aksoy, Tunç Ozan Aydın, Aljoša Smolić and Marc Pollefeys
ACM Transactions on Graphics, 2017
We present a new method for decomposing an image into a set of soft color segments, which are analogous to color layers with alpha channels that have been commonly utilized in modern image manipulation software. We show that the resulting decomposition serves as an effective intermediate image representation, which can be utilized for performing various, seemingly unrelated image manipulation tasks. We identify a set of requirements that soft color segmentation methods have to fulfill, and present an in-depth theoretical analysis of prior work. We propose an energy formulation for producing compact layers of homogeneous colors and a color refinement procedure, as well as a method for automatically estimating a statistical color model from an image. This results in a novel framework for automatic and high-quality soft color segmentation, which is efficient, parallelizable, and scalable. We show that our technique is superior in quality compared to previous methods through quantitative analysis as well as visually through an extensive set of examples. We demonstrate that our soft color segments can easily be exported to familiar image manipulation software packages and used to produce compelling results for numerous image manipulation applications without forcing the user to learn new tools and workflows.
author={Ya\u{g}{\i}z Aksoy and Tun\c{c} Ozan Ayd{\i}n and Aljo\v{s}a Smoli\'{c} and Marc Pollefeys},
title={Unmixing-Based Soft Color Segmentation for Image Manipulation},
journal={ACM Trans. Graph.},
pages = {19:1-19:19},
volume = {36},
number = {2}

Yağız Aksoy, Tunç Ozan Aydın, Marc Pollefeys and Aljoša Smolić
ACM Transactions on Graphics, 2016
Due to the widespread use of compositing in contemporary feature films, green-screen keying has become an essential part of post-production workflows. To comply with the ever-increasing quality requirements of the industry, specialized compositing artists spend countless hours using multiple commercial software tools, while eventually having to resort to manual painting because of the many shortcomings of these tools. Due to the sheer amount of manual labor involved in the process, new green-screen keying approaches that produce better keying results with less user interaction are welcome additions to the compositing artist's arsenal. We found that --- contrary to the common belief in the research community --- production-quality green-screen keying is still an unresolved problem with its unique challenges. In this paper, we propose a novel green-screen keying method utilizing a new energy minimization-based color unmixing algorithm. We present comprehensive comparisons with commercial software packages and relevant methods in literature, which show that the quality of our results is superior to any other currently available green-screen keying solution. Importantly, using the proposed method, these high-quality results can be generated using only one-tenth of the manual editing time that a professional compositing artist requires to process the same content having all previous state-of-the-art tools at his disposal.
author={Ya\u{g}{\i}z Aksoy and Tun\c{c} Ozan Ayd{\i}n and Marc Pollefeys and Aljo\v{s}a Smoli\'{c}},
title={Interactive High-Quality Green-Screen Keying via Color Unmixing},
journal={ACM Trans. Graph.},
volume = {35},
number = {5},
pages = {152:1--152:12},

Other conferences and workshops

Mattia Ryffel, Fabio Zünd, Yağız Aksoy, Alessia Marra, Maurizio Nitti, Tunç Ozan Aydın and Bob Sumner
International Conference on Game and Entertainment Technologies, 2017
We present a mobile augmented reality application that allows its users to modify colors of paintings via simple touch interactions. Our method is intended for museums and art exhibitions and aims to provide an entertaining way for interacting with paintings in a non-intrusive manner. Plausible color edits are achieved by utilizing a set of layers with corresponding alpha channels, which needs to be generated for each individual painting in a pre-processing step. Manually performing such a layer decomposition is a tedious process and makes the entire system infeasible for most practical use cases. In this work, we propose the use of a fully automatic soft color segmentation algorithm for content generation for such an augmented reality application. This way, we significantly reduce the amount of manual labor needed for deploying our system and thus make our system feasible for real-world use.
author={Mattia Ryffel and Fabio Z\"und and Ya\u{g}{\i}z Aksoy and Alessia Marra and Maurizio Nitti and Tun\c{c} Ozan Ayd{\i}n and Bob Sumner},
title={AR Museum: A Mobile Augmented Reality Application for Interactive Painting Recoloring},
booktitle={International Conference on Game and Entertainment Technologies},

Yağız Aksoy and A. Aydın Alatan
Most of the mobile applications require efficient and precise computation of the device pose, and almost every mobile device has inertial sensors already equipped together with a camera. This fact makes sensor fusion quite attractive for increasing efficiency during pose tracking. However, the state-of-the-art fusion algorithms have a major shortcoming: lack of well-defined uncertainty introduced to the system during the prediction stage of the fusion filters. Such a drawback results in determining covariances heuristically, and hence, requirement for data-dependent tuning to achieve high performance or even convergence of these filters. In this paper, we propose an inertially-aided visual odometry system that requires neither heuristics nor parameter tuning; computation of the required uncertainties on all the estimated variables are obtained after minimum number of assumptions. Moreover, the proposed system simultaneously estimates the metric scale of the pose computed from a monocular image stream. The experimental results indicate that the proposed scale estimation outperforms the state-of-the-art methods, whereas the pose estimation step yields quite acceptable results in real-time on resource constrained systems.
author={Aksoy, Ya\u{g}{\i}z and Alatan, A. Ayd{\i}n},
title={Uncertainty Modeling for Efficient Visual Odometry via Inertial Sensors on Mobile Devices},
booktitle={IEEE International Conference on Image Processing (ICIP)},

Florian Angehrn, Oliver Wang, Yağız Aksoy, Markus Gross and Aljosa Smolic
Free viewpoint video enables interactive viewpoint selection in real world scenes, which is attractive for many applications such as sports visualization. Multi-camera registration is one of the difficult tasks in such systems. We introduce the concept of a static high resolution master camera for improved long-term multiview alignment. All broadcast cameras are aligned to a common reference. Our approach builds on frame-to-frame alignment, extended into a recursive long-term estimation process, which is shown to be accurate, robust and stable over long sequences.
author={Florian Angehrn and Oliver Wang and Ya\u{g}{\i}z Aksoy and Markus Gross and Aljo\v{s}a Smoli\'{c}},
title={MasterCam FVV: Robust Registration of Multiview Sports Video to a Static High-Resolution Master Camera for Free Viewpoint Video},
booktitle={IEEE International Conference on Image Processing (ICIP)},

Yağız Aksoy and A. Aydın Alatan
ECCV Workshops, 2012
Shadows are illuminated as a result of Rayleigh scattering phenomenon, which happens to be more effective for small wavelengths of light. We propose utilization of false color images for shadow detection, since the transformation eliminates high frequency blue component and introduces low frequency near-infrared channel. Effectiveness of the approach is tested by using several shadow-variant texture and color-related cues proposed in the literature. Performances of these cues in regular and false color images are compared and analyzed within a supervised system by using a support vector machine classifier.
author={Aksoy, Yagiz and Alatan, A. Aydin},
title={Utilization of False Color Images in Shadow Detection},
booktitle={European Conference on Computer Vision (ECCV) Workshops},

Yağız Aksoy, Ozan Şener, A. Aydın Alatan and Kemal Uğur
We propose a complete still image based 2D-3D mobile conversion system for touch screen use. The system consists of interactive segmentation followed by 3D rendering. The interactive segmentation is conducted dynamically by color Gaussian mixture model updates and dynamic-iterative graph-cut. A coloring gesture is used to guide the way and entertain the user during the process. Output of the image segmentation is then fed to the 3D rendering stage of the system. For rendering stage, two novel improvements are proposed to handle holes resulting from depth image based rendering process. These improvements are also expected to enhance the 3D perception. These two methods are subjectively tested and their results are presented.
author={Aksoy, Yagiz and Sener, Ozan and Alatan, A. Aydin and Ugur, Kemal},
title={Interactive 2D-3D Image Conversion for Mobile Devices},
booktitle={IEEE International Conference on Image Processing (ICIP)},


Yağız Aksoy
Master's Thesis, Middle East Technical University, 2013
With the increase in the number and computational power of commercial mobile devices like smart phones and tablet computers, augmented reality applications are gaining more and more volume. In order to augment virtual objects effectively in real scenes, pose of the camera should be estimated with high precision and speed. Today, most of the mobile devices feature cameras and inertial measurement units which carry information on change in position and attitude of the camera. In this thesis, utilization of inertial sensors on mobile devices in aiding visual pose estimation is studied. Error characteristics of the inertial sensors on the utilized mobile device are analyzed. Gyroscope readings are utilized for aiding 2D feature tracking while accelerometer readings are used to help create a sparse 3D map of features later to be used for visual pose estimation. Metric velocity estimation is formulated using inertial readings and observations of a single 2D feature. Detailed formulations of uncertainties on all the estimated variables are provided. Finally, a novel, lightweight filter, which is capable of estimating the pose of the camera together with the metric scale, is proposed. The proposed filter runs without any heuristics needed for covariance propagation, which enables it to be used in different devices with different sensor characteristics without any modifications. Sensors with poor noise characteristics are successfully utilized to aid the visual pose estimation.
author = {Aksoy, Ya\u{g}{\i}z},
title = {Efficient Inertially Aided Visual Odometry towards Mobile Augmented Reality},
school = {Middle East Technical University},
year = {2013},
month = {August}}

Publications in medicine

Impact of transrectal prostate needle biopsy on erectile function: Results of power Doppler ultrasonography of the prostate
Altug Tuncel, Ugur Toprak, Melih Balci, Ersin Koseoglu, Yağız Aksoy, Alp Karademir and Ali Atan
The Kaohsiung Journal of Medical Sciences, 2014
We evaluated the impact of transrectal prostate needle biopsy (TPNB) on erectile function and on the prostate and bilateral neurovascular bundles using power Doppler ultrasonography imaging of the prostate. The study consisted of 42 patients who had undergone TPNB. Erectile function was evaluated prior to the biopsy, and in the 3rd month after the biopsy using the first five-item version of the International Index of Erectile Function (IIEF-5). Prior to and 3 months after the biopsy, the resistivity index of the prostate parenchyma and both neurovascular bundles was measured. The mean age of the men was 64.2 (47–78) years. Prior to TPNB, 10 (23.8%) patients did not have erectile dysfunction (ED) and 32 (76.2%) patients had ED. The mean IIEF-5 score was 20.8 (range: 2–25) prior to the biopsies, and the mean IIEF-5 score was 17.4 (range: 5–25; p < 0.001) after 3 months. For patients who were previously potent in the pre-biopsy period, the ED rate was 40% (n = 4/10) at the 3rd month evaluation. In these patients, all the resistivity index values were significantly decreased. Our results showed that TPNB may lead to an increased risk of ED. The presence of ED in men after TPNB might have an organic basis.
title = "Impact of transrectal prostate needle biopsy on erectile function: Results of power Doppler ultrasonography of the prostate ",
author = "Altug Tuncel and Ugur Toprak and Melih Balci and Ersin Koseoglu and Yagiz Aksoy and Alp Karademir and Ali Atan",
journal = "The Kaohsiung Journal of Medical Sciences ",
volume = "30",
number = "4",
pages = "194 - 199",
year = "2014",}

Transrectal prostate needle biopsy has a negative effect on neurovascular tissue and associated with an increased risk of erectile dysfunction: results of power doppler imaging of the prostate
Altug Tuncel, Ugur Toprak, Melih Balci, Ersin Koseoglu, Yağız Aksoy, Alp Karademir and Ali Atan
Congress of the European Society for Sexual Medicine (ESSM), 2012
booktitle={Congress of the European Society for Sexual Medicine},
title={Transrectal prostate needle biopsy has a negative effect on neurovascular tissue and associated with an increased risk of erectile dysfunction: results of power doppler imaging of the prostate},
author={Tuncel, A and Toprak, U. and Balci, M. and Koseoglu, E. and Aksoy, Y. and Karademir, A. and Atan, A.},}