Contact

About me

Since July 2019, I work full time at Nokia Bell Labs but I still keep close contact with the Institute for Pervasive Computing at ETH where I was a postdoctoral researcher and a PhD student before. I still actively supervise three PhD students at ETH.

My research interests include mobile and wearable computer vision, human computer interaction, and augmented reality. Between 2016 and 2019, I worked as the lead engineer of the ETH spin-off Kapanu on building cutting-edge augmented reality for dentistry. From 2011 to 2015, I was a scientific advisor to the ETH spin-off Scandit on visual code scanning with mobile and wearable devices. In 2014, I completed an R&D internship in mobile computer vision and augmented reality at Qualcomm Research.

I studied electrical engineering in Budapest and in Karlsruhe with focus on communication systems, and visual computing in Vienna with focus on mobile augmented reality. During my studies, I was working as a student assistant at the Computer and Automation Research Institute of the Hungarian Academy of Sciences. I joined ETH Zurich as a research and teaching assistant in March 2011 and defended my doctoral thesis in May 2016. (Full CV)

Research interests

Visual codes

Augmented reality

Interaction with smart objects

Mobile optical character recognition

Other activities

07/2019: I joined Nokia Bell Labs
11/2018: 3rd prize at ETH Entrepreneur Club startup pitching event 'From Science to Startup'
10/2018: Invited talk in the Art Museum of Liechtenstein at digitalTag, visitors including the prince of Liechtenstein
06/2018: Best poster award at the 16th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys), Munich, Germany
03/2018: Invited talk at digital Liechtenstein workshop on augmented reality
08/2017: We are featured on TechCrunch and The Verge
06/2017: Our startup Kapanu AG got acquired by Ivoclar Vivadent AG
03/2017: We presented our AR technology at IDS 2017, a dental trade show with over 155.000 visitors from 157 countries
07/2016: We are building cutting-edge augmented reality for dentistry at Kapanu
05/2016: I successfully defended my PhD thesis
01/2016: I attended the Dagstuhl seminar 16042 on eyewear computing
10/2015: I presented our research to the President of ETH and the Vice President of Switzerland
03/2015: Our work on gesture recognition was nominated for the ETH Spark Award 2015 as one of the 20 most promising inventions
03/2015: I presented our research to the Rector and the President of ETH
10/2014: I completed a 4-months internship on Mobile Augmented Reality at Qualcomm Research Vienna, Austria
09/2014: Our work is featured in ETH unterwegs
07/2014: Our work on gesture recognition won the Swisscom ICT Award 2014
10/2014: We are featured in Neue Zürcher Zeitung
09/2014: I was a student volunteer at the ECCV 2014 conference in Zurich
03/2014: I visited the Interactive 3D Technologies group at Microsoft Research Cambridge, UK
02/2014: I participated in the ETH - Disney Research Augmented Reality Games Hackathon in Zurich
09/2013: Our work is featured in ETH unterwegs
09/2013: I was local co-organizer of the Ubicomp 2013 and ISWC 2013 conferences in Zurich
06/2013: I attended the 4th UBI Summer School on Mobile Augmented Reality Interfaces in Oulu, Finland
05/2013: I presented my poster on next generation barcode scanning at the Swiss Vision Day 2013
10/2012: I was a student volunteer at the 3DIMPVT conference in Zurich
07/2012: I attended the International Computer Vision Summer School in Sicily
05/2012: I was a finalist in Mobile Visual Computing for the Qualcomm Innovation Fellowship
07/2011: I attended the University of Zurich Summer School on Interaction, Visualization and Ubiquitous Computing
07/2011: I attended the Networked Embedded Systems Summer School in Bertinoro, Italy

I was a member of the program committee of ETRA'19

I was a reviewer of the following conferences:
Ubicomp'13, MUM'13, UIST'14, ISMAR'15, Ubicomp'15, ISWC'15, CHI'16, UIST'16, MobileHCI'16, Ubicomp'16, Ubicomp'17, ISMAR'17, ETRA'18, ISMAR'18, CHI'19, EPIC'19, ETRA'19, ISMAR'19, ISWC'19, ISS'19, SUI'19, VRST'19

I am or I was a reviewer of the following journals:
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Multimedia
IEEE Transactions on Computational Imaging
Elsevier Signal Processing: Image Communication
Elsevier Image and Vision Computing
Hindawi Advances in Human-Computer Interaction
SPIE Journal of Electronic Imaging
ACM Interactive, Multimedia, Wearable and Ubiquitous Technologies (IMWUT)

Teaching

FS 2017: Informatics II for Electrical Engineers

FS 2016: Informatics II for Electrical Engineers

FS 2015: Informatics II for Electrical Engineers, Ubiquitous Computing and Ubiquitous Computing Seminar

FS 2014: Informatics II for Electrical Engineers, Seminar 'Smart Environments'

FS 2013: Informatics II for Electrical Engineers, Ubiquitous Computing and Seminar 'Interaction in Intelligent Environments'

HS 2012: Distributed Systems

FS 2012: Informatics II for Electrical Engineers and Ubiquitous Computing

HS 2011: Distributed Systems

FS 2011: Informatics II for Electrical Engineers

Student projects

I am always looking for talented students with background in both distributed systems and visual computing. Electrical engineers with experience in signal and image processing are also welcome.

Available Projects
Type	Title		Supervisor	Semester
M	Machine Learning on Mobile GPU		Gábor Sörös	available
M	Mobile and Wearable Augmented Reality		Gábor Sörös	available
Assigned Projects
Type	Title	Student	Supervisor	Semester
L	Inside-out Acoustic Head Tracking Using Imaginary Anchor Sources	Felix Pfreundtner	Jing Yang, Gábor Sörös	FS 19
Completed Projects
Type	Title	Student	Supervisor	Semester
M	Spatial Audio for Human-Object Interactions	Yves Frank	Jing Yang, Gábor Sörös	FS 18
M	User-centric rendering for handheld augmented reality	Shiheng Wang	Jing Yang, Gábor Sörös	HS 17
L	Gaze Estimation with Convolutional Neural Networks	Andrey Ihnatov	Mihai Bâce, Gábor Sörös	HS 16
L	Facial Augmented Reality	Andreas Hess	Gábor Sörös	HS 16
B	Temporal Context in Human Pose Estimation	Susanne Keller	Jie Song, Gábor Sörös	HS 16
B	Visible Light Communication for Smartglasses	Pascal Josephy	Gábor Sörös, Stefan Schmid	FS 16
B	Gesture Recognition on Wearable Devices	Sander Staal	Mihai Bâce, Gábor Sörös	FS 16
B	Solfege Sign Recognition with a Wearable Camera	Julia Giger	Gábor Sörös	FS 16
M	Online Gyroscope-Camera Autocalibration for Image Enhancement on Smartphones	Luc Humair	Gábor Sörös	FS 15
L	Image Processing on Mobile GPU	Carlo Beltrame	Gábor Sörös	FS 15
B	Wearable Shopping Assistant	Mauro Guerini	Gábor Sörös	FS 15
B	Image Processing on Mobile GPU	Marc Fischer	Gábor Sörös	FS 15
M	Blur Removal from Document Images	Stephan Semmler	Gábor Sörös	HS 14
B	Developing a Multi-Level Visual Code	David Chettrit	Gábor Sörös, Leyna Sadamori	FS 14
B	Rapid Object Reconstruction for Product Augmented Reality	Sandro Lombardi	Gábor Sörös	FS 14
B	Multiframe Blur Estimation using Inertial Measurement Sensors	Severin Münger	Gábor Sörös	FS 14
B	Smart Playing Cards with Smart Glasses	Michael Franz	Gábor Sörös	FS 14
B	A Biking Assistant on a Wearable Computer	Andy Zimmermann	Gábor Sörös	HS 13
B	Motion Blur Compensation using Inertial Measurement Sensors	Carlo Beltrame	Gábor Sörös	FS 13
B	A Magic Lens for Visualizing Device Interactions	Markus Schalch	Simon Mayer, Gábor Sörös, Marian George	FS 13
B	Optical Character Recognition with a Head-Mounted Camera	Thomas Knell	Gábor Sörös	FS 13
M	Optical Character Recognition on Mobile Phones (in cooperation with MIT)	Ekansh Anand	Christian Floerkemeier, Gábor Sörös	FS 12
L	Automatic Topology Estimation in Wireless Lamp Networks	Moritz Hoffmann, Claudio Gargiulo	Gábor Sörös	FS 12
L	Uncovering Device Whispers	Bram Scheidegger, Claude Barthels	Simon Mayer, Christian Beckel, Gábor Sörös	FS 12
B	Optical Character Recognition on Mobile Phones	Luc Humair	Christian Floerkemeier, Gábor Sörös	FS 12

Publications

• Vincent Becker, Sandro Kalbermatter, Simon Mayer, Gábor Sörös
  Tailored Controls: Creating Personalized Tangible User Interfaces from Paper.
  Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces. ISS '19, ACM, ISBN 978-1-4503-6891-9/19/11, Daejeon, South Korea, November 2019
  doi: 10.1145/3343055.3359700
  Abstract, BibTeX, Paper (.pdf)
  
  
• Valentin Vasiliu, Gábor Sörös
  Coherent rendering of virtual smile previews with fast neural style transfer.
  Proceedings of IEEE Symposium on Mixed and Augmented Reality (ISMAR 2019), Beijing, China. IEEE, October 2019
  Abstract, BibTeX
  
  
• Vincent Becker, Linus Fessler, Gábor Sörös
  GestEar: Combining Audio and Motion Sensing for Gesture Recognition on Smartwatches.
  Proceedings of the 2019 International Symposium on Wearable Computers. ISWC '19, ACM, ISBN 978-1-4503-6870-4, London, UK, September 2019
  doi: 10.1145/3341163.3347735
  Abstract, BibTeX, Paper (.pdf)
  
  
• Vincent Becker, Felix Rauchenstein, Gábor Sörös
  Connecting and Controlling Appliances through Wearable Augmented Reality.
  Augmented Human Research, Springer, August 2019
  doi: 10.1007/s41133-019-0019-0
  Abstract, BibTeX
  
  
• Jing Yang, Yves Frank, Gábor Sörös
  Hearing Is Believing: Synthesizing Spatial Audio from Everyday Objects to Users.
  Proceedings of the 10th Augmented Human International Conference. AH'19, ACM, Reims, France, March 2019
  Abstract, BibTeX, Paper (.pdf)
  
  
• Vincent Becker, Felix Rauchenstein, Gábor Sörös
  Investigating Universal Appliance Control through Wearable Augmented Reality.
  Proceedings of the 10th Augmented Human International Conference. AH '19, ISBN 978-1-4503-6547-5, 2019
  doi: 10.1145/3311823.3311853
  Abstract, BibTeX, Paper (.pdf)
  
  
• Vincent Becker, Pietro Oldrati, Liliana Barrios, Gábor Sörös
  TouchSense: Classifying Finger Touches and Measuring their Force with an Electromyography Armband.
  Proceedings of the 2018 International Symposium on Wearable Computers. ISWC '18, ACM, ISBN 978-1-4503-4835-5, Singapore, October 2018
  doi: 10.1145/3267242.3267250
  Abstract, BibTeX, Paper (.pdf)
  For the code, go to https://github.com/vincentbecker/TouchSense
  
  
• Jing Yang, Shiheng Wang, Gábor Sörös
  User-Perspective Rendering for Handheld Applications.
  2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, Munich, Germany, October 2018
  Abstract, BibTeX, Paper (.pdf)
  
  
• Jing Yang, Gábor Sörös
  Poster: Spatial Audio for Human-Object Interactions in Small AR Workspaces.
  Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys). ACM, Munich, Germany, June 2018
  Abstract, BibTeX, Paper (.pdf)
  
  
• Mihai Bâce, Sander Staal, Gábor Sörös
  Wearable Eye Tracker Calibration at Your Fingertips.
  ACM Symposium on Eye Tracking Research & Applications. ETRA '18, ACM, Warsaw, Poland, June 2018
  Abstract, BibTeX, Paper (.pdf)
  
  
• Jing Yang, Gábor Sörös
  Augmenting Smart Object Interactions with Smart Audio.
  Proceedings of the 9th Augmented Human International Conference. No. 28, ACM, pp. 28:1 - 28:3, Seoul, Republic of Korea, February 2018
  Abstract, BibTeX, Paper (.pdf)
  
  
• Vincent Becker, Pietro Oldrati, Liliana Barrios, Gábor Sörös
  TouchSense: Classifying and Measuring the Force of Finger Touches with an Electromyography Armband.
  Proceedings of the 9th Augmented Human International Conference. AH '18, ISBN 978-1-4503-5415-8, pp. 34:1–34:3, 2018
  doi: 10.1145/3174910.3174947
  Abstract, BibTeX, Paper (.pdf)
  
  
• Mihai Bâce, Philippe Schlattner, Vincent Becker, Gábor Sörös
  Facilitating Object Detection and Recognition through Eye Gaze.
  Proceedings of the Workshop on Object Recognition for Input and Mobile Interaction at the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017). ACM, Vienna, Austria, September 2017
  Abstract, BibTeX, Paper (.pdf)
  
  
• Vincent Becker, Mihai Bâce, Gábor Sörös
  Wearable machine learning for recognizing and controlling smart devices.
  Proceedings of the Workshop on Object Recognition for Input and Mobile Interaction at the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017). ACM, pp. 1–3, Vienna, Austria, September 2017
  Abstract, BibTeX, Paper (.pdf)
  
  
• Mihai Bâce, Sander Staal, Gábor Sörös, Giorgio Corbellini
  Collocated Multi-user Gestural Interactions with Unmodified Wearable Devices.
  Augmented Human Research, Vol. 2, August 2017
  Abstract, BibTeX, Paper (.pdf)
  Other files: Video
  
  
• Mihai Bâce, Gábor Sörös, Sander Staal, Giorgio Corbellini
  HandshakAR: Wearable Augmented Reality System for Effortless Information Sharing.
  Proceedings of the Augmented Human 2017 Conference (AH 2017). ACM, Mountain View, CA, USA, March 2017
  Abstract, BibTeX, Paper (.pdf)
  Other files: Video
  
  
• Gábor Sörös, Marcel Lancelle, Nicolas Degen, Roland Mörzinger
  Demo: Mobile Augmented Reality for Dentistry: Virtual Try-On in Live 3D.
  Live demo and poster at the European Conference on Computer Vision (ECCV 2016), Amsterdam, the Netherlands. October 2016
  Abstract, BibTeX
  
  
• Gábor Sörös, Julia Giger, Jie Song
  Demo: Solfège Hand Sign Recognition with Smart Glasses.
  Live demo and poster at the First International Workshop on Egocentric Perception, Interaction, and Computing (EPIC 2016), in conjuction with the European Conference on Computer Vision (ECCV 2016), Amsterdam, the Netherlands. October 2016
  Abstract, BibTeX, Paper (.pdf)
  Other files: Poster, Video, Video
  
  
• Frederik Hermans, Liam McNamara, Gábor Sörös, Christian Rohner, Thiemo Voigt, Edith Ngai
  Focus: Robust Visual Codes for Everyone.
  Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys 2016), Singapore, Singapore. pp. 319-332, June 2016
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk
  
  
• Gábor Sörös
  Wearable Barcode Scanning: Advancements in Visual Code Localization, Motion Blur Compensation, and Gesture Control.
  PhD thesis No. 23464, ETH Zurich, Zurich, Switzerland, 2016
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk, Supplement
  
  
• Marian George, Dejan Mircic, Gábor Sörös, Christian Floerkemeier, Friedemann Mattern
  Fine-Grained Product Class Recognition for Assisted Shopping.
  Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops 2015), Santiago, Chile. IEEE, pp. 154–162, December 2015
  Abstract, BibTeX, Paper (.pdf)
  
  
• Gábor Sörös, Stephan Semmler, Luc Humair, Otmar Hilliges
  Fast Blur Removal for Wearable QR Code Scanners.
  Proceedings of the 19th International Symposium on Wearable Computers (ISWC 2015), Osaka, Japan. September 2015
  Abstract, BibTeX, Paper (.pdf)
  Other files: Supplement, Talk, Video
  
  
• Gábor Sörös, Severin Münger, Carlo Beltrame, Luc Humair
  Multiframe Visual-Inertial Blur Estimation and Removal for Unmodified Smartphones.
  Proceedings of the 23rd International Conference on Computer Graphics, Visualization and Computer Vision (WSCG 2015), Plzen, Czech Republic. June 2015
  Abstract, BibTeX, Paper (.pdf)
  Other files: Supplement, Talk
  
  
• Jie Song, Gábor Sörös, Fabrizio Pece, Otmar Hilliges
  Demo: Estimation of 3D Hand Position and Gestures on Unmodified Wearable Devices.
  Live demo and poster at the IEEE Workshop on Observing and Understanding Hands in Action (HANDS 2015), in conjunction with the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, MA, USA. IEEE, June 2015
  Abstract, BibTeX, Paper (.pdf)
  Other files: Poster
  
  
• Jie Song, Fabrizio Pece, Gábor Sörös, Marion Koelle, Otmar Hilliges
  Joint Estimation of 3D Hand Position and Gestures from Monocular Video for Mobile Interaction.
  Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), Seoul, South Korea. ACM, April 2015
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk, Video
  
  
• Simon Mayer, Yassin N. Hassan, Gábor Sörös
  Demo: Magic Lenses for Revealing Device Interactions in Smart Environments.
  Live demo at the SIGGRAPH Asia 2014 Symposium on Mobile Graphics and Interactive Applications (MGIA 2014), Shenzhen, China. ACM, December 2014
  Abstract, BibTeX, Paper (.pdf)
  Other files: Video
  
  
• Simon Mayer, Yassin N. Hassan, Gábor Sörös
  A Magic Lens for Revealing Device Interactions in Smart Environments.
  Proceedings of the SIGGRAPH Asia 2014 Symposium on Mobile Graphics and Interactive Applications (MGIA 2014), Shenzhen, China. ACM, December 2014
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk, Video
  
  
• Jie Song, Gábor Sörös, Fabrizio Pece, Sean Fanello, Shahram Izadi, Cem Keskin, Otmar Hilliges
  In-air Gestures Around Unmodified Mobile Devices.
  Proceedings of the 27th ACM User Interface Software and Technology Symposium (UIST 2014), Honolulu, Hawaii. ACM, pp. 319-329, October 2014
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk, Video
  
  
• Jie Song, Gábor Sörös, Fabrizio Pece, Sean Fanello, Shahram Izadi, Cem Keskin, Otmar Hilliges
  Demo: In-air Gestures Around Unmodified Mobile Devices.
  Live demo and poster at the 13th European Conference on Computer Vision (ECCV 2014), Zurich, Switzerland. September 2014
  Abstract, BibTeX
  
  
• Simon Mayer, Gábor Sörös
  User Interface Beaming - Seamless Interaction with Smart Things using Personal Wearable Computers.
  Proceedings of the 11th International Conference on Wearable and Implantable Body Sensor Networks (BSN 2014). pp. 46-49, Zurich, Switzerland, June 2014
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk
  
  
• Gábor Sörös
  GPU-Accelerated Joint 1D and 2D Barcode Localization on Smartphones.
  Proceedings of the 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2014), Florence, Italy. IEEE, pp. 5095-5099, May 2014
  Abstract, BibTeX, Paper (.pdf)
  Other files: Poster, Video
  
  
• Gábor Sörös, Christian Floerkemeier
  Blur-Resistant Joint 1D and 2D Barcode Localization for Smartphones.
  Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia (MUM 2013), Lulea, Sweden. ACM, December 2013
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk
  
  
• Gábor Sörös, Florian Daiber, Tomer Weller
  Cyclo - A Personal Bike Coach Through the Glass.
  Proceedings of SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications (MGIA 2013), Hong Kong, Hong Kong. ACM, November 2013
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk
  
  
• Simon Mayer, Markus Schalch, Marian George, Gábor Sörös
  Poster: Device Recognition for Intuitive Interaction with the Web of Things.
  Adjunct Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2013). Zurich, Switzerland, September 2013
  Abstract, BibTeX, Paper (.pdf)
  
  
• Simon Mayer, Christian Beckel, Bram Scheidegger, Claude Barthels, Gábor Sörös
  Demo: Uncovering Device Whispers in Smart Homes.
  Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM 2012), Ulm, Germany. December 2012
  Abstract, BibTeX, Paper (.pdf)
  Other files: Poster
  
  
• Gábor Sörös, Christian Floerkemeier
  Poster: Towards Next Generation Barcode Scanning.
  Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM 2012), Ulm, Germany. December 2012
  Abstract, BibTeX, Paper (.pdf)
  Other files: Poster
  
  
• Gábor Sörös, Peter Rautek, Hartmut Seichter, Eduard Gröller
  Augmented Visualization with Natural Feature Tracking.
  Proceedings of the 10th ACM International Conference on Mobile and Ubiquitous Multimedia (MUM 2011), Beijing, China. pp. 4-12, December 2011
  Abstract, BibTeX, Paper (.pdf)
  Other files: Talk
  
  
• Péter Galambos, András Róka, Péter Korondi, Gábor Sörös
  Perifériás látáson alapuló információ-megjelenítés vizsgálata (in Hungarian)
  Proceedings of the XXIV. microCAD International Conference, Miskolc, Hungary, 2010
  
  
• Péter Galambos, András Róka, Gábor Sörös, Péter Korondi
  Visual feedback techniques for telemanipulation and system status sensualization
  Proceedings of the 8th IEEE International Symposium on Applied Machine Intelligence and Informatics (SAMI 2010), Herlany, Slovakia, 2010
  
  
• Gábor Sörös, Barna Reskó, Péter Baranyi
  Cognitive supervision system for industrial robots
  Proceedings of the 2nd IEEE International Conference on Human System Interaction (HSI 2009), Catania, Italy, 2009
  
  
• Gábor Sörös, Barna Reskó, Bjørn Solvang, Péter Baranyi
  A cognitive robot supervision system
  Proceedings of the 7th IEEE International Symposium on Applied Machine Intelligence and Informatics (SAMI 2009), Herlany, Slovakia, 2009
  
  

Theses

• Gábor Sörös
  Wearable Barcode Scanning: Advancements in Visual Code Localization, Motion Blur Compensation, and Gesture Control
  PhD Thesis, ETH Zurich, Switzerland, 2016
  Other files: Talk, Supplement
  
  
• Gábor Sörös
  Augmented Visualization
  Master Thesis, Budapest University of Technology and Economics, Hungary, 2010
  
  
• Gábor Sörös
  Cognitive Infocommunication in Automated Manufacturing Systems
  Bachelor Thesis, Budapest University of Technology and Economics, Hungary, 2008