Interaction with smart devices
Together with Pietro Oldrati, Liliana Barrios, and Gábor Sörös
Identifying the finger used for touching and measuring the force of the touch provides valuable information on manual interactions. This information can be inferred from electromyography (EMG) of the forearm, measuring the activation of the muscles controlling the hand and fingers. We present Touch-Sense, which classifies the finger touches using a novel neural network architecture and estimates their force on a smartphone in real time based on data recorded from the sensors of an inexpensive and wireless EMG armband. Using data collected from 18 participants with force ground truth, we evaluate our system's performance and limitations. Our system could allow for new interaction paradigms with appliances and objects, which we exemplarily showcase in four applications.
Investigating Universal Appliance Control through Wearable Augmented Reality
Together with Felix Rauchenstein and Gábor Sörös
The number of interconnected devices around us is constantly growing. However, it may become challenging to control all these devices when control interfaces are distributed over mechanical elements, apps, and confguration webpages. We investigate interaction methods for smart devices in augmented reality. The physical objects are augmented with interaction widgets, which are generated ondemand and represent the connected devices along with their adjustable parameters. For example, a loudspeaker can be overlaid with a controller widget for its volume. We explore three ways of manipulating the virtual widgets: (a) in-air fnger pinching and sliding, (b) whole arm gestures rotating and waving, (c) incorporating physical objects in the surrounding and mapping their movements to the interaction primitives. We compare these methods in a user study with 25 participants and find significant diﬀerences in the preference of the users, the speed of executing commands, and the granularity of the type of control.
Exploring zero-training algorithms for occupancy detection based on smart meter measurements
Together with Wilhelm Kleiminger
Detecting the occupancy in households is becoming increasingly important for enabling context-aware applications in smart homes. For example, smart heating systems, which aim at optimising the heating energy, often use the occupancy to determine when to heat the home. The occupancy schedule of a household can be inferred from the electricity consumption, as its changes indicate the presence or absence of inhabitants. As smart meters become more widespread, the real-time electricity consumption of households is often available in digital form. For such data, supervised classifiers are typically employed as occupancy detection mechanisms. However, these have to be trained on data labelled with the occupancy ground truth. Labelling occupancy data requires a high effort, sometimes it even may be impossible, making it difficult to apply these methods in real-world settings. Alternatively, one could use unsupervised classifiers, which do not require any labelled data for training. In this work, we introduce and explain several unsupervised occupancy detection algorithms. We evaluate these algorithms by applying them to three publicly available datasets with ground truth occupancy data, and compare them to one existing unsupervised classifier and several supervised classifiers. Two unsupervised algorithms perform the best and we find that the unsupervised classifiers outperform the supervised ones we compared to. Interestingly, we achieve a similar classification performance on coarse-grained aggregated datasets and their fine-grained counterparts.
Estimating the Savings Potential of Occupancy-based Heating Strategies
Together with Wilhelm Kleiminger, Vlad C. Coroamă, and Friedemann Mattern
Because space heating causes a large fraction of energy consumed in households, occupancy-based heating systems have become more and more popular in recent years. However, there is still no practical method to estimate the potential energy savings before installing such a system. While substantial work has been done on occupancy detection, previous work does not address a combination with heating simulation in order to provide an easily applicable method to estimate this savings potential. In this paper we present such a combination of an occupancy detection algorithm based on smart electricity meter data and a building heating simulation, which only requires publicly available weather data and some relevant building characteristics. We apply our method to a dataset containing such data for several thousand households and show that when taking occupancy into account, a household can save over 9% heating energy on average, while certain groups, such as employed single-person households, can even save 14% on average. Using our approach, households with high potential for energy savings can be quickly identified and their inhabitants could be more easily convinced to adopt an occupancy-based heating strategy.