Research — CGL

Our research explores how we can capture, represent, model, animate, and interact with digital humans, AI characters, and 3D scenes.

Faces

Physics-Based Simulation

We develop physics-based methods for modeling and simulating faces, combining neural simulation, simulation-ready anatomical models, and differentiable physics. These methods enable physics-grounded facial animation for applications ranging from entertainment to clinical treatment prediction.

3D Patient Models

We develop joint models of facial appearance and internal anatomy to create person-specific anatomical face models from visual input. Our methods infer complete anatomical head models from video, 3D face scans, or partial medical imaging data, providing a foundation for treatment simulation and the design of patient-specific appliances and implants.

Infant and Craniofacial Models

We learn 3D anatomical shape models of infants, including the facial surface and underlying bones, from uncontrolled and incomplete data. Our methods address the challenges of modeling the anatomy of the youngest patients, where available data is often sparse, heterogeneous, and incomplete.

Plate Computation for Cleft

We develop computational methods for designing person-specific appliances for infants with cleft lip and palate. Starting from 3D geometry and anatomical information, we translate patient-specific anatomy into computational designs for presurgical treatment.

Anatomical Body Modeling

3D Scene Reconstruction

AI Characters

Interactive Character Animation

We develop methods for intelligent and interactive digital characters that can engage with users in conversational settings. Our research spans character identity and behavior modeling, animation synthesis, and interactive systems, with the aim of creating expressive and responsive virtual characters.

AI Health Coach

We explore embodied conversational agents as an interface for personal health reflection, enabling users to discuss data from wearables and biosensors through natural dialogue. These agents make trends and statistics more accessible while supporting active sensemaking of personal health data.

Affective State Modeling

Identification of affective states enables the design of emotionally sentient systems. We have developed methods for affective state prediction based on camera recordings, low cost mobile biosensors, handwriting data, and smartphone touch and sensor data.

Speech-Driven Animation

We develop methods for generating and controlling expressive 3D character animation from speech. By modeling speech content and emotional expression, we enable synchronized facial motion together with controllable expressions, speaking styles, and character behavior.

Art-Directed Fluid Simulation

Artistically controlling the shape, motion and appearance of fluid simulations pose major challenges in visual effects production. We use differentiable simulation and physics-informed neural networks for style transfer, keyframe matching and artistic deformation.

3D Fluid Reconstruction

We develop methods for reconstructing dynamic 3D and 4D fluid phenomena from video. By combining physical constraints, differentiable rendering, and neural representations, we recover volumetric geometry, appearance, and motion from sparse visual observations.