Experience
I am currently a Principal Research Scientist at Nvidia, mainly working on machine learning-based 3D content creation.
I was a Senior Research Scientist in Google AR/VR. I was also the tech lead for machine learning in ARCore [Video], where I worked on enabling AR on smartphones. I built the infrastructure for cross-dataset training and model distillation and compression. I also worked on efficient models for depth estimation from passive and active stereo, from motion, and from a single image, and I worked on human pose estimation, neural rendering, and image matting.
I was a Senior Scientist and a Founding Team Member at perceptiveIO. I built a state-of-the-art 360 performance capture system for real-time reconstruction of arbitrary non-rigid scenes. We were acquired by Google.
I was a Researcher in the Interactive 3D Technologies (I3D) group at Microsoft Research, where I worked on Holoportation [Video]. This work was covered by ZDNet, Gizmodo, VentureBeat, Mashable, Engadget, and Wired, among others. This technology was running live on stage at TED 2016. I also worked on hand tracking from depth cameras.
I received a PhD in computer science from the University of Maryland. I worked with Larry Davis at the Institute for Advanced Computer Studies (UMIACS). My thesis was on multi-person activity recognition from video sequences. I also interned at Siemens, IST Austria, MSR Cambridge, and SRI International.
I received a MSc in computer science from the University of Western Ontario, where I worked with Yuri Boykov. My thesis was on semantic image segmentation. I also wrote a parallel out-of-core push-relabel algorithm.
Publications
-
Eric Ryan Chan*, Connor Zhizhen Lin*, Matthew Aaron Chan*, Koki Nagano*, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein.
Efficient Geometry-aware 3D Generative Adversarial Networks.
arXiv preprint, 2021.
[arXiv] [Project Page] -
Matan Atzmon, Koki Nagano, Sanja Fidler, Sameh Khamis, Yaron Lipman.
Frame Averaging for Equivariant Shape Space Learning.
arXiv preprint, 2021.
[arXiv] [Project Page] -
Sourav Biswas, Kangxue Yin, Maria Shugrina, Sanja Fidler, Sameh Khamis.
Hierarchical Neural Implicit Pose Network for Animation and Motion Retargeting.
arXiv preprint, 2021.
[arXiv] [Project Page] -
Francis Williams*, Zan Gojcic*, Sameh Khamis, Denis Zorin, Joan Bruna, Sanja Fidler, Or Litany.
Neural Fields as Learnable Kernels for 3D Reconstruction.
arXiv preprint, 2021.
[arXiv] [Project Page] -
Wenzheng Chen, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khamis, Or Litany, Sanja Fidler.
DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer.
In Neural Information Processing Systems (NeurIPS), 2021.
[arXiv] [Project Page] -
Kangxue Yin, Jun Gao, Maria Shugrina, Sameh Khamis, Sanja Fidler.
3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations.
In International Conference on Computer Vision (ICCV), 2021 (Oral).
[arXiv] [Project Page] -
Ming-Yu Liu, Koki Nagano, Yeongho Seol, Rafael Valle, Jaewoo Seo, Ting-Chun Wang, Arun Mallya, Sameh Khamis, Wei Ping, Rohan Badlani, Kevin J. Shih, Bryan Catanzaro, Simon Yuen, Jan Kautz.
I am AI: AI-driven Digital Avatar Made Easy.
ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, 2021, Real-Time Live (Best-in-Show Winner).
[Link] [Video] -
Hossam Isack, Christian Haene, Cem Keskin, Sofien Bouaziz, Yuri Boykov, Shahram Izadi, Sameh Khamis.
RePose: Learning Deep Kinematic Priors for Fast Human Pose Estimation.
arXiv preprint, 2020.
[arXiv] -
Moustafa Meshry, Dan B. Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, Ricardo Martin-Brualla.
Neural Rerendering in the Wild.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 (Oral, Best Paper Award Finalist).
[arXiv] [Video] [Code] [Project Page] -
Julien Valentin, Adarsh Kowdle, Jonathan T. Barron, Neal Wadhwa, Max Dzitsiuk, Michael Schoenberg, Vivek Verma, Ambrus Csaszar, Eric Turner, Ivan Dryanovski, Joao Afonso, Jose Pascoal, Konstantine Tsotsos Mira Leung, Mirko Schmidt, Onur Guleryuz, Sameh Khamis, Vladimir Tankovich, Sean Fanello, Shahram Izadi, Christoph Rhemann.
Depth from Motion for Smartphone AR.
In ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques in Asia, 2018.
[PDF] [Bibtex]@inproceedings{valentin-siggraph-asia2018, author = {Julien Valentin and Adarsh Kowdle and Jonathan T Barron and Neal Wadhwa and Max Dzitsiuk and Michael Schoenberg and Vivek Verma and Ambrus Csaszar and Eric Turner and Ivan Dryanovski and Joao Afonso and Jose Pascoal and Konstantine Tsotsos and Mira Leung and Mirko Schmidt and Onur Guleryuz and Sameh Khamis and Vladimir Tankovitch and Sean Fanello and Shahram Izadi and Christoph Rhemann}, title = {Depth from Motion for Smartphone AR}, booktitle = {ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques in Asia}, year = {2018} }
-
Adarsh Kowdle, Christoph Rhemann, Sean Fanello, Andrea Tagliasacchi, Jonathan Taylor, Philip Davidson, Mingsong Dou, Kaiwen Guo, Cem Keskin, Sameh Khamis, David Kim, Danhang Tang, Vladimir Tankovich, Julien Valentin, Shahram Izadi.
The Need 4 Speed in Real-Time Dense Visual Tracking.
In ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques in Asia, 2018.
[PDF] [Bibtex]@inproceedings{kowdle-siggraph-asia2018, author = {Adarsh Kowdle and Christoph Rhemann and Sean Fanello and Andrea Tagliasacchi and Jonathan Taylor and Philip Davidson and Mingsong Dou and Kaiwen Guo and Cem Keskin and Sameh Khamis and David Kim and Danhang Tang and Vladimir Tankovich and Julien Valentin and Shahram Izadi}, title = {The Need 4 Speed in Real-Time Dense Visual Tracking}, booktitle = {ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques in Asia}, year = {2018} }
-
Ricardo Martin-Brualla, Rohit Pandey, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Julien Valentin, Sameh Khamis, Philip Davidson, Anastasia Tkach, Peter Lincoln, Adarsh Kowdle, Christoph Rhemann, Dan B. Goldman, Cem Keskin, Steve Seitz, Shahram Izadi, Sean Fanello.
LookinGood: Enhancing Performance Capture with Real-time Neural Re-Rendering.
In ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques in Asia, 2018.
[arXiv] [Video] -
Yinda Zhang, Sameh Khamis, Christoph Rhemann, Julien Valentin, Adarsh Kowdle, Vladimir Tankovich, Michael Schoenberg, Shahram Izadi, Thomas Funkhouser, Sean Fanello.
ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems.
In European Conference on Computer Vision (ECCV), 2018.
[arXiv] -
Sameh Khamis, Sean Fanello, Christoph Rhemann, Adarsh Kowdle, Julien Valentin, Shahram Izadi.
StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction.
In European Conference on Computer Vision (ECCV), 2018.
[arXiv] -
Mingsong Dou, Philip Davidson, Sean Fanello, Sameh Khamis, Adarsh Kowdle, Christoph Rhemann, Vladimir Tankovich, Shahram Izadi.
Motion2Fusion: Real-time Volumetric Performance Capture.
In ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, 2017.
[PDF] [Bibtex]@inproceedings{dou-siggraph2017, author = {Dou, Mingsong and Davidson, Philip and Fanello, Sean Ryan and Khamis, Sameh and Kowdle, Adarsh and Rhemann, Christoph and Tankovich, Vladimir and Izadi, Shahram}, title = {Motion2fusion: real-time volumetric performance capture}, booktitle = {ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques}, year = {2017} }
-
Sergio Orts Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, Shahram Izadi.
Holoportation: Virtual 3D Teleportation in Real-time.
In ACM User Interface Software and Technology Symposium (UIST), 2016.
[PDF] [Bibtex] [Video]@inproceedings{orts-escolano-uist2016, author = {Sergio Orts-Escolano and Christoph Rhemann and Sean Fanello and Wayne Chang and Adarsh Kowdle and Yury Degtyarev and David Kim and Philip Davidson and Sameh Khamis and Mingsong Dou and Vladimir Tankovich and Charles Loop and Qin Cai and Philip Chou and Sarah Mennicken and Julien Valentin and Vivek Pradeep and Shenlong Wang and Sing Bing Kang and Pushmeet Kohli and Yuliya Lutchyn and Cem Keskin and Shahram Izadi}, title = {Holoportation: Virtual 3D Teleportation in Real-time}, booktitle = {ACM User Interface Software and Technology Symposium}, year = {2016} }
-
Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, Pushmeet Kohli, Vladimir Tankovich, Shahram Izadi.
Fusion4D: Real-time Performance Capture of Challenging Scenes.
In ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, 2016.
[PDF] [Bibtex] [Video]@inproceedings{dou-siggraph2016, author = {Mingsong Dou and Sameh Khamis and Yury Degtyarev and Philip Davidson and Sean Fanello and Adarsh Kowdle and Sergio Orts Escolano and Christoph Rhemann and David Kim and Jonathan Taylor and Pushmeet Kohli and Vladimir Tankovich and Shahram Izadi}, title = {Fusion4D: Real-time Performance Capture of Challenging Scenes}, booktitle = {ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques}, year = {2016} }
-
Jonathan Taylor, Lucas Bordeaux, Thomas Cashman, Bob Corish, Cem Keskin, Toby Sharp, Eduardo Soto, David Sweeney, Julien Valentin, Benjamin Luff, Arran Topalian, Erroll Wood, Sameh Khamis, Pushmeet Kohli, Shahram Izadi, Richard Banks, Andrew Fitzgibbon, Jamie Shotton.
Efficient and Precise Interactive Hand Tracking through Joint, Continuous Optimization of Pose and Correspondences.
In ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, 2016.
[PDF] [Bibtex] [Video]@inproceedings{taylor-siggraph2016, author = {Jonathan Taylor and Lucas Bordeaux and Thomas Cashman and Bob Corish and Cem Keskin and Eduardo Soto and David Sweeney and Julien Valentin and Benjamin Luff and Arran Topalian and Erroll Wood and Sameh Khamis and Pushmeet Kohli and Toby Sharp and Shahram Izadi and Richard Banks and Andrew Fitzgibbon and Jamie Shotton}, title = {Efficient and Precise Interactive Hand Tracking through Joint, Continuous Optimization of Pose and Correspondences}, booktitle = {ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques}, year = {2016} }
-
David Joseph Tan, Thomas Cashman, Jonathan Taylor, Andrew Fitzgibbon, Daniel Tarlow, Sameh Khamis, Shahram Izadi, Jamie Shotton.
Fits Like a Glove: Rapid and Reliable Hand Shape Personalization.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 (spotlight).
[PDF] [Bibtex]@inproceedings{tan-cvpr2016, author = {David Joseph Tan and Thomas Cashman and Jonathan Taylor and Andrew Fitzgibbon and Daniel Tarlow and Sameh Khamis and Shahram Izadi and Jamie Shotton}, title = {Fits Like a Glove: Rapid and Reliable Hand Shape Personalization}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, year = {2016} }
-
Sameh Khamis, Jonathan Taylor, Jamie Shotton, Cem Keskin, Shahram Izadi, Andrew Fitzgibbon.
Learning an Efficient Model of Hand Shape Variation from Depth Images.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[PDF] [Bibtex] [Supp. PDF] [Video]@inproceedings{khamis-cvpr2015, author = {Sameh Khamis and Jonathan Taylor and Jamie Shotton and Cem Keskin and Shahram Izadi and Andrew Fitzgibbon}, title = {Learning an Efficient Model of Hand Shape Variation from Depth Images}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, year = {2015} }
-
Sameh Khamis, Larry S. Davis.
Walking and Talking: A Bilinear Approach to Multi-Label Action Recognition.
In CVPR Workshop on Group And Crowd Behavior Analysis And Understanding, 2015.
[PDF] [Bibtex]@inproceedings{khamis-cvprw2015, author = {Sameh Khamis and Larry S. Davis}, title = {Walking and Talking: A Bilinear Approach to Multi-Label Action Recognition}, booktitle = {CVPR Workshop on Group And Crowd Behavior Analysis And Understanding}, year = {2015} }
-
Sameh Khamis, Christoph H. Lampert.
CoConut: Co-Classification with Output Space Regularization.
In British Machine Vision Conference (BMVC), 2014.
[PDF] [Bibtex] [Poster]@inproceedings{khamis-bmvc2014, author = {Sameh Khamis and Christoph H. Lampert}, title = {CoConut: Co-Classification with Output Space Regularization}, booktitle = {British Machine Vision Conference}, year = {2014} }
-
Sameh Khamis, Cheng-Hao Kuo, Vivek K. Singh, Vinay Shet, Larry S. Davis.
Joint Learning for Attribute-Consistent Person Re-Identification.
In ECCV Workshop on Visual Surveillance and Re-Identification, 2014.
[PDF] [Bibtex]@inproceedings{khamis-eccvw2014, author = {Sameh Khamis and Cheng-Hao Kuo and Vivek K. Singh and Vinay Shet and Larry S. Davis}, title = {Joint Learning for Attribute-Consistent Person Re-Identification}, booktitle = {ECCV Workshop on Visual Surveillance and Re-Identification}, year = {2014} }
-
Cheng-Hao Kuo, Sameh Khamis, Vinay Shet.
Person Re-identification using Semantic Color Names and RankBoost.
In IEEE Workshop on the Applications of Computer Vision (WACV), 2013.
[PDF] [Bibtex]@inproceedings{kuo-wacv2013, author = {Cheng-Hao Kuo and Sameh Khamis and Vinay Shet}, title = {Person Re-identification using Semantic Color Names and RankBoost}, booktitle = {IEEE Workshop on the Applications of Computer Vision}, year = {2013} }
-
Ben London, Sameh Khamis, Stephen H. Bach, Bert Huang, Lise Getoor, Larry S. Davis.
Collective Activity Detection using Hinge-loss Markov Random Fields.
In CVPR Workshop on Structured Prediction: Tractability, Learning and Inference, 2013.
[PDF] [Bibtex] [Slides]@inproceedings{london-cvprw2013, author = {Ben London and Sameh Khamis and Stephen H. Bach and Bert Huang and Lise Getoor and Larry S. Davis}, title = {Collective Activity Detection using Hinge-loss Markov Random Fields}, booktitle = {CVPR Workshop on Structured Prediction: Tractability, Learning and Inference}, year = {2013} }
-
Sameh Khamis, Vlad I. Morariu, Larry S. Davis.
Combining Per-Frame and Per-Track Cues for Multi-Person Action Recognition.
In European Conference on Computer Vision (ECCV), 2012.
[PDF] [Bibtex] [Poster] [Errata]@inproceedings{khamis-eccv2012, author = {Sameh Khamis and Vlad I. Morariu and Larry S. Davis}, title = {Combining Per-Frame and Per-Track Cues for Multi-Person Action Recognition}, booktitle = {European Conference on Computer Vision}, year = {2012} }
In the published version, equation (3) should use the distance directly, and not its pseudo-probability: v_{ab}(i, j) = \lambda_d\ d(i, j) - \lambda_c \log(p_{ab})
-
Sameh Khamis, Vlad I. Morariu, Larry S. Davis.
A Flow Model for Joint Action Recognition and Identity Maintenance.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
[PDF] [Bibtex] [Poster] [Spotlight] [Errata]@inproceedings{khamis-cvpr2012, author = {Sameh Khamis and Vlad I. Morariu and Larry S. Davis}, title = {A Flow Model for Joint Action Recognition and Identity Maintenance}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, year = {2012} }
In the published version, equation (3) should use the distance directly, and not its pseudo-probability: v_{ab}(i, j) = \lambda_d\ d(i, j) - \lambda_c \log(p_{ab})