Arun Mallya

I was previously a Senior Research Scientist in the Deep Imagination Research (DIR) group at NVIDIA. I was a part of this group from its inception, when it had just 3 members!

I graduated with a PhD from the University of Illinois at Urbana-Champaign, where I explored computer vision under the guidance of Prof. Svetlana Lazebnik.
I previously completed my Master's in CS at UIUC, and my Bachelor's in CSE at IIT KGP.

My research focus is on generative content creation with neural networks.

Github / LinkedIn / Scholar / webpage



A M
Selected Research    (See Google Scholar for all publications)

SPACE: Speech-driven Portrait Animation with Controllable Expression
Siddharth Gururani, Arun Mallya, Ting-Chun Wang, Rafael Valle, Ming-Yu Liu
International Conference on Computer Vision (ICCV), 2023
[arxiv preprint] [project webpage]

Implicit Warping for Animation with Image Sets
Arun Mallya, Ting-Chun Wang, Ming-Yu Liu
Neural Information Processing Systems (NeurIPS), 2022
[arxiv preprint] [project webpage]

One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing (oral)
Ting-Chun Wang, Arun Mallya, Ming-Yu Liu
Computer Vision and Pattern Recognition (CVPR), 2021
[arxiv preprint] [project webpage]

Implicit Neural Representations with Levels-of-Experts
Zekun Hao, Arun Mallya, Serge Belongie, Ming-Yu Liu
Neural Information Processing Systems (NeurIPS), 2022
[paper]

GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds (oral)
Zekun Hao, Arun Mallya, Serge Belongie, Ming-Yu Liu
International Conference on Computer Vision (ICCV), 2021
[arxiv preprint] [project webpage and code]

See through Gradients: Image Batch Recovery via GradInversion
Hongxu Yin, Arun Mallya, Arash Vahdat, Jose Alvarez, Pavlo Molchanov, Jan Kautz
Computer Vision and Pattern Recognition (CVPR), 2021
[paper]

Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion (oral)
Hongxu Yin, Pavlo Molchanov, Zhizhong Li, Jose Alvarez, Arun Mallya, Derek Hoiem, Niraj Jha, Jan Kautz
Computer Vision and Pattern Recognition (CVPR), 2020
[arxiv preprint] [code]

Piggyback: Adding Multiple Tasks to a Single, Fixed Network by Learning to Mask
Arun Mallya, Dillon Davis, Svetlana Lazebnik
European Conference on Computer Vision (ECCV), 2018
[arxiv preprint] [code]

PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Arun Mallya, Svetlana Lazebnik
Computer Vision and Pattern Recognition (CVPR), 2018
[arxiv preprint] [code]

Tutorials/Workshops
  1. Machine Learning with Synthetic Data, CVPR 2022
  2. Accelerating Computer Vision with Mixed Precision, ECCV 2020
  3. Accelerating Computer Vision with Mixed Precision, ICCV 2019
Writeups/Notes
Hosted on github. Edit requests/additions/corrections are welcome.
  1. A Backpropagation Refresher
  2. An Illustrated Explanation of the LSTM Forward-Backward Pass
  3. Introduction to RNNs
  4. Introduction to RNNs - II
  5. Jupyter notebook to find Receptive Field Size and Effective Stride (supports dilated convs)
  6. Visualization of neuron connections and receptive field of a CNN (including dilation)!

(imitation is the sincerest form of flattery)