I am a Visiting Principal Scientist at the NTU-Alibaba Joint Research Institute, Singapore. My research interests lie mainly in Computational Narrative Intelligence and related computer vision and NLP problems.

Story is a powerful tool for communication, an exhibit of creativity, and a timeless form of entertainment. Computational Narrative Intelligence (CNI) aims to create intelligent machines that can understand and create stories, manage interactive narratives, and respond appropriately to stories told to them. I have made contributions to all major areas of CNI, ranging from story generation and interactive narratives to human cognition, from learning story knowledge to story understanding. [My Google Scholar Page]

In order to understand the complex artifact that we call a story, it is necessary to jointly employ multiple AI capabilities to construct meaning from the ground up. To this end, I also spend time on multimodal reasoning and the machinery of deep neural networks.

From 2017 to 2019, I was a Senior Research Scientist at Baidu Research USA. From 2015 to 2017, I was a Research Scientist at Disney Research, leading a group of postdocs and interns. Prior to that, I worked as a postdoc with Leon Sigal and Jill Lehman. In 2014, I obtained my Ph.D. degree from Georgia Institute of Technology, working with Mark Riedl.

What's New
Dec 2019: New preprint that applies the theory of Graphon to optimize the organization of CNN layers: Searching for Stage-wise Neural Graphs In the Limit
Dec 2019: New preprint on the relationship between adversarial attacks and interpretability of neural networks: Does Interpretability of Neural Networks Imply Adversarial Robustness?
Sep 2019: New preprint on the topic of interpreting deep neural networks: NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
May 2019: Our paper on emotion recognition and attribution from video was accepted by the IEEE Transactions on Multimedia.
Nov 2018: A paper on dense video captioning was accepted at WACV 2019.
Nov 2018: Learning Gaussian embeddings for actors and persona accepted at AAAI 2019 (~16.2% acceptance rate).
Feb 2018: Released the LREC 2018 paper, the annotation dataset, the presentation, and the annotation guidelines (v1.0).
Feb 2018: Spotlight presentation (~6.8% acceptance rate) at CVPR 2018.

moc.kooltuo@ilgnayob :liamE

Research Areas

Multimodal Reasoning

Story Understanding

Acquiring Story Knowledge

Story Generation