Yuki M. Asano

Computer Vision | Machine Learning | Complex Systems

I'm an assistant professor for computer vision and machine learning and Science Manager of QUVA lab at the University of Amsterdam, where I work with Cees Snoek, Max Welling and Efstratios Gavves. My PhD was at the Visual Geometry Group (VGG) at the University of Oxford where I worked with Andrea Vedaldi and Christian Rupprecht. Prior to this I studied physics at the University of Munich (LMU) and Economics in Hagen as well as a MSc in Mathematical Modelling and Scientific Computing at the Mathematical Institute in Oxford. Also, I love running, the mountains ⛰️ and their combination.

Email  /  Google Scholar  /  Github /  Twitter /  LinkedIn /  CV

• Two papers accepted to ICLR'23!
• Welcome to my new MSc thesis students: Lukas, Luc, Apostolos and Alfonso!
• I'm honored to serve as an Area Chair for CVPR'23!
• If you are a MSc in AI student at the UvA and want to write your thesis with me, please contact me. I have some exciting projects.
• Two papers accepted at ECCV'22!
• One paper accepted at ICML'22!
• ▼More..
  • I'm honored to serve as an Area Chair for ECCV'22!
  • Two papers accepted at CVPR'22!
  • One paper accepted at ICLR'22!
  • Qualcomm UvA Deep Vision public seminar talk in Dec. 2021. link
  • New preprint on single-image learning.
  • Starting as an Assistant Professor at the UvA from Oct 2021.
  • Two papers accepted at NeurIPS'21 (including the first as supervisor)
  • One paper accepted at NeurIPS'21-Datasets Track: the PASS dataset, incl. pretrained models.
  • Passed my PhD with "no corrections", my examiners were Phillip Isola and Philip Torr.
  • Two papers accepted to ICCV'21! (GDT and STiCA)
  • One paper accepted to PNAS, my Erdös Number is now 3 via Jobst Heitzig.
  • One paper I supervised accepted to ACL'21's Workshop on online abuse and harms. More details to follow.
  • New preprint: using clustering & contrastive SSL we find objects without any supervision
  • OxAI team I supervised has published its results at ICLR'21 SDG workshop
  • Our new preprint on intersectional occupational biases of GPT-2 is out
  • Our paper on video-text representation learning got accepted as a Spotlight into ICLR 2021!
  • I've started volunteering my time at OxAI to help interdisciplinary teams work on AI projects.
  • Our paper on Self-Labelling Videos (SeLaVi) was accepted as a paper to NeurIPS! Code
  • Starting my summer internship June 22nd at FAIR and working with Armand Joulin and Ishan Misra.
  • I am Co-PI on a Amazon Machine Learning Award project with Christian Rupprecht and Andrea Vedaldi.
  • Awarded with the 2020 Qualcomm Innovation Fellowship.
  • I'll be co-organizing a workshop at ECCV 2020: Self-Supervised Learning: What Is Next?
  • Two papers have been accepted into ICLR 2020 (incl. one Spotlight)

I'm teaching the Deep Learning Course for the MSc in AI at the University of Amsterdam.

I'm interested in computer vision, self-supervised and multi-modal learning as well as causality and privacy in AI.

We propose to use self-supervision to provide diffusion models a guidance signal, this works better than label guidance.

We show that it is possible to extrpolate to semantic classes such as those of ImageNet using just a single datum as visual inputs. We leverage knowledge distillation for this and achieve performances of 94%/74% on CIFAR-10/100, 66% on ImageNet and, by extending this method to audio, 84% on SpeechCommands.

A causal representation learning method that can identify causal variables with instantaneous effects and their graph from temporal sequences with interventions.

We propose to adapt frozen vision transformers by providing input-dependent prompts, computed by a light-weight network. We surpass linear- & full-finetuning in multiple benchmarks.

We propose to utlize the "comments" modality which is common for internet data and show that it can improve vision-language learning.

We propose to tackle the task of video instance segmentation by leveraging self-supervised learning to generate support samples at inference time for improved performances.

We investigate bias and mitigation strategies when using GPT-3 for generating job-advertisements.

We do visual causal representation learning using videos. Our method is able to identify causal variables by intervening on them and observing their effects in time.

We self-supervisedly learn how to detect objects by learning to detect and combine self-segmented object parts starting from SSL pretrained ViTs.

We detect objects without any supervisory signal by leveraging multi-modal signals from videos and combining self-supervised contrastive- and clustering-based learning. Our model learns from video and detects objects in images.

We propose quantized reverse probing as a information-theoretic measure to assess the degree to which self-supervised visual representations align with human-interpretable concepts. his measure is also able to detect when the representation correlates with combinations of labelled concepts (e.g. "red apple") instead of just individual attributes ("red" and "apple" separately).

We present trajectory attention, a drop-in self-attention block for video transformers that implicitly tracks space-time patches along motion paths. We set SOTA results on a number of action recognition datasets: Kinetics-400, Something-Something V2, and Epic-Kitchens.

We analyze the biases and distributions of GPT-2's output w.r.t. to occupations. Especially interesting as AI find its way into hiring and automated application assessments.

We introduce PASS, a large-scale image dataset that does not include any humans, and show that it can be used for high-quality model pretraining while significantly reducing privacy concerns.

We better leverage latent time and space for video representation learning by computing efficient multi-crops in embedding space and using a shallow transformer to model time. This yields SOTA performance and allows for training with longer videos.

We give transformations the prominence they deserve by introducing a systematic framework suitable for contrastive learning. SOTA video representation learning by learning (in)variances systematically.

We build an agent-based version of a fundamental macroeconomic model and include simple decision making heuristics. We find highly complex behavior and business cycles.

We evaluate the potential of conducting object detection with blurred and GAN-swapped faces. It works well and can potentially even alleviate biases.

We use a generative objective to improve the instance discrimination limitations of contrastive learning to set new state-of-the-art results in text-to-video retrieval.

Unsupervisedly clustering videos via self-supervision. We show clustering videos well does not come for free from good representations. Instead, we learn a multi-modal clustering function that treats the audio and visual-stream as augmentations.

We propose a self-supervised learning formulation that simultaneously learns feature representations and useful dataset labels by optimizing the common cross-entropy loss for features and labels, while maximizing information.

We evaluate self-supervised feature learning methods and find that with sufficient data augmentation early layers can be learned using just one image. This is informative about self-supervision and the role of augmentations.

We investigate dietary transitions by analysing a large scale dataset of recipes and user ratings. We detect a consistent increase in the number of users switching to vegetarian diets, and maintaining them. We show that the transition is eased by initially switching to vegetarian diets

In my BSc thesis, I investigated how we can model proton computation tomography (pCT) using Monte-Carlo based software. We simulated an ideal pCT scanner and scans of several cylindrical phantoms with various tissue equivalent inserts of different sizes.

Great template from Jon Barron