#55 Self-Supervised Vision Models (Dr. Ishan Misra - FAIR).

Dr. Ishan Misra is a Research Scientist at Facebook AI Research where he works on Computer Vision and Machine Learning. His main research interest is reducing the need for human supervision, and indeed, human knowledge in visual learning systems. He finished his PhD at the Robotics Institute at Carnegie Mellon. He has done stints at Microsoft Research, INRIA and Yale. His bachelors is in computer science where he achieved the highest GPA in his cohort. 

Ishan is fast becoming a prolific scientist, already with more than 3000 citations under his belt and co-authoring with Yann LeCun; the godfather of deep learning.  Today though we will be focusing an exciting cluster of recent papers around unsupervised representation learning for computer vision released from FAIR. These are; DINO: Emerging Properties in Self-Supervised Vision Transformers, BARLOW TWINS: Self-Supervised Learning via Redundancy Reduction and PAWS: Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with
Support Samples. All of these papers are hot off the press, just being officially released in the last month or so. Many of you will remember PIRL: Self-Supervised Learning of Pretext-Invariant Representations which Ishan was the primary author of in 2019.

References;

Shuffle and Learn - https://arxiv.org/abs/1603.08561
DepthContrast - https://arxiv.org/abs/2101.02691
DINO - https://arxiv.org/abs/2104.14294
Barlow Twins - https://arxiv.org/abs/2103.03230
SwAV - https://arxiv.org/abs/2006.09882
PIRL - https://arxiv.org/abs/1912.01991
AVID - https://arxiv.org/abs/2004.12943 (best paper candidate at CVPR'21 (just announced over the weekend) - http://cvpr2021.thecvf.com/node/290)
 
Alexei (Alyosha) Efros
http://people.eecs.berkeley.edu/~efros/
http://www.cs.cmu.edu/~tmalisie/projects/nips09/
 
Exemplar networks
https://arxiv.org/abs/1406.6909
 
The bitter lesson - Rich Sutton
http://www.incompleteideas.net/IncIdeas/BitterLesson.html
 
Machine Teaching: A New Paradigm for Building Machine Learning Systems
https://arxiv.org/abs/1707.06742
 
POET
https://arxiv.org/pdf/1901.01753.pdf