MIT Seminar, Spring 2023

Minimal Perception: A Vision for the Future of Tiny Autonomous Robot






Friday March 24th, 2023
10:30 a.m. EST
Room 33-419
Massachusetts Institute of Technology

Invited by Luca Carlone


Abstract: Although autonomous agents are inherently built for a myriad of applications, their perceptual systems are designed with a single line of thought. Utilizing these traditional methods on resource-constraint and/or small-scale robots (less than 3 inches long) is highly inefficient as these algorithms are generic and not parsimonious. In stark contrast, the perceptual systems in biological beings have evolved to be highly efficient based on their natural habitat and their day-to-day tasks. This talk deals with drawing inspiration from nature to build a minimalist perception framework for tiny robots using only onboard sensing and computation at scales that were never thought possible before. The solution to minimal perception in robots at such scales lies at the intersection of AI, computer vision and computational imaging. This talk will discuss how modifying the sensor suite and learning structures/geometry of the scene rather than textures can result in smaller and faster depth prediction models. Reimagining the perception system from the ground up based on a class of tasks holds the key to minimalism that leads to a set of strong constraints, aiding in tackling the problem of autonomy on tiny robots.




About Me



Ph.D. Student
Perception and Robotics Group
Computer Science

I am a third year Ph.D. student and a Dean's Fellow in the Perception & Robotics Group (PRG) at University of Maryland, College Park (UMD), advised by Prof. Yiannis Aloimonos and Dr. Cornelia Fermuller. PRG is associated with the University of Maryland Institute of Advanced Computer Science Studies (UMIACS) and Autonomy, Robotics and Cognition Lab (ARC).

Interests: Active perception and deep learning applications to boost multi-robot interaction and navigation ability in aerial robots.

Prior to pursuing Ph.D., I did my Masters in Robotics at UMD where I worked on active behaviour of aerial robots and published GapFlyt where we used motion cues of the aerial agent to detect an unknown-shaped gap using a monocular camera. Apart from research work, I love to teach! I, along with Nitin J. Sanket designed and taught an open-source graduate course: ENAE788M (Hands-On Autonomous Aerial Robotics) at UMD in Fall 2019. In my spare time, I love to capture nature on my camera, especially landscape and wildlife photographs; watch and play competitive video games — Counter Strike and Dota 2.

Teaching

Instructor

ENAE788M: Hands On Autonomous Aerial Robotics

Fall 2019

This is an advanced graduate course that exposes the students with mathematical foundations of computer vision, planning and control for aerial robots. This course was designed and taught by me and Nitin J. Sanket. The course is designed to balance theory with an application on hardware.

The entire course is open-source! The links to video lectures and projects are given below:

Teaching Assistant

CMSC733: Computer Processing of Pictorial Information

(Instructor: Prof. Yiannis Aloimonos)

Spring 2020 | Spring 2019

CMSC 733 is an advanced graduate course on classical and deep learning approaches for geometric computer vision and computational photography which explores through image formation, visual features, image segmentation, recognition, motion estimation and 3D point clouds. We redesigned this course to showcase how to model classical 3D geometry problems using Deep Learning!

The entire course is open-source! The link to projects and student outputs are given below:

Teaching Assistant

CMSC426: Computer Vision

(Instructor: Prof. Yiannis Aloimonos)

Fall 2020 | Fall 2019 | Fall 2018

CMSC 426 is an introductory course on computer vision and computational photography that explores image formation, image features, image segmentation, image stitching, image recognition, motion estimation, and visual SLAM.

The entire course is open-source! The link to projects and student outputs are given below:

Research

NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction

IROS 2021

Chahat Deep Singh*, Nitin J. Sanket*, Cornelia Fermuller, Yiannis Aloimonos, IEEE International Conference on Intelligent Robots and Systems (IROS), 2021.
* Equal Contribution

 UMD    Paper    Project Page    Code    UMD    Cite  

EVPropNet: Detecting Drones By Finding Propellers For Mid-Air Landing And Following

RSS 2021

Nitin J. Sanket, Chahat Deep Singh, Chethan M. Parameshwara, Cornelia Fermuller, Guido C.H.E. de Croon, Yiannis Aloimonos, Robotics Science and Systems (RSS), 2021.

 UMD  

MorphEyes: Variable Baseline Stereo For Quadrotor Navigation

ICRA 2021

Nitin J. Sanket, Chahat Deep Singh, Varun Asthana, Cornelia Fermuller, Yiannis Aloimonos, IEEE International Conference on Robotics and Automation (ICRA) , 2021.

 Paper    Project Page    Code    UMD    Cite  

PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry

T-RO 2020

Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermuller, Yiannis Aloimonos, IEEE Transactions on Robotics (Under Review), 2020.

 Paper    Project Page    Code    UMD    Cite  

0-MMS: Zero-Shot Multi-Motion Segmentation With A Monocular Event Camera

ICRA 2021

Chethan M. Parameshwara, Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermuller, Yiannis Aloimonos, IEEE International Conference on Robotics and Automation (ICRA), 2021.

 Paper    Project Page    UMD    Cite  

EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras

ICRA 2020

Nitin J. Sanket*, Chethan M. Parameshwara*, Chahat Deep Singh, Cornelia Fermuller, Davide Scaramuzza, Yiannis Aloimonos, IEEE International Confernce on Robotics and Automation, Paris, 2020.
* Equal Contribution

 Paper    Project Page    Code    UMD    Cite  

GapFlyt: Active Vision Based Minimalist Structure-less Gap Detection For Quadrotor Flight

RA-L 2018 | IROS 2018

Chahat Deep Singh*, Nitin J. Sanket*, Kanishka Ganguly, Cornelia Fermuller, Yiannis Aloimonos, IEEE Robotics and Automation Letters, 2018.
* Equal Contribution

Project Page  Paper  Video  Cite 

Video Tutorials

Portfolio

My Works

  • All
  • App
  • Card
  • Web

App 1

App

Web 3

Web

App 2

App

Card 2

Card

Web 2

Web

App 3

App

Card 1

Card

Card 3

Card

Web 3

Web

Services

Peer Review


  • RA-L IEEE Robotics and Automation Letters
  • ICRA International Conference on Robotics and Automation
  • IROS International Conference on Intelligent Robots and Systems
  • CVPR Conference on Computer Vision and Pattern Recognition
  • ICCV International Conference on Computer Vision
  • RSS Robotics Science and Systems
  • TVCJ The Visual Computer Journal, Springer

Contact

Contact Me

My Address

A108 Adam Street, New York, NY 535022

Social Profiles

Email Me

contact@example.com

Call Me

+1 5589 55488 55

Loading
Your message has been sent. Thank you!