Tonmoay Deb

I am a second-year Ph.D. student in Computer Science at Northwestern University. I am a member of Northwestern Security & AI Lab (NSAIL) and advised by Dr. V.S. Subrahmanian. I am exploring solutions on emerging defensive security by designing/applying algorithms in Artificial Intelligence, Machine Learning, Computer Vision, and Natural Language Processing.

Before joining Northwestern, I was a Master's (thesis-based) student in Computer Science at the University of New Hampshire. I spent Summer 2021 at the Center for Coastal and Ocean Mapping/NOAA-UNH Joint Hydrographic Center, working on Unsupervised Semantic Segmentation under Dr. Yuri Rzhanov and Dr. Kim Lowell.

Before that, I was a Research Associate (full-time) at the Artificial Intelligence and Cybernetics (AGenCy) Lab, Independent University, Bangladesh. I was supervised by Dr. Amin Ahsan Ali, Dr. A K M Mahbubur Rahman, and Dr. Iftekhar Tanveer. Some notable projects are Diversity in English Video Captioning, Video Captioning in the Bengali Language, and Scalable Bengali Speech-to-Text.

My research interest evolved since my Undergrad at North South University. One of my key projects was PATRON. I was blessed with excellent advisors, Dr. Mohammad Rashedur Rahman, Mr. Adnan Firoze, and Dr. Mohammad Ashrafuzzaman Khan.

I dream to give dreams to machines, to solve the real-world problems!

Email  /  Google Scholar  /  GitHub  /  LinkedIn  /  Old Site

profile photo
  • December 21, 2022: Received AAAI Conference Travel Grant!

  • December 10, 2022: Received MS in Computer Science at Northwestern!

  • December 02, 2022: I am on the Northwestern McCormick School of Engineering News! Link

  • October 25, 2022: DUCK Demo paper accepted at AAAI 2023!

  • October 20, 2022: Presented DUCK at Conference on AI & National Security Link, Tweet

  • February 02, 2022: Got married on 2/2/2022!

  • October 04, 2021: One paper accepted at WACV 2022!

  • September 20, 2021: Joined Northwestern for Ph.D. in Computer Science

  • May 18, 2021: Started Summer Research Internship at CCOM, University of New Hampshire

  • April 24, 2021: One paper from PATRON accepted at IJCNN 2021!

  • February 1, 2021: Starting Masters in Computer Science (thesis-based) at University of New Hampshire
DUCK: A Drone-Urban Cyber-Defense Framework Based on Pareto-Optimal Deontic Logic Agents

Tonmoay Deb, J├╝rgen Dix, Mingi Jeong, Cristian Molinaro, Andrea Pugliese, Alberto Quattrini Li, Eugene Santos, V.S. Subrahmanian, Shanchieh Yang, and Youzhi Zhang

(To appear at) Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI, 2023 (demonstration track)

We have developed the DUCK multi-agent testbed that security agencies can use to simulate drone-based attacks by diverse actors and develop a combination of surveillance camera, drone, and cyber defenses against them.

Variational Stacked Local Attention Networks for Diverse Video Captioning

Tonmoay Deb, Akib Sadmanee, Kishor Kumar Bhaumik, Amin Ahsan Ali, M Ashraful Amin, and A K M Mahbubur Rahman

IEEE Winter Conference on Applications of Computer Vision (WACV), 2022.

[Full Text pdf] / [Supp. Material] / [Presentation Video]

We proposed a novel approach for diverse video captioning followed by extensive analysis and comparison with recent works.

UUCT-HyMP: Towards Tracking Dispersed Crowd Groups from UAVs

Tonmoay Deb, Mahieyin Rahmun, Shahriar Bijoy, Mayamin Hamid Raha, Mohammad A Khan

2021 International Joint Conference on Neural Networks (IJCNN), 2021

[Full Text pdf] / [Presentaton Slides] / [Video Demo]

We proposed a novel crowd group tracking benchmark dataset + algorithm.

Research Projects
Segmentation Analysis of Underwater Coral Imagery

Supervisors: Dr. Yuri Rzhanov and Dr. Kim Lowell.

Center for Coastal and Ocean Mapping/NOAA-UNH Joint Hydrographic Center (Summer 2021)

The main goal was to reduce the effort of coral image annotation as it is a very expensive (requires domain experts) and time-consuming process. Also, one major limitation was that the existing annotations are sparse (200 pixels per image). Another major challenge was that the coral objects don't conform to any specific shape, so the regular classifier can not locate even with sufficient samples. I first tried to overcome the sample shortage issue by integrating few-shot learning for recognizing class labels because patch-based methods are erroneous due to having fixed patch sizes. After satisfactory output, I moved to Unsupervised Segmentation of the corals because we had to distinguish the coral objects precisely. However, due to high variance among samples, I later implemented weakly-supervised semantic segmentation, e.g., 10-pixel labels per training, which performed relatively better. The work is ongoing.

Reviewer, ICRA 2021, ICRA 2023
Reviewer, WACV 2023
Reviewer, IEEE ICMLA 2022. Programme Committee Member, IEEE ICMLA 2023
University of New Hampshire Graduate Teaching Assistant (Introduction to Computer Science I and II): Spring 2021
North South University Lab Instructor (Fundamentals of Programming, Data Structure and Algorithms): Spring 2020, Summer 2020, Fall 2020

Undergraduate and Graduate Teaching Assistant (Machine Learning, Database Systems): Spring 2019, Summer 2019, Fall 2020

The template is taken from Jon Barron