Traditional camera-based detection systems (e.g., wildlife cameras, security cameras) rely on pre-defined models that can only detect a fixed set of objects. This limits their adaptability and usefulness for specialized or changing needs. Imagine instead a system where the user can tell the camera exactly what they want to detect — from “a deer in my garden” to “a delivery box left at the front door” — and have it learn to recognize that target within hours.
Our project aims to develop the next generation of user-customizable AI vision systems that combine generative AI and real-time computer vision. Users will describe their target in plain language (e.g., through ChatGPT or similar interfaces) and optionally provide example images. The system will generate synthetic training data and fine-tune a detection model, which will then be deployed to an edge AI camera. When the camera detects the target, it will immediately send an alert with an image to the user’s smartphone.
This technology has wide-ranging applications in wildlife monitoring, home security, industrial safety, and personalized environmental monitoring. It leverages vision-language models, generative data augmentation, edge computing, and real-time alerting.
Overall Research Goal
To design, build, and evaluate a fully functional AI-enabled detection camera system that supports user-defined targets, from the data collection and training pipeline to live deployment on an embedded device with automated phone notifications.
The Vision on Demand team seeks for curious, adaptable, and team-oriented researchers to work across disciplines to bring a new AI-powered camera detection system to life. This opportunity is open to students from first-year undergraduates through master’s level, with the hope that members will remain on the team beyond the initial two-term commitment.
As members gain experience, they will have the chance to take on leadership responsibilities and serve as mentors to new teammates, further developing their technical expertise and management abilities. The project values a diverse range of backgrounds and interests, and encourages applications from students enthusiastic about any of the areas outlined below.
While our team is structured into specialized subgroups, applicants join the project as a whole rather than for a specific subteam. Subgroups remain flexible, fostering creativity, personal growth, and collaborative research. Students can rotate between subteams according to their workload and evolving interests, promoting diverse experiences and skill development. As members build expertise and confidence, they will be encouraged to assume greater responsibilities and contribute across multiple project areas.
Computer Vision & Model Development (4 students)
Preferred Skills: Train and fine-tune object detection models (e.g., YOLOv8, Detectron2, MMDetection). Use synthetic data generation and data augmentation techniques (Stable Diffusion, DALL·E, etc.). Evaluate model performance and deploy to edge hardware.
Likely Majors: CS, CSE, DATA, ROB, EE, ECE, ME, IOE.
AI/LLM & Data Pipeline Engineering (4 students)
Preferred Skills: Integrate LLM interfaces (e.g., ChatGPT API) to interpret user text prompts. Automate image generation pipelines. Connect AI-generated datasets to model training scripts.
Likely Majors: CS, CSE, DATA, IOE, EE, ECE, SI
Embedded Systems & Edge Development (4 students)
Preferred Skills: Work with embedded AI devices (e.g., NVIDIA Jetson Nano, Raspberry Pi + Coral TPU). Optimize inference for low-latency detection. Set up camera hardware, local storage, and wireless connectivity.
Likely Majors: EE, CE, ECE, ME, ROB, CS, CSE
Apprentice Researcher (5 students)
Preferred Skills: Interest in AI vision systems and willingness to learn technical tools. Assist subteams with testing, data preparation, and basic programming. Open to first- and second-year undergraduates only.
Likely Majors: Any STEM
Raed Al-Kontar, Ph.D.
Dr. Raed Al Kontar is an associate professor in the Industrial & Operations Engineering department. He is also an affiliate with both the Michigan Institutes for Data Science and Computational Discovery and Engineering. Dr. Al Kontar leads the “Data Science Lab,” which focuses on data science using probabilistic models, with an emphasis on precision/personalized data science where knowledge from diverse data sources is effectively integrated. Dr. Al Kontar’s research has been highly recognized, with his group winning 12 best paper awards since 2022 across the Institute for Operations Research and the Management Sciences (INFORMS), the American Statistical Association (ASA), and the Institute of Industrial and Systems Engineers (IISE). His research is currently supported by the National Science Foundation (NSF), including a 2022 CAREER award, the National Institutes of Health (NIH), the National Library of Medicine (NLM), and various industry collaborators.
Weekly Meeting Time and Location:
The MDP-Vision on Demand team meets in-person weekly on Thursdays, 5:00-7:00 pm on North Campus. Led by the faculty PI, these meetings cover technical developments, team management, and strategy planning, and include hands-on working sessions addressing current challenges. On alternate weeks the team will hold a work-together session. Subteam lead/PI meetings may be held at this time.
Subteams schedule weekly in person work together sessions at a mutually convenient time for the members. These sessions are focused on collaborative work, progressing towards semester goals, and addressing specific questions. The faculty PI may join subteam meetings as needed.
Subteam leaders will meet weekly with the faculty PI to regularly report the progress of each subteam. Since all students will be new to the project, subteam leaders will be chosen as members gain experience and demonstrate leadership skills.
The team will maintain a Slack channel to coordinate progress asynchronously, seek support from others, document challenges, and generally keep each other informed.
Enrollment: Students will enroll in ENGR 255,355,455, or 599 for 2 credits per term. Enrollment is annual, with two-term commitments starting each January.
Course Substitutions: Honors, CS-ENGR/DS-ENGR/EE/CE-ENGR 355 and higher can count toward Flex Tech.
These substitutions/departmental courses are available for students in these respective majors. MDP does not yet have a formal agreement with other departments for substitutions/departmental courses not listed. Please reach out to your home department’s academic advisor about how you might apply MDP credits to your degree plan.
Citizenship Requirements: This project is open to all students on campus.
IP/NDA: Students who successfully match to this project team will be required to sign an Intellectual Property (IP) Agreement prior to starting participation in January.
Location: In-person participation is required for weekly meetings and work sessions. Some team meetings and activities will also be held remotely.
Summer Opportunity: Some Summer research internships may be available for qualifying students.
Learn more about the expectations for this type of MDP project