Driver DSMS/ADAS - POC Demo

World-Class Real-Time Driver Monitoring System | Optimized for Raspberry Pi & Low-Spec CPUs

---

🚀 Quick Start

# Install dependencies
pip install -r requirements.txt

# Run POC Demo
./run_poc.sh
# OR
streamlit run src/poc_demo.py

---

📦 Technologies & Libraries

Core Framework

• Streamlit (v1.28+) - Web UI framework
• OpenCV (v4.8+) - Image processing & video capture
• NumPy (v1.24+) - Numerical operations

Deep Learning Models

• YOLOv8n (Ultralytics) - Object detection (ONNX optimized)
• ONNX Runtime (v1.15+) - Fast inference engine
• PyTorch (v2.0+) - Model training/export (not used in runtime)

Face & Pose Analysis

• MediaPipe Face Mesh (v0.10+) - Face landmarks, PERCLOS, head pose
• MediaPipe Pose (v0.10+) - Body landmarks for smoking/seatbelt

Utilities

• PyYAML (v6.0+) - Configuration management
• scikit-learn (v1.3+) - ML utilities (installed but not used in POC)

---

✅ Active Features (POC)

DSMS (Driver State Monitoring)

1. Drowsiness Detection - MediaPipe Face Mesh (PERCLOS algorithm)
2. Distraction Detection - MediaPipe Face Mesh (head pose yaw/pitch)
3. Driver Absent Detection - MediaPipe Face Mesh (face presence)
4. Phone Detection - YOLOv8n ONNX (COCO class 67: cell phone)
5. Smoking Detection - MediaPipe Pose (hand-to-mouth gesture)
6. Seatbelt Detection - MediaPipe Pose (shoulder/chest analysis)

UI Features

• Real-time video feed (camera or uploaded file)
• Camera ON/OFF toggle
• Video file upload (MP4, AVI, MOV, MKV, WebM, FLV, WMV, M4V)
• Live alerts display
• Performance statistics

---

❌ Disabled Features (Not in POC)

Removed from Original Implementation

1. Vehicle Detection - YOLOv8n (COCO classes 2,3,5,7) - Removed for POC
2. Pedestrian Detection - YOLOv8n (COCO class 0) - Removed for POC
3. VideoMAE - Action recognition model - Too heavy for low-spec CPUs
4. Roboflow API - External seatbelt detection - Replaced with MediaPipe Pose
5. Isolation Forest - Anomaly detection - Not reliable without training data
6. Optical Flow - OpenCV Farneback - Removed (was for speed/braking estimation)

ADAS Features (Not Implemented)

• Forward Collision Warning (FCW)
• Lane Departure Warning (LDW)
• Tailgating Detection
• Hard Braking/Acceleration Detection
• Overspeed Detection

---

🎯 Model Details

YOLOv8n (ONNX)

• Model: yolov8n.onnx (auto-exported from PyTorch)
• Input: 640x640 RGB image
• Output: 84x8400 (4 bbox + 80 class scores)
• Classes Used: 67 (cell phone only)
• Confidence Threshold: 0.5
• Inference: Every 2nd frame (skip=2)

MediaPipe Face Mesh

• Landmarks: 468 points (refined)
• Features: PERCLOS, head yaw/pitch, face presence
• Confidence: 0.5 (detection), 0.5 (tracking)
• Max Faces: 1

MediaPipe Pose

• Landmarks: 33 body points
• Complexity: 1 (balanced)
• Features: Smoking (hand-to-mouth), Seatbelt (shoulder/chest)
• Inference: Every 6th frame (optimized)
• Confidence: 0.5 (detection), 0.5 (tracking)

---

⚙️ Configuration

File: config/poc_config.yaml

Key Settings:

• Frame size: 640x480
• Inference skip: 2 frames
• PERCLOS threshold: 0.3
• Head pose threshold: 25°
• Confidence threshold: 0.5

---

📊 Performance

Target Hardware: Raspberry Pi 4 / Low-spec CPU (4 cores, 2GHz, 8GB RAM)

Optimizations:

• ONNX inference (faster than PyTorch)
• Frame skipping (process every 2nd frame)
• MediaPipe Pose runs every 6th frame
• Queue-based threading (non-blocking UI)
• Optimized frame size (640x480)

Expected Performance:

• FPS: 15-25 (with frame skipping)
• Memory: 1-2GB
• CPU: 60-80%

---

📁 Project Structure

Driver_DSMS_ADAS/
├── src/
│   └── poc_demo.py          # Main POC application
├── config/
│   └── poc_config.yaml      # Configuration file
├── models/                  # Auto-created: YOLO ONNX models
├── logs/                    # Auto-created: Application logs
├── requirements.txt         # Python dependencies
├── run_poc.sh              # Quick start script
└── README.md               # This file

---

🔧 Dependencies

Required (see requirements.txt):

• streamlit>=1.28.0,<2.0.0
• opencv-python>=4.8.0,<5.0.0
• numpy>=1.24.0,<2.0.0
• ultralytics>=8.0.0,<9.0.0
• torch>=2.0.0,<3.0.0 (for YOLO export only)
• onnxruntime>=1.15.0,<2.0.0
• mediapipe>=0.10.0,<1.0.0
• pyyaml>=6.0,<7.0

Optional (installed but not used in POC):

• transformers>=4.30.0,<5.0.0 (VideoMAE - disabled)
• roboflow>=1.1.0,<2.0.0 (API - disabled)
• scikit-learn>=1.3.0,<2.0.0 (Isolation Forest - disabled)

---

🐛 Known Limitations

1. Smoking Detection: Heuristic-based (hand-to-mouth distance), may have false positives
2. Seatbelt Detection: Heuristic-based (shoulder/chest analysis), accuracy depends on camera angle
3. Phone Detection: Only detects visible phones (not in pockets)
4. Frame Skipping: Predictions update every 2nd frame (smooth video, delayed alerts)

---

📝 Notes

• Original File: track_drive.py (full implementation with disabled features)
• POC File: src/poc_demo.py (streamlined, optimized version)
• Models: Auto-downloaded on first run (YOLOv8n ~6MB)
• ONNX Export: Automatic on first run (creates models/yolov8n.onnx)

---

🎯 Use Cases

• Driver Monitoring: Real-time drowsiness, distraction, phone use
• Safety Compliance: Seatbelt, smoking detection
• Demo/POC: Lightweight, accurate features for presentations
• Raspberry Pi Deployment: Optimized for low-spec hardware

---

Last Updated: 2024
Status: ✅ POC Ready - Production Optimized