203 lines
5.7 KiB
Markdown
203 lines
5.7 KiB
Markdown
# Driver DSMS/ADAS - POC Demo
|
|
|
|
**World-Class Real-Time Driver Monitoring System** | Optimized for Raspberry Pi & Low-Spec CPUs
|
|
|
|
---
|
|
|
|
## 🚀 Quick Start
|
|
|
|
```bash
|
|
# Install dependencies
|
|
pip install -r requirements.txt
|
|
|
|
# Run POC Demo
|
|
./run_poc.sh
|
|
# OR
|
|
streamlit run src/poc_demo.py
|
|
```
|
|
|
|
---
|
|
|
|
## 📦 Technologies & Libraries
|
|
|
|
### **Core Framework**
|
|
- **Streamlit** (v1.28+) - Web UI framework
|
|
- **OpenCV** (v4.8+) - Image processing & video capture
|
|
- **NumPy** (v1.24+) - Numerical operations
|
|
|
|
### **Deep Learning Models**
|
|
- **YOLOv8n** (Ultralytics) - Object detection (ONNX optimized)
|
|
- **ONNX Runtime** (v1.15+) - Fast inference engine
|
|
- **PyTorch** (v2.0+) - Model training/export (not used in runtime)
|
|
|
|
### **Face & Pose Analysis**
|
|
- **MediaPipe Face Mesh** (v0.10+) - Face landmarks, PERCLOS, head pose
|
|
- **MediaPipe Pose** (v0.10+) - Body landmarks for smoking/seatbelt
|
|
|
|
### **Utilities**
|
|
- **PyYAML** (v6.0+) - Configuration management
|
|
- **scikit-learn** (v1.3+) - ML utilities (installed but not used in POC)
|
|
|
|
---
|
|
|
|
## ✅ Active Features (POC)
|
|
|
|
### **DSMS (Driver State Monitoring)**
|
|
1. **Drowsiness Detection** - MediaPipe Face Mesh (PERCLOS algorithm)
|
|
2. **Distraction Detection** - MediaPipe Face Mesh (head pose yaw/pitch)
|
|
3. **Driver Absent Detection** - MediaPipe Face Mesh (face presence)
|
|
4. **Phone Detection** - YOLOv8n ONNX (COCO class 67: cell phone)
|
|
5. **Smoking Detection** - MediaPipe Pose (hand-to-mouth gesture)
|
|
6. **Seatbelt Detection** - MediaPipe Pose (shoulder/chest analysis)
|
|
|
|
### **UI Features**
|
|
- Real-time video feed (camera or uploaded file)
|
|
- Camera ON/OFF toggle
|
|
- Video file upload (MP4, AVI, MOV, MKV, WebM, FLV, WMV, M4V)
|
|
- Live alerts display
|
|
- Performance statistics
|
|
|
|
---
|
|
|
|
## ❌ Disabled Features (Not in POC)
|
|
|
|
### **Removed from Original Implementation**
|
|
1. **Vehicle Detection** - YOLOv8n (COCO classes 2,3,5,7) - Removed for POC
|
|
2. **Pedestrian Detection** - YOLOv8n (COCO class 0) - Removed for POC
|
|
3. **VideoMAE** - Action recognition model - Too heavy for low-spec CPUs
|
|
4. **Roboflow API** - External seatbelt detection - Replaced with MediaPipe Pose
|
|
5. **Isolation Forest** - Anomaly detection - Not reliable without training data
|
|
6. **Optical Flow** - OpenCV Farneback - Removed (was for speed/braking estimation)
|
|
|
|
### **ADAS Features (Not Implemented)**
|
|
- Forward Collision Warning (FCW)
|
|
- Lane Departure Warning (LDW)
|
|
- Tailgating Detection
|
|
- Hard Braking/Acceleration Detection
|
|
- Overspeed Detection
|
|
|
|
---
|
|
|
|
## 🎯 Model Details
|
|
|
|
### **YOLOv8n (ONNX)**
|
|
- **Model**: `yolov8n.onnx` (auto-exported from PyTorch)
|
|
- **Input**: 640x640 RGB image
|
|
- **Output**: 84x8400 (4 bbox + 80 class scores)
|
|
- **Classes Used**: 67 (cell phone only)
|
|
- **Confidence Threshold**: 0.5
|
|
- **Inference**: Every 2nd frame (skip=2)
|
|
|
|
### **MediaPipe Face Mesh**
|
|
- **Landmarks**: 468 points (refined)
|
|
- **Features**: PERCLOS, head yaw/pitch, face presence
|
|
- **Confidence**: 0.5 (detection), 0.5 (tracking)
|
|
- **Max Faces**: 1
|
|
|
|
### **MediaPipe Pose**
|
|
- **Landmarks**: 33 body points
|
|
- **Complexity**: 1 (balanced)
|
|
- **Features**: Smoking (hand-to-mouth), Seatbelt (shoulder/chest)
|
|
- **Inference**: Every 6th frame (optimized)
|
|
- **Confidence**: 0.5 (detection), 0.5 (tracking)
|
|
|
|
---
|
|
|
|
## ⚙️ Configuration
|
|
|
|
**File**: `config/poc_config.yaml`
|
|
|
|
**Key Settings**:
|
|
- Frame size: 640x480
|
|
- Inference skip: 2 frames
|
|
- PERCLOS threshold: 0.3
|
|
- Head pose threshold: 25°
|
|
- Confidence threshold: 0.5
|
|
|
|
---
|
|
|
|
## 📊 Performance
|
|
|
|
**Target Hardware**: Raspberry Pi 4 / Low-spec CPU (4 cores, 2GHz, 8GB RAM)
|
|
|
|
**Optimizations**:
|
|
- ONNX inference (faster than PyTorch)
|
|
- Frame skipping (process every 2nd frame)
|
|
- MediaPipe Pose runs every 6th frame
|
|
- Queue-based threading (non-blocking UI)
|
|
- Optimized frame size (640x480)
|
|
|
|
**Expected Performance**:
|
|
- FPS: 15-25 (with frame skipping)
|
|
- Memory: 1-2GB
|
|
- CPU: 60-80%
|
|
|
|
---
|
|
|
|
## 📁 Project Structure
|
|
|
|
```
|
|
Driver_DSMS_ADAS/
|
|
├── src/
|
|
│ └── poc_demo.py # Main POC application
|
|
├── config/
|
|
│ └── poc_config.yaml # Configuration file
|
|
├── models/ # Auto-created: YOLO ONNX models
|
|
├── logs/ # Auto-created: Application logs
|
|
├── requirements.txt # Python dependencies
|
|
├── run_poc.sh # Quick start script
|
|
└── README.md # This file
|
|
```
|
|
|
|
---
|
|
|
|
## 🔧 Dependencies
|
|
|
|
**Required** (see `requirements.txt`):
|
|
- streamlit>=1.28.0,<2.0.0
|
|
- opencv-python>=4.8.0,<5.0.0
|
|
- numpy>=1.24.0,<2.0.0
|
|
- ultralytics>=8.0.0,<9.0.0
|
|
- torch>=2.0.0,<3.0.0 (for YOLO export only)
|
|
- onnxruntime>=1.15.0,<2.0.0
|
|
- mediapipe>=0.10.0,<1.0.0
|
|
- pyyaml>=6.0,<7.0
|
|
|
|
**Optional** (installed but not used in POC):
|
|
- transformers>=4.30.0,<5.0.0 (VideoMAE - disabled)
|
|
- roboflow>=1.1.0,<2.0.0 (API - disabled)
|
|
- scikit-learn>=1.3.0,<2.0.0 (Isolation Forest - disabled)
|
|
|
|
---
|
|
|
|
## 🐛 Known Limitations
|
|
|
|
1. **Smoking Detection**: Heuristic-based (hand-to-mouth distance), may have false positives
|
|
2. **Seatbelt Detection**: Heuristic-based (shoulder/chest analysis), accuracy depends on camera angle
|
|
3. **Phone Detection**: Only detects visible phones (not in pockets)
|
|
4. **Frame Skipping**: Predictions update every 2nd frame (smooth video, delayed alerts)
|
|
|
|
---
|
|
|
|
## 📝 Notes
|
|
|
|
- **Original File**: `track_drive.py` (full implementation with disabled features)
|
|
- **POC File**: `src/poc_demo.py` (streamlined, optimized version)
|
|
- **Models**: Auto-downloaded on first run (YOLOv8n ~6MB)
|
|
- **ONNX Export**: Automatic on first run (creates `models/yolov8n.onnx`)
|
|
|
|
---
|
|
|
|
## 🎯 Use Cases
|
|
|
|
- **Driver Monitoring**: Real-time drowsiness, distraction, phone use
|
|
- **Safety Compliance**: Seatbelt, smoking detection
|
|
- **Demo/POC**: Lightweight, accurate features for presentations
|
|
- **Raspberry Pi Deployment**: Optimized for low-spec hardware
|
|
|
|
---
|
|
|
|
**Last Updated**: 2024
|
|
**Status**: ✅ POC Ready - Production Optimized
|
|
|