# Driver DSMS/ADAS - POC Demo

**World-Class Real-Time Driver Monitoring System** | Optimized for Raspberry Pi & Low-Spec CPUs

---

## 🚀 Quick Start

```bash
# Install dependencies
pip install -r requirements.txt

# Run POC Demo
./run_poc.sh
# OR
streamlit run src/poc_demo.py
```

---

## 📦 Technologies & Libraries

### **Core Framework**
- **Streamlit** (v1.28+) - Web UI framework
- **OpenCV** (v4.8+) - Image processing & video capture
- **NumPy** (v1.24+) - Numerical operations

### **Deep Learning Models**
- **YOLOv8n** (Ultralytics) - Object detection (ONNX optimized)
- **ONNX Runtime** (v1.15+) - Fast inference engine
- **PyTorch** (v2.0+) - Model training/export (not used in runtime)

### **Face & Pose Analysis**
- **MediaPipe Face Mesh** (v0.10+) - Face landmarks, PERCLOS, head pose
- **MediaPipe Pose** (v0.10+) - Body landmarks for smoking/seatbelt

### **Utilities**
- **PyYAML** (v6.0+) - Configuration management
- **scikit-learn** (v1.3+) - ML utilities (installed but not used in POC)

---

## ✅ Active Features (POC)

### **DSMS (Driver State Monitoring)**
1. **Drowsiness Detection** - MediaPipe Face Mesh (PERCLOS algorithm)
2. **Distraction Detection** - MediaPipe Face Mesh (head pose yaw/pitch)
3. **Driver Absent Detection** - MediaPipe Face Mesh (face presence)
4. **Phone Detection** - YOLOv8n ONNX (COCO class 67: cell phone)
5. **Smoking Detection** - MediaPipe Pose (hand-to-mouth gesture)
6. **Seatbelt Detection** - MediaPipe Pose (shoulder/chest analysis)

### **UI Features**
- Real-time video feed (camera or uploaded file)
- Camera ON/OFF toggle
- Video file upload (MP4, AVI, MOV, MKV, WebM, FLV, WMV, M4V)
- Live alerts display
- Performance statistics

---

## ❌ Disabled Features (Not in POC)

### **Removed from Original Implementation**
1. **Vehicle Detection** - YOLOv8n (COCO classes 2,3,5,7) - Removed for POC
2. **Pedestrian Detection** - YOLOv8n (COCO class 0) - Removed for POC
3. **VideoMAE** - Action recognition model - Too heavy for low-spec CPUs
4. **Roboflow API** - External seatbelt detection - Replaced with MediaPipe Pose
5. **Isolation Forest** - Anomaly detection - Not reliable without training data
6. **Optical Flow** - OpenCV Farneback - Removed (was for speed/braking estimation)

### **ADAS Features (Not Implemented)**
- Forward Collision Warning (FCW)
- Lane Departure Warning (LDW)
- Tailgating Detection
- Hard Braking/Acceleration Detection
- Overspeed Detection

---

## 🎯 Model Details

### **YOLOv8n (ONNX)**
- **Model**: `yolov8n.onnx` (auto-exported from PyTorch)
- **Input**: 640x640 RGB image
- **Output**: 84x8400 (4 bbox + 80 class scores)
- **Classes Used**: 67 (cell phone only)
- **Confidence Threshold**: 0.5
- **Inference**: Every 2nd frame (skip=2)

### **MediaPipe Face Mesh**
- **Landmarks**: 468 points (refined)
- **Features**: PERCLOS, head yaw/pitch, face presence
- **Confidence**: 0.5 (detection), 0.5 (tracking)
- **Max Faces**: 1

### **MediaPipe Pose**
- **Landmarks**: 33 body points
- **Complexity**: 1 (balanced)
- **Features**: Smoking (hand-to-mouth), Seatbelt (shoulder/chest)
- **Inference**: Every 6th frame (optimized)
- **Confidence**: 0.5 (detection), 0.5 (tracking)

---

## ⚙️ Configuration

**File**: `config/poc_config.yaml`

**Key Settings**:
- Frame size: 640x480
- Inference skip: 2 frames
- PERCLOS threshold: 0.3
- Head pose threshold: 25°
- Confidence threshold: 0.5

---

## 📊 Performance

**Target Hardware**: Raspberry Pi 4 / Low-spec CPU (4 cores, 2GHz, 8GB RAM)

**Optimizations**:
- ONNX inference (faster than PyTorch)
- Frame skipping (process every 2nd frame)
- MediaPipe Pose runs every 6th frame
- Queue-based threading (non-blocking UI)
- Optimized frame size (640x480)

**Expected Performance**:
- FPS: 15-25 (with frame skipping)
- Memory: 1-2GB
- CPU: 60-80%

---

## 📁 Project Structure

```
Driver_DSMS_ADAS/
├── src/
│   └── poc_demo.py          # Main POC application
├── config/
│   └── poc_config.yaml      # Configuration file
├── models/                  # Auto-created: YOLO ONNX models
├── logs/                    # Auto-created: Application logs
├── requirements.txt         # Python dependencies
├── run_poc.sh              # Quick start script
└── README.md               # This file
```

---

## 🔧 Dependencies

**Required** (see `requirements.txt`):
- streamlit>=1.28.0,<2.0.0
- opencv-python>=4.8.0,<5.0.0
- numpy>=1.24.0,<2.0.0
- ultralytics>=8.0.0,<9.0.0
- torch>=2.0.0,<3.0.0 (for YOLO export only)
- onnxruntime>=1.15.0,<2.0.0
- mediapipe>=0.10.0,<1.0.0
- pyyaml>=6.0,<7.0

**Optional** (installed but not used in POC):
- transformers>=4.30.0,<5.0.0 (VideoMAE - disabled)
- roboflow>=1.1.0,<2.0.0 (API - disabled)
- scikit-learn>=1.3.0,<2.0.0 (Isolation Forest - disabled)

---

## 🐛 Known Limitations

1. **Smoking Detection**: Heuristic-based (hand-to-mouth distance), may have false positives
2. **Seatbelt Detection**: Heuristic-based (shoulder/chest analysis), accuracy depends on camera angle
3. **Phone Detection**: Only detects visible phones (not in pockets)
4. **Frame Skipping**: Predictions update every 2nd frame (smooth video, delayed alerts)

---

## 📝 Notes

- **Original File**: `track_drive.py` (full implementation with disabled features)
- **POC File**: `src/poc_demo.py` (streamlined, optimized version)
- **Models**: Auto-downloaded on first run (YOLOv8n ~6MB)
- **ONNX Export**: Automatic on first run (creates `models/yolov8n.onnx`)

---

## 🎯 Use Cases

- **Driver Monitoring**: Real-time drowsiness, distraction, phone use
- **Safety Compliance**: Seatbelt, smoking detection
- **Demo/POC**: Lightweight, accurate features for presentations
- **Raspberry Pi Deployment**: Optimized for low-spec hardware

---

**Last Updated**: 2024  
**Status**: ✅ POC Ready - Production Optimized