412 lines
8.8 KiB
Markdown
412 lines
8.8 KiB
Markdown
# TAT Notification Testing Guide
|
|
|
|
## Quick Setup for Testing
|
|
|
|
### Step 1: Setup Redis
|
|
|
|
**You MUST have Redis for TAT notifications to work.**
|
|
|
|
#### 🚀 Option A: Upstash (RECOMMENDED - No Installation!)
|
|
|
|
**Best choice for Windows development:**
|
|
|
|
1. Go to: https://console.upstash.com/
|
|
2. Sign up (free)
|
|
3. Create Database:
|
|
- Name: `redis-tat-dev`
|
|
- Type: Regional
|
|
- Region: Choose closest
|
|
4. Copy Redis URL (format: `rediss://default:...@host.upstash.io:6379`)
|
|
5. Add to `Re_Backend/.env`:
|
|
```bash
|
|
REDIS_URL=rediss://default:YOUR_PASSWORD@YOUR_HOST.upstash.io:6379
|
|
```
|
|
|
|
**✅ Done!** No installation, works everywhere!
|
|
|
|
See detailed guide: `docs/UPSTASH_SETUP_GUIDE.md`
|
|
|
|
#### Option B: Docker (If you prefer local)
|
|
```bash
|
|
docker run -d --name redis-tat -p 6379:6379 redis:latest
|
|
```
|
|
|
|
Then in `.env`:
|
|
```bash
|
|
REDIS_URL=redis://localhost:6379
|
|
```
|
|
|
|
#### Option C: Linux Production
|
|
```bash
|
|
sudo apt install redis-server -y
|
|
sudo systemctl start redis-server
|
|
```
|
|
|
|
#### Verify Connection
|
|
- **Upstash**: Use Console CLI → `PING` → should return `PONG`
|
|
- **Local**: `Test-NetConnection localhost -Port 6379`
|
|
|
|
---
|
|
|
|
### Step 2: Enable Test Mode (Optional but Recommended)
|
|
|
|
For faster testing, enable test mode where **1 hour = 1 minute**:
|
|
|
|
1. **Edit your `.env` file**:
|
|
```bash
|
|
TAT_TEST_MODE=true
|
|
```
|
|
|
|
2. **Restart your backend**:
|
|
```bash
|
|
cd Re_Backend
|
|
npm run dev
|
|
```
|
|
|
|
3. **Verify test mode is enabled** - You should see:
|
|
```
|
|
⏰ TAT Configuration:
|
|
- Test Mode: ENABLED (1 hour = 1 minute)
|
|
- Working Hours: 9:00 - 18:00
|
|
- Working Days: Monday - Friday
|
|
- Redis: redis://localhost:6379
|
|
```
|
|
|
|
---
|
|
|
|
### Step 3: Create a Test Workflow
|
|
|
|
#### Production Mode (TAT_TEST_MODE=false)
|
|
- Create a request with **2 hours TAT**
|
|
- Notifications will come at:
|
|
- **1 hour** (50%)
|
|
- **1.5 hours** (75%)
|
|
- **2 hours** (100% breach)
|
|
|
|
#### Test Mode (TAT_TEST_MODE=true) ⚡ FASTER
|
|
- Create a request with **6 hours TAT** (becomes 6 minutes)
|
|
- Notifications will come at:
|
|
- **3 minutes** (50%)
|
|
- **4.5 minutes** (75%)
|
|
- **6 minutes** (100% breach)
|
|
|
|
---
|
|
|
|
### Step 4: Submit and Monitor
|
|
|
|
1. **Create and Submit Request** via your frontend or API
|
|
|
|
2. **Check Backend Logs** - You should see:
|
|
```
|
|
[TAT Scheduler] Calculating TAT milestones for request...
|
|
[TAT Scheduler] Start: 2025-11-04 12:00
|
|
[TAT Scheduler] 50%: 2025-11-04 12:03
|
|
[TAT Scheduler] 75%: 2025-11-04 12:04
|
|
[TAT Scheduler] 100%: 2025-11-04 12:06
|
|
[TAT Scheduler] Scheduled tat50 for level...
|
|
[TAT Scheduler] Scheduled tat75 for level...
|
|
[TAT Scheduler] Scheduled tatBreach for level...
|
|
[TAT Scheduler] ✅ TAT jobs scheduled for request...
|
|
```
|
|
|
|
3. **Wait for Notifications**
|
|
- Watch the logs
|
|
- Check push notifications
|
|
- Verify database updates
|
|
|
|
4. **Verify Notifications** - Look for:
|
|
```
|
|
[TAT Processor] Processing tat50 for request...
|
|
[TAT Processor] tat50 notification sent for request...
|
|
```
|
|
|
|
---
|
|
|
|
## Testing Scenarios
|
|
|
|
### Scenario 1: Normal Flow (Happy Path)
|
|
```
|
|
1. Create request with TAT = 6 hours (6 min in test mode)
|
|
2. Submit request
|
|
3. Wait for 50% notification (3 min)
|
|
4. Wait for 75% notification (4.5 min)
|
|
5. Wait for 100% breach (6 min)
|
|
```
|
|
|
|
**Expected Result:**
|
|
- ✅ 3 notifications sent
|
|
- ✅ Database flags updated
|
|
- ✅ Activity logs created
|
|
|
|
---
|
|
|
|
### Scenario 2: Early Approval
|
|
```
|
|
1. Create request with TAT = 6 hours
|
|
2. Submit request
|
|
3. Wait for 50% notification (3 min)
|
|
4. Approve immediately
|
|
5. Remaining notifications should be cancelled
|
|
```
|
|
|
|
**Expected Result:**
|
|
- ✅ 50% notification received
|
|
- ✅ 75% and 100% notifications cancelled
|
|
- ✅ TAT jobs for next level scheduled
|
|
|
|
---
|
|
|
|
### Scenario 3: Multi-Level Approval
|
|
```
|
|
1. Create request with 3 approval levels (2 hours each)
|
|
2. Submit request
|
|
3. Level 1: Wait for notifications, then approve
|
|
4. Level 2: Should schedule new TAT jobs
|
|
5. Level 2: Wait for notifications, then approve
|
|
6. Level 3: Should schedule new TAT jobs
|
|
```
|
|
|
|
**Expected Result:**
|
|
- ✅ Each level gets its own TAT monitoring
|
|
- ✅ Previous level jobs cancelled on approval
|
|
- ✅ New level jobs scheduled
|
|
|
|
---
|
|
|
|
### Scenario 4: Rejection
|
|
```
|
|
1. Create request with TAT = 6 hours
|
|
2. Submit request
|
|
3. Wait for 50% notification
|
|
4. Reject the request
|
|
5. All remaining notifications should be cancelled
|
|
```
|
|
|
|
**Expected Result:**
|
|
- ✅ TAT jobs cancelled
|
|
- ✅ No further notifications
|
|
|
|
---
|
|
|
|
## Verification Checklist
|
|
|
|
### Backend Logs ✅
|
|
```bash
|
|
# Should see these messages:
|
|
✓ [TAT Queue] Connected to Redis
|
|
✓ [TAT Worker] Initialized and listening
|
|
✓ [TAT Scheduler] TAT jobs scheduled
|
|
✓ [TAT Processor] Processing tat50
|
|
✓ [TAT Processor] tat50 notification sent
|
|
```
|
|
|
|
### Database Check ✅
|
|
```sql
|
|
-- Check approval level TAT status
|
|
SELECT
|
|
request_id,
|
|
level_number,
|
|
approver_name,
|
|
tat_hours,
|
|
tat_percentage_used,
|
|
tat50_alert_sent,
|
|
tat75_alert_sent,
|
|
tat_breached,
|
|
tat_start_time,
|
|
status
|
|
FROM approval_levels
|
|
WHERE request_id = '<YOUR_REQUEST_ID>';
|
|
```
|
|
|
|
**Expected Fields:**
|
|
- `tat_start_time`: Should be set when level starts
|
|
- `tat50_alert_sent`: true after 50% notification
|
|
- `tat75_alert_sent`: true after 75% notification
|
|
- `tat_breached`: true after 100% notification
|
|
- `tat_percentage_used`: 50, 75, or 100
|
|
|
|
### Activity Logs ✅
|
|
```sql
|
|
-- Check activity timeline
|
|
SELECT
|
|
activity_type,
|
|
activity_description,
|
|
user_name,
|
|
created_at
|
|
FROM activities
|
|
WHERE request_id = '<YOUR_REQUEST_ID>'
|
|
ORDER BY created_at DESC;
|
|
```
|
|
|
|
**Expected Entries:**
|
|
- "50% of TAT time has elapsed"
|
|
- "75% of TAT time has elapsed - Escalation warning"
|
|
- "TAT deadline reached - Breach notification"
|
|
|
|
### Redis Queue ✅
|
|
```bash
|
|
# Connect to Redis
|
|
redis-cli
|
|
|
|
# Check scheduled jobs
|
|
KEYS bull:tatQueue:*
|
|
LRANGE bull:tatQueue:delayed 0 -1
|
|
|
|
# Check job details
|
|
HGETALL bull:tatQueue:tat50-<REQUEST_ID>-<LEVEL_ID>
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### ❌ No Notifications Received
|
|
|
|
**Problem:** TAT jobs scheduled but no notifications
|
|
|
|
**Solutions:**
|
|
1. Check Redis is running:
|
|
```powershell
|
|
Test-NetConnection localhost -Port 6379
|
|
```
|
|
|
|
2. Check worker is running:
|
|
```bash
|
|
# Look for in backend logs:
|
|
[TAT Worker] Worker is ready and listening
|
|
```
|
|
|
|
3. Check job delays:
|
|
```bash
|
|
redis-cli
|
|
> LRANGE bull:tatQueue:delayed 0 -1
|
|
```
|
|
|
|
4. Verify VAPID keys for push notifications:
|
|
```bash
|
|
# In .env file:
|
|
VAPID_PUBLIC_KEY=...
|
|
VAPID_PRIVATE_KEY=...
|
|
```
|
|
|
|
---
|
|
|
|
### ❌ Jobs Not Executing
|
|
|
|
**Problem:** Jobs scheduled but never execute
|
|
|
|
**Solutions:**
|
|
1. Check system time is correct
|
|
2. Verify test mode settings
|
|
3. Check worker logs for errors
|
|
4. Restart worker:
|
|
```bash
|
|
# Restart backend server
|
|
npm run dev
|
|
```
|
|
|
|
---
|
|
|
|
### ❌ Duplicate Notifications
|
|
|
|
**Problem:** Receiving multiple notifications for same milestone
|
|
|
|
**Solutions:**
|
|
1. Check database flags are being set:
|
|
```sql
|
|
SELECT tat50_alert_sent, tat75_alert_sent FROM approval_levels;
|
|
```
|
|
|
|
2. Verify job cancellation on approval:
|
|
```bash
|
|
# Should see in logs:
|
|
[Approval] TAT jobs cancelled for level...
|
|
```
|
|
|
|
3. Check for duplicate job IDs in Redis
|
|
|
|
---
|
|
|
|
### ❌ Redis Connection Errors
|
|
|
|
**Problem:** `ECONNREFUSED` errors
|
|
|
|
**Solutions:**
|
|
1. **Start Redis** - See Step 1
|
|
2. Check Redis URL in `.env`:
|
|
```bash
|
|
REDIS_URL=redis://localhost:6379
|
|
```
|
|
3. Verify port 6379 is not blocked:
|
|
```powershell
|
|
Test-NetConnection localhost -Port 6379
|
|
```
|
|
|
|
---
|
|
|
|
## Testing Timeline Examples
|
|
|
|
### Test Mode Enabled (1 hour = 1 minute)
|
|
|
|
| TAT Hours | Real Time | 50% | 75% | 100% |
|
|
|-----------|-----------|-----|-----|------|
|
|
| 2 hours | 2 minutes | 1m | 1.5m| 2m |
|
|
| 6 hours | 6 minutes | 3m | 4.5m| 6m |
|
|
| 24 hours | 24 minutes| 12m | 18m | 24m |
|
|
| 48 hours | 48 minutes| 24m | 36m | 48m |
|
|
|
|
### Production Mode (Normal)
|
|
|
|
| TAT Hours | 50% | 75% | 100% |
|
|
|-----------|--------|--------|--------|
|
|
| 2 hours | 1h | 1.5h | 2h |
|
|
| 6 hours | 3h | 4.5h | 6h |
|
|
| 24 hours | 12h | 18h | 24h |
|
|
| 48 hours | 24h | 36h | 48h |
|
|
|
|
---
|
|
|
|
## Quick Test Commands
|
|
|
|
```powershell
|
|
# 1. Check Redis
|
|
Test-NetConnection localhost -Port 6379
|
|
|
|
# 2. Start Backend (with test mode)
|
|
cd Re_Backend
|
|
$env:TAT_TEST_MODE="true"
|
|
npm run dev
|
|
|
|
# 3. Monitor Logs (in another terminal)
|
|
cd Re_Backend
|
|
Get-Content -Path "logs/app.log" -Wait -Tail 50
|
|
|
|
# 4. Check Redis Jobs
|
|
redis-cli KEYS "bull:tatQueue:*"
|
|
|
|
# 5. Query Database
|
|
psql -U laxman -d re_workflow_db -c "SELECT * FROM approval_levels WHERE tat_start_time IS NOT NULL;"
|
|
```
|
|
|
|
---
|
|
|
|
## Support
|
|
|
|
If you encounter issues:
|
|
|
|
1. **Check Logs**: `Re_Backend/logs/`
|
|
2. **Enable Debug**: Set `LOG_LEVEL=debug` in `.env`
|
|
3. **Redis Status**: `redis-cli ping` should return `PONG`
|
|
4. **Worker Status**: Look for "TAT Worker: Initialized" in logs
|
|
5. **Database**: Verify TAT fields exist in `approval_levels` table
|
|
|
|
---
|
|
|
|
**Happy Testing!** 🎉
|
|
|
|
For more information, see:
|
|
- `TAT_NOTIFICATION_SYSTEM.md` - Full system documentation
|
|
- `INSTALL_REDIS.txt` - Redis installation guide
|
|
- `backend_structure.txt` - Database schema reference
|
|
|