Re_Backend/docs/UPSTASH_SETUP_GUIDE.md

# Upstash Redis Setup Guide

## Why Upstash?

✅ **No Installation**: Works instantly on Windows, Mac, Linux  
✅ **100% Free Tier**: 10,000 commands/day (more than enough for dev)  
✅ **Production Ready**: Same service for dev and production  
✅ **Global CDN**: Fast from anywhere  
✅ **Zero Maintenance**: No Redis server to manage  

---

## Step-by-Step Setup (3 minutes)

### 1. Create Upstash Account

1. Go to: https://console.upstash.com/
2. Sign up with GitHub, Google, or Email
3. Verify your email (if required)

### 2. Create Redis Database

1. **Click "Create Database"**
2. **Fill in details**:
   - **Name**: `redis-tat-dev` (or any name you like)
   - **Type**: Select "Regional"
   - **Region**: Choose closest to you (e.g., US East, EU West)
   - **TLS**: Keep enabled (recommended)
   - **Eviction**: Choose "No Eviction"

3. **Click "Create"**

### 3. Copy Connection URL

After creation, you'll see your database dashboard:

1. **Find "REST API" section**
2. **Look for "Redis URL"** - it looks like:
   ```
   rediss://default:AbCdEfGh1234567890XyZ@us1-mighty-shark-12345.upstash.io:6379
   ```

3. **Click the copy button** 📋

---

## Configure Your Application

### Edit `.env` File

Open `Re_Backend/.env` and add/update:

```bash
# Upstash Redis URL
REDIS_URL=rediss://default:YOUR_PASSWORD@YOUR_URL.upstash.io:6379

# Enable test mode for faster testing
TAT_TEST_MODE=true
```

**Important**: 
- Note the **double `s`** in `rediss://` (TLS enabled)
- Copy the entire URL including the password

---

## Verify Connection

### Start Your Backend

```bash
cd Re_Backend
npm run dev
```

### Check Logs

You should see:
```
✅ [TAT Queue] Connected to Redis
✅ [TAT Worker] Initialized and listening
⏰ TAT Configuration:
   - Test Mode: ENABLED (1 hour = 1 minute)
   - Redis: rediss://***@upstash.io:6379
```

---

## Test Using Upstash Console

### Method 1: Web CLI (Easiest)

1. Go to your database in Upstash Console
2. Click the **"CLI"** tab
3. Type commands:
   ```redis
   PING
   # → PONG
   
   KEYS *
   # → Shows all keys (should see TAT jobs after submitting request)
   
   INFO
   # → Shows Redis server info
   ```

### Method 2: Redis CLI (Optional)

If you have `redis-cli` installed:

```bash
redis-cli -u "rediss://default:YOUR_PASSWORD@YOUR_URL.upstash.io:6379" ping
# → PONG
```

---

## Monitor Your TAT Jobs

### View Queued Jobs

In Upstash Console CLI:

```redis
# List all TAT jobs
KEYS bull:tatQueue:*

# See delayed jobs
LRANGE bull:tatQueue:delayed 0 -1

# Get specific job details
HGETALL bull:tatQueue:tat50-<REQUEST_ID>-<LEVEL_ID>
```

### Example Output

After submitting a request, you should see:
```redis
KEYS bull:tatQueue:*
# Returns:
# 1) "bull:tatQueue:id"
# 2) "bull:tatQueue:delayed"
# 3) "bull:tatQueue:tat50-abc123-xyz789"
# 4) "bull:tatQueue:tat75-abc123-xyz789"
# 5) "bull:tatQueue:tatBreach-abc123-xyz789"
```

---

## Upstash Features for Development

### 1. Data Browser
- View all keys and values
- Edit data directly
- Delete specific keys

### 2. CLI Tab
- Run Redis commands
- Test queries
- Debug issues

### 3. Metrics
- Monitor requests/sec
- Track data usage
- View connection count

### 4. Logs
- See all commands executed
- Debug connection issues
- Monitor performance

---

## Free Tier Limits

**Upstash Free Tier includes:**
- ✅ 10,000 commands per day
- ✅ 256 MB storage
- ✅ TLS/SSL encryption
- ✅ Global edge caching
- ✅ REST API access

**Perfect for:**
- ✅ Development
- ✅ Testing
- ✅ Small production apps (up to ~100 users)

---

## Production Considerations

### Upgrade When Needed

For production with high traffic:
- **Pro Plan**: $0.2 per 100K commands
- **Pay as you go**: No monthly fee
- **Auto-scaling**: Handles any load

### Security Best Practices

1. **Use TLS**: Always use `rediss://` (double s)
2. **Rotate Passwords**: Change regularly in production
3. **IP Restrictions**: Add allowed IPs in Upstash console
4. **Environment Variables**: Never commit REDIS_URL to Git

### Production Setup

```bash
# .env.production
REDIS_URL=rediss://default:PROD_PASSWORD@prod-region.upstash.io:6379
TAT_TEST_MODE=false  # Use real hours in production
WORK_START_HOUR=9
WORK_END_HOUR=18
```

---

## Troubleshooting

### Connection Refused Error

**Problem**: `ECONNREFUSED` or timeout

**Solutions**:

1. **Check URL format**:
   ```bash
   # Should be:
   rediss://default:password@host.upstash.io:6379
   
   # NOT:
   redis://... (missing second 's' for TLS)
   ```

2. **Verify database is active**:
   - Go to Upstash Console
   - Check database status (should be green "Active")

3. **Test connection**:
   - Use Upstash Console CLI tab
   - Type `PING` - should return `PONG`

### Slow Response Times

**Problem**: High latency

**Solutions**:

1. **Choose closer region**:
   - Delete database
   - Create new one in region closer to you

2. **Use REST API** (alternative):
   ```bash
   UPSTASH_REDIS_REST_URL=https://YOUR_URL.upstash.io
   UPSTASH_REDIS_REST_TOKEN=YOUR_TOKEN
   ```

### Command Limit Exceeded

**Problem**: "Daily request limit exceeded"

**Solutions**:

1. **Check usage**:
   - Go to Upstash Console → Metrics
   - See command count

2. **Optimize**:
   - Remove unnecessary Redis calls
   - Batch operations where possible

3. **Upgrade** (if needed):
   - Pro plan: $0.2 per 100K commands
   - No monthly fee

---

## Comparison: Upstash vs Local Redis

| Feature | Upstash | Local Redis |
|---------|---------|-------------|
| **Setup Time** | 2 minutes | 10-30 minutes |
| **Installation** | None | Docker/Memurai |
| **Maintenance** | Zero | Manual updates |
| **Cost (Dev)** | Free | Free |
| **Works Offline** | No | Yes |
| **Production** | Same setup | Need migration |
| **Monitoring** | Built-in | Setup required |
| **Backup** | Automatic | Manual |

**Verdict**: 
- ✅ **Upstash for most cases** (especially Windows dev)
- Local Redis only if you need offline development

---

## Migration from Local Redis

If you were using local Redis:

### 1. Export Data (Optional)

```bash
# From local Redis
redis-cli --rdb dump.rdb

# Import to Upstash (use Upstash REST API or CLI)
```

### 2. Update Configuration

```bash
# Old (.env)
REDIS_URL=redis://localhost:6379

# New (.env)
REDIS_URL=rediss://default:PASSWORD@host.upstash.io:6379
```

### 3. Restart Application

```bash
npm run dev
```

**That's it!** No code changes needed - BullMQ works identically.

---

## FAQs

### Q: Is Upstash free forever?
**A**: Yes, 10,000 commands/day free tier is permanent.

### Q: Can I use it in production?
**A**: Absolutely! Many companies use Upstash in production.

### Q: What if I exceed free tier?
**A**: You get notified. Either optimize or upgrade to pay-as-you-go.

### Q: Is my data secure?
**A**: Yes, TLS encryption by default, SOC 2 compliant.

### Q: Can I have multiple databases?
**A**: Yes, unlimited databases on free tier.

### Q: What about data persistence?
**A**: Full Redis persistence (RDB + AOF) with automatic backups.

---

## Resources

- **Upstash Docs**: https://docs.upstash.com/redis
- **Redis Commands**: https://redis.io/commands
- **BullMQ Docs**: https://docs.bullmq.io/
- **Our TAT System**: See `TAT_NOTIFICATION_SYSTEM.md`

---

## Next Steps

✅ Upstash setup complete? Now:

1. **Enable Test Mode**: Set `TAT_TEST_MODE=true` in `.env`
2. **Create Test Request**: Submit a 6-hour TAT request
3. **Watch Logs**: See notifications at 3min, 4.5min, 6min
4. **Check Upstash CLI**: Monitor jobs in real-time

---

**Setup Complete!** 🎉

Your TAT notification system is now powered by Upstash Redis!

---

**Last Updated**: November 4, 2025  
**Contact**: Royal Enfield Workflow Team