122 lines
3.8 KiB
Markdown
122 lines
3.8 KiB
Markdown
# Deployment Fix Guide - Requirement Processor Migration Issue
|
|
|
|
## Problem Summary
|
|
The deployment failed due to a database migration constraint issue in the requirement processor service. The error was:
|
|
```
|
|
❌ Migration failed: 001_business_context_tables.sql - null value in column "service" of relation "schema_migrations" violates not-null constraint
|
|
```
|
|
|
|
## Root Cause
|
|
The requirement processor's migration system was using an outdated schema for the `schema_migrations` table that didn't include the required `service` field, while the main database migration system expected this field to be present and non-null.
|
|
|
|
## Fix Applied
|
|
|
|
### 1. Updated Migration Script (`migrate.py`)
|
|
- ✅ Updated `schema_migrations` table schema to include `service` field
|
|
- ✅ Modified `is_applied()` function to check by both version and service
|
|
- ✅ Updated `mark_applied()` function to include service and description
|
|
- ✅ Fixed `run_migration()` function to use service parameter
|
|
|
|
### 2. Fixed Migration Files
|
|
- ✅ Removed foreign key constraint from initial migration to avoid dependency issues
|
|
- ✅ The second migration already handles the constraint properly
|
|
|
|
### 3. Created Fix Script
|
|
- ✅ Created `scripts/fix-requirement-processor-migration.sh` to clean up and restart the service
|
|
|
|
## Deployment Steps
|
|
|
|
### Option 1: Use the Fix Script (Recommended)
|
|
```bash
|
|
cd /home/ubuntu/codenuk-backend-live
|
|
./scripts/fix-requirement-processor-migration.sh
|
|
```
|
|
|
|
### Option 2: Manual Fix
|
|
```bash
|
|
# 1. Stop the requirement processor
|
|
docker compose stop requirement-processor
|
|
|
|
# 2. Clean up failed migration records
|
|
PGPASSWORD="password" psql -h localhost -p 5432 -U postgres -d dev_pipeline << 'EOF'
|
|
DELETE FROM schema_migrations WHERE service = 'requirement-processor' OR version LIKE '%.sql';
|
|
EOF
|
|
|
|
# 3. Restart the service
|
|
docker compose up -d requirement-processor
|
|
|
|
# 4. Check status
|
|
docker compose ps requirement-processor
|
|
```
|
|
|
|
### Option 3: Full Redeploy
|
|
```bash
|
|
# Stop all services
|
|
docker compose down
|
|
|
|
# Clean up database (if needed)
|
|
PGPASSWORD="password" psql -h localhost -p 5432 -U postgres -d dev_pipeline << 'EOF'
|
|
DELETE FROM schema_migrations WHERE service = 'requirement-processor';
|
|
EOF
|
|
|
|
# Start all services
|
|
docker compose up -d
|
|
```
|
|
|
|
## Verification Steps
|
|
|
|
1. **Check Service Status**
|
|
```bash
|
|
docker compose ps requirement-processor
|
|
```
|
|
|
|
2. **Check Migration Records**
|
|
```bash
|
|
PGPASSWORD="password" psql -h localhost -p 5432 -U postgres -d dev_pipeline << 'EOF'
|
|
SELECT service, version, applied_at, description
|
|
FROM schema_migrations
|
|
WHERE service = 'requirement-processor'
|
|
ORDER BY applied_at;
|
|
EOF
|
|
```
|
|
|
|
3. **Check Service Logs**
|
|
```bash
|
|
docker compose logs requirement-processor
|
|
```
|
|
|
|
4. **Test Health Endpoint**
|
|
```bash
|
|
curl http://localhost:8001/health
|
|
```
|
|
|
|
## Expected Results
|
|
|
|
After the fix:
|
|
- ✅ Requirement processor service should start successfully
|
|
- ✅ Migration records should show proper service field
|
|
- ✅ Health endpoint should return 200 OK
|
|
- ✅ All other services should continue running normally
|
|
|
|
## Prevention
|
|
|
|
To prevent this issue in the future:
|
|
1. Always ensure migration scripts use the correct `schema_migrations` table schema
|
|
2. Include service field in all migration tracking
|
|
3. Test migrations in development before deploying to production
|
|
4. Use the shared migration system consistently across all services
|
|
|
|
## Troubleshooting
|
|
|
|
If the issue persists:
|
|
1. Check database connectivity
|
|
2. Verify PostgreSQL is running
|
|
3. Check disk space and memory
|
|
4. Review all service logs
|
|
5. Consider a full database reset if necessary
|
|
|
|
## Files Modified
|
|
- `services/requirement-processor/migrations/migrate.py` - Updated migration system
|
|
- `services/requirement-processor/migrations/001_business_context_tables.sql` - Removed FK constraint
|
|
- `scripts/fix-requirement-processor-migration.sh` - Created fix script
|