codenuk_backend_mine/REQUIREMENT_PROCESSOR_MIGRATION_FIX.md

Deployment Fix Guide - Requirement Processor Migration Issue

Problem Summary

The deployment failed due to a database migration constraint issue in the requirement processor service. The error was:

❌ Migration failed: 001_business_context_tables.sql - null value in column "service" of relation "schema_migrations" violates not-null constraint

Root Cause

The requirement processor's migration system was using an outdated schema for the schema_migrations table that didn't include the required service field, while the main database migration system expected this field to be present and non-null.

Fix Applied

1. Updated Migration Script (migrate.py)

• ✅ Updated schema_migrations table schema to include service field
• ✅ Modified is_applied() function to check by both version and service
• ✅ Updated mark_applied() function to include service and description
• ✅ Fixed run_migration() function to use service parameter

2. Fixed Migration Files

• ✅ Removed foreign key constraint from initial migration to avoid dependency issues
• ✅ The second migration already handles the constraint properly

3. Created Fix Script

• ✅ Created scripts/fix-requirement-processor-migration.sh to clean up and restart the service

Deployment Steps

Option 1: Use the Fix Script (Recommended)

cd /home/ubuntu/codenuk-backend-live
./scripts/fix-requirement-processor-migration.sh

Option 2: Manual Fix

# 1. Stop the requirement processor
docker compose stop requirement-processor

# 2. Clean up failed migration records
PGPASSWORD="password" psql -h localhost -p 5432 -U postgres -d dev_pipeline << 'EOF'
DELETE FROM schema_migrations WHERE service = 'requirement-processor' OR version LIKE '%.sql';
EOF

# 3. Restart the service
docker compose up -d requirement-processor

# 4. Check status
docker compose ps requirement-processor

Option 3: Full Redeploy

# Stop all services
docker compose down

# Clean up database (if needed)
PGPASSWORD="password" psql -h localhost -p 5432 -U postgres -d dev_pipeline << 'EOF'
DELETE FROM schema_migrations WHERE service = 'requirement-processor';
EOF

# Start all services
docker compose up -d

Verification Steps

1. Check Service Status

   docker compose ps requirement-processor
   

2. Check Migration Records

   PGPASSWORD="password" psql -h localhost -p 5432 -U postgres -d dev_pipeline << 'EOF'
   SELECT service, version, applied_at, description 
   FROM schema_migrations 
   WHERE service = 'requirement-processor' 
   ORDER BY applied_at;
   EOF
   

3. Check Service Logs

   docker compose logs requirement-processor
   

4. Test Health Endpoint

   curl http://localhost:8001/health
   

Expected Results

After the fix:

• ✅ Requirement processor service should start successfully
• ✅ Migration records should show proper service field
• ✅ Health endpoint should return 200 OK
• ✅ All other services should continue running normally

Prevention

To prevent this issue in the future:

1. Always ensure migration scripts use the correct schema_migrations table schema
2. Include service field in all migration tracking
3. Test migrations in development before deploying to production
4. Use the shared migration system consistently across all services

Troubleshooting

If the issue persists:

1. Check database connectivity
2. Verify PostgreSQL is running
3. Check disk space and memory
4. Review all service logs
5. Consider a full database reset if necessary

Files Modified

• services/requirement-processor/migrations/migrate.py - Updated migration system
• services/requirement-processor/migrations/001_business_context_tables.sql - Removed FK constraint
• scripts/fix-requirement-processor-migration.sh - Created fix script