backend changes

GIT_INTEGRATION_FIX.md

@@ -0,0 +1,132 @@
+# Git Integration Service Fix - Build #27 Failure
+
+## 🚨 Issue Summary
+The git-integration service is failing with permission errors when trying to create the `/app/git-repos/diffs` directory. This is happening because the volume mount from the host doesn't have the correct ownership for the container user.
+
+## 🔧 Root Cause
+- **Error**: `EACCES: permission denied, mkdir '/app/git-repos/diffs'`
+- **Cause**: Host directory `/home/ubuntu/codenuk-backend-live/git-repos` doesn't exist or has wrong ownership
+- **Container User**: git-integration (UID 1001)
+- **Required**: Directory must be owned by UID 1001 to match container user
+
+## 🚀 **IMMEDIATE FIX - Run on Server**
+
+SSH to your server and run the fix script:
+
+```bash
+# SSH to the server
+ssh ubuntu@160.187.166.39
+
+# Navigate to the project directory
+cd /home/ubuntu/codenuk-backend-live
+
+# Run the fix script
+./scripts/server-fix-git-integration.sh
+```
+
+## 📋 **Manual Fix Steps** (if script doesn't work)
+
+If the automated script fails, run these commands manually:
+
+```bash
+# 1. Stop the failing service
+docker compose stop git-integration
+docker compose rm -f git-integration
+
+# 2. Create directories with proper permissions
+mkdir -p git-repos/diffs
+sudo chown -R 1001:1001 git-repos/
+chmod -R 755 git-repos/
+
+# 3. Verify permissions
+ls -la git-repos/
+
+# 4. Rebuild and restart service
+docker compose build --no-cache git-integration
+docker compose up -d git-integration
+
+# 5. Check service status
+docker compose ps git-integration
+docker compose logs git-integration
+```
+
+## 🔍 **Verification Steps**
+
+After running the fix, verify the service is working:
+
+```bash
+# Check service status
+docker compose ps git-integration
+
+# Check service health
+curl http://localhost:8012/health
+
+# Check logs for any errors
+docker compose logs --tail=50 git-integration
+```
+
+## 📊 **Expected Results**
+
+After the fix, you should see:
+- ✅ git-integration service status: `Up`
+- ✅ Health check returns HTTP 200
+- ✅ No permission errors in logs
+- ✅ Service starts successfully
+
+## 🛠️ **What Was Fixed**
+
+### 1. **Updated Dockerfile** (`services/git-integration/Dockerfile`)
+- Added better error handling in entrypoint script
+- Added logging to show permission fix attempts
+- Uses `su-exec` to properly switch users after fixing permissions
+
+### 2. **Created Fix Scripts**
+- `scripts/server-fix-git-integration.sh`: Comprehensive server-side fix
+- `scripts/setup-git-repos-directories.sh`: Simple directory setup
+- `scripts/fix-git-integration-deployment.sh`: Full deployment fix
+
+### 3. **Directory Structure**
+```
+/home/ubuntu/codenuk-backend-live/
+├── git-repos/           # Owner: 1001:1001, Permissions: 755
+│   └── diffs/          # Owner: 1001:1001, Permissions: 755
+└── docker-compose.yml
+```
+
+## 🚨 **If Still Failing**
+
+If the service still fails after running the fix:
+
+1. **Check Docker logs**:
+   ```bash
+   docker compose logs git-integration
+   ```
+
+2. **Check directory permissions**:
+   ```bash
+   ls -la git-repos/
+   stat git-repos/diffs/
+   ```
+
+3. **Verify container user**:
+   ```bash
+   docker compose exec git-integration id
+   ```
+
+4. **Check volume mount**:
+   ```bash
+   docker compose exec git-integration ls -la /app/git-repos/
+   ```
+
+## 📞 **Support**
+
+If you continue to experience issues:
+1. Run the verification steps above
+2. Collect the output from all commands
+3. Check the Jenkins build logs at: http://160.187.166.94:8080/job/codenuk-backend-live/27/console
+
+---
+
+**Last Updated**: October 2, 2025  
+**Build**: #27  
+**Status**: Fix Ready ✅

scripts/server-fix-git-integration.sh

@@ -0,0 +1,96 @@
+#!/bin/bash
+
+# Server-side script to fix git-integration deployment issues
+# Run this script on ubuntu@160.187.166.39
+
+set -e
+
+echo "🚀 Fixing git-integration service deployment on server..."
+echo "============================================================"
+
+# Get current directory
+CURRENT_DIR=$(pwd)
+echo "📍 Current directory: $CURRENT_DIR"
+
+# Check if we're in the right directory
+if [[ ! -f "docker-compose.yml" ]]; then
+    echo "❌ Error: docker-compose.yml not found. Please run this script from the codenuk-backend-live directory."
+    echo "Expected path: /home/ubuntu/codenuk-backend-live"
+    exit 1
+fi
+
+echo "✅ Found docker-compose.yml - proceeding with fix..."
+
+# Step 1: Stop the failing git-integration service
+echo ""
+echo "🛑 Step 1: Stopping git-integration service..."
+docker compose stop git-integration 2>/dev/null || true
+docker compose rm -f git-integration 2>/dev/null || true
+
+# Step 2: Create the git-repos directory structure
+echo ""
+echo "📁 Step 2: Creating git-repos directory structure..."
+mkdir -p git-repos
+mkdir -p git-repos/diffs
+
+# Step 3: Set proper ownership and permissions
+echo ""
+echo "👤 Step 3: Setting proper ownership and permissions..."
+echo "Setting ownership to 1001:1001 (matches container user)..."
+sudo chown -R 1001:1001 git-repos/
+echo "Setting permissions to 755..."
+chmod -R 755 git-repos/
+
+# Step 4: Verify the directory setup
+echo ""
+echo "✅ Step 4: Verifying directory setup..."
+echo "Directory listing:"
+ls -la git-repos/
+echo ""
+echo "Permissions check:"
+stat git-repos/
+stat git-repos/diffs/
+
+# Step 5: Rebuild the git-integration service
+echo ""
+echo "🔨 Step 5: Rebuilding git-integration service..."
+docker compose build --no-cache git-integration
+
+# Step 6: Start the git-integration service
+echo ""
+echo "🚀 Step 6: Starting git-integration service..."
+docker compose up -d git-integration
+
+# Step 7: Wait for service to start
+echo ""
+echo "⏳ Step 7: Waiting for service to start (30 seconds)..."
+sleep 30
+
+# Step 8: Check service status
+echo ""
+echo "🏥 Step 8: Checking service status..."
+echo "Service status:"
+docker compose ps git-integration
+
+echo ""
+echo "Service health check:"
+docker compose exec git-integration curl -f http://localhost:8012/health 2>/dev/null || echo "Health check failed - service may still be starting"
+
+# Step 9: Show recent logs
+echo ""
+echo "📋 Step 9: Recent service logs:"
+docker compose logs --tail=30 git-integration
+
+echo ""
+echo "============================================================"
+echo "🎉 Git-integration service fix completed!"
+echo "============================================================"
+echo ""
+echo "✅ Directories created with proper permissions"
+echo "✅ Service rebuilt and restarted"
+echo ""
+echo "If the service is still failing, check the logs with:"
+echo "docker compose logs git-integration"
+echo ""
+echo "To check if the service is healthy:"
+echo "curl http://localhost:8012/health"

services/git-integration/Dockerfile

@@ -26,10 +26,15 @@ RUN chmod -R 755 /app/git-repos
 # Create entrypoint script to handle volume permissions
 RUN echo '#!/bin/sh' > /app/entrypoint.sh && \
     echo '# Fix volume mount permissions' >> /app/entrypoint.sh && \
+    echo 'echo "🔧 Fixing git-repos directory permissions..."' >> /app/entrypoint.sh && \
     echo 'mkdir -p /app/git-repos/diffs' >> /app/entrypoint.sh && \
-    echo 'chown -R git-integration:nodejs /app/git-repos 2>/dev/null || true' >> /app/entrypoint.sh && \
+    echo 'chown -R git-integration:nodejs /app/git-repos 2>/dev/null || echo "⚠️  Could not change ownership (expected in some environments)"' >> /app/entrypoint.sh && \
-    echo 'chmod -R 755 /app/git-repos 2>/dev/null || true' >> /app/entrypoint.sh && \
+    echo 'chmod -R 755 /app/git-repos 2>/dev/null || echo "⚠️  Could not change permissions (expected in some environments)"' >> /app/entrypoint.sh && \
+    echo 'echo "✅ Directory setup completed"' >> /app/entrypoint.sh && \
+    echo 'echo "📁 Directory listing:"' >> /app/entrypoint.sh && \
+    echo 'ls -la /app/git-repos/ 2>/dev/null || echo "Could not list directory"' >> /app/entrypoint.sh && \
     echo '# Switch to git-integration user and execute command' >> /app/entrypoint.sh && \
+    echo 'echo "🚀 Starting git-integration service as user git-integration..."' >> /app/entrypoint.sh && \
     echo 'exec su-exec git-integration "$@"' >> /app/entrypoint.sh && \
     chmod +x /app/entrypoint.sh
 

services/git-integration/src/migrations/019_add_provider_name_to_repository_tables.sql

@@ -0,0 +1,57 @@
+-- Migration 019: Add provider_name column to repository tables
+-- This migration adds provider_name column to repository-related tables for multi-provider support
+
+-- Add provider_name column to repository_commit_details table
+ALTER TABLE repository_commit_details 
+ADD COLUMN IF NOT EXISTS provider_name VARCHAR(50) DEFAULT 'github' NOT NULL;
+
+-- Add provider_name column to repository_commit_files table
+ALTER TABLE repository_commit_files 
+ADD COLUMN IF NOT EXISTS provider_name VARCHAR(50) DEFAULT 'github' NOT NULL;
+
+-- Add provider_name column to repository_directories table
+ALTER TABLE repository_directories 
+ADD COLUMN IF NOT EXISTS provider_name VARCHAR(50) DEFAULT 'github' NOT NULL;
+
+-- Add provider_name column to repository_files table
+ALTER TABLE repository_files 
+ADD COLUMN IF NOT EXISTS provider_name VARCHAR(50) DEFAULT 'github' NOT NULL;
+
+-- Add provider_name column to repository_storage table
+ALTER TABLE repository_storage 
+ADD COLUMN IF NOT EXISTS provider_name VARCHAR(50) DEFAULT 'github' NOT NULL;
+
+-- Create indexes for provider_name columns for better query performance
+CREATE INDEX IF NOT EXISTS idx_repository_commit_details_provider_name ON repository_commit_details(provider_name);
+CREATE INDEX IF NOT EXISTS idx_repository_commit_files_provider_name ON repository_commit_files(provider_name);
+CREATE INDEX IF NOT EXISTS idx_repository_directories_provider_name ON repository_directories(provider_name);
+CREATE INDEX IF NOT EXISTS idx_repository_files_provider_name ON repository_files(provider_name);
+CREATE INDEX IF NOT EXISTS idx_repository_storage_provider_name ON repository_storage(provider_name);
+
+-- Add comments to document the column purpose
+COMMENT ON COLUMN repository_commit_details.provider_name IS 'Repository provider (github, gitlab, bitbucket, etc.)';
+COMMENT ON COLUMN repository_commit_files.provider_name IS 'Repository provider (github, gitlab, bitbucket, etc.)';
+COMMENT ON COLUMN repository_directories.provider_name IS 'Repository provider (github, gitlab, bitbucket, etc.)';
+COMMENT ON COLUMN repository_files.provider_name IS 'Repository provider (github, gitlab, bitbucket, etc.)';
+COMMENT ON COLUMN repository_storage.provider_name IS 'Repository provider (github, gitlab, bitbucket, etc.)';
+
+-- Update existing records to have 'github' as provider_name (if any exist without it)
+UPDATE repository_commit_details 
+SET provider_name = 'github' 
+WHERE provider_name IS NULL OR provider_name = '';
+
+UPDATE repository_commit_files 
+SET provider_name = 'github' 
+WHERE provider_name IS NULL OR provider_name = '';
+
+UPDATE repository_directories 
+SET provider_name = 'github' 
+WHERE provider_name IS NULL OR provider_name = '';
+
+UPDATE repository_files 
+SET provider_name = 'github' 
+WHERE provider_name IS NULL OR provider_name = '';
+
+UPDATE repository_storage 
+SET provider_name = 'github' 
+WHERE provider_name IS NULL OR provider_name = '';