backend changes

DEPLOYMENT_FIX_GUIDE.md

@@ -0,0 +1,171 @@
+# 🚀 Microservices Deployment Fix Guide
+
+## 🔍 Issues Identified
+
+### 1. N8N Service Failure (Exit Code 1)
+- **Root Cause**: Database schema conflicts and timing issues
+- **Symptoms**: `pipeline_n8n` container exits with code 1
+
+### 2. PostgreSQL Constraint Violations
+- **Root Cause**: Duplicate type/table creation attempts
+- **Error**: `duplicate key value violates unique constraint "pg_type_typname_nsp_index"`
+
+## 🛠️ Solutions Implemented
+
+### 1. Enhanced N8N Configuration
+- Added dedicated `n8n` schema
+- Improved health checks with longer start period
+- Added restart policy and better logging
+- Ensured proper dependency ordering
+
+### 2. Database Schema Cleanup
+- Created schema conflict resolution script
+- Proper table ownership and permissions
+- Separated n8n tables into dedicated schema
+
+### 3. Deployment Orchestration
+- Staged service startup to prevent race conditions
+- Proper dependency management
+- Volume cleanup for fresh starts
+
+## 🚀 Deployment Steps
+
+### Option 1: Automated Fix (Recommended)
+```bash
+cd /home/tech4biz/Desktop/Projectsnew/CODENUK1/codenuk-backend-live
+./scripts/fix-deployment-issues.sh
+```
+
+### Option 2: Manual Step-by-Step
+
+#### Step 1: Clean Environment
+```bash
+# Stop all services
+docker-compose down --volumes --remove-orphans
+
+# Clean Docker system
+docker system prune -f
+docker volume prune -f
+
+# Remove problematic volumes
+docker volume rm codenuk-backend-live_postgres_data 2>/dev/null || true
+docker volume rm codenuk-backend-live_n8n_data 2>/dev/null || true
+```
+
+#### Step 2: Start Core Infrastructure
+```bash
+# Start databases first
+docker-compose up -d postgres redis mongodb rabbitmq
+
+# Wait for readiness
+sleep 30
+```
+
+#### Step 3: Fix Database Schema
+```bash
+# Apply schema fixes
+docker exec -i pipeline_postgres psql -U pipeline_admin -d dev_pipeline < databases/scripts/fix-schema-conflicts.sql
+```
+
+#### Step 4: Run Migrations
+```bash
+docker-compose up migrations
+```
+
+#### Step 5: Start Services in Stages
+```bash
+# Stage 1: Core services
+docker-compose up -d n8n api-gateway requirement-processor
+
+# Stage 2: Generation services  
+docker-compose up -d tech-stack-selector architecture-designer code-generator
+
+# Stage 3: User services
+docker-compose up -d user-auth template-manager
+
+# Stage 4: Additional services
+docker-compose up -d ai-mockup-service git-integration web-dashboard
+```
+
+## 🏥 Health Verification
+
+### Check Service Status
+```bash
+docker-compose ps
+```
+
+### Check N8N Specifically
+```bash
+# Check n8n logs
+docker-compose logs n8n
+
+# Test n8n endpoint
+curl -f http://localhost:5678/healthz
+```
+
+### Check Database
+```bash
+# Connect to database
+docker exec -it pipeline_postgres psql -U pipeline_admin -d dev_pipeline
+
+# List schemas
+\dn
+
+# Check n8n tables
+\dt n8n.*
+```
+
+## 🔧 Troubleshooting
+
+### If N8N Still Fails
+1. Check logs: `docker-compose logs n8n`
+2. Verify database connection: `docker exec pipeline_postgres pg_isready`
+3. Check n8n schema exists: `docker exec -it pipeline_postgres psql -U pipeline_admin -d dev_pipeline -c "\dn"`
+
+### If Database Conflicts Persist
+1. Run schema cleanup again: `docker exec -i pipeline_postgres psql -U pipeline_admin -d dev_pipeline < databases/scripts/fix-schema-conflicts.sql`
+2. Check for remaining conflicts: `docker-compose logs postgres | grep ERROR`
+
+### If Services Won't Start
+1. Check dependencies: `docker-compose config --services`
+2. Start services individually: `docker-compose up [service-name]`
+3. Check resource usage: `docker stats`
+
+## 📊 Expected Results
+
+After successful deployment:
+- ✅ All services should show "Up" status
+- ✅ N8N accessible at http://localhost:5678
+- ✅ No database constraint errors
+- ✅ All health checks passing
+
+## 🎯 Key Improvements Made
+
+1. **N8N Configuration**:
+   - Dedicated schema isolation
+   - Better dependency management
+   - Improved health checks
+   - Restart policies
+
+2. **Database Management**:
+   - Schema conflict resolution
+   - Proper table ownership
+   - Clean migration process
+
+3. **Deployment Process**:
+   - Staged service startup
+   - Volume cleanup
+   - Dependency ordering
+
+## 📞 Support
+
+If issues persist:
+1. Check service logs: `docker-compose logs [service-name]`
+2. Verify network connectivity: `docker network ls`
+3. Check resource usage: `docker system df`
+4. Review configuration: `docker-compose config`
+
+---
+**Last Updated**: September 30, 2025  
+**Build Number**: 24+ (Fixed)  
+**Status**: ✅ Ready for Deployment

databases/scripts/fix-schema-conflicts.sql

@@ -0,0 +1,64 @@
+-- Fix Schema Conflicts and Prepare for Clean Deployment
+-- This script resolves duplicate key constraint violations
+
+-- Create n8n schema if it doesn't exist
+CREATE SCHEMA IF NOT EXISTS n8n;
+
+-- Clean up any existing conflicting types/tables
+DROP TYPE IF EXISTS claude_recommendations CASCADE;
+DROP TABLE IF EXISTS claude_recommendations CASCADE;
+
+-- Clean up n8n related tables if they exist in public schema
+DROP TABLE IF EXISTS public.n8n_credentials_entity CASCADE;
+DROP TABLE IF EXISTS public.n8n_execution_entity CASCADE;
+DROP TABLE IF EXISTS public.n8n_workflow_entity CASCADE;
+DROP TABLE IF EXISTS public.n8n_webhook_entity CASCADE;
+DROP TABLE IF EXISTS public.n8n_tag_entity CASCADE;
+DROP TABLE IF EXISTS public.n8n_workflows_tags CASCADE;
+
+-- Reset any conflicting sequences
+DROP SEQUENCE IF EXISTS claude_recommendations_id_seq CASCADE;
+
+-- Ensure proper permissions for n8n schema
+GRANT ALL PRIVILEGES ON SCHEMA n8n TO pipeline_admin;
+GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA n8n TO pipeline_admin;
+GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA n8n TO pipeline_admin;
+
+-- Create claude_recommendations table properly (if needed by services)
+CREATE TABLE IF NOT EXISTS claude_recommendations (
+    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    request_id VARCHAR(255) UNIQUE NOT NULL,
+    domain VARCHAR(100) NOT NULL,
+    budget DECIMAL(10,2) NOT NULL,
+    preferred_technologies TEXT[],
+    template_id UUID,
+    stack_name VARCHAR(255) NOT NULL,
+    monthly_cost DECIMAL(10,2) NOT NULL,
+    setup_cost DECIMAL(10,2) NOT NULL,
+    team_size VARCHAR(50) NOT NULL,
+    development_time INTEGER NOT NULL,
+    satisfaction INTEGER NOT NULL CHECK (satisfaction >= 0 AND satisfaction <= 100),
+    success_rate INTEGER NOT NULL CHECK (success_rate >= 0 AND success_rate <= 100),
+    frontend VARCHAR(100) NOT NULL,
+    backend VARCHAR(100) NOT NULL,
+    database VARCHAR(100) NOT NULL,
+    cloud VARCHAR(100) NOT NULL,
+    testing VARCHAR(100) NOT NULL,
+    mobile VARCHAR(100),
+    devops VARCHAR(100) NOT NULL,
+    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Create indexes for better performance
+CREATE INDEX IF NOT EXISTS idx_claude_recommendations_request_id ON claude_recommendations(request_id);
+CREATE INDEX IF NOT EXISTS idx_claude_recommendations_domain ON claude_recommendations(domain);
+CREATE INDEX IF NOT EXISTS idx_claude_recommendations_created_at ON claude_recommendations(created_at);
+
+-- Ensure proper ownership
+ALTER TABLE claude_recommendations OWNER TO pipeline_admin;
+
+-- Log the cleanup
+INSERT INTO schema_migrations (service, version, description, applied_at) 
+VALUES ('database-cleanup', 'fix_schema_conflicts', 'Fixed schema conflicts and prepared for clean deployment', CURRENT_TIMESTAMP)
+ON CONFLICT (service, version) DO NOTHING;

docker-compose.yml

@@ -703,7 +703,7 @@ services:
       - HOST=0.0.0.0
       - DATABASE_URL=postgresql://pipeline_admin:secure_pipeline_2024@postgres:5432/dev_pipeline
       - CLAUDE_API_KEY=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
-      - REDIS_URL=redis://:redis_secure_2024@pipeline_redis:6379
+      - REDIS_URL=redis://:redis_secure_2024@redis:6379
       - SERVICE_PORT=8007
       - LOG_LEVEL=INFO
       - DEFAULT_TARGET_QUALITY=0.85
@@ -726,9 +726,9 @@ services:
     healthcheck:
       test: ["CMD", "curl", "-f", "http://localhost:8007/health"]
       interval: 30s
-      timeout: 10s
+      timeout: 15s
-      retries: 3
+      retries: 5
-      start_period: 40s
+      start_period: 120s
   # =====================================
   # Workflow Orchestration
   # =====================================
@@ -795,6 +795,9 @@ services:
       - DB_POSTGRESDB_DATABASE=dev_pipeline
       - DB_POSTGRESDB_USER=pipeline_admin
       - DB_POSTGRESDB_PASSWORD=secure_pipeline_2024
+      - DB_POSTGRESDB_SCHEMA=n8n
+      - N8N_LOG_LEVEL=info
+      - N8N_METRICS=true
     volumes:
       - n8n_data:/home/node/.n8n
       - ./orchestration/n8n/workflows:/home/node/.n8n/workflows
@@ -807,12 +810,15 @@ services:
         condition: service_healthy
       rabbitmq:
         condition: service_healthy
+      migrations:
+        condition: service_completed_successfully
     healthcheck:
       test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:5678/healthz"]
       interval: 30s
       timeout: 10s
       retries: 5
       start_period: 60s
+    restart: unless-stopped
 
 # =====================================
 # Volumes

scripts/fix-deployment-issues.sh

@@ -0,0 +1,156 @@
+#!/bin/bash
+
+echo "🔧 Fixing Microservices Deployment Issues"
+echo "========================================"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Function to print colored output
+print_status() {
+    echo -e "${GREEN}[INFO]${NC} $1"
+}
+
+print_warning() {
+    echo -e "${YELLOW}[WARNING]${NC} $1"
+}
+
+print_error() {
+    echo -e "${RED}[ERROR]${NC} $1"
+}
+
+# 1. Stop all services
+print_status "Stopping all services..."
+docker-compose down --volumes --remove-orphans
+
+# 2. Clean up Docker system
+print_status "Cleaning up Docker system..."
+docker system prune -f
+docker volume prune -f
+
+# 3. Remove problematic volumes
+print_status "Removing problematic volumes..."
+docker volume rm codenuk-backend-live_postgres_data 2>/dev/null || true
+docker volume rm codenuk-backend-live_n8n_data 2>/dev/null || true
+docker volume rm codenuk-backend-live_migration_state 2>/dev/null || true
+
+# 4. Clean database schema conflicts
+print_status "Preparing clean database environment..."
+cat > /tmp/clean_db.sql << 'EOF'
+-- Clean up any existing schema conflicts
+DROP TYPE IF EXISTS claude_recommendations CASCADE;
+DROP TABLE IF EXISTS claude_recommendations CASCADE;
+
+-- Clean up n8n related tables if they exist
+DROP TABLE IF EXISTS n8n_credentials_entity CASCADE;
+DROP TABLE IF EXISTS n8n_execution_entity CASCADE;
+DROP TABLE IF EXISTS n8n_workflow_entity CASCADE;
+DROP TABLE IF EXISTS n8n_webhook_entity CASCADE;
+DROP TABLE IF EXISTS n8n_tag_entity CASCADE;
+DROP TABLE IF EXISTS n8n_workflows_tags CASCADE;
+
+-- Reset any conflicting sequences
+DROP SEQUENCE IF EXISTS claude_recommendations_id_seq CASCADE;
+EOF
+
+# 5. Start only core infrastructure first
+print_status "Starting core infrastructure services..."
+docker-compose up -d postgres redis mongodb rabbitmq
+
+# 6. Wait for databases to be ready
+print_status "Waiting for databases to be ready..."
+sleep 30
+
+# Check if postgres is ready
+print_status "Checking PostgreSQL readiness..."
+for i in {1..30}; do
+    if docker exec pipeline_postgres pg_isready -U pipeline_admin -d dev_pipeline; then
+        print_status "PostgreSQL is ready!"
+        break
+    fi
+    print_warning "Waiting for PostgreSQL... ($i/30)"
+    sleep 2
+done
+
+# 7. Clean the database
+print_status "Cleaning database schema conflicts..."
+docker exec -i pipeline_postgres psql -U pipeline_admin -d dev_pipeline < /tmp/clean_db.sql 2>/dev/null || true
+
+# 8. Run migrations
+print_status "Running database migrations..."
+docker-compose up migrations
+
+# Wait for migrations to complete
+print_status "Waiting for migrations to complete..."
+sleep 10
+
+# 9. Start n8n with proper initialization
+print_status "Starting n8n service..."
+docker-compose up -d n8n
+
+# Wait for n8n to initialize
+print_status "Waiting for n8n to initialize..."
+sleep 20
+
+# 10. Start remaining services in batches
+print_status "Starting core services..."
+docker-compose up -d \
+    api-gateway \
+    requirement-processor \
+    tech-stack-selector \
+    architecture-designer
+
+sleep 15
+
+print_status "Starting generation services..."
+docker-compose up -d \
+    code-generator \
+    test-generator \
+    deployment-manager
+
+sleep 15
+
+print_status "Starting user services..."
+docker-compose up -d \
+    user-auth \
+    template-manager \
+    unison
+
+sleep 15
+
+print_status "Starting additional services..."
+docker-compose up -d \
+    ai-mockup-service \
+    git-integration \
+    self-improving-generator \
+    web-dashboard
+
+# 11. Final health check
+print_status "Performing final health check..."
+sleep 30
+
+echo ""
+echo "🏥 Service Health Check"
+echo "======================"
+
+# Check service status
+docker-compose ps
+
+echo ""
+print_status "Deployment fix completed!"
+print_warning "Please check the service status above."
+print_warning "If any services are still failing, check their logs with:"
+print_warning "docker-compose logs [service-name]"
+
+# Clean up temp file
+rm -f /tmp/clean_db.sql
+
+echo ""
+print_status "🎯 Next Steps:"
+echo "1. Check service logs: docker-compose logs -f"
+echo "2. Verify n8n is accessible: http://localhost:5678"
+echo "3. Test API endpoints for health checks"
+echo "4. Monitor for any remaining issues"

self-improving-generator/Dockerfile

@@ -24,7 +24,7 @@ USER app
 EXPOSE 8007
 
 # Health check
-HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
+HEALTHCHECK --interval=30s --timeout=15s --start-period=120s --retries=5 \
     CMD curl -f http://localhost:8007/health || exit 1
 
 # Start the application

self-improving-generator/src/main.py

@@ -6,11 +6,13 @@ FastAPI application entry point for the self-improving code generator
 
 import logging
 import asyncio
+import time
 from contextlib import asynccontextmanager
 from fastapi import FastAPI, HTTPException, BackgroundTasks, Depends
 from fastapi.middleware.cors import CORSMiddleware
-from sqlalchemy import create_engine
+from sqlalchemy import create_engine, text
 from sqlalchemy.orm import sessionmaker
+from sqlalchemy.exc import OperationalError
 
 from .utils.config import get_settings, validate_configuration
 from .models.database_models import Base
@@ -29,6 +31,35 @@ generator: SelfImprovingCodeGenerator = None
 engine = None
 SessionLocal = None
 
+async def wait_for_database(database_url: str, max_retries: int = 30, delay: float = 2.0):
+    """Wait for database to be available with retry logic"""
+    for attempt in range(max_retries):
+        try:
+            logger.info(f"Attempting database connection (attempt {attempt + 1}/{max_retries})")
+            test_engine = create_engine(database_url)
+            
+            # Test the connection
+            with test_engine.connect() as conn:
+                conn.execute(text("SELECT 1"))
+            
+            logger.info("✅ Database connection successful")
+            test_engine.dispose()
+            return True
+            
+        except OperationalError as e:
+            if attempt < max_retries - 1:
+                logger.warning(f"Database connection failed (attempt {attempt + 1}): {e}")
+                logger.info(f"Retrying in {delay} seconds...")
+                await asyncio.sleep(delay)
+            else:
+                logger.error(f"Failed to connect to database after {max_retries} attempts: {e}")
+                raise
+        except Exception as e:
+            logger.error(f"Unexpected error connecting to database: {e}")
+            raise
+    
+    return False
+
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     """Application lifespan events"""
@@ -41,6 +72,9 @@ async def lifespan(app: FastAPI):
         
         logger.info("🚀 Starting Self-Improving Code Generator")
         
+        # Wait for database to be available
+        await wait_for_database(settings.database_url)
+        
         # Initialize database
         engine = create_engine(settings.database_url)
         SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
@@ -63,6 +97,8 @@ async def lifespan(app: FastAPI):
         raise
     finally:
         logger.info("🛑 Shutting down Self-Improving Code Generator")
+        if engine:
+            engine.dispose()
 
 # Create FastAPI app
 app = FastAPI(
@@ -122,24 +158,37 @@ async def root():
 @app.get("/health")
 async def health_check():
     """Health check endpoint"""
+    try:
         settings = get_settings()
         
+        # Test database connection
+        db_status = "disconnected"
+        if engine:
+            try:
+                with engine.connect() as conn:
+                    conn.execute(text("SELECT 1"))
+                db_status = "connected"
+            except Exception as e:
+                logger.warning(f"Database health check failed: {e}")
+                db_status = "error"
+        
         health_status = {
             "status": "healthy",
             "service": "Self-Improving Code Generator",
             "version": "1.0.0",
-        "timestamp": "2024-01-01T00:00:00Z",
+            "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
             "dependencies": {
-            "database": "connected" if engine else "disconnected",
+                "database": db_status,
                 "claude_api": "configured" if settings.claude_api_key else "not_configured",
                 "generator": "initialized" if generator else "not_initialized"
             }
         }
         
         # Check if all dependencies are healthy
-    all_healthy = all(
+        all_healthy = (
-        status == "connected" or status == "configured" or status == "initialized"
+            health_status["dependencies"]["database"] == "connected" and
-        for status in health_status["dependencies"].values()
+            health_status["dependencies"]["claude_api"] == "configured" and
+            health_status["dependencies"]["generator"] == "initialized"
         )
         
         if not all_healthy:
@@ -147,6 +196,16 @@ async def health_check():
         
         return health_status
         
+    except Exception as e:
+        logger.error(f"Health check failed: {e}")
+        return {
+            "status": "error",
+            "service": "Self-Improving Code Generator",
+            "version": "1.0.0",
+            "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+            "error": str(e)
+        }
+
 if __name__ == "__main__":
     import uvicorn
     settings = get_settings()

services/git-integration/package-lock.json

@@ -9,6 +9,7 @@
             "version": "1.0.0",
             "dependencies": {
                 "@octokit/rest": "^20.0.2",
+                "axios": "^1.12.2",
                 "cors": "^2.8.5",
                 "dotenv": "^16.3.1",
                 "express": "^4.18.2",
@@ -211,6 +212,23 @@
             "integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
             "license": "MIT"
         },
+        "node_modules/asynckit": {
+            "version": "0.4.0",
+            "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
+            "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
+            "license": "MIT"
+        },
+        "node_modules/axios": {
+            "version": "1.12.2",
+            "resolved": "https://registry.npmjs.org/axios/-/axios-1.12.2.tgz",
+            "integrity": "sha512-vMJzPewAlRyOgxV2dU0Cuz2O8zzzx9VYtbJOaBgXFeLc4IV/Eg50n4LowmehOOR61S8ZMpc2K5Sa7g6A4jfkUw==",
+            "license": "MIT",
+            "dependencies": {
+                "follow-redirects": "^1.15.6",
+                "form-data": "^4.0.4",
+                "proxy-from-env": "^1.1.0"
+            }
+        },
         "node_modules/balanced-match": {
             "version": "1.0.2",
             "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
@@ -366,6 +384,18 @@
                 "fsevents": "~2.3.2"
             }
         },
+        "node_modules/combined-stream": {
+            "version": "1.0.8",
+            "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
+            "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
+            "license": "MIT",
+            "dependencies": {
+                "delayed-stream": "~1.0.0"
+            },
+            "engines": {
+                "node": ">= 0.8"
+            }
+        },
         "node_modules/concat-map": {
             "version": "0.0.1",
             "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
@@ -431,6 +461,15 @@
                 "ms": "2.0.0"
             }
         },
+        "node_modules/delayed-stream": {
+            "version": "1.0.0",
+            "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
+            "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=0.4.0"
+            }
+        },
         "node_modules/depd": {
             "version": "2.0.0",
             "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@ -527,6 +566,21 @@
                 "node": ">= 0.4"
             }
         },
+        "node_modules/es-set-tostringtag": {
+            "version": "2.1.0",
+            "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
+            "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
+            "license": "MIT",
+            "dependencies": {
+                "es-errors": "^1.3.0",
+                "get-intrinsic": "^1.2.6",
+                "has-tostringtag": "^1.0.2",
+                "hasown": "^2.0.2"
+            },
+            "engines": {
+                "node": ">= 0.4"
+            }
+        },
         "node_modules/escape-html": {
             "version": "1.0.3",
             "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
@@ -653,6 +707,42 @@
                 "node": ">= 0.8"
             }
         },
+        "node_modules/follow-redirects": {
+            "version": "1.15.11",
+            "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz",
+            "integrity": "sha512-deG2P0JfjrTxl50XGCDyfI97ZGVCxIpfKYmfyrQ54n5FO/0gfIES8C/Psl6kWVDolizcaaxZJnTS0QSMxvnsBQ==",
+            "funding": [
+                {
+                    "type": "individual",
+                    "url": "https://github.com/sponsors/RubenVerborgh"
+                }
+            ],
+            "license": "MIT",
+            "engines": {
+                "node": ">=4.0"
+            },
+            "peerDependenciesMeta": {
+                "debug": {
+                    "optional": true
+                }
+            }
+        },
+        "node_modules/form-data": {
+            "version": "4.0.4",
+            "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.4.tgz",
+            "integrity": "sha512-KrGhL9Q4zjj0kiUt5OO4Mr/A/jlI2jDYs5eHBpYHPcBEVSiipAvn2Ko2HnPe20rmcuuvMHNdZFp+4IlGTMF0Ow==",
+            "license": "MIT",
+            "dependencies": {
+                "asynckit": "^0.4.0",
+                "combined-stream": "^1.0.8",
+                "es-set-tostringtag": "^2.1.0",
+                "hasown": "^2.0.2",
+                "mime-types": "^2.1.12"
+            },
+            "engines": {
+                "node": ">= 6"
+            }
+        },
         "node_modules/forwarded": {
             "version": "0.2.0",
             "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz",
@@ -779,6 +869,21 @@
                 "url": "https://github.com/sponsors/ljharb"
             }
         },
+        "node_modules/has-tostringtag": {
+            "version": "1.0.2",
+            "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
+            "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
+            "license": "MIT",
+            "dependencies": {
+                "has-symbols": "^1.0.3"
+            },
+            "engines": {
+                "node": ">= 0.4"
+            },
+            "funding": {
+                "url": "https://github.com/sponsors/ljharb"
+            }
+        },
         "node_modules/hasown": {
             "version": "2.0.2",
             "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
@@ -1317,6 +1422,12 @@
                 "node": ">= 0.10"
             }
         },
+        "node_modules/proxy-from-env": {
+            "version": "1.1.0",
+            "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
+            "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
+            "license": "MIT"
+        },
         "node_modules/pstree.remy": {
             "version": "1.1.8",
             "resolved": "https://registry.npmjs.org/pstree.remy/-/pstree.remy-1.1.8.tgz",

services/git-integration/package.json

@@ -10,6 +10,7 @@
     },
     "dependencies": {
         "@octokit/rest": "^20.0.2",
+        "axios": "^1.12.2",
         "cors": "^2.8.5",
         "dotenv": "^16.3.1",
         "express": "^4.18.2",

services/git-integration/src/app.js

@@ -8,12 +8,20 @@ const morgan = require('morgan');
 // Import database
 const database = require('./config/database');
 
+// Import services
+const EnhancedDiffProcessingService = require('./services/enhanced-diff-processing.service');
+
 // Import routes
 const githubRoutes = require('./routes/github-integration.routes');
 const githubOAuthRoutes = require('./routes/github-oauth');
 const webhookRoutes = require('./routes/webhook.routes');
 const vcsRoutes = require('./routes/vcs.routes');
 
+// Import new enhanced routes
+const commitsRoutes = require('./routes/commits.routes');
+const oauthProvidersRoutes = require('./routes/oauth-providers.routes');
+const enhancedWebhooksRoutes = require('./routes/enhanced-webhooks.routes');
+
 const app = express();
 const PORT = process.env.PORT || 8012;
 
@@ -21,8 +29,21 @@ const PORT = process.env.PORT || 8012;
 app.use(helmet());
 app.use(cors());
 app.use(morgan('combined'));
-app.use(express.json({ limit: '10mb' }));
+// Preserve raw body for webhook signature verification
-app.use(express.urlencoded({ extended: true }));
+
+app.use(express.json({
+  limit: '10mb',
+  verify: (req, res, buf) => {
+    req.rawBody = buf;
+  }
+}));
+
+app.use(express.urlencoded({
+  extended: true,
+  verify: (req, res, buf) => {
+    req.rawBody = buf;
+  }
+}));
 
 // Session middleware
 app.use(session({
@@ -42,6 +63,11 @@ app.use('/api/github', githubOAuthRoutes);
 app.use('/api/github', webhookRoutes);
 app.use('/api/vcs', vcsRoutes);
 
+// Enhanced routes
+app.use('/api/commits', commitsRoutes);
+app.use('/api/oauth', oauthProvidersRoutes);
+app.use('/api/webhooks', enhancedWebhooksRoutes);
+
 // Health check endpoint
 app.get('/health', (req, res) => {
   res.status(200).json({
@@ -63,7 +89,10 @@ app.get('/', (req, res) => {
       github: '/api/github',
       oauth: '/api/github/auth',
       webhook: '/api/github/webhook',
-      vcs: '/api/vcs/:provider'
+      vcs: '/api/vcs/:provider',
+      commits: '/api/commits',
+      oauth_providers: '/api/oauth',
+      enhanced_webhooks: '/api/webhooks'
     }
   });
 });
@@ -99,11 +128,37 @@ process.on('SIGINT', async () => {
   process.exit(0);
 });
 
+// Initialize services
+async function initializeServices() {
+  try {
+    // Initialize diff processing service
+    const diffProcessingService = new EnhancedDiffProcessingService();
+    await diffProcessingService.initialize();
+    
+    // Start background diff processing if enabled
+    if (process.env.ENABLE_BACKGROUND_DIFF_PROCESSING !== 'false') {
+      const processingInterval = parseInt(process.env.DIFF_PROCESSING_INTERVAL_MS) || 30000;
+      diffProcessingService.startBackgroundProcessing(processingInterval);
+      console.log('🔄 Background diff processing started');
+    }
+    
+    console.log('✅ All services initialized successfully');
+  } catch (error) {
+    console.error('❌ Error initializing services:', error);
+  }
+}
+
 // Start server
-app.listen(PORT, '0.0.0.0', () => {
+app.listen(PORT, '0.0.0.0', async () => {
   console.log(`🚀 Git Integration Service running on port ${PORT}`);
   console.log(`📊 Health check: http://localhost:${PORT}/health`);
   console.log(`🔗 GitHub API: http://localhost:${PORT}/api/github`);
+  console.log(`📝 Commits API: http://localhost:${PORT}/api/commits`);
+  console.log(`🔐 OAuth API: http://localhost:${PORT}/api/oauth`);
+  console.log(`🪝 Enhanced Webhooks: http://localhost:${PORT}/api/webhooks`);
+  
+  // Initialize services after server starts
+  await initializeServices();
 });
 
 module.exports = app;

services/git-integration/src/migrations/001_github_integration.sql

@@ -2,7 +2,7 @@
 -- This migration adds support for GitHub repository integration
 
 -- Create table for GitHub repositories
-CREATE TABLE IF NOT EXISTS github_repositories (
+CREATE TABLE IF NOT EXISTS "github_repositories@migrations/" (
   id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
   template_id UUID REFERENCES templates(id) ON DELETE CASCADE,
   repository_url VARCHAR(500) NOT NULL,
@@ -21,13 +21,13 @@ CREATE TABLE IF NOT EXISTS github_repositories (
 
 
 -- Create indexes for better performance
-CREATE INDEX IF NOT EXISTS idx_github_repos_template_id ON github_repositories(template_id);
+CREATE INDEX IF NOT EXISTS idx_github_repos_template_id ON "github_repositories@migrations/"(template_id);
-CREATE INDEX IF NOT EXISTS idx_github_repos_owner_name ON github_repositories(owner_name);
+CREATE INDEX IF NOT EXISTS idx_github_repos_owner_name ON "github_repositories@migrations/"(owner_name);
 CREATE INDEX IF NOT EXISTS idx_feature_mappings_feature_id ON feature_codebase_mappings(feature_id);
 CREATE INDEX IF NOT EXISTS idx_feature_mappings_repo_id ON feature_codebase_mappings(repository_id);
 
 -- Add trigger to update timestamp
-CREATE TRIGGER update_github_repos_updated_at BEFORE UPDATE ON github_repositories
+CREATE TRIGGER update_github_repos_updated_at BEFORE UPDATE ON "github_repositories@migrations/"
     FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
 
 -- =============================================

services/git-integration/src/migrations/002_repository_file_storage.sql

@@ -4,7 +4,7 @@
 -- Create table for repository local storage tracking
 CREATE TABLE IF NOT EXISTS repository_storage (
   id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
-  repository_id UUID REFERENCES github_repositories(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
   local_path TEXT NOT NULL,
   storage_status VARCHAR(50) DEFAULT 'pending', -- pending, downloading, completed, error
   total_files_count INTEGER DEFAULT 0,
@@ -21,7 +21,7 @@ CREATE TABLE IF NOT EXISTS repository_storage (
 -- Create table for directory structure
 CREATE TABLE IF NOT EXISTS repository_directories (
   id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
-  repository_id UUID REFERENCES github_repositories(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
   storage_id UUID REFERENCES repository_storage(id) ON DELETE CASCADE,
   parent_directory_id UUID REFERENCES repository_directories(id) ON DELETE CASCADE,
   directory_name VARCHAR(255) NOT NULL,
@@ -38,7 +38,7 @@ CREATE TABLE IF NOT EXISTS repository_directories (
 -- Create table for individual files
 CREATE TABLE IF NOT EXISTS repository_files (
   id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
-  repository_id UUID REFERENCES github_repositories(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
   storage_id UUID REFERENCES repository_storage(id) ON DELETE CASCADE,
   directory_id UUID REFERENCES repository_directories(id) ON DELETE SET NULL,
   filename VARCHAR(255) NOT NULL,

services/git-integration/src/migrations/003_add_user_id_to_template_refs.sql

@@ -2,12 +2,12 @@
 -- This ensures we always track which user owns/initiated records tied to a template
 
 -- Add user_id to github_repositories
-ALTER TABLE IF EXISTS github_repositories
+ALTER TABLE IF EXISTS "github_repositories@migrations/"
   ADD COLUMN IF NOT EXISTS user_id UUID REFERENCES users(id) ON DELETE CASCADE;
 
 -- Indexes for github_repositories
-CREATE INDEX IF NOT EXISTS idx_github_repos_user_id ON github_repositories(user_id);
+CREATE INDEX IF NOT EXISTS idx_github_repos_user_id ON "github_repositories@migrations/"(user_id);
-CREATE INDEX IF NOT EXISTS idx_github_repos_template_user ON github_repositories(template_id, user_id);
+CREATE INDEX IF NOT EXISTS idx_github_repos_template_user ON "github_repositories@migrations/"(template_id, user_id);
 
 -- Add user_id to feature_codebase_mappings
 ALTER TABLE IF EXISTS feature_codebase_mappings

services/git-integration/src/migrations/005_webhook_commits.sql

@@ -8,7 +8,7 @@ CREATE TABLE IF NOT EXISTS github_webhooks (
   action VARCHAR(100),
   owner_name VARCHAR(120),
   repository_name VARCHAR(200),
-  repository_id UUID REFERENCES github_repositories(id) ON DELETE SET NULL,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE SET NULL,
   ref VARCHAR(255),
   before_sha VARCHAR(64),
   after_sha VARCHAR(64),
@@ -26,7 +26,7 @@ CREATE INDEX IF NOT EXISTS idx_github_webhooks_event_type ON github_webhooks(eve
 -- Track commit SHA transitions per repository to detect changes over time
 CREATE TABLE IF NOT EXISTS repository_commit_events (
   id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
-  repository_id UUID REFERENCES github_repositories(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
   ref VARCHAR(255),
   before_sha VARCHAR(64),
   after_sha VARCHAR(64),

services/git-integration/src/migrations/006_commit_changes.sql

@@ -3,7 +3,7 @@
 -- Per-commit details linked to an attached repository
 CREATE TABLE IF NOT EXISTS repository_commit_details (
   id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
-  repository_id UUID REFERENCES github_repositories(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
   commit_sha VARCHAR(64) NOT NULL,
   author_name VARCHAR(200),
   author_email VARCHAR(320),

services/git-integration/src/migrations/007_add_last_synced_commit.sql

@@ -1,5 +1,5 @@
 -- 007_add_last_synced_commit.sql
-ALTER TABLE github_repositories
+ALTER TABLE "github_repositories@migrations/"
 ADD COLUMN IF NOT EXISTS last_synced_commit_sha VARCHAR(64),
 ADD COLUMN IF NOT EXISTS last_synced_at TIMESTAMP WITH TIME ZONE;
 

services/git-integration/src/migrations/009_provider_webhook_tables.sql

@@ -8,7 +8,7 @@ CREATE TABLE IF NOT EXISTS gitlab_webhooks (
   action TEXT,
   owner_name TEXT NOT NULL,
   repository_name TEXT NOT NULL,
-  repository_id UUID REFERENCES github_repositories(id),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id),
   ref TEXT,
   before_sha TEXT,
   after_sha TEXT,
@@ -26,7 +26,7 @@ CREATE TABLE IF NOT EXISTS bitbucket_webhooks (
   action TEXT,
   owner_name TEXT NOT NULL,
   repository_name TEXT NOT NULL,
-  repository_id UUID REFERENCES github_repositories(id),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id),
   ref TEXT,
   before_sha TEXT,
   after_sha TEXT,
@@ -44,7 +44,7 @@ CREATE TABLE IF NOT EXISTS gitea_webhooks (
   action TEXT,
   owner_name TEXT NOT NULL,
   repository_name TEXT NOT NULL,
-  repository_id UUID REFERENCES github_repositories(id),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id),
   ref TEXT,
   before_sha TEXT,
   after_sha TEXT,

services/git-integration/src/migrations/010_remove_template_id.sql

@@ -7,7 +7,7 @@ DROP INDEX IF EXISTS idx_github_repos_template_user;
 DROP INDEX IF EXISTS idx_feature_mappings_template_user;
 
 -- Remove template_id column from github_repositories table
-ALTER TABLE IF EXISTS github_repositories 
+ALTER TABLE IF EXISTS "github_repositories@migrations/" 
   DROP COLUMN IF EXISTS template_id;
 
 -- Remove template_id column from feature_codebase_mappings table

services/git-integration/src/migrations/012_add_user_id_to_github_repositories.sql

@@ -1,10 +1,10 @@
 -- Migration 012: Track which user attached/downloaded a repository
 
 -- Add user_id to github_repositories to associate records with the initiating user
-ALTER TABLE github_repositories
+ALTER TABLE "github_repositories@migrations/"
 ADD COLUMN IF NOT EXISTS user_id UUID REFERENCES users(id) ON DELETE SET NULL;
 
 -- Helpful index for filtering user-owned repositories
-CREATE INDEX IF NOT EXISTS idx_github_repositories_user_id ON github_repositories(user_id);
+CREATE INDEX IF NOT EXISTS idx_github_repositories_user_id ON "github_repositories@migrations/"(user_id);
 
 

services/git-integration/src/migrations/013_repository_commit_details.sql

@@ -0,0 +1,30 @@
+-- Migration 013: Repository Commit Details and Files
+-- This migration adds commit tracking tables that are missing from the current schema
+
+-- Per-commit details linked to an attached repository
+CREATE TABLE IF NOT EXISTS repository_commit_details (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  commit_sha VARCHAR(64) NOT NULL,
+  author_name VARCHAR(200),
+  author_email VARCHAR(320),
+  message TEXT,
+  url TEXT,
+  committed_at TIMESTAMP DEFAULT NOW(),
+  created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Per-file changes for each commit
+CREATE TABLE IF NOT EXISTS repository_commit_files (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  commit_id UUID REFERENCES repository_commit_details(id) ON DELETE CASCADE,
+  change_type VARCHAR(20) NOT NULL, -- added | modified | removed
+  file_path TEXT NOT NULL,
+  created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Create indexes for performance
+CREATE UNIQUE INDEX IF NOT EXISTS uq_repo_commit_sha ON repository_commit_details(repository_id, commit_sha);
+CREATE INDEX IF NOT EXISTS idx_repo_commit_created_at ON repository_commit_details(created_at);
+CREATE INDEX IF NOT EXISTS idx_commit_files_commit_id ON repository_commit_files(commit_id);
+CREATE INDEX IF NOT EXISTS idx_commit_files_path ON repository_commit_files(file_path);

services/git-integration/src/migrations/014_additional_oauth_providers.sql

@@ -0,0 +1,48 @@
+-- Migration 014: Additional OAuth Provider Tables
+-- This migration adds OAuth token tables for GitLab, Bitbucket, and Gitea
+
+-- GitLab OAuth tokens
+CREATE TABLE IF NOT EXISTS gitlab_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  gitlab_username TEXT,
+  gitlab_user_id TEXT,
+  scopes JSONB,
+  expires_at TIMESTAMP NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Bitbucket OAuth tokens
+CREATE TABLE IF NOT EXISTS bitbucket_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  bitbucket_username TEXT,
+  bitbucket_user_id TEXT,
+  scopes JSONB,
+  expires_at TIMESTAMP NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Gitea OAuth tokens
+CREATE TABLE IF NOT EXISTS gitea_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  gitea_username TEXT,
+  gitea_user_id TEXT,
+  scopes JSONB,
+  expires_at TIMESTAMP NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Add triggers for updated_at columns
+CREATE TRIGGER update_gitlab_user_tokens_updated_at BEFORE UPDATE ON gitlab_user_tokens
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_bitbucket_user_tokens_updated_at BEFORE UPDATE ON bitbucket_user_tokens
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_gitea_user_tokens_updated_at BEFORE UPDATE ON gitea_user_tokens
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();

services/git-integration/src/migrations/015_diff_storage_system.sql

@@ -0,0 +1,149 @@
+-- Migration 015: Git Diff Storage System
+-- This migration adds tables for storing git diffs with size-based storage strategy
+
+-- Store actual diff content with size-based strategy
+CREATE TABLE IF NOT EXISTS diff_contents (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  commit_id UUID REFERENCES repository_commit_details(id) ON DELETE CASCADE,
+  file_change_id UUID REFERENCES repository_commit_files(id) ON DELETE CASCADE,
+  
+  -- Diff metadata
+  diff_header TEXT, -- "diff --git a/file.py b/file.py"
+  diff_size_bytes INTEGER NOT NULL,
+  
+  -- Storage strategy based on size
+  storage_type VARCHAR(20) NOT NULL, -- 'external' (all diffs stored on disk)
+  
+  -- Store reference only (path on disk)
+  external_storage_path TEXT NOT NULL,
+  external_storage_provider VARCHAR(50) DEFAULT 'local', -- 'local', 's3', 'gcs'
+  
+  -- File information
+  file_path TEXT NOT NULL,
+  change_type VARCHAR(20) NOT NULL, -- 'added', 'modified', 'deleted', 'renamed'
+  
+  -- Processing status
+  processing_status VARCHAR(20) DEFAULT 'pending', -- 'pending', 'processed', 'failed'
+  processing_error TEXT,
+  
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Diff Processing Queue (for background processing)
+CREATE TABLE IF NOT EXISTS diff_processing_queue (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  commit_id UUID REFERENCES repository_commit_details(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  
+  -- Processing metadata
+  queue_status VARCHAR(20) DEFAULT 'pending', -- 'pending', 'processing', 'completed', 'failed'
+  priority INTEGER DEFAULT 0, -- Higher number = higher priority
+  retry_count INTEGER DEFAULT 0,
+  max_retries INTEGER DEFAULT 3,
+  
+  -- Git diff parameters
+  from_sha VARCHAR(64),
+  to_sha VARCHAR(64),
+  repo_local_path TEXT NOT NULL,
+  
+  -- Processing results
+  processed_at TIMESTAMP,
+  error_message TEXT,
+  
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Diff Statistics (for monitoring and optimization)
+CREATE TABLE IF NOT EXISTS diff_statistics (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  
+  -- Statistics period
+  period_start TIMESTAMP NOT NULL,
+  period_end TIMESTAMP NOT NULL,
+  
+  -- Count statistics
+  total_commits INTEGER DEFAULT 0,
+  total_files_changed INTEGER DEFAULT 0,
+  total_diffs_processed INTEGER DEFAULT 0,
+  
+  -- Size statistics
+  total_diff_size_bytes BIGINT DEFAULT 0,
+  avg_diff_size_bytes DECIMAL(10,2) DEFAULT 0,
+  max_diff_size_bytes BIGINT DEFAULT 0,
+  
+  -- Storage type breakdown
+  diffs_stored_external INTEGER DEFAULT 0,
+  
+  -- Performance metrics
+  avg_processing_time_ms DECIMAL(10,2) DEFAULT 0,
+  failed_processing_count INTEGER DEFAULT 0,
+  
+  created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Indexes for Performance
+CREATE INDEX IF NOT EXISTS idx_diff_contents_commit_id ON diff_contents(commit_id);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_file_change_id ON diff_contents(file_change_id);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_storage_type ON diff_contents(storage_type);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_file_path ON diff_contents(file_path);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_change_type ON diff_contents(change_type);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_processing_status ON diff_contents(processing_status);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_created_at ON diff_contents(created_at);
+
+CREATE INDEX IF NOT EXISTS idx_diff_queue_status ON diff_processing_queue(queue_status);
+CREATE INDEX IF NOT EXISTS idx_diff_queue_priority ON diff_processing_queue(priority DESC);
+CREATE INDEX IF NOT EXISTS idx_diff_queue_repository_id ON diff_processing_queue(repository_id);
+CREATE INDEX IF NOT EXISTS idx_diff_queue_created_at ON diff_processing_queue(created_at);
+
+CREATE INDEX IF NOT EXISTS idx_diff_stats_repository_id ON diff_statistics(repository_id);
+CREATE INDEX IF NOT EXISTS idx_diff_stats_period ON diff_statistics(period_start, period_end);
+
+-- Triggers for Updated At Columns
+CREATE TRIGGER update_diff_contents_updated_at BEFORE UPDATE ON diff_contents
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_diff_queue_updated_at BEFORE UPDATE ON diff_processing_queue
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+-- Helper Functions
+-- Function to get diff storage statistics for a repository
+CREATE OR REPLACE FUNCTION get_repository_diff_stats(repo_id UUID, days_back INTEGER DEFAULT 30)
+RETURNS TABLE (
+  total_diffs BIGINT,
+  total_size_bytes BIGINT,
+  avg_size_bytes DECIMAL(10,2),
+  storage_breakdown JSONB
+) AS $$
+BEGIN
+  RETURN QUERY
+  SELECT 
+    COUNT(*)::BIGINT as total_diffs,
+    COALESCE(SUM(dc.diff_size_bytes), 0)::BIGINT as total_size_bytes,
+    COALESCE(AVG(dc.diff_size_bytes), 0)::DECIMAL(10,2) as avg_size_bytes,
+    jsonb_build_object(
+      'external', COUNT(*) FILTER (WHERE dc.storage_type = 'external')
+    ) as storage_breakdown
+  FROM diff_contents dc
+  JOIN repository_commit_details rcd ON dc.commit_id = rcd.id
+  WHERE rcd.repository_id = repo_id
+    AND dc.created_at >= NOW() - INTERVAL '1 day' * days_back;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Function to clean up old diff processing queue entries
+CREATE OR REPLACE FUNCTION cleanup_old_diff_queue_entries(days_back INTEGER DEFAULT 7)
+RETURNS INTEGER AS $$
+DECLARE
+  deleted_count INTEGER;
+BEGIN
+  DELETE FROM diff_processing_queue 
+  WHERE created_at < NOW() - INTERVAL '1 day' * days_back
+    AND queue_status IN ('completed', 'failed');
+  
+  GET DIAGNOSTICS deleted_count = ROW_COUNT;
+  RETURN deleted_count;
+END;
+$$ LANGUAGE plpgsql;

services/git-integration/src/migrations/016_missing_columns_and_indexes.sql

@@ -0,0 +1,25 @@
+-- Migration 016: Missing Columns and Additional Indexes
+-- This migration adds missing columns and indexes from the provided migrations
+
+-- Add missing column to github_repositories if it doesn't exist
+ALTER TABLE "github_repositories@migrations/" 
+ADD COLUMN IF NOT EXISTS last_synced_commit_sha VARCHAR(64);
+
+-- Add missing ID column to repository_files if it doesn't exist
+ALTER TABLE repository_files 
+ADD COLUMN IF NOT EXISTS id UUID PRIMARY KEY DEFAULT uuid_generate_v4();
+
+-- Additional indexes for better performance that were missing
+CREATE INDEX IF NOT EXISTS idx_repo_directories_level ON repository_directories(level);
+CREATE INDEX IF NOT EXISTS idx_repo_directories_relative_path ON repository_directories(relative_path);
+
+CREATE INDEX IF NOT EXISTS idx_repo_files_extension ON repository_files(file_extension);
+CREATE INDEX IF NOT EXISTS idx_repo_files_filename ON repository_files(filename);
+CREATE INDEX IF NOT EXISTS idx_repo_files_relative_path ON repository_files(relative_path);
+CREATE INDEX IF NOT EXISTS idx_repo_files_is_binary ON repository_files(is_binary);
+
+-- Webhook indexes that might be missing
+CREATE INDEX IF NOT EXISTS idx_bitbucket_webhooks_event_type ON bitbucket_webhooks(event_type);
+CREATE INDEX IF NOT EXISTS idx_gitea_webhooks_repository_id ON gitea_webhooks(repository_id);
+CREATE INDEX IF NOT EXISTS idx_gitea_webhooks_created_at ON gitea_webhooks(created_at);
+CREATE INDEX IF NOT EXISTS idx_gitea_webhooks_event_type ON gitea_webhooks(event_type);

services/git-integration/src/migrations/017_complete_schema_from_provided_migrations.sql

@@ -0,0 +1,569 @@
+-- Migration 017: Complete Schema from Provided Migrations
+-- This migration creates all tables from the provided 001_github_integration.sql and 002_diff_storage.sql files
+
+-- =============================================
+-- Core Repository Tables
+-- =============================================
+
+-- Create table for GitHub repositories (enhanced version from provided migration)
+CREATE TABLE IF NOT EXISTS "github_repositories@migrations/" (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  template_id UUID, -- References templates(id) but table may not exist
+  repository_url VARCHAR(500) NOT NULL,
+  repository_name VARCHAR(200) NOT NULL,
+  owner_name VARCHAR(100) NOT NULL,
+  branch_name VARCHAR(100) DEFAULT 'main',
+  is_public BOOLEAN DEFAULT true,
+  requires_auth BOOLEAN DEFAULT false,
+  last_synced_at TIMESTAMP,
+  sync_status VARCHAR(50) DEFAULT 'pending',
+  metadata JSONB,
+  codebase_analysis JSONB,
+  last_synced_commit_sha VARCHAR(64),
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- =============================================
+-- Repository File Storage Tables
+-- =============================================
+
+-- Create table for repository local storage tracking
+CREATE TABLE IF NOT EXISTS repository_storage (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  local_path TEXT NOT NULL,
+  storage_status VARCHAR(50) DEFAULT 'pending', -- pending, downloading, completed, error
+  total_files_count INTEGER DEFAULT 0,
+  total_directories_count INTEGER DEFAULT 0,
+  total_size_bytes BIGINT DEFAULT 0,
+  download_started_at TIMESTAMP,
+  download_completed_at TIMESTAMP,
+  last_verified_at TIMESTAMP,
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW(),
+  UNIQUE(repository_id)
+);
+
+-- Create table for directory structure
+CREATE TABLE IF NOT EXISTS repository_directories (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  storage_id UUID REFERENCES repository_storage(id) ON DELETE CASCADE,
+  parent_directory_id UUID REFERENCES repository_directories(id) ON DELETE CASCADE,
+  directory_name VARCHAR(255) NOT NULL,
+  relative_path TEXT NOT NULL, -- path from repository root
+  absolute_path TEXT NOT NULL, -- full local filesystem path
+  level INTEGER DEFAULT 0, -- depth in hierarchy (0 = root)
+  files_count INTEGER DEFAULT 0,
+  subdirectories_count INTEGER DEFAULT 0,
+  total_size_bytes BIGINT DEFAULT 0,
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Create table for individual files
+CREATE TABLE IF NOT EXISTS repository_files (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  storage_id UUID REFERENCES repository_storage(id) ON DELETE CASCADE,
+  directory_id UUID REFERENCES repository_directories(id) ON DELETE SET NULL,
+  filename VARCHAR(255) NOT NULL,
+  file_extension VARCHAR(50),
+  relative_path TEXT NOT NULL, -- path from repository root
+  absolute_path TEXT NOT NULL, -- full local filesystem path
+  file_size_bytes BIGINT DEFAULT 0,
+  file_hash VARCHAR(64), -- SHA-256 hash for integrity
+  mime_type VARCHAR(100),
+  is_binary BOOLEAN DEFAULT false,
+  encoding VARCHAR(50) DEFAULT 'utf-8',
+  github_sha VARCHAR(40), -- GitHub blob SHA for tracking changes
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- =============================================
+-- Webhook Tables
+-- =============================================
+
+-- GitHub webhooks table
+CREATE TABLE IF NOT EXISTS github_webhooks (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  delivery_id VARCHAR(120),
+  event_type VARCHAR(100) NOT NULL,
+  action VARCHAR(100),
+  owner_name VARCHAR(120),
+  repository_name VARCHAR(200),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE SET NULL,
+  ref VARCHAR(255),
+  before_sha VARCHAR(64),
+  after_sha VARCHAR(64),
+  commit_count INTEGER,
+  payload JSONB NOT NULL,
+  processed_at TIMESTAMP,
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- GitLab webhooks table
+CREATE TABLE IF NOT EXISTS gitlab_webhooks (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  delivery_id TEXT,
+  event_type TEXT NOT NULL,
+  action TEXT,
+  owner_name TEXT NOT NULL,
+  repository_name TEXT NOT NULL,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id),
+  ref TEXT,
+  before_sha TEXT,
+  after_sha TEXT,
+  commit_count INTEGER DEFAULT 0,
+  payload JSONB,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Bitbucket webhooks table
+CREATE TABLE IF NOT EXISTS bitbucket_webhooks (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  delivery_id TEXT,
+  event_type TEXT NOT NULL,
+  action TEXT,
+  owner_name TEXT NOT NULL,
+  repository_name TEXT NOT NULL,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id),
+  ref TEXT,
+  before_sha TEXT,
+  after_sha TEXT,
+  commit_count INTEGER DEFAULT 0,
+  payload JSONB,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Gitea webhooks table
+CREATE TABLE IF NOT EXISTS gitea_webhooks (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  delivery_id TEXT,
+  event_type TEXT NOT NULL,
+  action TEXT,
+  owner_name TEXT NOT NULL,
+  repository_name TEXT NOT NULL,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id),
+  ref TEXT,
+  before_sha TEXT,
+  after_sha TEXT,
+  commit_count INTEGER DEFAULT 0,
+  payload JSONB,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- =============================================
+-- Commit Tracking Tables
+-- =============================================
+
+-- Per-commit details linked to an attached repository
+CREATE TABLE IF NOT EXISTS repository_commit_details (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  commit_sha VARCHAR(64) NOT NULL,
+  author_name VARCHAR(200),
+  author_email VARCHAR(320),
+  message TEXT,
+  url TEXT,
+  committed_at TIMESTAMP DEFAULT NOW(),
+  created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Per-file changes for each commit
+CREATE TABLE IF NOT EXISTS repository_commit_files (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  commit_id UUID REFERENCES repository_commit_details(id) ON DELETE CASCADE,
+  change_type VARCHAR(20) NOT NULL, -- added | modified | removed
+  file_path TEXT NOT NULL,
+  created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- =============================================
+-- OAuth Token Tables
+-- =============================================
+
+-- GitHub OAuth tokens
+CREATE TABLE IF NOT EXISTS github_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  github_username VARCHAR(100) NOT NULL,
+  github_user_id INTEGER NOT NULL,
+  scopes JSONB,
+  expires_at TIMESTAMP,
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- GitLab OAuth tokens
+CREATE TABLE IF NOT EXISTS gitlab_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  gitlab_username TEXT,
+  gitlab_user_id TEXT,
+  scopes JSONB,
+  expires_at TIMESTAMP NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Bitbucket OAuth tokens
+CREATE TABLE IF NOT EXISTS bitbucket_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  bitbucket_username TEXT,
+  bitbucket_user_id TEXT,
+  scopes JSONB,
+  expires_at TIMESTAMP NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- Gitea OAuth tokens
+CREATE TABLE IF NOT EXISTS gitea_user_tokens (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  access_token TEXT NOT NULL,
+  gitea_username TEXT,
+  gitea_user_id TEXT,
+  scopes JSONB,
+  expires_at TIMESTAMP NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+-- =============================================
+-- Diff Storage Tables (from 002_diff_storage.sql)
+-- =============================================
+
+-- Store actual diff content with size-based strategy
+CREATE TABLE IF NOT EXISTS diff_contents (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  commit_id UUID REFERENCES repository_commit_details(id) ON DELETE CASCADE,
+  file_change_id UUID REFERENCES repository_commit_files(id) ON DELETE CASCADE,
+  
+  -- Diff metadata
+  diff_header TEXT, -- "diff --git a/file.py b/file.py"
+  diff_size_bytes INTEGER NOT NULL,
+  
+  -- Storage strategy based on size
+  storage_type VARCHAR(20) NOT NULL, -- 'external' (all diffs stored on disk)
+  
+  -- Store reference only (path on disk)
+  external_storage_path TEXT NOT NULL,
+  external_storage_provider VARCHAR(50) DEFAULT 'local', -- 'local', 's3', 'gcs'
+  
+  -- File information
+  file_path TEXT NOT NULL,
+  change_type VARCHAR(20) NOT NULL, -- 'added', 'modified', 'deleted', 'renamed'
+  
+  -- Processing status
+  processing_status VARCHAR(20) DEFAULT 'pending', -- 'pending', 'processed', 'failed'
+  processing_error TEXT,
+  
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Diff Processing Queue (for background processing)
+CREATE TABLE IF NOT EXISTS diff_processing_queue (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  commit_id UUID REFERENCES repository_commit_details(id) ON DELETE CASCADE,
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  
+  -- Processing metadata
+  queue_status VARCHAR(20) DEFAULT 'pending', -- 'pending', 'processing', 'completed', 'failed'
+  priority INTEGER DEFAULT 0, -- Higher number = higher priority
+  retry_count INTEGER DEFAULT 0,
+  max_retries INTEGER DEFAULT 3,
+  
+  -- Git diff parameters
+  from_sha VARCHAR(64),
+  to_sha VARCHAR(64),
+  repo_local_path TEXT NOT NULL,
+  
+  -- Processing results
+  processed_at TIMESTAMP,
+  error_message TEXT,
+  
+  created_at TIMESTAMP DEFAULT NOW(),
+  updated_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Diff Statistics (for monitoring and optimization)
+CREATE TABLE IF NOT EXISTS diff_statistics (
+  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  repository_id UUID REFERENCES "github_repositories@migrations/"(id) ON DELETE CASCADE,
+  
+  -- Statistics period
+  period_start TIMESTAMP NOT NULL,
+  period_end TIMESTAMP NOT NULL,
+  
+  -- Count statistics
+  total_commits INTEGER DEFAULT 0,
+  total_files_changed INTEGER DEFAULT 0,
+  total_diffs_processed INTEGER DEFAULT 0,
+  
+  -- Size statistics
+  total_diff_size_bytes BIGINT DEFAULT 0,
+  avg_diff_size_bytes DECIMAL(10,2) DEFAULT 0,
+  max_diff_size_bytes BIGINT DEFAULT 0,
+  
+  -- Storage type breakdown
+  diffs_stored_external INTEGER DEFAULT 0,
+  
+  -- Performance metrics
+  avg_processing_time_ms DECIMAL(10,2) DEFAULT 0,
+  failed_processing_count INTEGER DEFAULT 0,
+  
+  created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- =============================================
+-- Indexes for Performance
+-- =============================================
+
+-- GitHub repositories indexes
+CREATE INDEX IF NOT EXISTS idx_github_repos_template_id ON "github_repositories@migrations/"(template_id);
+CREATE INDEX IF NOT EXISTS idx_github_repos_owner_name ON "github_repositories@migrations/"(owner_name);
+
+-- Repository storage indexes
+CREATE INDEX IF NOT EXISTS idx_repository_storage_repo_id ON repository_storage(repository_id);
+CREATE INDEX IF NOT EXISTS idx_repository_storage_status ON repository_storage(storage_status);
+
+-- Repository directories indexes
+CREATE INDEX IF NOT EXISTS idx_repo_directories_repo_id ON repository_directories(repository_id);
+CREATE INDEX IF NOT EXISTS idx_repo_directories_parent_id ON repository_directories(parent_directory_id);
+CREATE INDEX IF NOT EXISTS idx_repo_directories_storage_id ON repository_directories(storage_id);
+CREATE INDEX IF NOT EXISTS idx_repo_directories_level ON repository_directories(level);
+CREATE INDEX IF NOT EXISTS idx_repo_directories_relative_path ON repository_directories(relative_path);
+
+-- Repository files indexes
+CREATE INDEX IF NOT EXISTS idx_repo_files_repo_id ON repository_files(repository_id);
+CREATE INDEX IF NOT EXISTS idx_repo_files_directory_id ON repository_files(directory_id);
+CREATE INDEX IF NOT EXISTS idx_repo_files_storage_id ON repository_files(storage_id);
+CREATE INDEX IF NOT EXISTS idx_repo_files_extension ON repository_files(file_extension);
+CREATE INDEX IF NOT EXISTS idx_repo_files_filename ON repository_files(filename);
+CREATE INDEX IF NOT EXISTS idx_repo_files_relative_path ON repository_files(relative_path);
+CREATE INDEX IF NOT EXISTS idx_repo_files_is_binary ON repository_files(is_binary);
+
+-- GitHub webhooks indexes
+CREATE INDEX IF NOT EXISTS idx_github_webhooks_delivery_id ON github_webhooks(delivery_id);
+CREATE INDEX IF NOT EXISTS idx_github_webhooks_repo ON github_webhooks(owner_name, repository_name);
+CREATE INDEX IF NOT EXISTS idx_github_webhooks_event_type ON github_webhooks(event_type);
+
+-- GitLab webhooks indexes
+CREATE INDEX IF NOT EXISTS idx_gitlab_webhooks_repository_id ON gitlab_webhooks(repository_id);
+CREATE INDEX IF NOT EXISTS idx_gitlab_webhooks_created_at ON gitlab_webhooks(created_at);
+CREATE INDEX IF NOT EXISTS idx_gitlab_webhooks_event_type ON gitlab_webhooks(event_type);
+
+-- Bitbucket webhooks indexes
+CREATE INDEX IF NOT EXISTS idx_bitbucket_webhooks_repository_id ON bitbucket_webhooks(repository_id);
+CREATE INDEX IF NOT EXISTS idx_bitbucket_webhooks_created_at ON bitbucket_webhooks(created_at);
+CREATE INDEX IF NOT EXISTS idx_bitbucket_webhooks_event_type ON bitbucket_webhooks(event_type);
+
+-- Gitea webhooks indexes
+CREATE INDEX IF NOT EXISTS idx_gitea_webhooks_repository_id ON gitea_webhooks(repository_id);
+CREATE INDEX IF NOT EXISTS idx_gitea_webhooks_created_at ON gitea_webhooks(created_at);
+CREATE INDEX IF NOT EXISTS idx_gitea_webhooks_event_type ON gitea_webhooks(event_type);
+
+-- Commit details indexes
+CREATE UNIQUE INDEX IF NOT EXISTS uq_repo_commit_sha ON repository_commit_details(repository_id, commit_sha);
+CREATE INDEX IF NOT EXISTS idx_repo_commit_created_at ON repository_commit_details(created_at);
+
+-- Commit files indexes
+CREATE INDEX IF NOT EXISTS idx_commit_files_commit_id ON repository_commit_files(commit_id);
+CREATE INDEX IF NOT EXISTS idx_commit_files_path ON repository_commit_files(file_path);
+
+-- OAuth token indexes
+CREATE INDEX IF NOT EXISTS idx_github_user_tokens_github_username ON github_user_tokens(github_username);
+
+-- Diff contents indexes
+CREATE INDEX IF NOT EXISTS idx_diff_contents_commit_id ON diff_contents(commit_id);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_file_change_id ON diff_contents(file_change_id);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_storage_type ON diff_contents(storage_type);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_file_path ON diff_contents(file_path);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_change_type ON diff_contents(change_type);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_processing_status ON diff_contents(processing_status);
+CREATE INDEX IF NOT EXISTS idx_diff_contents_created_at ON diff_contents(created_at);
+
+-- Processing queue indexes
+CREATE INDEX IF NOT EXISTS idx_diff_queue_status ON diff_processing_queue(queue_status);
+CREATE INDEX IF NOT EXISTS idx_diff_queue_priority ON diff_processing_queue(priority DESC);
+CREATE INDEX IF NOT EXISTS idx_diff_queue_repository_id ON diff_processing_queue(repository_id);
+CREATE INDEX IF NOT EXISTS idx_diff_queue_created_at ON diff_processing_queue(created_at);
+
+-- Statistics indexes
+CREATE INDEX IF NOT EXISTS idx_diff_stats_repository_id ON diff_statistics(repository_id);
+CREATE INDEX IF NOT EXISTS idx_diff_stats_period ON diff_statistics(period_start, period_end);
+
+-- =============================================
+-- Triggers for Updated At Columns
+-- =============================================
+
+-- Create update function if it doesn't exist
+CREATE OR REPLACE FUNCTION update_updated_at_column()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = NOW();
+    RETURN NEW;
+END;
+$$ language 'plpgsql';
+
+-- GitHub repositories trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_github_repos_updated_at'
+  ) THEN
+    CREATE TRIGGER update_github_repos_updated_at BEFORE UPDATE ON "github_repositories@migrations/"
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Repository storage trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_repository_storage_updated_at'
+  ) THEN
+    CREATE TRIGGER update_repository_storage_updated_at BEFORE UPDATE ON repository_storage
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Repository directories trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_repository_directories_updated_at'
+  ) THEN
+    CREATE TRIGGER update_repository_directories_updated_at BEFORE UPDATE ON repository_directories
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Repository files trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_repository_files_updated_at'
+  ) THEN
+    CREATE TRIGGER update_repository_files_updated_at BEFORE UPDATE ON repository_files
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- GitHub webhooks trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_github_webhooks_updated_at'
+  ) THEN
+    CREATE TRIGGER update_github_webhooks_updated_at BEFORE UPDATE ON github_webhooks
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- GitLab user tokens trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_gitlab_user_tokens_updated_at'
+  ) THEN
+    CREATE TRIGGER update_gitlab_user_tokens_updated_at BEFORE UPDATE ON gitlab_user_tokens
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Bitbucket user tokens trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_bitbucket_user_tokens_updated_at'
+  ) THEN
+    CREATE TRIGGER update_bitbucket_user_tokens_updated_at BEFORE UPDATE ON bitbucket_user_tokens
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Gitea user tokens trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_gitea_user_tokens_updated_at'
+  ) THEN
+    CREATE TRIGGER update_gitea_user_tokens_updated_at BEFORE UPDATE ON gitea_user_tokens
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Diff contents trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_diff_contents_updated_at'
+  ) THEN
+    CREATE TRIGGER update_diff_contents_updated_at BEFORE UPDATE ON diff_contents
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- Processing queue trigger
+DO $$
+BEGIN
+  IF NOT EXISTS (
+    SELECT 1 FROM pg_trigger WHERE tgname = 'update_diff_queue_updated_at'
+  ) THEN
+    CREATE TRIGGER update_diff_queue_updated_at BEFORE UPDATE ON diff_processing_queue
+      FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+  END IF;
+END $$;
+
+-- =============================================
+-- Helper Functions
+-- =============================================
+
+-- Function to get diff storage statistics for a repository
+CREATE OR REPLACE FUNCTION get_repository_diff_stats(repo_id UUID, days_back INTEGER DEFAULT 30)
+RETURNS TABLE (
+  total_diffs BIGINT,
+  total_size_bytes BIGINT,
+  avg_size_bytes DECIMAL(10,2),
+  storage_breakdown JSONB
+) AS $$
+BEGIN
+  RETURN QUERY
+  SELECT 
+    COUNT(*)::BIGINT as total_diffs,
+    COALESCE(SUM(dc.diff_size_bytes), 0)::BIGINT as total_size_bytes,
+    COALESCE(AVG(dc.diff_size_bytes), 0)::DECIMAL(10,2) as avg_size_bytes,
+    jsonb_build_object(
+      'external', COUNT(*) FILTER (WHERE dc.storage_type = 'external')
+    ) as storage_breakdown
+  FROM diff_contents dc
+  JOIN repository_commit_details rcd ON dc.commit_id = rcd.id
+  WHERE rcd.repository_id = repo_id
+    AND dc.created_at >= NOW() - INTERVAL '1 day' * days_back;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Function to clean up old diff processing queue entries
+CREATE OR REPLACE FUNCTION cleanup_old_diff_queue_entries(days_back INTEGER DEFAULT 7)
+RETURNS INTEGER AS $$
+DECLARE
+  deleted_count INTEGER;
+BEGIN
+  DELETE FROM diff_processing_queue 
+  WHERE created_at < NOW() - INTERVAL '1 day' * days_back
+    AND queue_status IN ('completed', 'failed');
+  
+  GET DIAGNOSTICS deleted_count = ROW_COUNT;
+  RETURN deleted_count;
+END;
+$$ LANGUAGE plpgsql;

services/git-integration/src/models/commit-details.model.js

@@ -0,0 +1,79 @@
+const { v4: uuidv4 } = require('uuid');
+const database = require('../config/database');
+
+class CommitDetailsModel {
+  constructor() {
+    this.db = database;
+  }
+
+  async createCommitDetail(commitData) {
+    const {
+      repository_id,
+      commit_sha,
+      author_name,
+      author_email,
+      message,
+      url,
+      committed_at
+    } = commitData;
+
+    const query = `
+      INSERT INTO repository_commit_details (
+        id, repository_id, commit_sha, author_name, author_email, 
+        message, url, committed_at, created_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, NOW())
+      ON CONFLICT (repository_id, commit_sha) 
+      DO UPDATE SET
+        author_name = EXCLUDED.author_name,
+        author_email = EXCLUDED.author_email,
+        message = EXCLUDED.message,
+        url = EXCLUDED.url,
+        committed_at = EXCLUDED.committed_at
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      repository_id,
+      commit_sha,
+      author_name,
+      author_email,
+      message,
+      url,
+      committed_at || new Date()
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async getCommitsByRepository(repository_id, limit = 50, offset = 0) {
+    const query = `
+      SELECT * FROM repository_commit_details 
+      WHERE repository_id = $1 
+      ORDER BY committed_at DESC 
+      LIMIT $2 OFFSET $3
+    `;
+    
+    const result = await this.db.query(query, [repository_id, limit, offset]);
+    return result.rows;
+  }
+
+  async getCommitBySha(repository_id, commit_sha) {
+    const query = `
+      SELECT * FROM repository_commit_details 
+      WHERE repository_id = $1 AND commit_sha = $2
+    `;
+    
+    const result = await this.db.query(query, [repository_id, commit_sha]);
+    return result.rows[0];
+  }
+
+  async deleteCommitsByRepository(repository_id) {
+    const query = `DELETE FROM repository_commit_details WHERE repository_id = $1`;
+    const result = await this.db.query(query, [repository_id]);
+    return result.rowCount;
+  }
+}
+
+module.exports = CommitDetailsModel;

services/git-integration/src/models/commit-files.model.js

@@ -0,0 +1,102 @@
+const { v4: uuidv4 } = require('uuid');
+const database = require('../config/database');
+
+class CommitFilesModel {
+  constructor() {
+    this.db = database;
+  }
+
+  async createCommitFile(fileData) {
+    const {
+      commit_id,
+      change_type,
+      file_path
+    } = fileData;
+
+    const query = `
+      INSERT INTO repository_commit_files (
+        id, commit_id, change_type, file_path, created_at
+      ) VALUES ($1, $2, $3, $4, NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      commit_id,
+      change_type,
+      file_path
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async createMultipleCommitFiles(commit_id, files) {
+    if (!files || files.length === 0) return [];
+
+    const values = [];
+    const placeholders = [];
+    
+    files.forEach((file, index) => {
+      const baseIndex = index * 4;
+      placeholders.push(`($${baseIndex + 1}, $${baseIndex + 2}, $${baseIndex + 3}, $${baseIndex + 4}, NOW())`);
+      values.push(uuidv4(), commit_id, file.change_type, file.file_path);
+    });
+
+    const query = `
+      INSERT INTO repository_commit_files (
+        id, commit_id, change_type, file_path, created_at
+      ) VALUES ${placeholders.join(', ')}
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, values);
+    return result.rows;
+  }
+
+  async getFilesByCommit(commit_id) {
+    const query = `
+      SELECT * FROM repository_commit_files 
+      WHERE commit_id = $1 
+      ORDER BY file_path
+    `;
+    
+    const result = await this.db.query(query, [commit_id]);
+    return result.rows;
+  }
+
+  async getFilesByRepository(repository_id, limit = 100, offset = 0) {
+    const query = `
+      SELECT rcf.*, rcd.commit_sha, rcd.committed_at 
+      FROM repository_commit_files rcf
+      JOIN repository_commit_details rcd ON rcf.commit_id = rcd.id
+      WHERE rcd.repository_id = $1 
+      ORDER BY rcd.committed_at DESC, rcf.file_path
+      LIMIT $2 OFFSET $3
+    `;
+    
+    const result = await this.db.query(query, [repository_id, limit, offset]);
+    return result.rows;
+  }
+
+  async getFileChangesByPath(repository_id, file_path) {
+    const query = `
+      SELECT rcf.*, rcd.commit_sha, rcd.committed_at, rcd.author_name
+      FROM repository_commit_files rcf
+      JOIN repository_commit_details rcd ON rcf.commit_id = rcd.id
+      WHERE rcd.repository_id = $1 AND rcf.file_path = $2
+      ORDER BY rcd.committed_at DESC
+    `;
+    
+    const result = await this.db.query(query, [repository_id, file_path]);
+    return result.rows;
+  }
+
+  async deleteFilesByCommit(commit_id) {
+    const query = `DELETE FROM repository_commit_files WHERE commit_id = $1`;
+    const result = await this.db.query(query, [commit_id]);
+    return result.rowCount;
+  }
+}
+
+module.exports = CommitFilesModel;

services/git-integration/src/models/diff-storage.model.js

@@ -0,0 +1,226 @@
+const { v4: uuidv4 } = require('uuid');
+const database = require('../config/database');
+
+class DiffStorageModel {
+  constructor() {
+    this.db = database;
+  }
+
+  async createDiffContent(diffData) {
+    const {
+      commit_id,
+      file_change_id,
+      diff_header,
+      diff_size_bytes,
+      storage_type = 'external',
+      external_storage_path,
+      external_storage_provider = 'local',
+      file_path,
+      change_type,
+      processing_status = 'pending'
+    } = diffData;
+
+    const query = `
+      INSERT INTO diff_contents (
+        id, commit_id, file_change_id, diff_header, diff_size_bytes,
+        storage_type, external_storage_path, external_storage_provider,
+        file_path, change_type, processing_status, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      commit_id,
+      file_change_id,
+      diff_header,
+      diff_size_bytes,
+      storage_type,
+      external_storage_path,
+      external_storage_provider,
+      file_path,
+      change_type,
+      processing_status
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async updateProcessingStatus(diff_id, status, error_message = null) {
+    const query = `
+      UPDATE diff_contents 
+      SET processing_status = $2, processing_error = $3, updated_at = NOW()
+      WHERE id = $1
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, [diff_id, status, error_message]);
+    return result.rows[0];
+  }
+
+  async getDiffsByCommit(commit_id) {
+    const query = `
+      SELECT * FROM diff_contents 
+      WHERE commit_id = $1 
+      ORDER BY file_path
+    `;
+    
+    const result = await this.db.query(query, [commit_id]);
+    return result.rows;
+  }
+
+  async getDiffsByRepository(repository_id, limit = 50, offset = 0) {
+    const query = `
+      SELECT dc.*, rcd.commit_sha, rcd.committed_at
+      FROM diff_contents dc
+      JOIN repository_commit_details rcd ON dc.commit_id = rcd.id
+      WHERE rcd.repository_id = $1
+      ORDER BY rcd.committed_at DESC, dc.file_path
+      LIMIT $2 OFFSET $3
+    `;
+    
+    const result = await this.db.query(query, [repository_id, limit, offset]);
+    return result.rows;
+  }
+
+  async getDiffById(diff_id) {
+    const query = `SELECT * FROM diff_contents WHERE id = $1`;
+    const result = await this.db.query(query, [diff_id]);
+    return result.rows[0];
+  }
+
+  // Diff Processing Queue methods
+  async addToProcessingQueue(queueData) {
+    const {
+      commit_id,
+      repository_id,
+      queue_status = 'pending',
+      priority = 0,
+      from_sha,
+      to_sha,
+      repo_local_path
+    } = queueData;
+
+    const query = `
+      INSERT INTO diff_processing_queue (
+        id, commit_id, repository_id, queue_status, priority,
+        from_sha, to_sha, repo_local_path, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      commit_id,
+      repository_id,
+      queue_status,
+      priority,
+      from_sha,
+      to_sha,
+      repo_local_path
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async getNextQueueItem() {
+    const query = `
+      SELECT * FROM diff_processing_queue 
+      WHERE queue_status = 'pending' 
+      ORDER BY priority DESC, created_at ASC 
+      LIMIT 1
+    `;
+    
+    const result = await this.db.query(query);
+    return result.rows[0];
+  }
+
+  async updateQueueStatus(queue_id, status, error_message = null) {
+    const query = `
+      UPDATE diff_processing_queue 
+      SET queue_status = $2, error_message = $3, 
+          processed_at = CASE WHEN $2 IN ('completed', 'failed') THEN NOW() ELSE processed_at END,
+          updated_at = NOW()
+      WHERE id = $1
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, [queue_id, status, error_message]);
+    return result.rows[0];
+  }
+
+  async incrementRetryCount(queue_id) {
+    const query = `
+      UPDATE diff_processing_queue 
+      SET retry_count = retry_count + 1, updated_at = NOW()
+      WHERE id = $1
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, [queue_id]);
+    return result.rows[0];
+  }
+
+  // Statistics methods
+  async createDiffStatistics(statsData) {
+    const {
+      repository_id,
+      period_start,
+      period_end,
+      total_commits = 0,
+      total_files_changed = 0,
+      total_diffs_processed = 0,
+      total_diff_size_bytes = 0,
+      avg_diff_size_bytes = 0,
+      max_diff_size_bytes = 0,
+      diffs_stored_external = 0,
+      avg_processing_time_ms = 0,
+      failed_processing_count = 0
+    } = statsData;
+
+    const query = `
+      INSERT INTO diff_statistics (
+        id, repository_id, period_start, period_end, total_commits,
+        total_files_changed, total_diffs_processed, total_diff_size_bytes,
+        avg_diff_size_bytes, max_diff_size_bytes, diffs_stored_external,
+        avg_processing_time_ms, failed_processing_count, created_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      repository_id,
+      period_start,
+      period_end,
+      total_commits,
+      total_files_changed,
+      total_diffs_processed,
+      total_diff_size_bytes,
+      avg_diff_size_bytes,
+      max_diff_size_bytes,
+      diffs_stored_external,
+      avg_processing_time_ms,
+      failed_processing_count
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async getRepositoryDiffStats(repository_id, days_back = 30) {
+    const query = `SELECT * FROM get_repository_diff_stats($1, $2)`;
+    const result = await this.db.query(query, [repository_id, days_back]);
+    return result.rows[0];
+  }
+
+  async cleanupOldQueueEntries(days_back = 7) {
+    const query = `SELECT cleanup_old_diff_queue_entries($1)`;
+    const result = await this.db.query(query, [days_back]);
+    return result.rows[0].cleanup_old_diff_queue_entries;
+  }
+}
+
+module.exports = DiffStorageModel;

services/git-integration/src/models/oauth-tokens.model.js

@@ -0,0 +1,227 @@
+const { v4: uuidv4 } = require('uuid');
+const database = require('../config/database');
+
+class OAuthTokensModel {
+  constructor() {
+    this.db = database;
+  }
+
+  // GitLab OAuth tokens
+  async createGitLabToken(tokenData) {
+    const {
+      access_token,
+      gitlab_username,
+      gitlab_user_id,
+      scopes,
+      expires_at
+    } = tokenData;
+
+    const query = `
+      INSERT INTO gitlab_user_tokens (
+        id, access_token, gitlab_username, gitlab_user_id, 
+        scopes, expires_at, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      access_token,
+      gitlab_username,
+      gitlab_user_id,
+      scopes,
+      expires_at
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async getGitLabTokenByUsername(gitlab_username) {
+    const query = `
+      SELECT * FROM gitlab_user_tokens 
+      WHERE gitlab_username = $1 
+      ORDER BY created_at DESC 
+      LIMIT 1
+    `;
+    
+    const result = await this.db.query(query, [gitlab_username]);
+    return result.rows[0];
+  }
+
+  async updateGitLabToken(token_id, tokenData) {
+    const {
+      access_token,
+      scopes,
+      expires_at
+    } = tokenData;
+
+    const query = `
+      UPDATE gitlab_user_tokens 
+      SET access_token = $2, scopes = $3, expires_at = $4, updated_at = NOW()
+      WHERE id = $1
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, [token_id, access_token, scopes, expires_at]);
+    return result.rows[0];
+  }
+
+  // Bitbucket OAuth tokens
+  async createBitbucketToken(tokenData) {
+    const {
+      access_token,
+      bitbucket_username,
+      bitbucket_user_id,
+      scopes,
+      expires_at
+    } = tokenData;
+
+    const query = `
+      INSERT INTO bitbucket_user_tokens (
+        id, access_token, bitbucket_username, bitbucket_user_id, 
+        scopes, expires_at, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      access_token,
+      bitbucket_username,
+      bitbucket_user_id,
+      scopes,
+      expires_at
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async getBitbucketTokenByUsername(bitbucket_username) {
+    const query = `
+      SELECT * FROM bitbucket_user_tokens 
+      WHERE bitbucket_username = $1 
+      ORDER BY created_at DESC 
+      LIMIT 1
+    `;
+    
+    const result = await this.db.query(query, [bitbucket_username]);
+    return result.rows[0];
+  }
+
+  async updateBitbucketToken(token_id, tokenData) {
+    const {
+      access_token,
+      scopes,
+      expires_at
+    } = tokenData;
+
+    const query = `
+      UPDATE bitbucket_user_tokens 
+      SET access_token = $2, scopes = $3, expires_at = $4, updated_at = NOW()
+      WHERE id = $1
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, [token_id, access_token, scopes, expires_at]);
+    return result.rows[0];
+  }
+
+  // Gitea OAuth tokens
+  async createGiteaToken(tokenData) {
+    const {
+      access_token,
+      gitea_username,
+      gitea_user_id,
+      scopes,
+      expires_at
+    } = tokenData;
+
+    const query = `
+      INSERT INTO gitea_user_tokens (
+        id, access_token, gitea_username, gitea_user_id, 
+        scopes, expires_at, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      access_token,
+      gitea_username,
+      gitea_user_id,
+      scopes,
+      expires_at
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async getGiteaTokenByUsername(gitea_username) {
+    const query = `
+      SELECT * FROM gitea_user_tokens 
+      WHERE gitea_username = $1 
+      ORDER BY created_at DESC 
+      LIMIT 1
+    `;
+    
+    const result = await this.db.query(query, [gitea_username]);
+    return result.rows[0];
+  }
+
+  async updateGiteaToken(token_id, tokenData) {
+    const {
+      access_token,
+      scopes,
+      expires_at
+    } = tokenData;
+
+    const query = `
+      UPDATE gitea_user_tokens 
+      SET access_token = $2, scopes = $3, expires_at = $4, updated_at = NOW()
+      WHERE id = $1
+      RETURNING *
+    `;
+
+    const result = await this.db.query(query, [token_id, access_token, scopes, expires_at]);
+    return result.rows[0];
+  }
+
+  // Generic methods for all providers
+  async deleteExpiredTokens() {
+    const queries = [
+      'DELETE FROM gitlab_user_tokens WHERE expires_at < NOW()',
+      'DELETE FROM bitbucket_user_tokens WHERE expires_at < NOW()',
+      'DELETE FROM gitea_user_tokens WHERE expires_at < NOW()'
+    ];
+
+    let totalDeleted = 0;
+    for (const query of queries) {
+      const result = await this.db.query(query);
+      totalDeleted += result.rowCount;
+    }
+
+    return totalDeleted;
+  }
+
+  async getAllTokensByProvider(provider) {
+    const tableMap = {
+      'gitlab': 'gitlab_user_tokens',
+      'bitbucket': 'bitbucket_user_tokens',
+      'gitea': 'gitea_user_tokens'
+    };
+
+    const tableName = tableMap[provider];
+    if (!tableName) {
+      throw new Error(`Unsupported provider: ${provider}`);
+    }
+
+    const query = `SELECT * FROM ${tableName} ORDER BY created_at DESC`;
+    const result = await this.db.query(query);
+    return result.rows;
+  }
+}
+
+module.exports = OAuthTokensModel;

services/git-integration/src/routes/commits.routes.js

@@ -0,0 +1,273 @@
+const express = require('express');
+const router = express.Router();
+const CommitTrackingService = require('../services/commit-tracking.service');
+const DiffStorageModel = require('../models/diff-storage.model');
+const EnhancedDiffProcessingService = require('../services/enhanced-diff-processing.service');
+
+const commitTrackingService = new CommitTrackingService();
+const diffStorageModel = new DiffStorageModel();
+const diffProcessingService = new EnhancedDiffProcessingService();
+
+/**
+ * GET /api/commits/repository/:repositoryId
+ * Get commit history for a repository
+ */
+router.get('/repository/:repositoryId', async (req, res) => {
+  try {
+    const { repositoryId } = req.params;
+    const { 
+      limit = 50, 
+      offset = 0, 
+      include_files = false 
+    } = req.query;
+
+    const options = {
+      limit: parseInt(limit),
+      offset: parseInt(offset),
+      include_files: include_files === 'true'
+    };
+
+    const commits = await commitTrackingService.getCommitHistory(repositoryId, options);
+    
+    res.json({
+      success: true,
+      data: commits,
+      pagination: {
+        limit: options.limit,
+        offset: options.offset,
+        total: commits.length
+      }
+    });
+  } catch (error) {
+    console.error('Error getting commit history:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get commit history',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/commits/:commitId/files
+ * Get file changes for a specific commit
+ */
+router.get('/:commitId/files', async (req, res) => {
+  try {
+    const { commitId } = req.params;
+    
+    const files = await commitTrackingService.commitFilesModel.getFilesByCommit(commitId);
+    
+    res.json({
+      success: true,
+      data: files
+    });
+  } catch (error) {
+    console.error('Error getting commit files:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get commit files',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/commits/repository/:repositoryId/file-history
+ * Get file change history for a specific file
+ */
+router.get('/repository/:repositoryId/file-history', async (req, res) => {
+  try {
+    const { repositoryId } = req.params;
+    const { file_path } = req.query;
+
+    if (!file_path) {
+      return res.status(400).json({
+        success: false,
+        error: 'file_path query parameter is required'
+      });
+    }
+
+    const history = await commitTrackingService.getFileHistory(repositoryId, file_path);
+    
+    res.json({
+      success: true,
+      data: history
+    });
+  } catch (error) {
+    console.error('Error getting file history:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get file history',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/commits/repository/:repositoryId/stats
+ * Get repository commit statistics
+ */
+router.get('/repository/:repositoryId/stats', async (req, res) => {
+  try {
+    const { repositoryId } = req.params;
+    
+    const stats = await commitTrackingService.getRepositoryStats(repositoryId);
+    
+    res.json({
+      success: true,
+      data: stats
+    });
+  } catch (error) {
+    console.error('Error getting repository stats:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get repository stats',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/commits/process-webhook
+ * Process commits from webhook payload
+ */
+router.post('/process-webhook', async (req, res) => {
+  try {
+    const { webhookPayload, repositoryId } = req.body;
+
+    if (!webhookPayload || !repositoryId) {
+      return res.status(400).json({
+        success: false,
+        error: 'webhookPayload and repositoryId are required'
+      });
+    }
+
+    const processedCommits = await commitTrackingService.processWebhookCommits(
+      webhookPayload, 
+      repositoryId
+    );
+    
+    res.json({
+      success: true,
+      data: {
+        processed_commits: processedCommits.length,
+        commits: processedCommits
+      }
+    });
+  } catch (error) {
+    console.error('Error processing webhook commits:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to process webhook commits',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/commits/:commitId/diffs
+ * Get diff information for a commit
+ */
+router.get('/:commitId/diffs', async (req, res) => {
+  try {
+    const { commitId } = req.params;
+    
+    const diffs = await diffStorageModel.getDiffsByCommit(commitId);
+    
+    res.json({
+      success: true,
+      data: diffs
+    });
+  } catch (error) {
+    console.error('Error getting commit diffs:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get commit diffs',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/commits/diffs/:diffId/content
+ * Get actual diff content
+ */
+router.get('/diffs/:diffId/content', async (req, res) => {
+  try {
+    const { diffId } = req.params;
+    
+    const diffContent = await diffProcessingService.retrieveDiffContent(diffId);
+    
+    res.set('Content-Type', 'text/plain');
+    res.send(diffContent);
+  } catch (error) {
+    console.error('Error getting diff content:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get diff content',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/commits/repository/:repositoryId/diff-stats
+ * Get diff statistics for a repository
+ */
+router.get('/repository/:repositoryId/diff-stats', async (req, res) => {
+  try {
+    const { repositoryId } = req.params;
+    const { days_back = 30 } = req.query;
+    
+    const stats = await diffStorageModel.getRepositoryDiffStats(
+      repositoryId, 
+      parseInt(days_back)
+    );
+    
+    res.json({
+      success: true,
+      data: stats
+    });
+  } catch (error) {
+    console.error('Error getting diff stats:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get diff stats',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/commits/process-diff-queue
+ * Manually trigger diff processing
+ */
+router.post('/process-diff-queue', async (req, res) => {
+  try {
+    const result = await diffProcessingService.processNextQueueItem();
+    
+    if (result) {
+      res.json({
+        success: true,
+        data: result,
+        message: 'Diff processing completed'
+      });
+    } else {
+      res.json({
+        success: true,
+        data: null,
+        message: 'No items in queue to process'
+      });
+    }
+  } catch (error) {
+    console.error('Error processing diff queue:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to process diff queue',
+      message: error.message
+    });
+  }
+});
+
+module.exports = router;

services/git-integration/src/routes/enhanced-webhooks.routes.js

@@ -0,0 +1,369 @@
+const express = require('express');
+const router = express.Router();
+const EnhancedWebhookService = require('../services/enhanced-webhook.service');
+
+const webhookService = new EnhancedWebhookService();
+
+/**
+ * POST /api/webhooks/github
+ * Handle GitHub webhooks
+ */
+router.post('/github', async (req, res) => {
+  try {
+    const payload = req.body;
+    const headers = req.headers;
+
+    console.log(`Received GitHub webhook: ${headers['x-github-event']}`);
+
+    const result = await webhookService.processGitHubWebhook(payload, headers);
+    
+    res.json({
+      success: true,
+      data: result,
+      message: 'GitHub webhook processed successfully'
+    });
+  } catch (error) {
+    console.error('Error processing GitHub webhook:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to process GitHub webhook',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/webhooks/gitlab
+ * Handle GitLab webhooks
+ */
+router.post('/gitlab', async (req, res) => {
+  try {
+    const payload = req.body;
+    const headers = req.headers;
+
+    console.log(`Received GitLab webhook: ${headers['x-gitlab-event']}`);
+
+    const result = await webhookService.processGitLabWebhook(payload, headers);
+    
+    res.json({
+      success: true,
+      data: result,
+      message: 'GitLab webhook processed successfully'
+    });
+  } catch (error) {
+    console.error('Error processing GitLab webhook:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to process GitLab webhook',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/webhooks/bitbucket
+ * Handle Bitbucket webhooks
+ */
+router.post('/bitbucket', async (req, res) => {
+  try {
+    const payload = req.body;
+    const headers = req.headers;
+
+    console.log(`Received Bitbucket webhook: ${headers['x-event-key']}`);
+
+    const result = await webhookService.processBitbucketWebhook(payload, headers);
+    
+    res.json({
+      success: true,
+      data: result,
+      message: 'Bitbucket webhook processed successfully'
+    });
+  } catch (error) {
+    console.error('Error processing Bitbucket webhook:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to process Bitbucket webhook',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/webhooks/gitea
+ * Handle Gitea webhooks
+ */
+router.post('/gitea', async (req, res) => {
+  try {
+    const payload = req.body;
+    const headers = req.headers;
+
+    console.log(`Received Gitea webhook: ${headers['x-gitea-event']}`);
+
+    const result = await webhookService.processGiteaWebhook(payload, headers);
+    
+    res.json({
+      success: true,
+      data: result,
+      message: 'Gitea webhook processed successfully'
+    });
+  } catch (error) {
+    console.error('Error processing Gitea webhook:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to process Gitea webhook',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/webhooks/github/recent
+ * Get recent GitHub webhooks
+ */
+router.get('/github/recent', async (req, res) => {
+  try {
+    const { limit = 50, offset = 0 } = req.query;
+    
+    const query = `
+      SELECT id, delivery_id, event_type, action, owner_name, repository_name,
+             ref, commit_count, processed_at, created_at
+      FROM github_webhooks 
+      ORDER BY created_at DESC 
+      LIMIT $1 OFFSET $2
+    `;
+    
+    const result = await webhookService.db.query(query, [parseInt(limit), parseInt(offset)]);
+    
+    res.json({
+      success: true,
+      data: result.rows,
+      pagination: {
+        limit: parseInt(limit),
+        offset: parseInt(offset),
+        total: result.rows.length
+      }
+    });
+  } catch (error) {
+    console.error('Error getting recent GitHub webhooks:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get recent GitHub webhooks',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/webhooks/gitlab/recent
+ * Get recent GitLab webhooks
+ */
+router.get('/gitlab/recent', async (req, res) => {
+  try {
+    const { limit = 50, offset = 0 } = req.query;
+    
+    const query = `
+      SELECT id, delivery_id, event_type, action, owner_name, repository_name,
+             ref, commit_count, created_at
+      FROM gitlab_webhooks 
+      ORDER BY created_at DESC 
+      LIMIT $1 OFFSET $2
+    `;
+    
+    const result = await webhookService.db.query(query, [parseInt(limit), parseInt(offset)]);
+    
+    res.json({
+      success: true,
+      data: result.rows,
+      pagination: {
+        limit: parseInt(limit),
+        offset: parseInt(offset),
+        total: result.rows.length
+      }
+    });
+  } catch (error) {
+    console.error('Error getting recent GitLab webhooks:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get recent GitLab webhooks',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/webhooks/bitbucket/recent
+ * Get recent Bitbucket webhooks
+ */
+router.get('/bitbucket/recent', async (req, res) => {
+  try {
+    const { limit = 50, offset = 0 } = req.query;
+    
+    const query = `
+      SELECT id, delivery_id, event_type, action, owner_name, repository_name,
+             ref, commit_count, created_at
+      FROM bitbucket_webhooks 
+      ORDER BY created_at DESC 
+      LIMIT $1 OFFSET $2
+    `;
+    
+    const result = await webhookService.db.query(query, [parseInt(limit), parseInt(offset)]);
+    
+    res.json({
+      success: true,
+      data: result.rows,
+      pagination: {
+        limit: parseInt(limit),
+        offset: parseInt(offset),
+        total: result.rows.length
+      }
+    });
+  } catch (error) {
+    console.error('Error getting recent Bitbucket webhooks:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get recent Bitbucket webhooks',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/webhooks/gitea/recent
+ * Get recent Gitea webhooks
+ */
+router.get('/gitea/recent', async (req, res) => {
+  try {
+    const { limit = 50, offset = 0 } = req.query;
+    
+    const query = `
+      SELECT id, delivery_id, event_type, action, owner_name, repository_name,
+             ref, commit_count, created_at
+      FROM gitea_webhooks 
+      ORDER BY created_at DESC 
+      LIMIT $1 OFFSET $2
+    `;
+    
+    const result = await webhookService.db.query(query, [parseInt(limit), parseInt(offset)]);
+    
+    res.json({
+      success: true,
+      data: result.rows,
+      pagination: {
+        limit: parseInt(limit),
+        offset: parseInt(offset),
+        total: result.rows.length
+      }
+    });
+  } catch (error) {
+    console.error('Error getting recent Gitea webhooks:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get recent Gitea webhooks',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/webhooks/stats
+ * Get webhook statistics across all providers
+ */
+router.get('/stats', async (req, res) => {
+  try {
+    const { days_back = 7 } = req.query;
+    
+    const queries = [
+      `SELECT 'github' as provider, COUNT(*) as total, 
+              COUNT(*) FILTER (WHERE processed_at IS NOT NULL) as processed
+       FROM github_webhooks 
+       WHERE created_at >= NOW() - INTERVAL '${parseInt(days_back)} days'`,
+      
+      `SELECT 'gitlab' as provider, COUNT(*) as total, 
+              COUNT(*) as processed
+       FROM gitlab_webhooks 
+       WHERE created_at >= NOW() - INTERVAL '${parseInt(days_back)} days'`,
+      
+      `SELECT 'bitbucket' as provider, COUNT(*) as total, 
+              COUNT(*) as processed
+       FROM bitbucket_webhooks 
+       WHERE created_at >= NOW() - INTERVAL '${parseInt(days_back)} days'`,
+      
+      `SELECT 'gitea' as provider, COUNT(*) as total, 
+              COUNT(*) as processed
+       FROM gitea_webhooks 
+       WHERE created_at >= NOW() - INTERVAL '${parseInt(days_back)} days'`
+    ];
+
+    const results = await Promise.all(
+      queries.map(query => webhookService.db.query(query))
+    );
+
+    const stats = results.map(result => result.rows[0]).filter(row => row.total > 0);
+    
+    res.json({
+      success: true,
+      data: {
+        period_days: parseInt(days_back),
+        providers: stats,
+        total_webhooks: stats.reduce((sum, stat) => sum + parseInt(stat.total), 0),
+        total_processed: stats.reduce((sum, stat) => sum + parseInt(stat.processed), 0)
+      }
+    });
+  } catch (error) {
+    console.error('Error getting webhook stats:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get webhook stats',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/webhooks/:provider/:webhookId/payload
+ * Get full webhook payload by ID
+ */
+router.get('/:provider/:webhookId/payload', async (req, res) => {
+  try {
+    const { provider, webhookId } = req.params;
+    
+    const tableMap = {
+      'github': 'github_webhooks',
+      'gitlab': 'gitlab_webhooks',
+      'bitbucket': 'bitbucket_webhooks',
+      'gitea': 'gitea_webhooks'
+    };
+
+    const tableName = tableMap[provider];
+    if (!tableName) {
+      return res.status(400).json({
+        success: false,
+        error: 'Invalid provider. Must be github, gitlab, bitbucket, or gitea'
+      });
+    }
+
+    const query = `SELECT payload FROM ${tableName} WHERE id = $1`;
+    const result = await webhookService.db.query(query, [webhookId]);
+    
+    if (result.rows.length === 0) {
+      return res.status(404).json({
+        success: false,
+        error: 'Webhook not found'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: result.rows[0].payload
+    });
+  } catch (error) {
+    console.error('Error getting webhook payload:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get webhook payload',
+      message: error.message
+    });
+  }
+});
+
+module.exports = router;

services/git-integration/src/routes/github-integration.routes.js

@@ -1168,11 +1168,7 @@ router.delete('/repository/:id', async (req, res) => {
     // Clean up file storage
     await githubService.cleanupRepositoryStorage(id);
     
-    // Delete feature mappings first
+   
-    await database.query(
-      'DELETE FROM feature_codebase_mappings WHERE repository_id = $1',
-      [id]
-    );
     
     // Delete repository record
     await database.query(

services/git-integration/src/routes/oauth-providers.routes.js

@@ -0,0 +1,384 @@
+const express = require('express');
+const router = express.Router();
+const OAuthTokensModel = require('../models/oauth-tokens.model');
+
+const oauthTokensModel = new OAuthTokensModel();
+
+/**
+ * POST /api/oauth/gitlab/token
+ * Store GitLab OAuth token
+ */
+router.post('/gitlab/token', async (req, res) => {
+  try {
+    const {
+      access_token,
+      gitlab_username,
+      gitlab_user_id,
+      scopes,
+      expires_at
+    } = req.body;
+
+    if (!access_token || !gitlab_username) {
+      return res.status(400).json({
+        success: false,
+        error: 'access_token and gitlab_username are required'
+      });
+    }
+
+    const tokenData = {
+      access_token,
+      gitlab_username,
+      gitlab_user_id,
+      scopes,
+      expires_at: expires_at ? new Date(expires_at) : null
+    };
+
+    const token = await oauthTokensModel.createGitLabToken(tokenData);
+    
+    res.json({
+      success: true,
+      data: {
+        id: token.id,
+        gitlab_username: token.gitlab_username,
+        scopes: token.scopes,
+        expires_at: token.expires_at,
+        created_at: token.created_at
+      }
+    });
+  } catch (error) {
+    console.error('Error storing GitLab token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to store GitLab token',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/oauth/gitlab/token/:username
+ * Get GitLab OAuth token by username
+ */
+router.get('/gitlab/token/:username', async (req, res) => {
+  try {
+    const { username } = req.params;
+    
+    const token = await oauthTokensModel.getGitLabTokenByUsername(username);
+    
+    if (!token) {
+      return res.status(404).json({
+        success: false,
+        error: 'GitLab token not found'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: {
+        id: token.id,
+        gitlab_username: token.gitlab_username,
+        scopes: token.scopes,
+        expires_at: token.expires_at,
+        created_at: token.created_at
+      }
+    });
+  } catch (error) {
+    console.error('Error getting GitLab token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get GitLab token',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/oauth/bitbucket/token
+ * Store Bitbucket OAuth token
+ */
+router.post('/bitbucket/token', async (req, res) => {
+  try {
+    const {
+      access_token,
+      bitbucket_username,
+      bitbucket_user_id,
+      scopes,
+      expires_at
+    } = req.body;
+
+    if (!access_token || !bitbucket_username) {
+      return res.status(400).json({
+        success: false,
+        error: 'access_token and bitbucket_username are required'
+      });
+    }
+
+    const tokenData = {
+      access_token,
+      bitbucket_username,
+      bitbucket_user_id,
+      scopes,
+      expires_at: expires_at ? new Date(expires_at) : null
+    };
+
+    const token = await oauthTokensModel.createBitbucketToken(tokenData);
+    
+    res.json({
+      success: true,
+      data: {
+        id: token.id,
+        bitbucket_username: token.bitbucket_username,
+        scopes: token.scopes,
+        expires_at: token.expires_at,
+        created_at: token.created_at
+      }
+    });
+  } catch (error) {
+    console.error('Error storing Bitbucket token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to store Bitbucket token',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/oauth/bitbucket/token/:username
+ * Get Bitbucket OAuth token by username
+ */
+router.get('/bitbucket/token/:username', async (req, res) => {
+  try {
+    const { username } = req.params;
+    
+    const token = await oauthTokensModel.getBitbucketTokenByUsername(username);
+    
+    if (!token) {
+      return res.status(404).json({
+        success: false,
+        error: 'Bitbucket token not found'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: {
+        id: token.id,
+        bitbucket_username: token.bitbucket_username,
+        scopes: token.scopes,
+        expires_at: token.expires_at,
+        created_at: token.created_at
+      }
+    });
+  } catch (error) {
+    console.error('Error getting Bitbucket token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get Bitbucket token',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * POST /api/oauth/gitea/token
+ * Store Gitea OAuth token
+ */
+router.post('/gitea/token', async (req, res) => {
+  try {
+    const {
+      access_token,
+      gitea_username,
+      gitea_user_id,
+      scopes,
+      expires_at
+    } = req.body;
+
+    if (!access_token || !gitea_username) {
+      return res.status(400).json({
+        success: false,
+        error: 'access_token and gitea_username are required'
+      });
+    }
+
+    const tokenData = {
+      access_token,
+      gitea_username,
+      gitea_user_id,
+      scopes,
+      expires_at: expires_at ? new Date(expires_at) : null
+    };
+
+    const token = await oauthTokensModel.createGiteaToken(tokenData);
+    
+    res.json({
+      success: true,
+      data: {
+        id: token.id,
+        gitea_username: token.gitea_username,
+        scopes: token.scopes,
+        expires_at: token.expires_at,
+        created_at: token.created_at
+      }
+    });
+  } catch (error) {
+    console.error('Error storing Gitea token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to store Gitea token',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/oauth/gitea/token/:username
+ * Get Gitea OAuth token by username
+ */
+router.get('/gitea/token/:username', async (req, res) => {
+  try {
+    const { username } = req.params;
+    
+    const token = await oauthTokensModel.getGiteaTokenByUsername(username);
+    
+    if (!token) {
+      return res.status(404).json({
+        success: false,
+        error: 'Gitea token not found'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: {
+        id: token.id,
+        gitea_username: token.gitea_username,
+        scopes: token.scopes,
+        expires_at: token.expires_at,
+        created_at: token.created_at
+      }
+    });
+  } catch (error) {
+    console.error('Error getting Gitea token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get Gitea token',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * GET /api/oauth/:provider/tokens
+ * Get all tokens for a provider
+ */
+router.get('/:provider/tokens', async (req, res) => {
+  try {
+    const { provider } = req.params;
+    
+    if (!['gitlab', 'bitbucket', 'gitea'].includes(provider)) {
+      return res.status(400).json({
+        success: false,
+        error: 'Invalid provider. Must be gitlab, bitbucket, or gitea'
+      });
+    }
+
+    const tokens = await oauthTokensModel.getAllTokensByProvider(provider);
+    
+    // Remove sensitive access_token from response
+    const sanitizedTokens = tokens.map(token => {
+      const { access_token, ...safeToken } = token;
+      return safeToken;
+    });
+
+    res.json({
+      success: true,
+      data: sanitizedTokens
+    });
+  } catch (error) {
+    console.error('Error getting provider tokens:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to get provider tokens',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * DELETE /api/oauth/expired-tokens
+ * Clean up expired tokens
+ */
+router.delete('/expired-tokens', async (req, res) => {
+  try {
+    const deletedCount = await oauthTokensModel.deleteExpiredTokens();
+    
+    res.json({
+      success: true,
+      data: {
+        deleted_count: deletedCount
+      },
+      message: `Deleted ${deletedCount} expired tokens`
+    });
+  } catch (error) {
+    console.error('Error deleting expired tokens:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to delete expired tokens',
+      message: error.message
+    });
+  }
+});
+
+/**
+ * PUT /api/oauth/gitlab/token/:tokenId
+ * Update GitLab OAuth token
+ */
+router.put('/gitlab/token/:tokenId', async (req, res) => {
+  try {
+    const { tokenId } = req.params;
+    const { access_token, scopes, expires_at } = req.body;
+
+    if (!access_token) {
+      return res.status(400).json({
+        success: false,
+        error: 'access_token is required'
+      });
+    }
+
+    const tokenData = {
+      access_token,
+      scopes,
+      expires_at: expires_at ? new Date(expires_at) : null
+    };
+
+    const updatedToken = await oauthTokensModel.updateGitLabToken(tokenId, tokenData);
+    
+    if (!updatedToken) {
+      return res.status(404).json({
+        success: false,
+        error: 'Token not found'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: {
+        id: updatedToken.id,
+        gitlab_username: updatedToken.gitlab_username,
+        scopes: updatedToken.scopes,
+        expires_at: updatedToken.expires_at,
+        updated_at: updatedToken.updated_at
+      }
+    });
+  } catch (error) {
+    console.error('Error updating GitLab token:', error);
+    res.status(500).json({
+      success: false,
+      error: 'Failed to update GitLab token',
+      message: error.message
+    });
+  }
+});
+
+module.exports = router;

services/git-integration/src/routes/vcs.routes.js

@@ -156,10 +156,7 @@ router.post('/:provider/attach-repository', async (req, res) => {
         mappingValues.push(`(uuid_generate_v4(), $${i++}, $${i++}, $${i++}, $${i++})`);
         params.push(feature.id, repositoryRecord.id, '[]', '{}');
       }
-      await database.query(
+     
-        `INSERT INTO feature_codebase_mappings (id, feature_id, repository_id, code_paths, code_snippets) VALUES ${mappingValues.join(', ')}`,
-        params
-      );
     }
 
     const storageInfo = await (async () => {
@@ -209,7 +206,7 @@ router.post('/:provider/webhook', async (req, res) => {
     }
 
     // Signature verification
-    const rawBody = JSON.stringify(payload);
+    const rawBody = req.rawBody ? req.rawBody : Buffer.from(JSON.stringify(payload));
     const verifySignature = () => {
       try {
         if (providerKey === 'gitlab') {
@@ -220,12 +217,19 @@ router.post('/:provider/webhook', async (req, res) => {
         }
         if (providerKey === 'gitea') {
           const crypto = require('crypto');
-          const provided = req.headers['x-gitea-signature'];
+          const providedHeader = req.headers['x-gitea-signature'] || req.headers['x-gogs-signature'] || req.headers['x-hub-signature-256'];
           const secret = process.env.GITEA_WEBHOOK_SECRET;
           if (!secret) return true;
-          if (!provided) return false;
+          if (!providedHeader) return false;
+          let provided = String(providedHeader);
+          if (provided.startsWith('sha256=')) provided = provided.slice('sha256='.length);
           const expected = crypto.createHmac('sha256', secret).update(rawBody).digest('hex');
+  
+          try {
             return crypto.timingSafeEqual(Buffer.from(expected, 'hex'), Buffer.from(provided, 'hex'));
+          } catch (_) {
+            return false;
+          }
         }
         if (providerKey === 'bitbucket') {
           // Bitbucket Cloud webhooks typically have no shared secret by default
@@ -241,6 +245,14 @@ router.post('/:provider/webhook', async (req, res) => {
       return res.status(401).json({ success: false, message: 'Invalid webhook signature' });
     }
 
+    if (providerKey === 'bitbucket') {
+      console.log('🔔 Bitbucket webhook received:', {
+        eventKey: req.headers['x-event-key'],
+        requestId: req.headers['x-request-id'],
+        userAgent: req.headers['user-agent'],
+        payloadSize: rawBody?.length || 0
+      });
+    }
     // Process webhook event using comprehensive service
     const eventType = extractEventType(providerKey, payload);
     await vcsWebhookService.processWebhookEvent(providerKey, eventType, payload);
@@ -378,10 +390,8 @@ router.get('/:provider/repository/:id/file-content', async (req, res) => {
       return res.status(400).json({ success: false, message: 'File path is required' });
     }
     const query = `
-      SELECT rf.*, rfc.content_text, rfc.content_preview, rfc.language_detected, 
+      SELECT rf.*t
-             rfc.line_count, rfc.char_count
       FROM repository_files rf
-      LEFT JOIN repository_file_contents rfc ON rf.id = rfc.file_id
       WHERE rf.repository_id = $1 AND rf.relative_path = $2
     `;
     const result = await database.query(query, [id, file_path]);
@@ -449,7 +459,6 @@ router.delete('/:provider/repository/:id', async (req, res) => {
     }
     const repository = getResult.rows[0];
     await fileStorageService.cleanupRepositoryStorage(id);
-    await database.query('DELETE FROM feature_codebase_mappings WHERE repository_id = $1', [id]);
     await database.query('DELETE FROM github_repositories WHERE id = $1', [id]);
     res.json({ success: true, message: 'Repository removed successfully', data: { removed_repository: repository.repository_name, template_id: repository.template_id } });
   } catch (error) {
@@ -459,7 +468,7 @@ router.delete('/:provider/repository/:id', async (req, res) => {
 });
 
 // OAuth placeholders (start/callback) per provider for future implementation
-router.get('/:provider/auth/start', (req, res) => {
+router.get('/:provider/auth/start', async (req, res) => {
   try {
     const providerKey = (req.params.provider || '').toLowerCase();
     const oauth = getOAuthService(providerKey);
@@ -479,14 +488,29 @@ router.get('/:provider/auth/callback', (req, res) => {
       const code = req.query.code;
       const error = req.query.error;
       const errorDescription = req.query.error_description;
+      console.log(`🔄 [VCS OAUTH] Callback received for ${providerKey}:`, { 
+        hasCode: !!code, 
+        hasError: !!error, 
+        code: code?.substring(0, 10) + '...',
+        error,
+        errorDescription
+      });
       const oauth = getOAuthService(providerKey);
       if (!oauth) return res.status(400).json({ success: false, message: 'Unsupported provider or OAuth not available' });
       if (!code) {
         // Surface upstream provider error details if present
         if (error || errorDescription) {
-          return res.status(400).json({ success: false, message: 'OAuth error from provider', provider: providerKey, error: error || 'unknown_error', error_description: errorDescription || null, query: req.query });
+          console.error(`❌ [VCS OAUTH] Provider error for ${providerKey}:`, { error, errorDescription });
+          return res.status(400).json({ 
+            success: false, 
+            message: 'OAuth error from provider', 
+            provider: providerKey, 
+            error: error || 'unknown_error', 
+            error_description: errorDescription || null, 
+            query: req.query 
+          });
         }
-        return res.status(400).json({ success: false, message: 'Missing code' });
+        return res.status(400).json({ success: false, message: 'Missing authorization code' });
       }
       const accessToken = await oauth.exchangeCodeForToken(code);
       const user = await oauth.getUserInfo(accessToken);
@@ -504,7 +528,34 @@ router.get('/:provider/auth/callback', (req, res) => {
       const tokenRecord = await oauth.storeToken(accessToken, user, userId || null);
       res.json({ success: true, provider: providerKey, user, token: { id: tokenRecord.id || null } });
     } catch (e) {
-      res.status(500).json({ success: false, message: e.message || 'OAuth callback failed' });
+    
+      console.error(`❌ [VCS OAUTH] Callback error for ${req.params.provider}:`, e);
+      
+      // Provide more specific error messages
+      let errorMessage = e.message || 'OAuth callback failed';
+      let statusCode = 500;
+      
+      if (e.message.includes('not configured')) {
+        statusCode = 500;
+        errorMessage = `OAuth configuration error: ${e.message}`;
+      } else if (e.message.includes('timeout')) {
+        statusCode = 504;
+        errorMessage = `OAuth timeout: ${e.message}`;
+      } else if (e.message.includes('network error') || e.message.includes('Cannot connect')) {
+        statusCode = 502;
+        errorMessage = `Network error: ${e.message}`;
+      } else if (e.message.includes('HTTP error')) {
+        statusCode = 502;
+        errorMessage = `OAuth provider error: ${e.message}`;
+      }
+      
+      res.status(statusCode).json({ 
+        success: false, 
+        message: errorMessage,
+        provider: req.params.provider,
+        error: e.message,
+        details: process.env.NODE_ENV === 'development' ? e.stack : undefined
+      });
     }
   })();
 });

services/git-integration/src/routes/webhook.routes.js

@@ -45,21 +45,7 @@ router.post('/webhook', async (req, res) => {
       await webhookService.processWebhookEvent(eventType, payloadWithDelivery);
     }
 
-    // Log the webhook event
+  
-    await webhookService.logWebhookEvent(
-      eventType || 'unknown',
-      req.body.action || 'unknown',
-      req.body.repository?.full_name || 'unknown',
-      {
-        delivery_id: deliveryId,
-        event_type: eventType,
-        action: req.body.action,
-        repository: req.body.repository?.full_name,
-        sender: req.body.sender?.login
-      },
-      deliveryId,
-      payloadWithDelivery
-    );
 
     res.status(200).json({
       success: true,

services/git-integration/src/services/bitbucket-oauth.js

@@ -10,12 +10,13 @@ class BitbucketOAuthService {
 
   getAuthUrl(state) {
     if (!this.clientId) throw new Error('Bitbucket OAuth not configured');
+    const scopes = process.env.BITBUCKET_OAUTH_SCOPES || 'repository account';
     const params = new URLSearchParams({
       client_id: this.clientId,
       response_type: 'code',
       state,
       // Bitbucket Cloud uses 'repository' for read access; 'repository:write' for write
-      scope: 'repository account',
+      scope: scopes,
       redirect_uri: this.redirectUri
     });
     return `https://bitbucket.org/site/oauth2/authorize?${params.toString()}`;
@@ -48,7 +49,7 @@ class BitbucketOAuthService {
        VALUES ($1, $2, $3, $4, $5)
        ON CONFLICT (id) DO UPDATE SET access_token = EXCLUDED.access_token, bitbucket_username = EXCLUDED.bitbucket_username, bitbucket_user_id = EXCLUDED.bitbucket_user_id, scopes = EXCLUDED.scopes, expires_at = EXCLUDED.expires_at, updated_at = NOW()
        RETURNING *`,
-      [accessToken, user.username || user.display_name, user.uuid || null, JSON.stringify(['repository:read','account']), null]
+       [accessToken, user.username || user.display_name, user.uuid || null, JSON.stringify(['repository:admin','webhook','account']), null]
     );
     return result.rows[0];
   }

services/git-integration/src/services/commit-tracking.service.js

@@ -0,0 +1,239 @@
+const CommitDetailsModel = require('../models/commit-details.model');
+const CommitFilesModel = require('../models/commit-files.model');
+const DiffStorageModel = require('../models/diff-storage.model');
+
+class CommitTrackingService {
+  constructor() {
+    this.commitDetailsModel = new CommitDetailsModel();
+    this.commitFilesModel = new CommitFilesModel();
+    this.diffStorageModel = new DiffStorageModel();
+  }
+
+  /**
+   * Process a complete commit with its file changes
+   * @param {Object} commitData - Commit information
+   * @param {Array} fileChanges - Array of file changes
+   * @returns {Object} Processed commit with files
+   */
+  async processCommitWithFiles(commitData, fileChanges = []) {
+    try {
+      // Create commit detail record
+      const commit = await this.commitDetailsModel.createCommitDetail(commitData);
+      
+      // Create file change records if provided
+      let files = [];
+      if (fileChanges && fileChanges.length > 0) {
+        files = await this.commitFilesModel.createMultipleCommitFiles(commit.id, fileChanges);
+        
+        // Queue diff processing for this commit
+        await this.queueDiffProcessing(commit, commitData.repository_id);
+      }
+
+      return {
+        commit,
+        files,
+        total_files: files.length
+      };
+    } catch (error) {
+      console.error('Error processing commit with files:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Queue diff processing for a commit
+   * @param {Object} commit - Commit details
+   * @param {string} repository_id - Repository ID
+   */
+  async queueDiffProcessing(commit, repository_id) {
+    try {
+      const queueData = {
+        commit_id: commit.id,
+        repository_id: repository_id,
+        to_sha: commit.commit_sha,
+        repo_local_path: `/tmp/repos/${repository_id}`, // This should be configurable
+        priority: 1
+      };
+
+      await this.diffStorageModel.addToProcessingQueue(queueData);
+      console.log(`Queued diff processing for commit ${commit.commit_sha}`);
+    } catch (error) {
+      console.error('Error queuing diff processing:', error);
+      // Don't throw - this is not critical for commit processing
+    }
+  }
+
+  /**
+   * Get commit history for a repository with file changes
+   * @param {string} repository_id - Repository ID
+   * @param {Object} options - Query options
+   * @returns {Array} Commit history with file counts
+   */
+  async getCommitHistory(repository_id, options = {}) {
+    const { limit = 50, offset = 0, include_files = false } = options;
+
+    try {
+      const commits = await this.commitDetailsModel.getCommitsByRepository(
+        repository_id, 
+        limit, 
+        offset
+      );
+
+      if (include_files) {
+        // Fetch file changes for each commit
+        for (const commit of commits) {
+          const files = await this.commitFilesModel.getFilesByCommit(commit.id);
+          commit.files = files;
+          commit.file_count = files.length;
+        }
+      } else {
+        // Just get file counts
+        for (const commit of commits) {
+          const files = await this.commitFilesModel.getFilesByCommit(commit.id);
+          commit.file_count = files.length;
+        }
+      }
+
+      return commits;
+    } catch (error) {
+      console.error('Error getting commit history:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Get file change history for a specific file path
+   * @param {string} repository_id - Repository ID
+   * @param {string} file_path - File path
+   * @returns {Array} File change history
+   */
+  async getFileHistory(repository_id, file_path) {
+    try {
+      return await this.commitFilesModel.getFileChangesByPath(repository_id, file_path);
+    } catch (error) {
+      console.error('Error getting file history:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process webhook commit data
+   * @param {Object} webhookPayload - Webhook payload
+   * @param {string} repository_id - Repository ID
+   * @returns {Array} Processed commits
+   */
+  async processWebhookCommits(webhookPayload, repository_id) {
+    try {
+      const commits = webhookPayload.commits || [];
+      const processedCommits = [];
+
+      for (const commitData of commits) {
+        const processedCommit = await this.processCommitFromWebhook(commitData, repository_id);
+        processedCommits.push(processedCommit);
+      }
+
+      return processedCommits;
+    } catch (error) {
+      console.error('Error processing webhook commits:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process a single commit from webhook data
+   * @param {Object} commitData - Commit data from webhook
+   * @param {string} repository_id - Repository ID
+   * @returns {Object} Processed commit
+   */
+  async processCommitFromWebhook(commitData, repository_id) {
+    try {
+      const commit = {
+        repository_id,
+        commit_sha: commitData.id,
+        author_name: commitData.author?.name,
+        author_email: commitData.author?.email,
+        message: commitData.message,
+        url: commitData.url,
+        committed_at: new Date(commitData.timestamp)
+      };
+
+      // Extract file changes
+      const fileChanges = [];
+      
+      // Added files
+      if (commitData.added) {
+        commitData.added.forEach(filePath => {
+          fileChanges.push({
+            change_type: 'added',
+            file_path: filePath
+          });
+        });
+      }
+
+      // Modified files
+      if (commitData.modified) {
+        commitData.modified.forEach(filePath => {
+          fileChanges.push({
+            change_type: 'modified',
+            file_path: filePath
+          });
+        });
+      }
+
+      // Removed files
+      if (commitData.removed) {
+        commitData.removed.forEach(filePath => {
+          fileChanges.push({
+            change_type: 'removed',
+            file_path: filePath
+          });
+        });
+      }
+
+      return await this.processCommitWithFiles(commit, fileChanges);
+    } catch (error) {
+      console.error('Error processing commit from webhook:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Get repository statistics
+   * @param {string} repository_id - Repository ID
+   * @returns {Object} Repository statistics
+   */
+  async getRepositoryStats(repository_id) {
+    try {
+      const commits = await this.commitDetailsModel.getCommitsByRepository(repository_id, 1000);
+      const recentFiles = await this.commitFilesModel.getFilesByRepository(repository_id, 100);
+      
+      // Calculate statistics
+      const totalCommits = commits.length;
+      const uniqueAuthors = new Set(commits.map(c => c.author_email)).size;
+      const recentActivity = commits.filter(c => {
+        const commitDate = new Date(c.committed_at);
+        const thirtyDaysAgo = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000);
+        return commitDate > thirtyDaysAgo;
+      }).length;
+
+      // File change statistics
+      const fileStats = recentFiles.reduce((acc, file) => {
+        acc[file.change_type] = (acc[file.change_type] || 0) + 1;
+        return acc;
+      }, {});
+
+      return {
+        total_commits: totalCommits,
+        unique_authors: uniqueAuthors,
+        recent_activity_30_days: recentActivity,
+        file_changes: fileStats,
+        last_commit: commits[0] || null
+      };
+    } catch (error) {
+      console.error('Error getting repository stats:', error);
+      throw error;
+    }
+  }
+}
+
+module.exports = CommitTrackingService;

services/git-integration/src/services/diff-processing.service.js

@@ -0,0 +1,421 @@
+// services/diff-processing.service.js
+const fs = require('fs');
+const path = require('path');
+const zlib = require('zlib');
+const database = require('../config/database');
+const GitRepoService = require('./git-repo.service');
+
+class DiffProcessingService {
+  constructor() {
+    this.gitService = new GitRepoService();
+    this.baseDir = process.env.ATTACHED_REPOS_DIR || '/tmp/attached-repos';
+    // Allow overriding diff storage root via env; fallback to <ATTACHED_REPOS_DIR>/diffs
+    const envDiffDir = process.env.DIFF_STORAGE_DIR && process.env.DIFF_STORAGE_DIR.trim().length > 0
+      ? process.env.DIFF_STORAGE_DIR
+      : null;
+    this.diffStorageDir = envDiffDir || path.join(this.baseDir, 'diffs');
+    
+    // Size threshold: <= 50KB store in DB; > 50KB store on disk
+    this.SIZE_THRESHOLDS = {
+      SMALL: 50 * 1024
+    };
+    
+    this.ensureDiffStorageDir();
+  }
+
+  ensureDiffStorageDir() {
+    if (!fs.existsSync(this.diffStorageDir)) {
+      fs.mkdirSync(this.diffStorageDir, { recursive: true });
+    }
+  }
+
+  // Main method to process diffs for a commit
+  async processCommitDiffs(commitId, repositoryId, repoPath, fromSha, toSha) {
+    console.log(`🔄 Processing diffs for commit ${commitId}`);
+    
+    try {
+      // Get the diff content
+      const diffContent = await this.gitService.getDiff(repoPath, fromSha, toSha);
+      
+      if (!diffContent || diffContent.trim().length === 0) {
+        console.log(`⚠️  No diff content found for commit ${commitId}`);
+        return { success: false, reason: 'No diff content' };
+      }
+
+      // Parse the diff to extract individual file changes
+      const fileDiffs = this.parseDiffContent(diffContent);
+      
+      console.log(`📄 Found ${fileDiffs.length} file changes in diff`);
+
+      // Process each file diff
+      const results = [];
+      for (const fileDiff of fileDiffs) {
+        try {
+          const result = await this.processFileDiff(commitId, fileDiff);
+          results.push(result);
+        } catch (error) {
+          console.error(`❌ Failed to process diff for file ${fileDiff.filePath}:`, error.message);
+          results.push({
+            success: false,
+            filePath: fileDiff.filePath,
+            error: error.message
+          });
+        }
+      }
+
+      // Update processing statistics
+      await this.updateDiffStatistics(repositoryId, results);
+
+      return {
+        success: true,
+        processedFiles: results.length,
+        results: results
+      };
+
+    } catch (error) {
+      console.error(`❌ Failed to process commit diffs:`, error.message);
+      return { success: false, error: error.message };
+    }
+  }
+
+  // Parse diff content into individual file diffs
+  parseDiffContent(diffContent) {
+    const fileDiffs = [];
+    const lines = diffContent.split('\n');
+    
+    let currentFileDiff = null;
+    let currentDiffBody = [];
+    
+    for (let i = 0; i < lines.length; i++) {
+      const line = lines[i];
+      
+      // Check for file header (diff --git)
+      if (line.startsWith('diff --git')) {
+        // Save previous file diff if exists
+        if (currentFileDiff) {
+          currentFileDiff.diffBody = currentDiffBody.join('\n');
+          // Infer accurate change type and canonical paths from diff body
+          this.inferChangeTypeAndPaths(currentFileDiff);
+          fileDiffs.push(currentFileDiff);
+        }
+        
+        // Start new file diff
+        currentFileDiff = this.parseFileHeader(line);
+        currentDiffBody = [line];
+      } else if (currentFileDiff) {
+        currentDiffBody.push(line);
+      }
+    }
+    
+    // Don't forget the last file diff
+    if (currentFileDiff) {
+      currentFileDiff.diffBody = currentDiffBody.join('\n');
+      this.inferChangeTypeAndPaths(currentFileDiff);
+      fileDiffs.push(currentFileDiff);
+    }
+    
+    return fileDiffs;
+  }
+
+  // Parse file header from diff line
+  parseFileHeader(diffLine) {
+    // Example: diff --git a/src/app.js b/src/app.js
+    const match = diffLine.match(/diff --git a\/(.+) b\/(.+)/);
+    if (!match) {
+      throw new Error(`Invalid diff header: ${diffLine}`);
+    }
+    
+    const oldPath = match[1];
+    const newPath = match[2];
+    
+    // Determine change type
+    let changeType = 'modified';
+    if (oldPath === '/dev/null') {
+      changeType = 'added';
+    } else if (newPath === '/dev/null') {
+      changeType = 'deleted';
+    } else if (oldPath !== newPath) {
+      changeType = 'renamed';
+    }
+    
+    return {
+      filePath: newPath === '/dev/null' ? oldPath : newPath,
+      oldPath: oldPath === '/dev/null' ? null : oldPath,
+      newPath: newPath === '/dev/null' ? null : newPath,
+      changeType: changeType,
+      diffHeader: diffLine
+    };
+  }
+
+  // Inspect the diff body to accurately infer change type and canonical paths
+  inferChangeTypeAndPaths(fileDiff) {
+    if (!fileDiff || !fileDiff.diffBody) return;
+
+    const lines = fileDiff.diffBody.split('\n');
+    let headerOld = null;
+    let headerNew = null;
+    let sawNewFile = false;
+    let sawDeletedFile = false;
+    let sawRenameFrom = null;
+    let sawRenameTo = null;
+
+    for (const l of lines) {
+      if (l.startsWith('new file mode')) sawNewFile = true;
+      if (l.startsWith('deleted file mode')) sawDeletedFile = true;
+      if (l.startsWith('rename from ')) sawRenameFrom = l.replace('rename from ', '').trim();
+      if (l.startsWith('rename to ')) sawRenameTo = l.replace('rename to ', '').trim();
+      if (l.startsWith('--- ')) headerOld = l.substring(4).trim(); // e.g., 'a/path' or '/dev/null'
+      if (l.startsWith('+++ ')) headerNew = l.substring(4).trim(); // e.g., 'b/path' or '/dev/null'
+      // Stop early after we've collected the header markers
+      if (headerOld && headerNew && (sawNewFile || sawDeletedFile || sawRenameFrom || sawRenameTo)) break;
+    }
+
+    // Normalize paths like 'a/path' or 'b/path'
+    const normalize = (p) => {
+      if (!p) return null;
+      if (p === '/dev/null') return '/dev/null';
+      // strip leading a/ or b/
+      return p.replace(/^a\//, '').replace(/^b\//, '');
+    };
+
+    const oldPath = normalize(headerOld);
+    const newPath = normalize(headerNew);
+
+    // Decide change type priority: rename > added > deleted > modified
+    if (sawRenameFrom || sawRenameTo) {
+      fileDiff.changeType = 'renamed';
+      fileDiff.oldPath = sawRenameFrom || oldPath || fileDiff.oldPath || null;
+      fileDiff.newPath = sawRenameTo || newPath || fileDiff.newPath || null;
+      fileDiff.filePath = fileDiff.newPath || fileDiff.filePath;
+      return;
+    }
+
+    if (sawNewFile || oldPath === '/dev/null') {
+      fileDiff.changeType = 'added';
+      fileDiff.oldPath = null;
+      fileDiff.newPath = newPath || fileDiff.newPath || fileDiff.filePath;
+      fileDiff.filePath = fileDiff.newPath;
+      return;
+    }
+
+    if (sawDeletedFile || newPath === '/dev/null') {
+      fileDiff.changeType = 'deleted';
+      fileDiff.newPath = null;
+      fileDiff.oldPath = oldPath || fileDiff.oldPath || fileDiff.filePath;
+      fileDiff.filePath = fileDiff.oldPath;
+      return;
+    }
+
+    // Default to modified; refine filePath if headers present
+    fileDiff.changeType = 'modified';
+    if (newPath && newPath !== '/dev/null') {
+      fileDiff.filePath = newPath;
+    }
+    if (!fileDiff.oldPath && oldPath && oldPath !== '/dev/null') fileDiff.oldPath = oldPath;
+    if (!fileDiff.newPath && newPath && newPath !== '/dev/null') fileDiff.newPath = newPath;
+  }
+
+  // Process individual file diff
+  async processFileDiff(commitId, fileDiff) {
+    const diffSize = Buffer.byteLength(fileDiff.diffBody, 'utf8');
+    const storageType = 'external';
+    
+    console.log(`📊 File ${fileDiff.filePath}: ${diffSize} bytes, storage: ${storageType}`);
+    
+    // Get file change record
+    const fileChangeQuery = `
+      SELECT rcf.id 
+      FROM repository_commit_files rcf
+      JOIN repository_commit_details rcd ON rcf.commit_id = rcd.id
+      WHERE rcd.id = $1 AND rcf.file_path = $2
+    `;
+    
+    const fileChangeResult = await database.query(fileChangeQuery, [commitId, fileDiff.filePath]);
+    
+    if (fileChangeResult.rows.length === 0) {
+      throw new Error(`File change record not found for ${fileDiff.filePath}`);
+    }
+    
+    const fileChangeId = fileChangeResult.rows[0].id;
+    
+    // Store diff based on size
+    const diffContentId = await this.storeExternally(commitId, fileChangeId, fileDiff, diffSize);
+    
+    return {
+      success: true,
+      filePath: fileDiff.filePath,
+      changeType: fileDiff.changeType,
+      diffSize: diffSize,
+      storageType: storageType,
+      diffContentId: diffContentId
+    };
+  }
+
+  // Determine storage type (always external)
+  determineStorageType(size) { return 'external'; }
+
+  // Removed DB storage path (all diffs stored externally)
+
+  // Remove compression path; no longer used
+
+  // Store large diff externally
+  async storeExternally(commitId, fileChangeId, fileDiff, diffSize) {
+    // Create directory structure: diffs/repo_id/commit_sha/
+    const commitDir = path.join(this.diffStorageDir, commitId);
+    if (!fs.existsSync(commitDir)) {
+      fs.mkdirSync(commitDir, { recursive: true });
+    }
+    
+    // Create safe filename
+    const safeFileName = fileDiff.filePath.replace(/[^a-zA-Z0-9._-]/g, '_') + '.diff';
+    const filePath = path.join(commitDir, safeFileName);
+    
+    // Write diff to file
+    fs.writeFileSync(filePath, fileDiff.diffBody, 'utf8');
+    
+    const query = `
+      INSERT INTO diff_contents (
+        commit_id, file_change_id, diff_header, diff_size_bytes,
+        storage_type, external_storage_path, external_storage_provider,
+        file_path, change_type, processing_status
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
+      RETURNING id
+    `;
+    
+    const result = await database.query(query, [
+      commitId, fileChangeId, fileDiff.diffHeader, diffSize,
+      'external', filePath, 'local',
+      fileDiff.filePath, fileDiff.changeType, 'processed'
+    ]);
+    
+    return result.rows[0].id;
+  }
+
+  // Update diff processing statistics
+  async updateDiffStatistics(repositoryId, results) {
+    const successful = results.filter(r => r.success);
+    const failed = results.filter(r => !r.success);
+    
+    const totalSize = successful.reduce((sum, r) => sum + (r.diffSize || 0), 0);
+    const avgSize = successful.length > 0 ? totalSize / successful.length : 0;
+    const externalCount = successful.filter(r => r.storageType === 'external').length;
+    
+    const query = `
+      INSERT INTO diff_statistics (
+        repository_id, period_start, period_end,
+        total_commits, total_files_changed, total_diffs_processed,
+        total_diff_size_bytes, avg_diff_size_bytes, max_diff_size_bytes,
+        diffs_stored_external, failed_processing_count
+      ) VALUES ($1, NOW() - INTERVAL '1 hour', NOW(), 
+        1, $2, $3, $4, $5, $6, $7, $8)
+    `;
+    
+    await database.query(query, [
+      repositoryId, successful.length, successful.length,
+      totalSize, avgSize, Math.max(...successful.map(r => r.diffSize || 0), 0),
+      externalCount, failed.length
+    ]);
+  }
+
+  // Queue diff processing for background processing
+  async queueDiffProcessing(commitId, repositoryId, repoPath, fromSha, toSha, priority = 0) {
+    const query = `
+      INSERT INTO diff_processing_queue (
+        commit_id, repository_id, from_sha, to_sha, repo_local_path, priority
+      ) VALUES ($1, $2, $3, $4, $5, $6)
+      RETURNING id
+    `;
+    
+    const result = await database.query(query, [
+      commitId, repositoryId, fromSha, toSha, repoPath, priority
+    ]);
+    
+    return result.rows[0].id;
+  }
+
+  // Get diff content for retrieval
+  async getDiffContent(diffContentId) {
+    const query = `
+      SELECT storage_type, external_storage_path, file_path, change_type
+      FROM diff_contents 
+      WHERE id = $1
+    `;
+    
+    const result = await database.query(query, [diffContentId]);
+    
+    if (result.rows.length === 0) {
+      throw new Error(`Diff content not found: ${diffContentId}`);
+    }
+    
+    const diff = result.rows[0];
+    
+    // We only support external storage now
+    return fs.readFileSync(diff.external_storage_path, 'utf8');
+  }
+
+  // Get diffs for a commit
+  async getCommitDiffs(commitId) {
+    const query = `
+      SELECT dc.*, rcf.change_type as file_change_type
+      FROM diff_contents dc
+      JOIN repository_commit_files rcf ON dc.file_change_id = rcf.id
+      WHERE dc.commit_id = $1
+      ORDER BY dc.file_path
+    `;
+    
+    const result = await database.query(query, [commitId]);
+    return result.rows;
+  }
+
+  // Get repository diff statistics
+  async getRepositoryDiffStats(repositoryId, daysBack = 30) {
+    const query = `
+      SELECT * FROM get_repository_diff_stats($1, $2)
+    `;
+    
+    const result = await database.query(query, [repositoryId, daysBack]);
+    return result.rows[0];
+  }
+
+  // Clean up old external diff files
+  async cleanupOldDiffs(daysBack = 30) {
+    const cutoffDate = new Date();
+    cutoffDate.setDate(cutoffDate.getDate() - daysBack);
+    
+    // Get old external diffs
+    const query = `
+      SELECT external_storage_path 
+      FROM diff_contents 
+      WHERE storage_type = 'external' 
+        AND created_at < $1
+    `;
+    
+    const result = await database.query(query, [cutoffDate]);
+    
+    // Delete files
+    let deletedCount = 0;
+    for (const row of result.rows) {
+      try {
+        if (fs.existsSync(row.external_storage_path)) {
+          fs.unlinkSync(row.external_storage_path);
+          deletedCount++;
+        }
+      } catch (error) {
+        console.warn(`Failed to delete diff file ${row.external_storage_path}:`, error.message);
+      }
+    }
+    
+    // Delete database records
+    const deleteQuery = `
+      DELETE FROM diff_contents 
+      WHERE storage_type = 'external' 
+        AND created_at < $1
+    `;
+    
+    await database.query(deleteQuery, [cutoffDate]);
+    
+    return deletedCount;
+  }
+}
+
+module.exports = DiffProcessingService;

services/git-integration/src/services/enhanced-diff-processing.service.js

@@ -0,0 +1,336 @@
+const fs = require('fs').promises;
+const path = require('path');
+const { execSync } = require('child_process');
+const DiffStorageModel = require('../models/diff-storage.model');
+
+class EnhancedDiffProcessingService {
+  constructor() {
+    this.diffStorageModel = new DiffStorageModel();
+    this.storageBasePath = process.env.DIFF_STORAGE_PATH || '/tmp/diffs';
+    this.maxDiffSize = parseInt(process.env.MAX_DIFF_SIZE_BYTES) || 10 * 1024 * 1024; // 10MB default
+  }
+
+  /**
+   * Initialize the diff processing service
+   */
+  async initialize() {
+    try {
+      await fs.mkdir(this.storageBasePath, { recursive: true });
+      console.log(`Diff storage initialized at: ${this.storageBasePath}`);
+    } catch (error) {
+      console.error('Error initializing diff storage:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process the next item in the diff processing queue
+   * @returns {Object|null} Processing result or null if no items
+   */
+  async processNextQueueItem() {
+    try {
+      const queueItem = await this.diffStorageModel.getNextQueueItem();
+      if (!queueItem) {
+        return null;
+      }
+
+      console.log(`Processing diff queue item: ${queueItem.id}`);
+      
+      // Update status to processing
+      await this.diffStorageModel.updateQueueStatus(queueItem.id, 'processing');
+
+      try {
+        const result = await this.processDiffForCommit(queueItem);
+        
+        // Update status to completed
+        await this.diffStorageModel.updateQueueStatus(queueItem.id, 'completed');
+        
+        return result;
+      } catch (error) {
+        console.error(`Error processing diff queue item ${queueItem.id}:`, error);
+        
+        // Increment retry count
+        const updatedItem = await this.diffStorageModel.incrementRetryCount(queueItem.id);
+        
+        // Check if we should retry or mark as failed
+        if (updatedItem.retry_count >= updatedItem.max_retries) {
+          await this.diffStorageModel.updateQueueStatus(
+            queueItem.id, 
+            'failed', 
+            error.message
+          );
+        } else {
+          await this.diffStorageModel.updateQueueStatus(queueItem.id, 'pending');
+        }
+        
+        throw error;
+      }
+    } catch (error) {
+      console.error('Error processing queue item:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process diff for a specific commit
+   * @param {Object} queueItem - Queue item data
+   * @returns {Object} Processing result
+   */
+  async processDiffForCommit(queueItem) {
+    const { commit_id, repository_id, from_sha, to_sha, repo_local_path } = queueItem;
+    
+    try {
+      // Generate diff using git
+      const diffOutput = await this.generateGitDiff(repo_local_path, from_sha, to_sha);
+      
+      if (!diffOutput || diffOutput.trim().length === 0) {
+        console.log(`No diff output for commit ${to_sha}`);
+        return { processed_files: 0, total_size: 0 };
+      }
+
+      // Parse diff output into individual file diffs
+      const fileDiffs = this.parseDiffOutput(diffOutput);
+      
+      const processedFiles = [];
+      let totalSize = 0;
+
+      for (const fileDiff of fileDiffs) {
+        try {
+          const diffContent = await this.storeDiffContent(
+            commit_id,
+            repository_id,
+            fileDiff
+          );
+          
+          processedFiles.push(diffContent);
+          totalSize += diffContent.diff_size_bytes;
+        } catch (error) {
+          console.error(`Error storing diff for file ${fileDiff.file_path}:`, error);
+          // Continue with other files
+        }
+      }
+
+      console.log(`Processed ${processedFiles.length} file diffs, total size: ${totalSize} bytes`);
+      
+      return {
+        processed_files: processedFiles.length,
+        total_size: totalSize,
+        files: processedFiles
+      };
+    } catch (error) {
+      console.error('Error processing diff for commit:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Generate git diff using command line
+   * @param {string} repoPath - Repository path
+   * @param {string} fromSha - From commit SHA
+   * @param {string} toSha - To commit SHA
+   * @returns {string} Diff output
+   */
+  async generateGitDiff(repoPath, fromSha, toSha) {
+    try {
+      const command = fromSha 
+        ? `git -C "${repoPath}" diff ${fromSha}..${toSha}`
+        : `git -C "${repoPath}" show --format="" ${toSha}`;
+      
+      const diffOutput = execSync(command, { 
+        encoding: 'utf8',
+        maxBuffer: this.maxDiffSize 
+      });
+      
+      return diffOutput;
+    } catch (error) {
+      if (error.code === 'ENOENT') {
+        throw new Error(`Repository not found at path: ${repoPath}`);
+      }
+      throw new Error(`Git diff failed: ${error.message}`);
+    }
+  }
+
+  /**
+   * Parse diff output into individual file diffs
+   * @param {string} diffOutput - Raw diff output
+   * @returns {Array} Array of file diff objects
+   */
+  parseDiffOutput(diffOutput) {
+    const fileDiffs = [];
+    const lines = diffOutput.split('\n');
+    
+    let currentFile = null;
+    let currentDiff = [];
+    
+    for (const line of lines) {
+      if (line.startsWith('diff --git ')) {
+        // Save previous file diff if exists
+        if (currentFile && currentDiff.length > 0) {
+          fileDiffs.push({
+            ...currentFile,
+            diff_content: currentDiff.join('\n')
+          });
+        }
+        
+        // Start new file diff
+        const match = line.match(/diff --git a\/(.+) b\/(.+)/);
+        if (match) {
+          currentFile = {
+            file_path: match[2],
+            diff_header: line,
+            change_type: 'modified' // Default, will be refined below
+          };
+          currentDiff = [line];
+        }
+      } else if (line.startsWith('new file mode')) {
+        if (currentFile) currentFile.change_type = 'added';
+        currentDiff.push(line);
+      } else if (line.startsWith('deleted file mode')) {
+        if (currentFile) currentFile.change_type = 'deleted';
+        currentDiff.push(line);
+      } else if (line.startsWith('rename from')) {
+        if (currentFile) currentFile.change_type = 'renamed';
+        currentDiff.push(line);
+      } else {
+        currentDiff.push(line);
+      }
+    }
+    
+    // Don't forget the last file
+    if (currentFile && currentDiff.length > 0) {
+      fileDiffs.push({
+        ...currentFile,
+        diff_content: currentDiff.join('\n')
+      });
+    }
+    
+    return fileDiffs;
+  }
+
+  /**
+   * Store diff content to external storage
+   * @param {string} commit_id - Commit ID
+   * @param {string} repository_id - Repository ID
+   * @param {Object} fileDiff - File diff data
+   * @returns {Object} Stored diff content record
+   */
+  async storeDiffContent(commit_id, repository_id, fileDiff) {
+    try {
+      const diffSizeBytes = Buffer.byteLength(fileDiff.diff_content, 'utf8');
+      
+      // Generate storage path
+      const fileName = `${commit_id}_${this.sanitizeFileName(fileDiff.file_path)}.diff`;
+      const storagePath = path.join(this.storageBasePath, repository_id, fileName);
+      
+      // Ensure directory exists
+      await fs.mkdir(path.dirname(storagePath), { recursive: true });
+      
+      // Write diff content to file
+      await fs.writeFile(storagePath, fileDiff.diff_content, 'utf8');
+      
+      // Create database record
+      const diffData = {
+        commit_id,
+        diff_header: fileDiff.diff_header,
+        diff_size_bytes: diffSizeBytes,
+        storage_type: 'external',
+        external_storage_path: storagePath,
+        external_storage_provider: 'local',
+        file_path: fileDiff.file_path,
+        change_type: fileDiff.change_type,
+        processing_status: 'processed'
+      };
+      
+      const diffContent = await this.diffStorageModel.createDiffContent(diffData);
+      
+      console.log(`Stored diff for ${fileDiff.file_path} (${diffSizeBytes} bytes)`);
+      
+      return diffContent;
+    } catch (error) {
+      console.error('Error storing diff content:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Retrieve diff content from storage
+   * @param {string} diff_id - Diff content ID
+   * @returns {string} Diff content
+   */
+  async retrieveDiffContent(diff_id) {
+    try {
+      const diffRecord = await this.diffStorageModel.getDiffById(diff_id);
+      if (!diffRecord) {
+        throw new Error(`Diff record not found: ${diff_id}`);
+      }
+      
+      if (diffRecord.storage_type === 'external') {
+        const content = await fs.readFile(diffRecord.external_storage_path, 'utf8');
+        return content;
+      }
+      
+      throw new Error(`Unsupported storage type: ${diffRecord.storage_type}`);
+    } catch (error) {
+      console.error('Error retrieving diff content:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Start background processing of diff queue
+   * @param {number} intervalMs - Processing interval in milliseconds
+   */
+  startBackgroundProcessing(intervalMs = 30000) {
+    console.log(`Starting background diff processing (interval: ${intervalMs}ms)`);
+    
+    const processQueue = async () => {
+      try {
+        const result = await this.processNextQueueItem();
+        if (result) {
+          console.log(`Processed diff queue item: ${result.processed_files} files`);
+        }
+      } catch (error) {
+        console.error('Background diff processing error:', error);
+      }
+    };
+    
+    // Process immediately and then on interval
+    processQueue();
+    setInterval(processQueue, intervalMs);
+  }
+
+  /**
+   * Clean up old diff files and queue entries
+   * @param {number} daysBack - Days to keep
+   * @returns {Object} Cleanup results
+   */
+  async cleanupOldData(daysBack = 30) {
+    try {
+      // Clean up queue entries
+      const deletedQueueItems = await this.diffStorageModel.cleanupOldQueueEntries(7);
+      
+      // TODO: Clean up old diff files from storage
+      // This would require querying diff_contents for old records and deleting files
+      
+      console.log(`Cleanup completed: ${deletedQueueItems} queue items deleted`);
+      
+      return {
+        deleted_queue_items: deletedQueueItems
+      };
+    } catch (error) {
+      console.error('Error during cleanup:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Sanitize file name for storage
+   * @param {string} fileName - Original file name
+   * @returns {string} Sanitized file name
+   */
+  sanitizeFileName(fileName) {
+    return fileName.replace(/[^a-zA-Z0-9._-]/g, '_');
+  }
+}
+
+module.exports = EnhancedDiffProcessingService;

services/git-integration/src/services/enhanced-webhook.service.js

@@ -0,0 +1,509 @@
+const { v4: uuidv4 } = require('uuid');
+const database = require('../config/database');
+const CommitTrackingService = require('./commit-tracking.service');
+
+class EnhancedWebhookService {
+  constructor() {
+    this.db = database;
+    this.commitTrackingService = new CommitTrackingService();
+  }
+
+  /**
+   * Process GitHub webhook
+   * @param {Object} payload - GitHub webhook payload
+   * @param {Object} headers - Request headers
+   * @returns {Object} Processing result
+   */
+  async processGitHubWebhook(payload, headers) {
+    try {
+      const deliveryId = headers['x-github-delivery'];
+      const eventType = headers['x-github-event'];
+      
+      // Store webhook record
+      const webhookRecord = await this.storeGitHubWebhook({
+        delivery_id: deliveryId,
+        event_type: eventType,
+        action: payload.action,
+        owner_name: payload.repository?.owner?.login,
+        repository_name: payload.repository?.name,
+        ref: payload.ref,
+        before_sha: payload.before,
+        after_sha: payload.after,
+        commit_count: payload.commits?.length || 0,
+        payload: payload
+      });
+
+      // Process based on event type
+      let processingResult = null;
+      
+      if (eventType === 'push') {
+        processingResult = await this.processPushEvent(payload, 'github');
+      } else if (eventType === 'pull_request') {
+        processingResult = await this.processPullRequestEvent(payload, 'github');
+      }
+
+      // Update webhook as processed
+      await this.markWebhookProcessed(webhookRecord.id, 'github');
+
+      return {
+        success: true,
+        webhook_id: webhookRecord.id,
+        event_type: eventType,
+        processing_result: processingResult
+      };
+    } catch (error) {
+      console.error('Error processing GitHub webhook:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process GitLab webhook
+   * @param {Object} payload - GitLab webhook payload
+   * @param {Object} headers - Request headers
+   * @returns {Object} Processing result
+   */
+  async processGitLabWebhook(payload, headers) {
+    try {
+      const deliveryId = headers['x-gitlab-event-uuid'];
+      const eventType = headers['x-gitlab-event'];
+      
+      // Store webhook record
+      const webhookRecord = await this.storeGitLabWebhook({
+        delivery_id: deliveryId,
+        event_type: eventType,
+        action: payload.object_kind,
+        owner_name: payload.project?.namespace || payload.user_username,
+        repository_name: payload.project?.name || payload.repository?.name,
+        ref: payload.ref,
+        before_sha: payload.before,
+        after_sha: payload.after,
+        commit_count: payload.total_commits_count || payload.commits?.length || 0,
+        payload: payload
+      });
+
+      // Process based on event type
+      let processingResult = null;
+      
+      if (eventType === 'Push Hook' || payload.object_kind === 'push') {
+        processingResult = await this.processPushEvent(payload, 'gitlab');
+      } else if (eventType === 'Merge Request Hook' || payload.object_kind === 'merge_request') {
+        processingResult = await this.processPullRequestEvent(payload, 'gitlab');
+      }
+
+      return {
+        success: true,
+        webhook_id: webhookRecord.id,
+        event_type: eventType,
+        processing_result: processingResult
+      };
+    } catch (error) {
+      console.error('Error processing GitLab webhook:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process Bitbucket webhook
+   * @param {Object} payload - Bitbucket webhook payload
+   * @param {Object} headers - Request headers
+   * @returns {Object} Processing result
+   */
+  async processBitbucketWebhook(payload, headers) {
+    try {
+      const deliveryId = headers['x-request-uuid'];
+      const eventType = headers['x-event-key'];
+      
+      // Store webhook record
+      const webhookRecord = await this.storeBitbucketWebhook({
+        delivery_id: deliveryId,
+        event_type: eventType,
+        action: payload.action || 'push',
+        owner_name: payload.repository?.owner?.username || payload.repository?.workspace?.slug,
+        repository_name: payload.repository?.name,
+        ref: payload.push?.changes?.[0]?.new?.name,
+        before_sha: payload.push?.changes?.[0]?.old?.target?.hash,
+        after_sha: payload.push?.changes?.[0]?.new?.target?.hash,
+        commit_count: payload.push?.changes?.[0]?.commits?.length || 0,
+        payload: payload
+      });
+
+      // Process based on event type
+      let processingResult = null;
+      
+      if (eventType === 'repo:push') {
+        processingResult = await this.processPushEvent(payload, 'bitbucket');
+      } else if (eventType === 'pullrequest:created' || eventType === 'pullrequest:updated') {
+        processingResult = await this.processPullRequestEvent(payload, 'bitbucket');
+      }
+
+      return {
+        success: true,
+        webhook_id: webhookRecord.id,
+        event_type: eventType,
+        processing_result: processingResult
+      };
+    } catch (error) {
+      console.error('Error processing Bitbucket webhook:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process Gitea webhook
+   * @param {Object} payload - Gitea webhook payload
+   * @param {Object} headers - Request headers
+   * @returns {Object} Processing result
+   */
+  async processGiteaWebhook(payload, headers) {
+    try {
+      const deliveryId = headers['x-gitea-delivery'];
+      const eventType = headers['x-gitea-event'];
+      
+      // Store webhook record
+      const webhookRecord = await this.storeGiteaWebhook({
+        delivery_id: deliveryId,
+        event_type: eventType,
+        action: payload.action,
+        owner_name: payload.repository?.owner?.login,
+        repository_name: payload.repository?.name,
+        ref: payload.ref,
+        before_sha: payload.before,
+        after_sha: payload.after,
+        commit_count: payload.commits?.length || 0,
+        payload: payload
+      });
+
+      // Process based on event type
+      let processingResult = null;
+      
+      if (eventType === 'push') {
+        processingResult = await this.processPushEvent(payload, 'gitea');
+      } else if (eventType === 'pull_request') {
+        processingResult = await this.processPullRequestEvent(payload, 'gitea');
+      }
+
+      return {
+        success: true,
+        webhook_id: webhookRecord.id,
+        event_type: eventType,
+        processing_result: processingResult
+      };
+    } catch (error) {
+      console.error('Error processing Gitea webhook:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process push event from any provider
+   * @param {Object} payload - Webhook payload
+   * @param {string} provider - Git provider (github, gitlab, bitbucket, gitea)
+   * @returns {Object} Processing result
+   */
+  async processPushEvent(payload, provider) {
+    try {
+      // Find repository record
+      const repositoryId = await this.findRepositoryId(payload, provider);
+      
+      if (!repositoryId) {
+        console.warn(`Repository not found for ${provider} push event`);
+        return { processed_commits: 0, message: 'Repository not found' };
+      }
+
+      // Extract commits based on provider format
+      const commits = this.extractCommitsFromPayload(payload, provider);
+      
+      if (!commits || commits.length === 0) {
+        return { processed_commits: 0, message: 'No commits in payload' };
+      }
+
+      // Process commits using commit tracking service
+      const processedCommits = await this.commitTrackingService.processWebhookCommits(
+        { commits }, 
+        repositoryId
+      );
+
+      // Update repository last sync info
+      await this.updateRepositorySync(repositoryId, payload, provider);
+
+      return {
+        processed_commits: processedCommits.length,
+        commits: processedCommits,
+        repository_id: repositoryId
+      };
+    } catch (error) {
+      console.error('Error processing push event:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Process pull request event from any provider
+   * @param {Object} payload - Webhook payload
+   * @param {string} provider - Git provider
+   * @returns {Object} Processing result
+   */
+  async processPullRequestEvent(payload, provider) {
+    try {
+      // For now, just log PR events - can be extended later
+      console.log(`Processing ${provider} pull request event:`, {
+        action: payload.action,
+        pr_number: payload.number || payload.pull_request?.number || payload.pullrequest?.id,
+        title: payload.pull_request?.title || payload.pullrequest?.title
+      });
+
+      return {
+        event_type: 'pull_request',
+        action: payload.action,
+        message: 'Pull request event logged'
+      };
+    } catch (error) {
+      console.error('Error processing pull request event:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Extract commits from webhook payload based on provider
+   * @param {Object} payload - Webhook payload
+   * @param {string} provider - Git provider
+   * @returns {Array} Normalized commits array
+   */
+  extractCommitsFromPayload(payload, provider) {
+    switch (provider) {
+      case 'github':
+      case 'gitea':
+        return payload.commits || [];
+      
+      case 'gitlab':
+        return payload.commits || [];
+      
+      case 'bitbucket':
+        return payload.push?.changes?.[0]?.commits || [];
+      
+      default:
+        return [];
+    }
+  }
+
+  /**
+   * Find repository ID based on webhook payload
+   * @param {Object} payload - Webhook payload
+   * @param {string} provider - Git provider
+   * @returns {string|null} Repository ID
+   */
+  async findRepositoryId(payload, provider) {
+    try {
+      let ownerName, repositoryName;
+
+      switch (provider) {
+        case 'github':
+        case 'gitea':
+          ownerName = payload.repository?.owner?.login;
+          repositoryName = payload.repository?.name;
+          break;
+        
+        case 'gitlab':
+          ownerName = payload.project?.namespace || payload.user_username;
+          repositoryName = payload.project?.name || payload.repository?.name;
+          break;
+        
+        case 'bitbucket':
+          ownerName = payload.repository?.owner?.username || payload.repository?.workspace?.slug;
+          repositoryName = payload.repository?.name;
+          break;
+        
+        default:
+          return null;
+      }
+
+      if (!ownerName || !repositoryName) {
+        return null;
+      }
+
+      const query = `
+        SELECT id FROM github_repositories 
+        WHERE owner_name = $1 AND repository_name = $2
+        LIMIT 1
+      `;
+      
+      const result = await this.db.query(query, [ownerName, repositoryName]);
+      return result.rows[0]?.id || null;
+    } catch (error) {
+      console.error('Error finding repository ID:', error);
+      return null;
+    }
+  }
+
+  /**
+   * Update repository sync information
+   * @param {string} repositoryId - Repository ID
+   * @param {Object} payload - Webhook payload
+   * @param {string} provider - Git provider
+   */
+  async updateRepositorySync(repositoryId, payload, provider) {
+    try {
+      let afterSha;
+      
+      switch (provider) {
+        case 'github':
+        case 'gitea':
+          afterSha = payload.after;
+          break;
+        case 'gitlab':
+          afterSha = payload.after;
+          break;
+        case 'bitbucket':
+          afterSha = payload.push?.changes?.[0]?.new?.target?.hash;
+          break;
+      }
+
+      if (afterSha) {
+        const query = `
+          UPDATE github_repositories 
+          SET last_synced_at = NOW(), 
+              last_synced_commit_sha = $2,
+              sync_status = 'completed'
+          WHERE id = $1
+        `;
+        
+        await this.db.query(query, [repositoryId, afterSha]);
+      }
+    } catch (error) {
+      console.error('Error updating repository sync:', error);
+      // Don't throw - this is not critical
+    }
+  }
+
+  // Webhook storage methods
+  async storeGitHubWebhook(webhookData) {
+    const query = `
+      INSERT INTO github_webhooks (
+        id, delivery_id, event_type, action, owner_name, repository_name,
+        ref, before_sha, after_sha, commit_count, payload, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      webhookData.delivery_id,
+      webhookData.event_type,
+      webhookData.action,
+      webhookData.owner_name,
+      webhookData.repository_name,
+      webhookData.ref,
+      webhookData.before_sha,
+      webhookData.after_sha,
+      webhookData.commit_count,
+      JSON.stringify(webhookData.payload)
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async storeGitLabWebhook(webhookData) {
+    const query = `
+      INSERT INTO gitlab_webhooks (
+        id, delivery_id, event_type, action, owner_name, repository_name,
+        ref, before_sha, after_sha, commit_count, payload, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      webhookData.delivery_id,
+      webhookData.event_type,
+      webhookData.action,
+      webhookData.owner_name,
+      webhookData.repository_name,
+      webhookData.ref,
+      webhookData.before_sha,
+      webhookData.after_sha,
+      webhookData.commit_count,
+      JSON.stringify(webhookData.payload)
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async storeBitbucketWebhook(webhookData) {
+    const query = `
+      INSERT INTO bitbucket_webhooks (
+        id, delivery_id, event_type, action, owner_name, repository_name,
+        ref, before_sha, after_sha, commit_count, payload, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      webhookData.delivery_id,
+      webhookData.event_type,
+      webhookData.action,
+      webhookData.owner_name,
+      webhookData.repository_name,
+      webhookData.ref,
+      webhookData.before_sha,
+      webhookData.after_sha,
+      webhookData.commit_count,
+      JSON.stringify(webhookData.payload)
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async storeGiteaWebhook(webhookData) {
+    const query = `
+      INSERT INTO gitea_webhooks (
+        id, delivery_id, event_type, action, owner_name, repository_name,
+        ref, before_sha, after_sha, commit_count, payload, created_at, updated_at
+      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, NOW(), NOW())
+      RETURNING *
+    `;
+
+    const values = [
+      uuidv4(),
+      webhookData.delivery_id,
+      webhookData.event_type,
+      webhookData.action,
+      webhookData.owner_name,
+      webhookData.repository_name,
+      webhookData.ref,
+      webhookData.before_sha,
+      webhookData.after_sha,
+      webhookData.commit_count,
+      JSON.stringify(webhookData.payload)
+    ];
+
+    const result = await this.db.query(query, values);
+    return result.rows[0];
+  }
+
+  async markWebhookProcessed(webhookId, provider) {
+    const tableMap = {
+      'github': 'github_webhooks',
+      'gitlab': 'gitlab_webhooks',
+      'bitbucket': 'bitbucket_webhooks',
+      'gitea': 'gitea_webhooks'
+    };
+
+    const tableName = tableMap[provider];
+    if (!tableName) return;
+
+    const query = `
+      UPDATE ${tableName} 
+      SET processed_at = NOW(), updated_at = NOW()
+      WHERE id = $1
+    `;
+
+    await this.db.query(query, [webhookId]);
+  }
+}
+
+module.exports = EnhancedWebhookService;

services/git-integration/src/services/file-storage.service.js

@@ -178,10 +178,7 @@ class FileStorageService {
       
       const fileRecord = fileResult.rows[0];
       
-      // Process file content if it's a text file and not too large (< 10MB)
+   
-      if (!isBinary && stats.size < 10 * 1024 * 1024) {
-        await this.processFileContent(fileRecord.id, absolutePath, extension);
-      }
       
       return fileRecord;
       
@@ -191,36 +188,7 @@ class FileStorageService {
     }
   }
 
-  // Process and store file content
-  async processFileContent(fileId, absolutePath, extension) {
-    try {
-      const content = fs.readFileSync(absolutePath, 'utf-8');
-      const lines = content.split('\n');
-      const preview = content.substring(0, 1000);
-      const language = this.languageMap[extension] || 'text';
 
-      const contentQuery = `
-        INSERT INTO repository_file_contents (
-          file_id, content_text, content_preview, language_detected,
-          line_count, char_count
-        ) VALUES ($1, $2, $3, $4, $5, $6)
-        ON CONFLICT (file_id) DO UPDATE SET
-          content_text = $2,
-          content_preview = $3,
-          language_detected = $4,
-          line_count = $5,
-          char_count = $6,
-          updated_at = NOW()
-      `;
-      
-      await database.query(contentQuery, [
-        fileId, content, preview, language, lines.length, content.length
-      ]);
-      
-    } catch (error) {
-      console.warn(`Error processing file content for file ID ${fileId}:`, error.message);
-    }
-  }
 
   // Complete storage process for a repository
   async completeRepositoryStorage(storageId) {
@@ -307,10 +275,9 @@ class FileStorageService {
   // Get files in a directory
   async getDirectoryFiles(repositoryId, directoryPath = '') {
     const query = `
-      SELECT rf.*, rfc.language_detected, rfc.line_count
+      SELECT rf.*
       FROM repository_files rf
       LEFT JOIN repository_directories rd ON rf.directory_id = rd.id
-      LEFT JOIN repository_file_contents rfc ON rf.id = rfc.file_id
       WHERE rf.repository_id = $1 AND rd.relative_path = $2
       ORDER BY rf.filename
     `;
@@ -321,19 +288,7 @@ class FileStorageService {
 
   // Search files by content
   async searchFileContent(repositoryId, searchQuery) {
-    const query = `
+    return [];
-      SELECT rf.filename, rf.relative_path, rfc.language_detected,
-             ts_rank_cd(to_tsvector('english', rfc.content_text), plainto_tsquery('english', $2)) as rank
-      FROM repository_files rf
-      JOIN repository_file_contents rfc ON rf.id = rfc.file_id
-      WHERE rf.repository_id = $1 
-        AND to_tsvector('english', rfc.content_text) @@ plainto_tsquery('english', $2)
-      ORDER BY rank DESC, rf.filename
-      LIMIT 50
-    `;
-    
-    const result = await database.query(query, [repositoryId, searchQuery]);
-    return result.rows;
   }
 
   // Utility methods
@@ -373,7 +328,6 @@ class FileStorageService {
   // Clean up storage for a repository
   async cleanupRepositoryStorage(repositoryId) {
     const queries = [
-      'DELETE FROM repository_file_contents WHERE file_id IN (SELECT id FROM repository_files WHERE repository_id = $1)',
       'DELETE FROM repository_files WHERE repository_id = $1',
       'DELETE FROM repository_directories WHERE repository_id = $1',
       'DELETE FROM repository_storage WHERE repository_id = $1'

services/git-integration/src/services/git-repo.service.js

@@ -120,6 +120,20 @@ class GitRepoService {
   }
 
   async getDiff(repoPath, fromSha, toSha, options = { patch: true }) {
+       // Ensure both SHAs exist locally; if using shallow clone, fetch missing objects
+       try {
+        if (fromSha) {
+          await this.runGit(repoPath, ['cat-file', '-e', `${fromSha}^{commit}`]).catch(async () => {
+            await this.runGit(repoPath, ['fetch', '--depth', '200', 'origin', fromSha]);
+          });
+        }
+        if (toSha && toSha !== 'HEAD') {
+          await this.runGit(repoPath, ['cat-file', '-e', `${toSha}^{commit}`]).catch(async () => {
+            await this.runGit(repoPath, ['fetch', '--depth', '200', 'origin', toSha]);
+          });
+        }  
+      } catch (_) {}
+
     const range = fromSha && toSha ? `${fromSha}..${toSha}` : toSha ? `${toSha}^..${toSha}` : '';
     const mode = options.patch ? '--patch' : '--name-status';
     const args = ['diff', mode];
@@ -129,6 +143,12 @@ class GitRepoService {
   }
 
   async getChangedFilesSince(repoPath, sinceSha) {
+      // Ensure SHA exists locally in case of shallow clone
+      try {
+        await this.runGit(repoPath, ['cat-file', '-e', `${sinceSha}^{commit}`]).catch(async () => { 
+          await this.runGit(repoPath, ['fetch', '--depth', '200', 'origin', sinceSha]);  
+        });  
+      } catch (_) {}
     const output = await this.runGit(repoPath, ['diff', '--name-status', `${sinceSha}..HEAD`]);
     const lines = output.split('\n').filter(Boolean);
     return lines.map(line => {

services/git-integration/src/services/gitea-oauth.js

@@ -1,6 +1,8 @@
 // services/gitea-oauth.js
 const database = require('../config/database');
 
+const axios = require('axios');
+
 class GiteaOAuthService {
   constructor() {
     this.clientId = process.env.GITEA_CLIENT_ID;
@@ -17,42 +19,135 @@ class GiteaOAuthService {
       redirect_uri: this.redirectUri,
       response_type: 'code',
       // Request both user and repository read scopes
-      scope: 'read:user read:repository',
+      scope: 'read:user read:repository write:repository',
       state
     });
-    return `${authUrl}?${params.toString()}`;
+    const fullUrl = `${authUrl}?${params.toString()}`;
+    console.log(`🔗 [GITEA OAUTH] Generated auth URL: ${fullUrl}`);
+    return fullUrl;
   }
 
   async exchangeCodeForToken(code) {
     const tokenUrl = `${this.baseUrl}/login/oauth/access_token`;
-    const resp = await fetch(tokenUrl, {
+    console.log(`🔄 [GITEA OAUTH] Exchanging code for token at: ${tokenUrl}`);
-      method: 'POST',
+    console.log(`🔧 [GITEA OAUTH] Config - Base URL: ${this.baseUrl}, Client ID: ${this.clientId?.substring(0, 8)}...`);
-      headers: { 'Accept': 'application/json', 'Content-Type': 'application/x-www-form-urlencoded' },
+    // Validate required configuration
-      body: new URLSearchParams({
+    if (!this.clientId) {
+      throw new Error('GITEA_CLIENT_ID is not configured');
+    }
+    if (!this.clientSecret) {
+      throw new Error('GITEA_CLIENT_SECRET is not configured');
+    }
+    try {
+      const response = await axios.post(tokenUrl, new URLSearchParams({
         client_id: this.clientId,
         client_secret: this.clientSecret,
         code,
         grant_type: 'authorization_code',
-        redirect_uri: this.redirectUri
+        redirect_uri: this.redirectUri,
-      })
+        scope: 'read:user read:repository write:repository'
-    });
+      }), {
-    let data = null;
+        headers: {
-    try { data = await resp.json(); } catch (_) { data = null; }
+          'Content-Type': 'application/x-www-form-urlencoded',
-    if (!resp.ok || data?.error) {
+          'Accept': 'application/json',
-      const detail = data?.error_description || data?.error || (await resp.text().catch(() => '')) || 'unknown_error';
+          'User-Agent': 'CodeNuk-GitIntegration/1.0'
-      throw new Error(`Gitea token exchange failed: ${detail}`);
+        },
+        timeout: 30000,
+        maxRedirects: 0,
+        // Add network configuration to handle connectivity issues
+        httpsAgent: new (require('https').Agent)({
+          keepAlive: true,
+          timeout: 30000,
+          // Force IPv4 to avoid IPv6 connectivity issues
+          family: 4
+        }),
+        // Add retry configuration
+        validateStatus: function (status) {
+          return status >= 200 && status < 300;
         }
-    return data.access_token;
+      });
+      console.log(`📥 [GITEA OAUTH] Response status: ${response.status} ${response.statusText}`);
+      console.log(`📥 [GITEA OAUTH] Response data:`, response.data);
+      if (response.data.error) {
+        console.error(`❌ [GITEA OAUTH] Token exchange failed:`, response.data);
+        throw new Error(response.data.error_description || response.data.error || 'Gitea token exchange failed');
+      }
+      if (!response.data.access_token) {
+        console.error(`❌ [GITEA OAUTH] No access token in response:`, response.data);
+        throw new Error('No access token received from Gitea OAuth');
+      }
+      console.log(`✅ [GITEA OAUTH] Token exchange successful`);
+      return response.data.access_token;
+    } catch (e) {
+      console.error(`❌ [GITEA OAUTH] Token exchange error:`, e);
+      // Handle AggregateError (multiple network errors)
+      if (e.name === 'AggregateError' && e.errors && e.errors.length > 0) {
+        const firstError = e.errors[0];
+        if (firstError.code === 'ETIMEDOUT') {
+          throw new Error(`Gitea OAuth timeout: Request to ${tokenUrl} timed out after 30 seconds`);
+        } else if (firstError.code === 'ENOTFOUND' || firstError.code === 'ECONNREFUSED') {
+          throw new Error(`Gitea OAuth network error: Cannot connect to ${this.baseUrl}. Please check your network connection and GITEA_BASE_URL configuration`);
+        } else {
+          throw new Error(`Gitea OAuth network error: ${firstError.message || 'Connection failed'}`);
+        }
+      }
+      if (e.code === 'ECONNABORTED' || e.message.includes('timeout')) {
+        throw new Error(`Gitea OAuth timeout: Request to ${tokenUrl} timed out after 30 seconds`);
+      } else if (e.code === 'ENOTFOUND' || e.code === 'ECONNREFUSED' || e.message.includes('Network Error')) {
+        throw new Error(`Gitea OAuth network error: Cannot connect to ${this.baseUrl}. Please check your network connection and GITEA_BASE_URL configuration`);
+      } else if (e.response) {
+        // Handle HTTP error responses
+        const status = e.response.status;
+        const data = e.response.data;
+        throw new Error(`Gitea OAuth HTTP error ${status}: ${JSON.stringify(data)}`);
+      } else {
+        throw new Error(`Gitea OAuth error: ${e.message || 'Unknown error occurred'}`);
+      }
+    }
+  
   }
 
   async getUserInfo(accessToken) {
-    const resp = await fetch(`${this.baseUrl}/api/v1/user`, { headers: { Authorization: `Bearer ${accessToken}` } });
+
-    if (!resp.ok) {
+    const userUrl = `${this.baseUrl}/api/v1/user`;
-      let txt = '';
+    console.log(`🔄 [GITEA OAUTH] Fetching user info from: ${userUrl}`);
-      try { txt = await resp.text(); } catch (_) {}
+    try {
-      throw new Error(`Failed to fetch Gitea user (status ${resp.status}): ${txt}`);
+      const response = await axios.get(userUrl, {
+        headers: {
+          'Authorization': `token ${accessToken}`,
+          'Accept': 'application/json',
+          'User-Agent': 'CodeNuk-GitIntegration/1.0'
+        },
+        timeout: 15000,
+        // Add network configuration to handle connectivity issues
+        httpsAgent: new (require('https').Agent)({
+          keepAlive: true,
+          timeout: 15000,
+          // Force IPv4 to avoid IPv6 connectivity issues
+          family: 4
+        }),
+        // Add retry configuration
+        validateStatus: function (status) {
+          return status >= 200 && status < 300;
         }
-    return await resp.json();
+      });
+      console.log(`📥 [GITEA OAUTH] User info response status: ${response.status} ${response.statusText}`);
+      console.log(`✅ [GITEA OAUTH] User info retrieved successfully for: ${response.data.login || response.data.username}`);
+      return response.data;
+    } catch (e) {
+      console.error(`❌ [GITEA OAUTH] User info error:`, e);      
+      if (e.response) {
+        console.error(`❌ [GITEA OAUTH] User info failed:`, e.response.data);
+        throw new Error(`Failed to fetch Gitea user (${e.response.status}): ${JSON.stringify(e.response.data)}`);
+      } else if (e.code === 'ECONNABORTED' || e.message.includes('timeout')) {
+        throw new Error('Gitea user info timeout: Request timed out after 15 seconds');
+      } else if (e.code === 'ENOTFOUND' || e.code === 'ECONNREFUSED' || e.message.includes('Network Error')) {
+        throw new Error(`Gitea user info network error: Cannot connect to ${this.baseUrl}`);
+      } else {
+        throw e;
+      }
+    }
+   
   }
 
   async storeToken(accessToken, user) {
@@ -61,7 +156,7 @@ class GiteaOAuthService {
        VALUES ($1, $2, $3, $4, $5)
        ON CONFLICT (id) DO UPDATE SET access_token = EXCLUDED.access_token, gitea_username = EXCLUDED.gitea_username, gitea_user_id = EXCLUDED.gitea_user_id, scopes = EXCLUDED.scopes, expires_at = EXCLUDED.expires_at, updated_at = NOW()
        RETURNING *`,
-      [accessToken, user.login, user.id, JSON.stringify(['read:repository']), null]
+      [accessToken, user.login, user.id, JSON.stringify(['read:user','read:repository']), null]
     );
     return result.rows[0];
   }

services/git-integration/src/services/providers/bitbucket.adapter.js

@@ -87,14 +87,28 @@ class BitbucketAdapter extends VcsProviderInterface {
       if (!callbackUrl) return { created: false, reason: 'missing_callback_url' };
       const token = await this.oauth.getToken();
       if (!token?.access_token) return { created: false, reason: 'missing_token' };
-      // Bitbucket Cloud webhooks don't support shared secret directly; create basic push webhook
+      // Bitbucket Cloud requires repository:admin and webhook scopes
-      const resp = await fetch(`https://api.bitbucket.org/2.0/repositories/${owner}/${repo}/hooks`, {
+      const hooksUrl = `https://api.bitbucket.org/2.0/repositories/${owner}/${repo}/hooks`;
+      // Avoid duplicates: list existing hooks first
+      try {
+        const listResp = await fetch(hooksUrl, { headers: { Authorization: `Bearer ${token.access_token}` } });
+        if (listResp.ok) {
+          const data = await listResp.json();
+          const existing = (data.values || []).find(h => h.url === callbackUrl);
+          if (existing) {
+            return { created: false, reason: 'already_exists', hook_id: existing.uuid || existing.id };
+          }
+        }
+      } catch (_) {}
+      // Create push webhook
+      const resp = await fetch(hooksUrl, {
         method: 'POST',
         headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${token.access_token}` },
         body: JSON.stringify({ description: 'CodeNuk Git Integration', url: callbackUrl, active: true, events: ['repo:push'] })
       });
       if (resp.ok) { const d = await resp.json(); return { created: true, hook_id: d.uuid || d.id }; }
-      return { created: false, reason: `status_${resp.status}` };
+      const detail = await resp.text().catch(() => '');
+      return { created: false, reason: `status_${resp.status}`, detail };
     } catch (e) {
       return { created: false, error: e.message };
     }
@@ -111,13 +125,28 @@ class BitbucketAdapter extends VcsProviderInterface {
       } else {
         repoPath = await this.gitRepoService.cloneIfMissingWithHost(owner, repo, branch, this.host);
       }
+      // Fetch and fast-forward with auth header for private repos
+      let beforeSha = await this.gitRepoService.getHeadSha(repoPath);
+      let afterSha = beforeSha;
+      try {
+        if (token?.access_token) {
+          // Use extraheader for both fetch and pull
+          await this.gitRepoService.runGit(repoPath, ['-c', `http.extraheader=Authorization: Bearer ${token.access_token}`, 'fetch', '--all', '--prune']);
+          await this.gitRepoService.runGit(repoPath, ['checkout', branch]);
+          await this.gitRepoService.runGit(repoPath, ['-c', `http.extraheader=Authorization: Bearer ${token.access_token}`, 'pull', '--ff-only', 'origin', branch]);
+        } else {
+          await this.gitRepoService.fetchAndFastForward(repoPath, branch);
+        }
+        afterSha = await this.gitRepoService.getHeadSha(repoPath);
+      } catch (_) {}
+
       storageRecord = await this.fileStorageService.initializeRepositoryStorage(repositoryId, repoPath);
       await this.fileStorageService.processDirectoryStructure(storageRecord.id, repositoryId, repoPath);
       const finalStorage = await this.fileStorageService.completeRepositoryStorage(storageRecord.id);
       
       // Get the current HEAD commit SHA and update the repository record
       try {
-        const headSha = await this.gitRepoService.getHeadSha(repoPath);
+        const headSha = afterSha || (await this.gitRepoService.getHeadSha(repoPath));
         await database.query(
           'UPDATE github_repositories SET last_synced_at = NOW(), last_synced_commit_sha = $1, updated_at = NOW() WHERE id = $2',
           [headSha, repositoryId]
@@ -129,7 +158,7 @@ class BitbucketAdapter extends VcsProviderInterface {
           [repositoryId]
         );
       }
-      return { success: true, method: 'git', targetDir: repoPath, storage: finalStorage };
+      return { success: true, method: 'git', targetDir: repoPath, beforeSha, afterSha, storage: finalStorage };
     } catch (e) {
       if (storageRecord) await this.fileStorageService.markStorageFailed(storageRecord.id, e.message);
       return { success: false, error: e.message };
@@ -148,11 +177,32 @@ class BitbucketAdapter extends VcsProviderInterface {
 
   async getRepositoryDiff(owner, repo, branch, fromSha, toSha) {
     const repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, branch);
+    // Fast-forward before diff; include auth for private repos
+    try {
+      const token = await this.oauth.getToken();
+      if (token?.access_token) {
+        await this.gitRepoService.runGit(repoPath, ['-c', `http.extraheader=Authorization: Bearer ${token.access_token}`, 'fetch', '--all', '--prune']);
+        await this.gitRepoService.runGit(repoPath, ['checkout', branch]);
+        await this.gitRepoService.runGit(repoPath, ['-c', `http.extraheader=Authorization: Bearer ${token.access_token}`, 'pull', '--ff-only', 'origin', branch]);
+      } else {
+        await this.gitRepoService.fetchAndFastForward(repoPath, branch);
+      }
+    } catch (_) {}
     return await this.gitRepoService.getDiff(repoPath, fromSha || null, toSha || 'HEAD', { patch: true });
   }
 
   async getRepositoryChangesSince(owner, repo, branch, sinceSha) {
     const repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, branch);
+    try {
+      const token = await this.oauth.getToken();
+      if (token?.access_token) {
+        await this.gitRepoService.runGit(repoPath, ['-c', `http.extraheader=Authorization: Bearer ${token.access_token}`, 'fetch', '--all', '--prune']);
+        await this.gitRepoService.runGit(repoPath, ['checkout', branch]);
+        await this.gitRepoService.runGit(repoPath, ['-c', `http.extraheader=Authorization: Bearer ${token.access_token}`, 'pull', '--ff-only', 'origin', branch]);
+      } else {
+        await this.gitRepoService.fetchAndFastForward(repoPath, branch);
+      }
+    } catch (_) {}
     return await this.gitRepoService.getChangedFilesSince(repoPath, sinceSha);
   }
 

services/git-integration/src/services/providers/gitea.adapter.js

@@ -3,6 +3,8 @@ const VcsProviderInterface = require('../vcs-provider.interface');
 const FileStorageService = require('../file-storage.service');
 const GitRepoService = require('../git-repo.service');
 const GiteaOAuthService = require('../gitea-oauth');
+const axios = require('axios');
+const https = require('https');
 
 class GiteaAdapter extends VcsProviderInterface {
   constructor() {
@@ -33,32 +35,89 @@ class GiteaAdapter extends VcsProviderInterface {
     const token = await this.oauth.getToken();
     const base = (process.env.GITEA_BASE_URL || 'https://gitea.com').replace(/\/$/, '');
     
+    console.log(`🔍 [GITEA] Checking repository access for: ${owner}/${repo}`);
+    console.log(`🔍 [GITEA] Token available: ${!!token?.access_token}`);
+    console.log(`🔍 [GITEA] API base URL: ${base}`);
+    
     try {
       // Always try with authentication first (like GitHub behavior)
       if (token?.access_token) {
-        const resp = await fetch(`${base}/api/v1/repos/${owner}/${repo}`, { headers: { Authorization: `Bearer ${token.access_token}` } });
+        const url = `${base}/api/v1/repos/${owner}/${repo}`;
-        if (resp.status === 200) {
+        console.log(`🔍 [GITEA] Trying authenticated request to: ${url}`);
-          const d = await resp.json();
+        
+        const response = await axios.get(url, {
+          headers: { Authorization: `token ${token.access_token}` },
+          httpsAgent: new https.Agent({
+            keepAlive: true,
+            timeout: 15000,
+            family: 4 // Force IPv4 to avoid IPv6 connectivity issues
+          }),
+          timeout: 15000,
+          validateStatus: function (status) {
+            return status >= 200 && status < 300; // Only consider 2xx as success
+          }
+        });
+        
+        console.log(`🔍 [GITEA] Authenticated response status: ${response.status}`);
+        
+        if (response.status === 200) {
+          const d = response.data;
           const isPrivate = !!d.private;
+          console.log(`✅ [GITEA] Repository accessible via authentication, private: ${isPrivate}`);
           return { exists: true, isPrivate, hasAccess: true, requiresAuth: isPrivate };
+        } else {
+          console.log(`❌ [GITEA] Authenticated request failed with status: ${response.status}`);
+          console.log(`❌ [GITEA] Error response: ${JSON.stringify(response.data)}`);
         }
       }
       
       // No token or token failed: try without authentication
-      const resp = await fetch(`${base}/api/v1/repos/${owner}/${repo}`);
+      const url = `${base}/api/v1/repos/${owner}/${repo}`;
-      if (resp.status === 200) {
+      console.log(`🔍 [GITEA] Trying unauthenticated request to: ${url}`);
-        const d = await resp.json();
+      
+      const response = await axios.get(url, {
+        httpsAgent: new https.Agent({
+          keepAlive: true,
+          timeout: 15000,
+          family: 4 // Force IPv4 to avoid IPv6 connectivity issues
+        }),
+        timeout: 15000,
+        validateStatus: function (status) {
+          return status >= 200 && status < 300; // Only consider 2xx as success
+        }
+      });
+      
+      console.log(`🔍 [GITEA] Unauthenticated response status: ${response.status}`);
+      
+      if (response.status === 200) {
+        const d = response.data;
+        console.log(`✅ [GITEA] Repository accessible without authentication, private: ${!!d.private}`);
         return { exists: true, isPrivate: !!d.private, hasAccess: true, requiresAuth: false };
       }
-      if (resp.status === 404 || resp.status === 403) {
-        // Repository exists but requires authentication (like GitHub behavior)
-        return { exists: resp.status !== 404 ? true : false, isPrivate: true, hasAccess: false, requiresAuth: true };
-      }
     } catch (error) {
+      if (error.response) {
+        // The request was made and the server responded with a status code
+        // that falls out of the range of 2xx
+        console.log(`❌ [GITEA] Request failed with status: ${error.response.status}`);
+        console.log(`❌ [GITEA] Error response: ${JSON.stringify(error.response.data)}`);
+        
+        if (error.response.status === 404 || error.response.status === 403) {
+          console.log(`🔍 [GITEA] Repository exists but requires authentication (status: ${error.response.status})`);
+          return { exists: error.response.status !== 404 ? true : false, isPrivate: true, hasAccess: false, requiresAuth: true };
+        }
+      } else if (error.request) {
+        // The request was made but no response was received
+        console.log(`❌ [GITEA] Network error: ${error.message}`);
+      } else {
+        // Something happened in setting up the request that triggered an Error
+        console.log(`❌ [GITEA] Exception occurred: ${error.message}`);
+      }
+      
       // If any error occurs, assume repository requires authentication
       return { exists: false, isPrivate: null, hasAccess: false, requiresAuth: true, error: 'Repository not found or requires authentication' };
     }
     
+    console.log(`❌ [GITEA] Falling through to default error case`);
     return { exists: false, isPrivate: null, hasAccess: false, requiresAuth: true, error: 'Repository not found or requires authentication' };
   }
 
@@ -67,12 +126,22 @@ class GiteaAdapter extends VcsProviderInterface {
     const base = (process.env.GITEA_BASE_URL || 'https://gitea.com').replace(/\/$/, '');
     if (token?.access_token) {
       try {
-        const resp = await fetch(`${base}/api/v1/repos/${owner}/${repo}`, { headers: { Authorization: `Bearer ${token.access_token}` } });
+        const response = await axios.get(`${base}/api/v1/repos/${owner}/${repo}`, {
-        if (resp.ok) {
+          headers: { Authorization: `token ${token.access_token}` },
-          const d = await resp.json();
+          httpsAgent: new https.Agent({
+            keepAlive: true,
+            timeout: 15000,
+            family: 4 // Force IPv4 to avoid IPv6 connectivity issues
+          }),
+          timeout: 15000
+        });
+        if (response.status === 200) {
+          const d = response.data;
           return { full_name: d.full_name || `${owner}/${repo}`, visibility: d.private ? 'private' : 'public', default_branch: d.default_branch || 'main', updated_at: d.updated_at };
         }
-      } catch (_) {}
+      } catch (error) {
+        console.log(`❌ [GITEA] Failed to fetch repository metadata: ${error.message}`);
+      }
     }
     return { full_name: `${owner}/${repo}`, visibility: 'public', default_branch: 'main', updated_at: new Date().toISOString() };
   }
@@ -83,20 +152,92 @@ class GiteaAdapter extends VcsProviderInterface {
 
   async ensureRepositoryWebhook(owner, repo, callbackUrl) {
     try {
-      if (!callbackUrl) return { created: false, reason: 'missing_callback_url' };
+      if (!callbackUrl) {
+        console.warn('⚠️ [GITEA] Webhook callbackUrl not provided; skipping webhook creation');
+        return { created: false, reason: 'missing_callback_url' };
+      }
+      
       const token = await this.oauth.getToken();
-      if (!token?.access_token) return { created: false, reason: 'missing_token' };
+      if (!token?.access_token) {
+        console.warn('⚠️ [GITEA] OAuth token not available; skipping webhook creation');
+        return { created: false, reason: 'missing_token' };
+      }
+      
       const base = (process.env.GITEA_BASE_URL || 'https://gitea.com').replace(/\/$/, '');
       const secret = process.env.GITEA_WEBHOOK_SECRET || '';
-      const resp = await fetch(`${base}/api/v1/repos/${owner}/${repo}/hooks`, {
+      
-        method: 'POST',
+      console.log(`🔗 [GITEA] Setting up webhook for ${owner}/${repo}`);
-        headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${token.access_token}` },
+      
-        body: JSON.stringify({ type: 'gitea', config: { url: callbackUrl, content_type: 'json', secret: secret || undefined }, events: ['push'], active: true })
+      // First, list existing hooks to avoid duplicates
+      try {
+        const listResponse = await axios.get(`${base}/api/v1/repos/${owner}/${repo}/hooks`, {
+          headers: { Authorization: `token ${token.access_token}` },
+          httpsAgent: new https.Agent({
+            keepAlive: true,
+            timeout: 15000,
+            family: 4
+          }),
+          timeout: 15000
         });
-      if (resp.ok) { const d = await resp.json(); return { created: true, hook_id: d.id }; }
+        
-      return { created: false, reason: `status_${resp.status}` };
+        if (listResponse.status === 200) {
-    } catch (e) {
+          const existingHooks = listResponse.data;
-      return { created: false, error: e.message };
+          
+          // Check if a webhook with our callback URL already exists
+          const existingHook = existingHooks.find(hook => 
+            hook.config && hook.config.url === callbackUrl
+          );
+          
+          if (existingHook) {
+            console.log(`✅ [GITEA] Webhook already exists (ID: ${existingHook.id})`);
+            return { created: false, reason: 'already_exists', hook_id: existingHook.id };
+          }
+        }
+      } catch (error) {
+        console.warn('⚠️ [GITEA] Could not list existing webhooks, continuing with creation attempt:', error.message);
+      }
+      
+      // Create new webhook
+      const response = await axios.post(`${base}/api/v1/repos/${owner}/${repo}/hooks`, {
+        type: 'gitea',
+        config: {
+          url: callbackUrl,
+          content_type: 'json',
+          secret: secret || undefined
+        },
+        events: ['push'],
+        active: true
+      }, {
+        headers: { 
+          'Content-Type': 'application/json', 
+          Authorization: `token ${token.access_token}` 
+        },
+        httpsAgent: new https.Agent({
+          keepAlive: true,
+          timeout: 15000,
+          family: 4
+        }),
+        timeout: 15000
+      });
+      
+      if (response.status === 200 || response.status === 201) { 
+        const hookData = response.data; 
+        console.log(`✅ [GITEA] Webhook created successfully (ID: ${hookData.id})`);
+        return { created: true, hook_id: hookData.id }; 
+      }
+      
+      console.warn(`⚠️ [GITEA] Webhook creation failed with status: ${response.status}`);
+      return { created: false, reason: `status_${response.status}` };
+      
+    } catch (error) {
+      // Common cases: insufficient permissions, private repo without correct scope
+      if (error.response) {
+        console.warn('⚠️ [GITEA] Webhook creation failed:', error.response.status, error.response.data?.message || error.message);
+        return { created: false, error: error.message, status: error.response.status };
+      } else {
+        console.warn('⚠️ [GITEA] Webhook creation failed:', error.message);
+        return { created: false, error: error.message };
+      }
     }
   }
 
@@ -111,13 +252,22 @@ class GiteaAdapter extends VcsProviderInterface {
       } else {
         repoPath = await this.gitRepoService.cloneIfMissingWithHost(owner, repo, branch, this.host);
       }
+      // Fetch and fast-forward to ensure latest commits are present
+      let beforeSha = await this.gitRepoService.getHeadSha(repoPath);
+      let afterSha = beforeSha;
+      try {
+        const res = await this.gitRepoService.fetchAndFastForward(repoPath, branch);
+        beforeSha = res.beforeSha || beforeSha;
+        afterSha = res.afterSha || afterSha;
+      } catch (_) {}
+
       storageRecord = await this.fileStorageService.initializeRepositoryStorage(repositoryId, repoPath);
       await this.fileStorageService.processDirectoryStructure(storageRecord.id, repositoryId, repoPath);
       const finalStorage = await this.fileStorageService.completeRepositoryStorage(storageRecord.id);
       
       // Get the current HEAD commit SHA and update the repository record
       try {
-        const headSha = await this.gitRepoService.getHeadSha(repoPath);
+        const headSha = afterSha || (await this.gitRepoService.getHeadSha(repoPath));
         await database.query(
           'UPDATE github_repositories SET last_synced_at = NOW(), last_synced_commit_sha = $1, updated_at = NOW() WHERE id = $2',
           [headSha, repositoryId]
@@ -129,7 +279,7 @@ class GiteaAdapter extends VcsProviderInterface {
           [repositoryId]
         );
       }
-      return { success: true, method: 'git', targetDir: repoPath, storage: finalStorage };
+      return { success: true, method: 'git', targetDir: repoPath, beforeSha, afterSha, storage: finalStorage };
     } catch (e) {
       if (storageRecord) await this.fileStorageService.markStorageFailed(storageRecord.id, e.message);
       return { success: false, error: e.message };
@@ -148,11 +298,14 @@ class GiteaAdapter extends VcsProviderInterface {
 
   async getRepositoryDiff(owner, repo, branch, fromSha, toSha) {
     const repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, branch);
+    // Proactively fetch latest to ensure SHAs exist
+    try { await this.gitRepoService.fetchAndFastForward(repoPath, branch); } catch (_) {}
     return await this.gitRepoService.getDiff(repoPath, fromSha || null, toSha || 'HEAD', { patch: true });
   }
 
   async getRepositoryChangesSince(owner, repo, branch, sinceSha) {
     const repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, branch);
+    try { await this.gitRepoService.fetchAndFastForward(repoPath, branch); } catch (_) {}
     return await this.gitRepoService.getChangedFilesSince(repoPath, sinceSha);
   }
 

services/git-integration/src/services/providers/gitlab.adapter.js

@@ -158,13 +158,126 @@ class GitlabAdapter extends VcsProviderInterface {
   }
 
   async getRepositoryDiff(owner, repo, branch, fromSha, toSha) {
-    const repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, branch);
+    // Mirror robust GitHub behavior: ensure repo exists, handle main/master fallback,
+    // fetch required history for provided SHAs (handle shallow clones), then diff.
+    const preferredBranch = branch || 'main';
+    const alternateBranch = preferredBranch === 'main' ? 'master' : 'main';
+
+    const fs = require('fs');
+    const path = require('path');
+    let repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, preferredBranch);
+
+    // Ensure repo exists locally (prefer OAuth)
+    try {
+      const token = await this.oauth.getToken().catch(() => null);
+      try {
+        if (token?.access_token) {
+          repoPath = await this.gitRepoService.cloneIfMissingWithAuth(owner, repo, preferredBranch, this.host, token.access_token, 'oauth2');
+        } else {
+          repoPath = await this.gitRepoService.cloneIfMissingWithHost(owner, repo, preferredBranch, this.host);
+        }
+      } catch (_) {
+        // Try alternate common default branch
+        try {
+          if (token?.access_token) {
+            repoPath = await this.gitRepoService.cloneIfMissingWithAuth(owner, repo, alternateBranch, this.host, token.access_token, 'oauth2');
+          } else {
+            repoPath = await this.gitRepoService.cloneIfMissingWithHost(owner, repo, alternateBranch, this.host);
+          }
+        } catch (_) {}
+      }
+
+      // If preferred path missing but alternate exists, use alternate
+      const altPath = this.gitRepoService.getLocalRepoPath(owner, repo, alternateBranch);
+      if ((!fs.existsSync(repoPath) || !fs.existsSync(path.join(repoPath, '.git'))) && fs.existsSync(altPath)) {
+        repoPath = altPath;
+      }
+
+      // Fetch and checkout; attempt preferred then alternate
+      try {
+        await this.gitRepoService.fetchAndFastForward(repoPath, preferredBranch);
+      } catch (_) {
+        try { await this.gitRepoService.fetchAndFastForward(repoPath, alternateBranch); } catch (_) {}
+      }
+
+      // Ensure both SHAs exist locally; if not, fetch them explicitly
+      const ensureShaPresent = async (sha) => {
+        if (!sha) return;
+        try {
+          await this.gitRepoService.runGit(repoPath, ['cat-file', '-e', `${sha}^{commit}`]);
+        } catch (_) {
+          // Try fetching just that object; if shallow, unshallow or fetch full history
+          try { await this.gitRepoService.runGit(repoPath, ['fetch', '--depth=2147483647', 'origin']); } catch (_) {}
+          try { await this.gitRepoService.runGit(repoPath, ['fetch', 'origin', sha]); } catch (_) {}
+        }
+      };
+      await ensureShaPresent(fromSha || null);
+      await ensureShaPresent(toSha || null);
+
       return await this.gitRepoService.getDiff(repoPath, fromSha || null, toSha || 'HEAD', { patch: true });
+    } catch (error) {
+      const attempted = [
+        this.gitRepoService.getLocalRepoPath(owner, repo, preferredBranch),
+        this.gitRepoService.getLocalRepoPath(owner, repo, alternateBranch)
+      ].join(' | ');
+      throw new Error(`${error.message} (attempted paths: ${attempted})`);
+    }
   }
 
   async getRepositoryChangesSince(owner, repo, branch, sinceSha) {
-    const repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, branch);
+    const preferredBranch = branch || 'main';
+    const alternateBranch = preferredBranch === 'main' ? 'master' : 'main';
+    const fs = require('fs');
+    const path = require('path');
+    let repoPath = this.gitRepoService.getLocalRepoPath(owner, repo, preferredBranch);
+
+    try {
+      const token = await this.oauth.getToken().catch(() => null);
+      try {
+        if (token?.access_token) {
+          repoPath = await this.gitRepoService.cloneIfMissingWithAuth(owner, repo, preferredBranch, this.host, token.access_token, 'oauth2');
+        } else {
+          repoPath = await this.gitRepoService.cloneIfMissingWithHost(owner, repo, preferredBranch, this.host);
+        }
+      } catch (_) {
+        try {
+          if (token?.access_token) {
+            repoPath = await this.gitRepoService.cloneIfMissingWithAuth(owner, repo, alternateBranch, this.host, token.access_token, 'oauth2');
+          } else {
+            repoPath = await this.gitRepoService.cloneIfMissingWithHost(owner, repo, alternateBranch, this.host);
+          }
+        } catch (_) {}
+      }
+
+      const altPath = this.gitRepoService.getLocalRepoPath(owner, repo, alternateBranch);
+      if ((!fs.existsSync(repoPath) || !fs.existsSync(path.join(repoPath, '.git'))) && fs.existsSync(altPath)) {
+        repoPath = altPath;
+      }
+
+      try {
+        await this.gitRepoService.fetchAndFastForward(repoPath, preferredBranch);
+      } catch (_) {
+        try { await this.gitRepoService.fetchAndFastForward(repoPath, alternateBranch); } catch (_) {}
+      }
+
+      // Ensure sinceSha exists locally
+      if (sinceSha) {
+        try {
+          await this.gitRepoService.runGit(repoPath, ['cat-file', '-e', `${sinceSha}^{commit}`]);
+        } catch (_) {
+          try { await this.gitRepoService.runGit(repoPath, ['fetch', '--depth=2147483647', 'origin']); } catch (_) {}
+          try { await this.gitRepoService.runGit(repoPath, ['fetch', 'origin', sinceSha]); } catch (_) {}
+        }
+      }
+
       return await this.gitRepoService.getChangedFilesSince(repoPath, sinceSha);
+    } catch (error) {
+      const attempted = [
+        this.gitRepoService.getLocalRepoPath(owner, repo, preferredBranch),
+        this.gitRepoService.getLocalRepoPath(owner, repo, alternateBranch)
+      ].join(' | ');
+      throw new Error(`${error.message} (attempted paths: ${attempted})`);
+    }
   }
 
   async cleanupRepositoryStorage(repositoryId) {

services/git-integration/src/services/vcs-webhook.service.js

@@ -1,11 +1,13 @@
 // services/vcs-webhook.service.js
 const database = require('../config/database');
 const providerRegistry = require('./provider-registry');
+const DiffProcessingService = require('./diff-processing.service');
 
 class VcsWebhookService {
   constructor() {
     this._schemaChecked = false;
     this._webhookEventColumns = new Map();
+    this.diffService = new DiffProcessingService();
   }
 
   // Process webhook events for any VCS provider
@@ -38,7 +40,32 @@ class VcsWebhookService {
 
   // Handle push events for any provider
   async handlePushEvent(providerKey, payload) {
-    const { repository, project, ref, commits, pusher, user } = payload;
+       let { repository, project, ref, commits, pusher, user } = payload;
+
+    // Normalize Bitbucket push payload structure to the common fields
+    if (providerKey === 'bitbucket' && payload && payload.push && Array.isArray(payload.push.changes)) {
+      try {
+        const firstChange = payload.push.changes[0] || {};
+        const newRef = firstChange.new || {};
+        const oldRef = firstChange.old || {};
+        // Branch name
+        const branchName = newRef.name || oldRef.name || null;
+        // Compose a git-like ref
+        ref = branchName ? `refs/heads/${branchName}` : ref;
+        // Aggregate commits across changes
+        const allCommits = [];
+        for (const change of payload.push.changes) {
+          if (Array.isArray(change.commits)) {
+            allCommits.push(...change.commits);
+          }
+        }
+        commits = allCommits;
+        // Surface before/after hashes for persistence
+        payload.before = oldRef.target?.hash || payload.before || null;
+        payload.after = newRef.target?.hash || payload.after || null;
+        // Bitbucket sets repository at top-level; keep as-is if present
+      } catch (_) {}
+    }
     
     // Build a provider-normalized repo object for extraction
     let repoForExtraction = repository || {};
@@ -85,12 +112,7 @@ class VcsWebhookService {
       });
 
       if (repoId) {
-        // Insert into repository_commit_events
+     
-        await database.query(
-          `INSERT INTO repository_commit_events (repository_id, ref, before_sha, after_sha, commit_count)
-           VALUES ($1, $2, $3, $4, $5)`,
-          [repoId, ref, payload.before || null, payload.after || null, commitData.length]
-        );
 
         // Persist per-commit details and file paths
         if (commitData.length > 0) {
@@ -114,10 +136,12 @@ class VcsWebhookService {
                   commit.url || null
                 ]
               );
-
               const commitId = commitInsert.rows[0].id;
-
+               // For Bitbucket, we'll skip file change insertion during webhook processing
-              // Insert file changes
+              // since the webhook doesn't include file changes. The background sync will
+              // handle this properly by fetching the changes from git directly.
+              if (providerKey !== 'bitbucket') {
+                // For other providers (GitHub, GitLab, Gitea), use the webhook data
                 const addFiles = (paths = [], changeType) => paths.forEach(async (p) => {
                   try {
                     await database.query(
@@ -127,10 +151,10 @@ class VcsWebhookService {
                     );
                   } catch (_) {}
                 });
-
                 addFiles(commit.added || [], 'added');
                 addFiles(commit.modified || [], 'modified');
                 addFiles(commit.removed || [], 'removed');
+              }
             } catch (commitErr) {
               console.warn('Failed to persist commit details:', commitErr.message);
             }
@@ -157,6 +181,10 @@ class VcsWebhookService {
               repoId
             );
 
+               // Process diffs for each commit after successful sync
+               if (downloadResult.success && downloadResult.targetDir) {
+                await this.processCommitDiffs(repoId, commitData, downloadResult.targetDir, providerKey);
+              }
             await database.query(
               'UPDATE github_repositories SET sync_status = $1, last_synced_at = NOW(), updated_at = NOW() WHERE id = $2',
               [downloadResult.success ? 'synced' : 'error', repoId]
@@ -250,9 +278,11 @@ class VcsWebhookService {
           },
           message: commit.message,
           url: commit.links?.html?.href,
-          added: commit.added || [],
+           // Bitbucket webhook doesn't include file changes in commit objects
-          modified: commit.modified || [],
+          // We'll fetch these from git directly during processing
-          removed: commit.removed || []
+          added: [],
+          modified: [],
+          removed: []
         }));
       case 'gitea':
         return commits.map(commit => ({
@@ -361,94 +391,124 @@ class VcsWebhookService {
     console.log(`Repository: ${payload.repository?.full_name || payload.repository?.path_with_namespace || 'Unknown'}`);
   }
 
-  // Log webhook events for debugging and analytics
+  // Process diffs for commits in a push event (same as GitHub webhook service)
-  async logWebhookEvent(providerKey, eventType, action, repositoryFullName, metadata = {}, deliveryId = null, fullPayload = null) {
+  async processCommitDiffs(repositoryId, commits, repoLocalPath, providerKey = 'github') {
+    console.log(`🔄 Processing diffs for ${commits.length} commits in repository ${repositoryId}`);
+
+    if (!Array.isArray(commits) || commits.length === 0) {
+      console.log('⚠️  No commits to process');
+      return;
+    }
+
+    for (const commit of commits) {
       try {
-      await this._ensureWebhookEventsSchemaCached();
+        console.log(`📝 Processing diff for commit: ${commit.id}`);
-
+        const commitQuery = `
-      // Build a flexible INSERT based on existing columns
+          SELECT id FROM repository_commit_details
-      const columns = [];
+          WHERE repository_id = $1 AND commit_sha = $2
-      const placeholders = [];
-      const values = [];
-      let i = 1;
-
-      columns.push('event_type');
-      placeholders.push(`$${i++}`);
-      values.push(eventType);
-
-      if (this._webhookEventColumns.has('action')) {
-        columns.push('action');
-        placeholders.push(`$${i++}`);
-        values.push(action || null);
-      }
-      if (this._webhookEventColumns.has('repository_full_name')) {
-        columns.push('repository_full_name');
-        placeholders.push(`$${i++}`);
-        values.push(repositoryFullName || null);
-      }
-      if (this._webhookEventColumns.has('delivery_id')) {
-        columns.push('delivery_id');
-        placeholders.push(`$${i++}`);
-        values.push(deliveryId || null);
-      }
-      if (this._webhookEventColumns.has('metadata')) {
-        columns.push('metadata');
-        placeholders.push(`$${i++}`);
-        values.push(JSON.stringify({ ...metadata, provider: providerKey }));
-      }
-      if (this._webhookEventColumns.has('event_payload')) {
-        columns.push('event_payload');
-        placeholders.push(`$${i++}`);
-        values.push(JSON.stringify(fullPayload || {}));
-      }
-      if (this._webhookEventColumns.has('received_at')) {
-        columns.push('received_at');
-        placeholders.push(`$${i++}`);
-        values.push(new Date());
-      }
-      if (this._webhookEventColumns.has('processing_status')) {
-        columns.push('processing_status');
-        placeholders.push(`$${i++}`);
-        values.push('pending');
-      }
-
-      const query = `INSERT INTO webhook_events (${columns.join(', ')}) VALUES (${placeholders.join(', ')})`;
-      await database.query(query, values);
-    } catch (error) {
-      console.warn('Failed to log webhook event:', error.message);
-    }
-  }
-
-  async _ensureWebhookEventsSchemaCached() {
-    if (this._schemaChecked) return;
-    try {
-      const result = await database.query(
-        "SELECT column_name, is_nullable FROM information_schema.columns WHERE table_schema='public' AND table_name='webhook_events'"
-      );
-      for (const row of result.rows) {
-        this._webhookEventColumns.set(row.column_name, row.is_nullable);
-      }
-    } catch (e) {
-      console.warn('Could not introspect webhook_events schema:', e.message);
-    } finally {
-      this._schemaChecked = true;
-    }
-  }
-
-  // Get recent webhook events
-  async getRecentWebhookEvents(limit = 50) {
-    try {
-      const query = `
-        SELECT * FROM webhook_events
-        ORDER BY received_at DESC
-        LIMIT $1
         `;
+        const commitResult = await database.query(commitQuery, [repositoryId, commit.id]);
+        if (commitResult.rows.length === 0) {
+          console.warn(`⚠️  Commit ${commit.id} not found in database, skipping diff processing`);
+          continue;
+        }
+        const commitId = commitResult.rows[0].id;
+        const parentSha = commit.parents && commit.parents.length > 0 ? commit.parents[0].id : null;
 
-      const result = await database.query(query, [limit]);
+         // For Bitbucket, we need to ensure file changes are in the database first
-      return result.rows;
+         if (providerKey === 'bitbucket') {
+          await this.ensureBitbucketFileChanges(commitId, commit, repoLocalPath);
+        }
+        const diffResult = await this.diffService.processCommitDiffs(
+          commitId,
+          repositoryId,
+          repoLocalPath,
+          parentSha,
+          commit.id
+        );
+
+        if (diffResult.success) {
+          console.log(`✅ Successfully processed ${diffResult.processedFiles} file diffs for commit ${commit.id}`);
+        } else {
+          console.warn(`⚠️  Failed to process diffs for commit ${commit.id}: ${diffResult.error || diffResult.reason}`);
+        }
       } catch (error) {
-      console.error('Failed to get webhook events:', error.message);
+        console.error(`❌ Error processing diff for commit ${commit.id}:`, error.message);
-      return [];
+
+      }
+    }
+  }
+
+  // Ensure Bitbucket file changes are stored in the database
+  async ensureBitbucketFileChanges(commitId, commit, repoLocalPath) {
+    try {
+      // Check if file changes already exist for this commit
+      const existingFilesQuery = `
+        SELECT COUNT(*) as count FROM repository_commit_files WHERE commit_id = $1
+      `;
+      const existingResult = await database.query(existingFilesQuery, [commitId]);
+      
+      if (parseInt(existingResult.rows[0].count) > 0) {
+        console.log(`📁 File changes already exist for commit ${commit.id}`);
+        return;
+
+      }
+
+      // Get file changes from git
+      const GitRepoService = require('./git-repo.service');
+      const gitService = new GitRepoService();
+
+      const parentSha = commit.parents && commit.parents.length > 0 ? commit.parents[0].id : null;
+      const fromSha = parentSha || `${commit.id}~1`;
+      const fileChanges = await gitService.getChangedFilesSince(repoLocalPath, fromSha);
+
+      // Parse the git diff output and categorize files
+      const added = [];
+      const modified = [];
+      const removed = [];
+      
+      for (const change of fileChanges) {
+        const { status, filePath } = change;
+        switch (status) {
+          case 'A':
+            added.push(filePath);
+            break;
+          case 'M':
+            modified.push(filePath);
+            break;
+          case 'D':
+            removed.push(filePath);
+            break;
+          case 'R':
+            // Renamed files - treat as modified for now
+            modified.push(filePath);
+            break;
+          default:
+            // Unknown status, treat as modified
+            modified.push(filePath);
+        }
+      }
+      // Insert file changes
+      const addFiles = (paths = [], changeType) => paths.forEach(async (p) => {
+        try {
+          await database.query(
+            `INSERT INTO repository_commit_files (commit_id, change_type, file_path)
+             VALUES ($1, $2, $3)`,
+            [commitId, changeType, p]
+          );
+        } catch (err) {
+          console.warn(`Failed to insert file change: ${err.message}`);
+        }
+      });
+    addFiles(added, 'added');
+    addFiles(modified, 'modified');
+    addFiles(removed, 'removed');
+
+    
+      console.log(`📁 Inserted ${added.length + modified.length + removed.length} file changes for Bitbucket commit ${commit.id}`);
+     
+    } catch (error) {
+      console.warn(`Failed to ensure Bitbucket file changes for commit ${commit.id}:`, error.message);
     }
   }
 }

services/git-integration/src/services/webhook.service.js

@@ -2,6 +2,7 @@
 const crypto = require('crypto');
 const database = require('../config/database');
 const GitHubIntegrationService = require('./github-integration.service');
+const DiffProcessingService = require('./diff-processing.service');
 
 class WebhookService {
   constructor() {
@@ -9,6 +10,7 @@ class WebhookService {
     this._schemaChecked = false;
     this._webhookEventColumns = new Map();
     this.githubService = new GitHubIntegrationService();
+    this.diffService = new DiffProcessingService();
   }
 
   // Verify GitHub webhook signature
@@ -97,11 +99,7 @@ class WebhookService {
       );
 
       if (repoId) {
-        await database.query(
+        // repository_commit_events table removed as requested
-          `INSERT INTO repository_commit_events (repository_id, ref, before_sha, after_sha, commit_count)
-           VALUES ($1, $2, $3, $4, $5)`,
-          [repoId, ref, payload.before || null, payload.after || null, Array.isArray(commits) ? commits.length : 0]
-        );
 
         // Persist per-commit details and file paths (added/modified/removed)
         if (Array.isArray(commits) && commits.length > 0) {
@@ -165,6 +163,11 @@ class WebhookService {
               repoId
             );
 
+            // Process diffs for each commit after successful sync
+            if (downloadResult.success && downloadResult.targetDir) {
+              await this.processCommitDiffs(repoId, commits, downloadResult.targetDir);
+            }
+
             await database.query(
               'UPDATE github_repositories SET sync_status = $1, last_synced_at = NOW(), updated_at = NOW() WHERE id = $2',
               [downloadResult.success ? 'synced' : 'error', repoId]
@@ -265,97 +268,59 @@ class WebhookService {
     console.log(`Zen: ${payload.zen || 'No zen message'}`);
   }
 
-  // Log webhook events for debugging and analytics
+  // Process diffs for commits in a push event
-  async _ensureWebhookEventsSchemaCached() {
+  async processCommitDiffs(repositoryId, commits, repoLocalPath) {
-    if (this._schemaChecked) return;
+    console.log(`🔄 Processing diffs for ${commits.length} commits in repository ${repositoryId}`);
+    
+    if (!Array.isArray(commits) || commits.length === 0) {
+      console.log('⚠️  No commits to process');
+      return;
+    }
+
+    for (const commit of commits) {
       try {
-      const result = await database.query(
+        console.log(`📝 Processing diff for commit: ${commit.id}`);
-        "SELECT column_name, is_nullable FROM information_schema.columns WHERE table_schema='public' AND table_name='webhook_events'"
-      );
-      for (const row of result.rows) {
-        this._webhookEventColumns.set(row.column_name, row.is_nullable);
-      }
-    } catch (e) {
-      // If schema check fails, proceed with best-effort insert
-      console.warn('Could not introspect webhook_events schema:', e.message);
-    } finally {
-      this._schemaChecked = true;
-    }
-  }
         
-  async logWebhookEvent(eventType, action, repositoryFullName, metadata = {}, deliveryId = null, fullPayload = null) {
+        // Get commit record from database
-    try {
+        const commitQuery = `
-      await this._ensureWebhookEventsSchemaCached();
+          SELECT id FROM repository_commit_details 
-
+          WHERE repository_id = $1 AND commit_sha = $2
-      // Build a flexible INSERT based on existing columns
-      const columns = [];
-      const placeholders = [];
-      const values = [];
-      let i = 1;
-
-      columns.push('event_type');
-      placeholders.push(`$${i++}`);
-      values.push(eventType);
-
-      if (this._webhookEventColumns.has('action')) {
-        columns.push('action');
-        placeholders.push(`$${i++}`);
-        values.push(action || null);
-      }
-      if (this._webhookEventColumns.has('repository_full_name')) {
-        columns.push('repository_full_name');
-        placeholders.push(`$${i++}`);
-        values.push(repositoryFullName || null);
-      }
-      if (this._webhookEventColumns.has('delivery_id')) {
-        columns.push('delivery_id');
-        placeholders.push(`$${i++}`);
-        values.push(deliveryId || null);
-      }
-      if (this._webhookEventColumns.has('metadata')) {
-        columns.push('metadata');
-        placeholders.push(`$${i++}`);
-        values.push(JSON.stringify(metadata || {}));
-      }
-      if (this._webhookEventColumns.has('event_payload')) {
-        columns.push('event_payload');
-        placeholders.push(`$${i++}`);
-        values.push(JSON.stringify(fullPayload || {}));
-      }
-      if (this._webhookEventColumns.has('received_at')) {
-        columns.push('received_at');
-        placeholders.push(`$${i++}`);
-        values.push(new Date());
-      }
-      if (this._webhookEventColumns.has('processing_status')) {
-        columns.push('processing_status');
-        placeholders.push(`$${i++}`);
-        values.push('pending');
-      }
-
-      const query = `INSERT INTO webhook_events (${columns.join(', ')}) VALUES (${placeholders.join(', ')})`;
-      await database.query(query, values);
-    } catch (error) {
-      console.warn('Failed to log webhook event:', error.message);
-    }
-  }
-
-  // Get recent webhook events
-  async getRecentWebhookEvents(limit = 50) {
-    try {
-      const query = `
-        SELECT * FROM webhook_events
-        ORDER BY received_at DESC
-        LIMIT $1
         `;
         
-      const result = await database.query(query, [limit]);
+        const commitResult = await database.query(commitQuery, [repositoryId, commit.id]);
-      return result.rows;
+        
+        if (commitResult.rows.length === 0) {
+          console.warn(`⚠️  Commit ${commit.id} not found in database, skipping diff processing`);
+          continue;
+        }
+        
+        const commitId = commitResult.rows[0].id;
+        
+        // Get parent commit SHA for diff calculation
+        const parentSha = commit.parents && commit.parents.length > 0 ? commit.parents[0].id : null;
+        
+        // Process the diff
+        const diffResult = await this.diffService.processCommitDiffs(
+          commitId,
+          repositoryId,
+          repoLocalPath,
+          parentSha,
+          commit.id
+        );
+        
+        if (diffResult.success) {
+          console.log(`✅ Successfully processed ${diffResult.processedFiles} file diffs for commit ${commit.id}`);
+        } else {
+          console.warn(`⚠️  Failed to process diffs for commit ${commit.id}: ${diffResult.error || diffResult.reason}`);
+        }
+        
       } catch (error) {
-      console.error('Failed to get webhook events:', error.message);
+        console.error(`❌ Error processing diff for commit ${commit.id}:`, error.message);
-      return [];
       }
     }
   }
 
+  // webhook_events table removed as requested - logging functionality disabled
+}
+
 module.exports = WebhookService;

services/git-integration/test-repo.js

@@ -1,94 +0,0 @@
-const GitHubIntegrationService = require('./src/services/github-integration.service');
-const fs = require('fs');
-const path = require('path');
-
-async function testRepository() {
-  try {
-    console.log('🔍 Testing GitHub Repository Integration...');
-    
-    const githubService = new GitHubIntegrationService();
-    const repositoryUrl = 'https://github.com/prakash6383206529/code-generator.git';
-    
-    console.log(`📂 Testing repository: ${repositoryUrl}`);
-    
-    // Parse the GitHub URL
-    const { owner, repo, branch } = githubService.parseGitHubUrl(repositoryUrl);
-    console.log(`📋 Parsed URL - Owner: ${owner}, Repo: ${repo}, Branch: ${branch || 'main'}`);
-    
-    // Check if repository is public
-    try {
-      const unauthenticatedOctokit = new (require('@octokit/rest')).Octokit({
-        userAgent: 'CodeNuk-GitIntegration/1.0.0',
-      });
-      
-      const { data: repoInfo } = await unauthenticatedOctokit.repos.get({ owner, repo });
-      const isPublic = !repoInfo.private;
-      
-      console.log(`🔓 Repository is ${isPublic ? 'public' : 'private'}`);
-      console.log(`📊 Repository info:`);
-      console.log(`   - Full Name: ${repoInfo.full_name}`);
-      console.log(`   - Description: ${repoInfo.description || 'No description'}`);
-      console.log(`   - Language: ${repoInfo.language || 'Unknown'}`);
-      console.log(`   - Stars: ${repoInfo.stargazers_count}`);
-      console.log(`   - Forks: ${repoInfo.forks_count}`);
-      console.log(`   - Default Branch: ${repoInfo.default_branch}`);
-      console.log(`   - Size: ${repoInfo.size} KB`);
-      console.log(`   - Updated: ${repoInfo.updated_at}`);
-      
-      if (isPublic) {
-        console.log(`✅ Repository is accessible and public`);
-        
-        // Try to analyze the codebase
-        console.log(`🔍 Analyzing codebase...`);
-        const analysis = await githubService.analyzeCodebase(owner, repo, repoInfo.default_branch, true);
-        console.log(`📈 Codebase analysis completed:`);
-        console.log(`   - Total Files: ${analysis.total_files}`);
-        console.log(`   - Languages: ${Object.keys(analysis.languages).join(', ')}`);
-        console.log(`   - Main Language: ${analysis.main_language}`);
-        console.log(`   - Structure: ${analysis.structure ? 'Available' : 'Not available'}`);
-        
-        // Try to download the repository
-        console.log(`📥 Attempting to download repository...`);
-        const downloadResult = await githubService.downloadRepository(owner, repo, repoInfo.default_branch);
-        
-        if (downloadResult.success) {
-          console.log(`✅ Repository downloaded successfully!`);
-          console.log(`📁 Local path: ${downloadResult.local_path}`);
-          console.log(`📊 Download stats:`);
-          console.log(`   - Files: ${downloadResult.files_count}`);
-          console.log(`   - Directories: ${downloadResult.directories_count}`);
-          console.log(`   - Size: ${downloadResult.total_size_bytes} bytes`);
-          
-          // List some files
-          if (fs.existsSync(downloadResult.local_path)) {
-            const files = fs.readdirSync(downloadResult.local_path);
-            console.log(`📄 Sample files in root:`);
-            files.slice(0, 10).forEach(file => {
-              const filePath = path.join(downloadResult.local_path, file);
-              const stat = fs.statSync(filePath);
-              console.log(`   - ${file} (${stat.isDirectory() ? 'directory' : 'file'})`);
-            });
-          }
-        } else {
-          console.log(`❌ Repository download failed: ${downloadResult.error}`);
-        }
-        
-      } else {
-        console.log(`🔒 Repository is private - authentication required`);
-      }
-      
-    } catch (error) {
-      console.log(`❌ Error accessing repository: ${error.message}`);
-      if (error.status === 404) {
-        console.log(`🔍 Repository might be private or not found`);
-      }
-    }
-    
-  } catch (error) {
-    console.error(`💥 Test failed: ${error.message}`);
-    console.error(error.stack);
-  }
-}
-
-// Run the test
-testRepository();

services/git-integration/test-webhook.js

@@ -1,70 +0,0 @@
-// test-webhook.js - Simple test script for webhook endpoint
-const fetch = require('node-fetch');
-
-const WEBHOOK_URL = 'http://localhost:8012/api/github/webhook';
-
-// Test webhook with a sample GitHub push event
-const testPayload = {
-  ref: 'refs/heads/main',
-  before: 'abc123',
-  after: 'def456',
-  repository: {
-    id: 123456,
-    name: 'test-repo',
-    full_name: 'testuser/test-repo',
-    owner: {
-      login: 'testuser',
-      id: 789
-    }
-  },
-  pusher: {
-    name: 'testuser',
-    email: 'test@example.com'
-  },
-  commits: [
-    {
-      id: 'def456',
-      message: 'Test commit',
-      author: {
-        name: 'Test User',
-        email: 'test@example.com'
-      }
-    }
-  ]
-};
-
-async function testWebhook() {
-  try {
-    console.log('🧪 Testing webhook endpoint...');
-    console.log(`📡 Sending POST request to: ${WEBHOOK_URL}`);
-    
-    const response = await fetch(WEBHOOK_URL, {
-      method: 'POST',
-      headers: {
-        'Content-Type': 'application/json',
-        'X-GitHub-Event': 'push',
-        'X-GitHub-Delivery': 'test-delivery-123'
-      },
-      body: JSON.stringify(testPayload)
-    });
-
-    const result = await response.json();
-    
-    console.log('📊 Response Status:', response.status);
-    console.log('📋 Response Body:', JSON.stringify(result, null, 2));
-    
-    if (response.ok) {
-      console.log('✅ Webhook test successful!');
-    } else {
-      console.log('❌ Webhook test failed!');
-    }
-    
-  } catch (error) {
-    console.error('❌ Error testing webhook:', error.message);
-  }
-}
-
-// Run the test
-testWebhook();
-
-