codenuk_backend_mine/Readme-final.md

# Complete Deployment Guide for Junior Developers
## Automated Development Pipeline

### 🎯 **SYSTEM STATUS: FULLY OPERATIONAL**

**Good News!** Your automated development pipeline is already deployed and working! Here's what's currently running:

### **✅ CURRENT SYSTEM STATUS**
- **16 Services Running** - All core services are UP and HEALTHY
- **Complete Pipeline Active** - requirement-processor → tech-stack-selector → architecture-designer → code-generator
- **All Databases Connected** - PostgreSQL, MongoDB, Redis, Neo4j, ChromaDB
- **Backend API Working** - All services responding on their designated ports

### **🎭 ENTRY POINTS**
1. **Web Dashboard (React)** - Port 3001 (Main UI for creating requirements)
2. **n8n Workflow** - Port 5678 (Orchestration & automation)
3. **Main Dashboard Service** - Port 8008 (System monitoring)

---

## 📋 **SERVICES OVERVIEW**

| Service | Status | Port | Purpose | Health |
|---------|--------|------|---------|--------|
| **Frontend & UI** |
| web-dashboard (React) | ⚠️ Not Started | 3001 | Complete project builder with auth | Need to start |
| dashboard-service | ❌ Unhealthy | 8008 | System monitoring | Needs fixing |
| user-auth | ❌ Unhealthy | 8011 | User registration/login/JWT | CRITICAL - needed by frontend |
| **Core Pipeline** |
| requirement-processor | ✅ Healthy | 8001 | Process requirements → features | Working |
| tech-stack-selector | ✅ Healthy | 8002 | Features → tech recommendations | Working |
| architecture-designer | ✅ Healthy | 8003 | Tech stack → system architecture | Working |
| code-generator | ✅ Healthy | 8004 | Architecture → generated code | Working |
| **Supporting Services** |
| api-gateway | ✅ Healthy | 8000 | API routing & management | Working |
| test-generator | ✅ Healthy | 8005 | Generate test cases | Working |
| deployment-manager | ✅ Healthy | 8006 | Handle deployments | Working |
| self-improving-generator | ✅ Healthy | 8007 | Code quality improvement | Working |
| template-manager | ⚠️ Starting | 8009 | Dynamic templates & features | CRITICAL - needed by frontend |
| **Infrastructure** |
| postgres | ✅ Healthy | 5432 | Primary database | Working |
| neo4j | ✅ Healthy | 7474/7687 | Graph database | Working |
| chromadb | ✅ Healthy | 8010 | Vector database | Working |
| rabbitmq | ✅ Healthy | 5672/15672 | Message queue | Working |
| n8n | ✅ Healthy | 5678 | Workflow orchestration | Working |

---

## 🚀 **GETTING STARTED (3 Steps)**

### **Step 1: Start Web Dashboard (Main Entry Point)**
```bash
# Navigate to project directory
cd /Users/yasha/Documents/Tech4biz-Code-Generator/automated-dev-pipeline

# Start the React web dashboard
cd services/web-dashboard
npm start
```

**Expected Output:**
```
Compiled successfully!

You can now view web-dashboard in the browser.

  Local:            http://localhost:3001
  On Your Network:  http://192.168.x.x:3001
```

### **Step 2: Access the System**
Open your browser and go to:
- **Main Interface:** http://localhost:3001 (Web Dashboard - Requirements Creation)
- **System Monitor:** http://localhost:8008 (Dashboard Service - if healthy)
- **Workflow Manager:** http://localhost:5678 (n8n - username: pipeline_admin, password: pipeline_n8n_2024)

### **Step 3: Test the Pipeline**
1. Create requirements in Web Dashboard (port 3001)
2. Process through the pipeline
3. Monitor results in Dashboard Service (port 8008)

---

## 🔧 **BACKEND CREDENTIALS & CONNECTIONS**

### **Database Connections (For Development)**

#### **PostgreSQL (Primary Database)**
```bash
# Connection Details
Host: localhost (external) / postgres (internal)
Port: 5432
Database: dev_pipeline
Username: pipeline_admin
Password: secure_pipeline_2024

# Connect via Docker
docker exec -it pipeline_postgres psql -U pipeline_admin -d dev_pipeline

# Connection String
postgresql://pipeline_admin:secure_pipeline_2024@localhost:5432/dev_pipeline
```

#### **MongoDB (Document Storage)**
```bash
# Connection Details
Host: localhost (external) / mongodb (internal)
Port: 27017
Username: pipeline_user
Password: pipeline_password

# Connect via Docker
docker exec -it pipeline_mongodb mongosh -u pipeline_user -p pipeline_password

# Connection String
mongodb://pipeline_user:pipeline_password@localhost:27017/
```

#### **Redis (Cache & Sessions)**
```bash
# Connection Details
Host: localhost (external) / redis (internal)
Port: 6379
Password: redis_secure_2024

# Connect via Docker
docker exec -it pipeline_redis redis-cli -a redis_secure_2024

# Connection String
redis://redis:6379
```

#### **Neo4j (Graph Database)**
```bash
# Connection Details
Host: localhost
HTTP Port: 7474 (Neo4j Browser)
Bolt Port: 7687 (Application connections)
Username: neo4j
Password: password

# Access Neo4j Browser
http://localhost:7474
```

#### **ChromaDB (Vector Database)**
```bash
# Connection Details
Host: localhost
Port: 8010
API Endpoint: http://localhost:8010

# Test connection
curl http://localhost:8010/api/v1/heartbeat
```

### **API Keys & Environment**
```bash
# Anthropic Claude API
ANTHROPIC_API_KEY=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

# React App Environment
REACT_APP_ANTHROPIC_API_KEY=(same as above)
```

---

## 🧪 **TESTING THE SYSTEM**

### **Quick Health Check All Services**
```bash
#!/bin/bash
# Save this as check_health.sh and run it

echo "🔍 Checking all services..."

services=(
  "8001:requirement-processor"
  "8002:tech-stack-selector" 
  "8003:architecture-designer"
  "8004:code-generator"
  "8000:api-gateway"
  "8005:test-generator"
  "8006:deployment-manager"
  "8007:self-improving-generator"
  "8008:dashboard-service"
  "8009:template-manager"
  "8011:user-auth"
  "5678:n8n"
  "8010:chromadb"
)

for service in "${services[@]}"; do
  port=$(echo $service | cut -d: -f1)
  name=$(echo $service | cut -d: -f2)
  printf "%-25s " "$name:"
  if curl -s -f http://localhost:$port/health > /dev/null 2>&1; then
    echo "✅ Healthy"
  else
    echo "❌ Unhealthy or No Health Endpoint"
  fi
done
```

### **Test the Complete Pipeline**
```bash
# 1. Test Requirement Processing
curl -X POST http://localhost:8001/api/v1/process-requirements \
  -H "Content-Type: application/json" \
  -d '{
    "project_name": "Test E-commerce",
    "user_management": true,
    "payment_processing": true,
    "inventory_management": true,
    "reporting": true
  }'

# 2. Test Tech Stack Selection
curl -X POST http://localhost:8002/api/v1/select-tech-stack \
  -H "Content-Type: application/json" \
  -d '{
    "features": ["user_management", "payment_processing"],
    "scale": "medium",
    "complexity": "high"
  }'

# 3. Test Architecture Design
curl -X POST http://localhost:8003/api/v1/design-architecture \
  -H "Content-Type: application/json" \
  -d '{
    "tech_stack": {"backend": "python", "frontend": "react"},
    "requirements": {"features": ["user_management"]}
  }'

# 4. Test Code Generation
curl -X POST http://localhost:8004/api/v1/generate-code \
  -H "Content-Type: application/json" \
  -d '{
    "architecture": {"type": "microservices"},
    "tech_stack": {"backend": "python"},
    "requirements": {"project_name": "test"}
  }'
```

---

## 🚨 **FIXING UNHEALTHY SERVICES**

### **Fix Dashboard Service (Port 8008)**
```bash
# Check logs
docker logs pipeline_dashboard

# If unhealthy, restart
docker compose restart dashboard

# Check health again
curl http://localhost:8008/api/health
```

### **Fix User Auth Service (Port 8011)**
```bash
# Check logs
docker logs pipeline_user_auth

# If unhealthy, restart
docker compose restart user-auth

# Check health again
curl http://localhost:8011/health
```

---

## 🔄 **COMMON OPERATIONS**

### **Restart Entire System**
```bash
cd /Users/yasha/Documents/Tech4biz-Code-Generator/automated-dev-pipeline

# Stop all services
docker compose down

# Start all services
docker compose up -d

# Check status
docker compose ps
```

### **Restart Individual Service**
```bash
# Restart a specific service
docker compose restart requirement-processor

# Check its logs
docker logs pipeline_requirement_processor

# Check its health
curl http://localhost:8001/health
```

### **Update Service Code**
```bash
# If you modify service code, rebuild and restart
docker compose build requirement-processor
docker compose up -d requirement-processor
```

### **View Real-time Logs**
```bash
# View logs for all services
docker compose logs -f

# View logs for specific service
docker logs -f pipeline_requirement_processor
```

---

## 🛠️ **DEVELOPMENT WORKFLOW**

### **Making Changes to Services**

1. **Edit Code**
   ```bash
   # Edit service files
   nano services/requirement-processor/src/main.py
   ```

2. **Rebuild & Restart**
   ```bash
   docker compose build requirement-processor
   docker compose up -d requirement-processor
   ```

3. **Test Changes**
   ```bash
   curl http://localhost:8001/health
   ```

### **Adding New Features**

1. **Update Requirements**
   ```bash
   # Add to requirements.txt
   nano services/requirement-processor/requirements.txt
   ```

2. **Rebuild Container**
   ```bash
   docker compose build requirement-processor
   docker compose up -d requirement-processor
   ```

### **Database Operations**

```bash
# Connect to PostgreSQL
docker exec -it pipeline_postgres psql -U pipeline_admin -d dev_pipeline

# View tables
\dt

# Connect to MongoDB
docker exec -it pipeline_mongodb mongosh -u pipeline_user -p pipeline_password

# Show databases
show dbs

# Connect to Redis
docker exec -it pipeline_redis redis-cli -a redis_secure_2024

# View keys
keys *
```

---

## 📊 **MONITORING & DEBUGGING**

### **Check Resource Usage**
```bash
# View container resource usage
docker stats

# View system resources
docker system df
```

### **Debug Service Issues**
```bash
# Check container logs
docker logs pipeline_requirement_processor

# Check container environment
docker exec pipeline_requirement_processor env

# Check container filesystem
docker exec -it pipeline_requirement_processor ls -la /app
```

### **Performance Monitoring**
```bash
# Check database connections
docker exec pipeline_postgres psql -U pipeline_admin -d dev_pipeline -c "SELECT count(*) FROM pg_stat_activity;"

# Check Redis memory usage
docker exec pipeline_redis redis-cli -a redis_secure_2024 info memory
```

---

## 🎯 **SUCCESS CHECKLIST**

### ✅ **System is Ready When:**
- [ ] All 16 services show as "healthy" in `docker compose ps`
- [ ] Web dashboard accessible at http://localhost:3001
- [ ] All API health checks return successful responses
- [ ] Can create requirements through web interface
- [ ] Pipeline processes requirements → tech stack → architecture → code
- [ ] Generated code appears in dashboard

### ✅ **Development Environment Ready When:**
- [ ] Can modify service code and see changes after rebuild
- [ ] Database connections working from external tools
- [ ] Logs provide clear debugging information
- [ ] Health checks help identify issues quickly

---

## 🆘 **EMERGENCY PROCEDURES**

### **Complete System Reset**
```bash
# WARNING: This will delete all data!
docker compose down -v
docker system prune -a
docker compose up -d
```

### **Backup Important Data**
```bash
# Backup PostgreSQL
docker exec pipeline_postgres pg_dump -U pipeline_admin dev_pipeline > backup_$(date +%Y%m%d).sql

# Backup generated projects
cp -r generated-projects backup_projects_$(date +%Y%m%d)
```

### **Contact Information**
- Check logs first: `docker compose logs -f`
- Check service health: `curl http://localhost:PORT/health`
- Check database connections using provided credentials
- Review this guide's troubleshooting section

---

This guide provides everything a junior developer needs to deploy, operate, and maintain your automated development pipeline system. The system is already working - they just need to start the React web dashboard to access the main interface!