codenuk_backend_mine/services/tech-stack-selector/Readme.md

🚀 Enhanced AI Tech Stack Selector v4.0

📋 Overview

The Enhanced AI Tech Stack Selector is an enterprise-grade, AI-powered system that intelligently analyzes business requirements and recommends optimal technology stacks. It's designed as part of a 4-service automated development pipeline that takes natural language requirements and outputs complete, production-ready applications.

🎯 System Purpose

Input: Processed requirements from the Requirement Processor (Port 8001) Output: Intelligent technology stack recommendations with implementation roadmaps Integration: Part of n8n workflow orchestration system

🏗️ Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                    Enhanced Tech Stack Selector                 │
├─────────────────────────────────────────────────────────────────┤
│  🧠 Intelligence Layer                                          │
│  ├── ContextOptimizationEngine     (Token Management)          │
│  ├── HallucinationPreventionEngine (Quality Assurance)         │
│  ├── BusinessProblemAnalyzer       (AI Business Understanding) │
│  ├── TechnologyIntelligenceEngine  (Multi-AI Recommendations)  │
│  └── EnhancedTechStackSelector     (Main Orchestrator)         │
├─────────────────────────────────────────────────────────────────┤
│  💾 Storage Layer                                              │
│  ├── Redis           (Session Context - Fast Access)          │
│  ├── PostgreSQL      (Structured Decision History)            │
│  ├── Neo4j           (Technology Relationship Graphs)         │
│  └── ChromaDB        (Vector Similarity Search)               │
├─────────────────────────────────────────────────────────────────┤
│  🤖 AI Integration Layer                                       │
│  ├── Claude 3.5 Sonnet  (Primary Architecture Analysis)       │
│  ├── GPT-4 Turbo        (Secondary Validation)                │
│  └── Rule-Based Engine  (Baseline Validation)                 │
└─────────────────────────────────────────────────────────────────┘

🔧 Core Components

1. ContextOptimizationEngine

Purpose: Manages AI context within token limits while maximizing relevance

Key Features:

• Token Budget Management: Claude (180K), GPT-4 (100K), Local (8K)
• Intelligent Context Selection: Prioritizes most relevant information
• Hierarchical Context Structure: Level 1 (Critical) → Level 2 (Important) → Level 3 (Supporting)
• Progressive Disclosure: Adds context layers as needed

How It Works:

# 1. Calculate available tokens
available_tokens = self.max_tokens['claude']  # 180,000

# 2. Gather all context components
context_components = {
    'current_requirements': {'priority': 1.0, 'tokens': 5000},
    'similar_decisions': {'priority': 0.9, 'tokens': 8000},
    'technology_trends': {'priority': 0.7, 'tokens': 3000}
}

# 3. Select highest priority components that fit budget
selected_context = intelligent_selection(components, available_tokens)

# 4. Create hierarchical structure for progressive feeding
hierarchical_context = create_levels(selected_context)

2. HallucinationPreventionEngine

Purpose: Detects and prevents AI hallucinations in technology recommendations

Validation Layers:

• Technology Existence: Validates against known technology database
• Scale Appropriateness: Ensures tech choices match project scale
• Domain Fit: Validates domain-specific technology alignment
• Internal Consistency: Checks for contradictions in recommendations
• Implementation Feasibility: Validates team size vs complexity

Technology Knowledge Base:

technology_knowledge_base = {
    'frontend_frameworks': {
        'react': {'maturity': 'high', 'ecosystem': 'excellent'},
        'vue': {'maturity': 'high', 'ecosystem': 'good'},
        'angular': {'maturity': 'high', 'ecosystem': 'excellent'}
    },
    'backend_technologies': {
        'nodejs': {'performance': 'good', 'scalability': 'good'},
        'python': {'performance': 'medium', 'scalability': 'good'},
        'java': {'performance': 'excellent', 'scalability': 'excellent'}
    }
}

3. ContextPersistenceManager

Purpose: Manages context storage across multiple database systems

Storage Strategy:

• Redis: Fast session context (1-hour TTL)
• PostgreSQL: Structured decision history with versioning
• Neo4j: Technology relationship graphs and patterns
• ChromaDB: Vector embeddings for semantic similarity search

Data Flow:

User Request → Context Retrieval → AI Analysis → Decision Storage
     ↓              ↓                 ↓              ↓
Project ID → Redis Lookup → Multi-AI → All Databases
             Session Cache   Processing   Updated

4. BusinessProblemAnalyzer

Purpose: Uses AI to understand core business problems dynamically

Analysis Process:

1. Context Extraction: Pulls domain, complexity, requirements from processor output
2. AI Business Analysis: Claude analyzes business model, value proposition, constraints
3. Problem Characteristics: Assesses complexity, scale, performance, team needs
4. Fallback Logic: Rule-based analysis when AI unavailable

Business Model Detection:

# AI Prompt Example
"""
Analyze this business requirement:
Domain: ecommerce
Requirements: "Online marketplace for handmade crafts..."

Return JSON:
{
    "core_business_problem": "Enable artisans to sell online",
    "business_model": "marketplace",
    "value_proposition": "Connect craft buyers with artisan sellers",
    "success_criteria": ["vendor adoption", "transaction volume"]
}
"""

5. TechnologyIntelligenceEngine

Purpose: AI-driven technology recommendations with multi-model consensus

Recommendation Process:

1. Context Optimization: Prepare context for AI models
2. Primary Analysis: Claude generates comprehensive recommendations
3. Secondary Validation: GPT-4 validates and suggests improvements
4. Multi-AI Consensus: Synthesizes recommendations from multiple sources
5. Final Assessment: Risk analysis, implementation roadmap, success metrics

AI Consensus Logic:

# Weight different AI models based on reliability
model_weights = {
    'claude': 0.4,      # Primary for architecture
    'gpt4': 0.3,        # Secondary for validation
    'rule_based': 0.3   # Baseline validation
}

# Calculate consensus score
consensus_score = sum(model_confidence * weight for model, weight in weights)

6. EnhancedTechStackSelector

Purpose: Main orchestrator that coordinates all components

Selection Process:

1. Context Retrieval: Get conversation history for project continuity
2. Business Analysis: Understand the core business problem
3. Historical Learning: Find similar past decisions and success rates
4. AI Recommendations: Generate intelligent technology suggestions
5. Validation & Enhancement: Apply historical data and validation
6. Response Generation: Create comprehensive recommendation package
7. Context Storage: Store decision for future learning

🤖 AI Integration Details

API Keys Configuration

# Your actual API keys are configured in the code
CLAUDE_API_KEY = "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"
OPENAI_API_KEY = "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"

Model Selection Strategy

• Claude 3.5 Sonnet: Primary model for architecture analysis (180K context)
• GPT-4 Turbo: Secondary validation and cross-checking (100K context)
• Rule-Based: Fallback when AI models unavailable

Token Management

# Progressive context feeding based on model capacity
if needs_more_context(response):
    # Add Level 2 context
    enhanced_prompt = base_prompt + level_2_context
    response = call_ai_model(enhanced_prompt)
    
    if still_needs_context(response):
        # Add Level 3 context
        final_prompt = enhanced_prompt + level_3_context
        response = call_ai_model(final_prompt)

💾 Database Integration

Redis (Session Context)

Purpose: Fast access to current conversation state TTL: 1 hour Data Structure:

{
    "last_analysis": {...},
    "last_recommendations": {...},
    "context_version": 2
}

PostgreSQL (Structured Storage)

Purpose: Permanent storage of technology decisions Schema:

CREATE TABLE tech_decisions (
    id SERIAL PRIMARY KEY,
    project_id VARCHAR(255) UNIQUE,
    decision_data JSONB,
    timestamp TIMESTAMP,
    domain VARCHAR(100),
    complexity VARCHAR(100),
    version INTEGER DEFAULT 1
);

Neo4j (Graph Relationships)

Purpose: Technology relationship patterns and domain connections Graph Structure:

(Project)-[:HAS_DOMAIN]->(Domain)
(Project)-[:USES_FRONTEND]->(Frontend)
(Project)-[:USES_BACKEND]->(Backend)
(Frontend)-[:COMPATIBLE_WITH]->(Backend)

ChromaDB (Vector Similarity)

Purpose: Semantic search for similar projects Process:

1. Convert requirements to embeddings using SentenceTransformer
2. Store project embeddings with metadata
3. Query for similar projects using vector similarity
4. Return top 5 most similar past decisions

📡 API Endpoints

Main Endpoint: POST /api/v1/select

Purpose: Primary technology stack selection endpoint

Request Format:

{
    "processed_requirements": {
        "comprehensive_analysis": {...},
        "original_requirements": "Build an e-commerce platform...",
        "implementation_strategy": {...}
    },
    "project_name": "E-commerce Platform",
    "project_id": "optional-for-context-continuity"
}

Response Format:

{
    "success": true,
    "data": {
        "project_id": "uuid-generated",
        "analysis_metadata": {
            "processing_method": "multi_model_consensus",
            "confidence_score": 0.92,
            "ai_models_used": ["claude", "openai", "rule_based"]
        },
        "business_problem_analysis": {...},
        "technology_recommendations": {...},
        "actionable_recommendations": {
            "primary_stack": {...},
            "implementation_priorities": [...],
            "risk_mitigation_plan": {...}
        }
    }
}

Health Check: GET /health

Purpose: System health monitoring Returns: Component status, uptime, feature availability

Debug Endpoints:

• GET /api/v1/debug/ai-models - Test AI model connectivity
• GET /api/v1/context/{project_id} - Retrieve project context
• GET /api/v1/system-status - Comprehensive system status

🔄 Processing Methods

The system supports multiple processing methods based on available resources:

1. MULTI_MODEL_CONSENSUS (Preferred)

• Uses Claude + GPT-4 + Rule-based analysis
• Highest confidence and accuracy
• Cross-validates recommendations

2. CONTEXT_ENHANCED (Single AI Model)

• Uses one AI model with enhanced context
• Good performance when only one model available
• Still includes validation layers

3. RULE_BASED_ONLY (Fallback)

• Pure rule-based analysis
• No AI models required
• Basic but functional recommendations

🛡️ Quality Assurance

Hallucination Prevention

1. Technology Validation: Check against known technology database
2. Consistency Checking: Ensure internal logical consistency
3. Scale Validation: Match technology to project scale
4. Domain Validation: Ensure domain-appropriate choices

Confidence Scoring

# Multi-factor confidence calculation
base_confidence = ai_model_confidence  # 0.9
validation_boost = validation_score    # 0.85
historical_factor = success_rate       # 0.8

final_confidence = (base_confidence * 0.5) + 
                  (validation_boost * 0.3) + 
                  (historical_factor * 0.2)

🚀 Development Setup

Requirements

pip install fastapi uvicorn anthropic openai
pip install redis asyncpg neo4j chromadb
pip install sentence-transformers loguru

Environment Variables

# API Keys (also hardcoded in main.py)
CLAUDE_API_KEY=your-claude-key
OPENAI_API_KEY=your-openai-key

# Database Connections
REDIS_HOST=redis
REDIS_PORT=6379
POSTGRES_URL=postgresql://user:pass@postgres:5432/db
NEO4J_URI=bolt://neo4j:7687
CHROMA_HOST=chromadb
CHROMA_PORT=8000

Running the Service

# Development
python main.py

# Production
uvicorn main:app --host 0.0.0.0 --port 8002

# Docker
docker build -t tech-stack-selector .
docker run -p 8002:8002 tech-stack-selector

🔧 Integration with n8n Pipeline

Pipeline Flow

User Input → Requirement Processor (8001) → Tech Stack Selector (8002) → Architecture Designer (8003) → Code Generator (8004)

n8n Configuration

{
    "name": "Tech Stack Selection",
    "type": "HTTP Request",
    "url": "http://tech-stack-selector:8002/api/v1/select",
    "method": "POST",
    "body": "{{ $json.data }}"
}

📊 Monitoring & Debugging

Health Monitoring

• Component health checks for all databases
• AI model connectivity testing
• Feature availability status

Logging

• Structured logging with loguru
• Request/response logging
• Error tracking and debugging
• Performance metrics

Debug Tools

• AI model connectivity testing
• Context retrieval and inspection
• System status comprehensive view
• Storage system health checks

🎯 Future Enhancement Opportunities

For Junior Developers

1. Enhanced Business Logic

   • Add more domain-specific patterns
   • Improve complexity scoring algorithms
   • Add industry-specific recommendations

2. AI Model Improvements

   • Add more AI models (Gemini, etc.)
   • Implement custom fine-tuned models
   • Add specialized domain models

3. Context Optimization

   • Implement more sophisticated embedding models
   • Add semantic chunking algorithms
   • Improve relevance scoring

4. Storage Enhancements

   • Add time-series analysis
   • Implement better caching strategies
   • Add backup and recovery systems

5. API Improvements

   • Add streaming responses
   • Implement webhooks for updates
   • Add batch processing capabilities

🐛 Common Issues & Solutions

Issue: AI Model Not Responding

Symptoms: 500 errors, timeout responses Solution: Check API keys, test connectivity via debug endpoint

Issue: Context Not Persisting

Symptoms: No conversation history, recommendations not improving Solution: Verify database connections, check Redis TTL settings

Issue: Low Confidence Scores

Symptoms: Confidence < 0.7, validation warnings Solution: Check input quality, verify AI model responses, review validation rules

Issue: Poor Recommendations

Symptoms: Inappropriate technology choices, mismatched scale Solution: Review business problem analysis, check domain classification, verify complexity scoring

📝 Code Examples

Adding a New Domain

# In BusinessProblemAnalyzer._fallback_business_analysis()
elif 'gaming' in domain or 'game' in requirements:
    business_model = "gaming"
    core_problem = "Create engaging gaming experience"
    
# In ContextOptimizationEngine._get_business_indicators()
'gaming': ['real-time', 'multiplayer', 'graphics', 'performance']

Adding Custom Validation Rules

# In HallucinationPreventionEngine._build_validation_rules()
'gaming_validation': {
    'required_features': ['real_time', 'graphics', 'performance'],
    'recommended_tech': ['unity', 'unreal', 'webgl']
}

Extending AI Prompts

# In TechnologyIntelligenceEngine._build_context_optimized_prompt()
if domain == 'gaming':
    base_prompt += """
    ## Gaming-Specific Considerations:
    - Real-time performance requirements
    - Graphics and rendering needs
    - Multiplayer architecture considerations
    """

📚 Additional Resources

• FastAPI Documentation: https://fastapi.tiangolo.com/
• Claude API: https://docs.anthropic.com/
• OpenAI API: https://platform.openai.com/docs
• Neo4j Documentation: https://neo4j.com/docs/
• ChromaDB Guide: https://docs.trychroma.com/

---

Last Updated: July 3, 2025 Version: 4.0.0 Maintainer: AI Development Pipeline Team Status: Production Ready ✅

Normal startup (auto-detects if migration needed)

./start_migrated.sh

Force re-migration (useful when you add new data)

./start_migrated.sh --force-migration

Show help

./start_migrated.sh --help

healthcare, finance, gaming, education, media, iot, social, elearning, realestate, travel, manufacturing, ecommerce, saas