1275 lines
30 KiB
Markdown
1275 lines
30 KiB
Markdown
# AI Analysis Service - Communication Architecture
|
|
|
|
## Table of Contents
|
|
1. [System Architecture Overview](#system-architecture-overview)
|
|
2. [Service Components](#service-components)
|
|
3. [Communication Flow](#communication-flow)
|
|
4. [AI Analysis Service Details](#ai-analysis-service-details)
|
|
5. [API Gateway Integration](#api-gateway-integration)
|
|
6. [Git Integration Service](#git-integration-service)
|
|
7. [Frontend Integration](#frontend-integration)
|
|
8. [Request/Response Examples](#requestresponse-examples)
|
|
9. [Error Handling](#error-handling)
|
|
10. [Deployment Configuration](#deployment-configuration)
|
|
|
|
---
|
|
|
|
## System Architecture Overview
|
|
|
|
The AI Analysis Service is part of a microservices architecture with the following components:
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ FRONTEND │
|
|
│ (Next.js Application) │
|
|
│ http://localhost:3000 │
|
|
└────────────────────┬────────────────────────────────────────────┘
|
|
│
|
|
│ HTTP/REST
|
|
│
|
|
┌────────────────────▼────────────────────────────────────────────┐
|
|
│ API GATEWAY │
|
|
│ (Express.js Proxy Layer) │
|
|
│ Port: 8000 │
|
|
│ Routes: /api/ai-analysis/*, /api/github/*, /api/ai/repository/*│
|
|
└──────────┬─────────────────────────┬──────────────────────┬─────┘
|
|
│ │ │
|
|
│ HTTP │ HTTP │ HTTP
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌──────────────────┐ ┌──────────────────────┐ ┌───────────────┐
|
|
│ AI ANALYSIS │ │ GIT INTEGRATION │ │ OTHER │
|
|
│ SERVICE │◄──┤ SERVICE │ │ SERVICES │
|
|
│ Port: 8022 │ │ Port: 8012 │ │ │
|
|
│ (Python/FastAPI)│ │ (Node.js/Express) │ │ │
|
|
└──────────────────┘ └──────────────────────┘ └───────────────┘
|
|
│ │
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────┐ ┌──────────────────────┐
|
|
│ PostgreSQL DB │ │ Redis Cache │
|
|
│ (Repository │ │ (Analysis Results) │
|
|
│ Metadata) │ │ │
|
|
└──────────────────┘ └──────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Service Components
|
|
|
|
### 1. AI Analysis Service (Port 8022)
|
|
- **Technology**: Python 3.11, FastAPI, Anthropic Claude API
|
|
- **Purpose**: Analyzes repository code using Claude AI
|
|
- **Key Features**:
|
|
- Repository analysis with memory system
|
|
- File-by-file code quality assessment
|
|
- PDF/JSON report generation
|
|
- Rate limiting for Claude API (90 requests/minute)
|
|
- Redis caching for repeated analyses
|
|
- Content optimization for large files
|
|
|
|
### 2. API Gateway (Port 8000)
|
|
- **Technology**: Node.js, Express.js
|
|
- **Purpose**: Central entry point for all frontend requests
|
|
- **Key Features**:
|
|
- Request routing and proxying
|
|
- CORS handling
|
|
- Authentication middleware
|
|
- Service health monitoring
|
|
- WebSocket support
|
|
- Request logging
|
|
|
|
### 3. Git Integration Service (Port 8012)
|
|
- **Technology**: Node.js, Express.js
|
|
- **Purpose**: Manages repository connections and file access
|
|
- **Key Features**:
|
|
- GitHub OAuth integration
|
|
- Repository cloning and syncing
|
|
- File tree navigation
|
|
- Diff generation
|
|
- Commit history tracking
|
|
- Multi-VCS support (GitHub, GitLab, Bitbucket, Gitea)
|
|
|
|
### 4. Frontend (Port 3000)
|
|
- **Technology**: Next.js 14, React, TypeScript
|
|
- **Purpose**: User interface
|
|
- **Key Features**:
|
|
- Repository management
|
|
- AI analysis triggering
|
|
- Real-time progress tracking
|
|
- Report viewing
|
|
|
|
---
|
|
|
|
## Communication Flow
|
|
|
|
### Complete Analysis Flow
|
|
|
|
```
|
|
┌──────────┐
|
|
│ Frontend │
|
|
└────┬─────┘
|
|
│
|
|
│ 1. POST /api/ai-analysis/analyze-repository
|
|
│ Body: { repository_id, user_id, output_format, max_files }
|
|
│
|
|
▼
|
|
┌────────────────┐
|
|
│ API Gateway │
|
|
└────┬───────────┘
|
|
│
|
|
│ 2. Forward to AI Analysis Service
|
|
│ POST /analyze-repository
|
|
│ Timeout: 240 seconds
|
|
│
|
|
▼
|
|
┌──────────────────────┐
|
|
│ AI Analysis Service │
|
|
└────┬─────────────────┘
|
|
│
|
|
│ 3. Get Repository Info from Git Integration
|
|
│ GET http://git-integration:8012/api/github/repository/{id}/ui-view
|
|
│ Headers: { x-user-id: userId }
|
|
│
|
|
▼
|
|
┌──────────────────────┐
|
|
│ Git Integration │
|
|
└────┬─────────────────┘
|
|
│
|
|
│ 4. Return Repository Metadata
|
|
│ { repository_info, file_tree, local_path }
|
|
│
|
|
▼
|
|
┌──────────────────────┐
|
|
│ AI Analysis Service │
|
|
└────┬─────────────────┘
|
|
│
|
|
│ 5. For each file:
|
|
│ - Get file content via Git Integration API
|
|
│ - Check Redis cache
|
|
│ - Apply rate limiting
|
|
│ - Optimize content (truncate if > 8000 tokens)
|
|
│ - Send to Claude API
|
|
│ - Cache result
|
|
│
|
|
│ 6. Generate Repository-level Analysis
|
|
│ - Architecture assessment
|
|
│ - Security review
|
|
│ - Code quality metrics
|
|
│
|
|
│ 7. Create Report (PDF/JSON)
|
|
│ Path: /app/reports/{analysis_id}_analysis.pdf
|
|
│
|
|
│ 8. Return Analysis Results
|
|
│ { success, analysis_id, report_path, stats }
|
|
│
|
|
▼
|
|
┌────────────────┐
|
|
│ API Gateway │
|
|
└────┬───────────┘
|
|
│
|
|
│ 9. Forward response to Frontend
|
|
│
|
|
▼
|
|
┌──────────┐
|
|
│ Frontend │
|
|
└──────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## AI Analysis Service Details
|
|
|
|
### Base URL
|
|
```
|
|
Direct: http://localhost:8022
|
|
Via Gateway: http://localhost:8000/api/ai-analysis
|
|
```
|
|
|
|
### Key Endpoints
|
|
|
|
#### 1. Analyze Repository
|
|
**Endpoint**: `POST /analyze-repository`
|
|
|
|
**Request**:
|
|
```json
|
|
{
|
|
"repository_id": "uuid-string",
|
|
"user_id": "user-uuid",
|
|
"output_format": "pdf",
|
|
"max_files": 100
|
|
}
|
|
```
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"message": "Repository analysis completed successfully",
|
|
"analysis_id": "repo_analysis_uuid_20241216_143022",
|
|
"report_path": "/app/reports/repo_analysis_uuid_20241216_143022_analysis.pdf",
|
|
"stats": {
|
|
"repository_id": "uuid-string",
|
|
"total_files": 85,
|
|
"total_lines": 15420,
|
|
"languages": ["typescript", "javascript", "python"],
|
|
"code_quality_score": 7.8,
|
|
"high_quality_files": 45,
|
|
"medium_quality_files": 30,
|
|
"low_quality_files": 10,
|
|
"total_issues": 23
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Get Repository Info
|
|
**Endpoint**: `GET /repository/{repository_id}/info?user_id={user_id}`
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"repository_info": {
|
|
"id": "uuid-string",
|
|
"name": "my-repo",
|
|
"owner": "owner-name",
|
|
"local_path": "/app/git-repos/owner__my-repo__main",
|
|
"repository_url": "https://github.com/owner/my-repo"
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 3. Download Report
|
|
**Endpoint**: `GET /reports/{filename}`
|
|
|
|
Returns the generated PDF or JSON report file.
|
|
|
|
#### 4. Memory System Stats
|
|
**Endpoint**: `GET /memory/stats`
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"memory_stats": {
|
|
"total_memories": 150,
|
|
"recent_analyses": 5,
|
|
"cache_hit_rate": 0.65
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 5. Health Check
|
|
**Endpoint**: `GET /health`
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"status": "healthy",
|
|
"service": "ai-analysis-service",
|
|
"timestamp": "2024-12-16T14:30:22.000Z",
|
|
"version": "1.0.0"
|
|
}
|
|
```
|
|
|
|
### Internal Components
|
|
|
|
#### GitIntegrationClient
|
|
Handles communication with git-integration service:
|
|
|
|
```python
|
|
class GitIntegrationClient:
|
|
def __init__(self):
|
|
self.base_url = os.getenv('GIT_INTEGRATION_SERVICE_URL',
|
|
'http://git-integration:8012')
|
|
self.timeout = 30.0
|
|
|
|
async def get_repository_info(self, repository_id: str, user_id: str):
|
|
"""Get repository information from git-integration service."""
|
|
response = await client.get(
|
|
f"{self.base_url}/api/github/repository/{repository_id}/ui-view",
|
|
headers={'x-user-id': user_id}
|
|
)
|
|
return response.json()
|
|
```
|
|
|
|
#### ClaudeRateLimiter
|
|
Manages Claude API rate limits:
|
|
|
|
```python
|
|
class ClaudeRateLimiter:
|
|
def __init__(self, requests_per_minute: int = 90):
|
|
self.requests_per_minute = requests_per_minute
|
|
self.requests = []
|
|
self.lock = asyncio.Lock()
|
|
|
|
async def wait_if_needed(self):
|
|
"""Wait if rate limit would be exceeded."""
|
|
# Implements sliding window rate limiting
|
|
```
|
|
|
|
#### AnalysisCache
|
|
Redis-based caching for analysis results:
|
|
|
|
```python
|
|
class AnalysisCache:
|
|
def __init__(self):
|
|
self.redis = redis.Redis(
|
|
host=os.getenv('REDIS_HOST', 'redis'),
|
|
port=6379,
|
|
decode_responses=True
|
|
)
|
|
self.cache_ttl = 86400 # 24 hours
|
|
|
|
async def get_cached_analysis(self, file_hash: str):
|
|
"""Get cached analysis result."""
|
|
cache_key = f"analysis:{file_hash}"
|
|
return self.redis.get(cache_key)
|
|
```
|
|
|
|
#### ContentOptimizer
|
|
Optimizes large files for Claude API:
|
|
|
|
```python
|
|
class ContentOptimizer:
|
|
@staticmethod
|
|
def optimize_content_for_claude(content: str, max_tokens: int = 8000):
|
|
"""Optimize file content for Claude API limits."""
|
|
if len(content) > max_tokens * 4:
|
|
# Extract important lines (imports, functions, classes)
|
|
# Truncate intelligently
|
|
return optimized_content
|
|
return content
|
|
```
|
|
|
|
---
|
|
|
|
## API Gateway Integration
|
|
|
|
### AI Analysis Routes
|
|
|
|
The API Gateway proxies requests to the AI Analysis Service:
|
|
|
|
```javascript
|
|
// Route: /api/ai-analysis/*
|
|
app.use('/api/ai-analysis',
|
|
createServiceLimiter(200),
|
|
(req, res, next) => {
|
|
const aiAnalysisServiceUrl = process.env.AI_ANALYSIS_URL ||
|
|
'http://localhost:8022';
|
|
const rewrittenPath = req.originalUrl.replace(/^\/api\/ai-analysis/, '');
|
|
const targetUrl = `${aiAnalysisServiceUrl}${rewrittenPath}`;
|
|
|
|
// Proxy configuration
|
|
const options = {
|
|
method: req.method,
|
|
url: targetUrl,
|
|
headers: {
|
|
'Content-Type': 'application/json',
|
|
'User-Agent': 'API-Gateway/1.0',
|
|
'X-User-ID': req.user?.id || req.user?.userId,
|
|
},
|
|
timeout: 240000, // 4 minutes
|
|
maxContentLength: 100 * 1024 * 1024 // 100MB
|
|
};
|
|
|
|
// Forward request
|
|
axios(options)
|
|
.then(response => res.status(response.status).json(response.data))
|
|
.catch(error => handleError(error));
|
|
}
|
|
);
|
|
```
|
|
|
|
### Service Configuration
|
|
|
|
```javascript
|
|
const serviceTargets = {
|
|
AI_ANALYSIS_URL: process.env.AI_ANALYSIS_URL || 'http://localhost:8022',
|
|
GIT_INTEGRATION_URL: process.env.GIT_INTEGRATION_URL || 'http://localhost:8012',
|
|
// ... other services
|
|
};
|
|
```
|
|
|
|
### CORS Configuration
|
|
|
|
```javascript
|
|
app.use(cors({
|
|
origin: '*',
|
|
credentials: true,
|
|
methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'PATCH'],
|
|
allowedHeaders: ['Content-Type', 'Authorization', 'X-User-ID']
|
|
}));
|
|
```
|
|
|
|
---
|
|
|
|
## Git Integration Service
|
|
|
|
### Base URL
|
|
```
|
|
Direct: http://localhost:8012
|
|
Via Gateway: http://localhost:8000/api/github
|
|
```
|
|
|
|
### Key Endpoints Used by AI Analysis
|
|
|
|
#### 1. Get Repository UI View
|
|
**Endpoint**: `GET /api/github/repository/{id}/ui-view?view_type=tree`
|
|
|
|
**Headers**:
|
|
```
|
|
x-user-id: {userId}
|
|
```
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"data": {
|
|
"repository_info": {
|
|
"id": "uuid",
|
|
"name": "repo-name",
|
|
"owner": "owner-name",
|
|
"provider": "github",
|
|
"local_path": "/app/git-repos/owner__repo__main",
|
|
"repository_url": "https://github.com/owner/repo"
|
|
},
|
|
"ui_data": {
|
|
"left_panel": {
|
|
"file_tree": [
|
|
{
|
|
"type": "directory",
|
|
"path": "src",
|
|
"children": [
|
|
{
|
|
"type": "file",
|
|
"path": "src/index.ts",
|
|
"size": 1024
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Get File Content
|
|
**Endpoint**: `GET /api/github/repository/{id}/file-content?file_path={path}`
|
|
|
|
**Headers**:
|
|
```
|
|
x-user-id: {userId}
|
|
```
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"data": {
|
|
"file_info": {
|
|
"filename": "index.ts",
|
|
"file_extension": "ts",
|
|
"relative_path": "src/index.ts",
|
|
"file_size_bytes": 1024,
|
|
"is_binary": false,
|
|
"language_detected": "typescript"
|
|
},
|
|
"content": "import express from 'express';\n...",
|
|
"preview": "import express from 'express';\n..."
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 3. Attach Repository
|
|
**Endpoint**: `POST /api/github/attach-repository`
|
|
|
|
**Request**:
|
|
```json
|
|
{
|
|
"repository_url": "https://github.com/owner/repo",
|
|
"branch_name": "main",
|
|
"user_id": "user-uuid"
|
|
}
|
|
```
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"message": "Repository attached and synced successfully",
|
|
"data": {
|
|
"repository_id": "uuid",
|
|
"repository_name": "repo",
|
|
"owner_name": "owner",
|
|
"branch_name": "main",
|
|
"is_public": true,
|
|
"storage_info": {
|
|
"local_path": "/app/git-repos/owner__repo__main",
|
|
"total_files": 85,
|
|
"total_size_bytes": 1024000
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Communication Pattern
|
|
|
|
The AI Analysis Service communicates with Git Integration through HTTP REST APIs:
|
|
|
|
```python
|
|
async def get_repository_files_from_api(repository_id: str, user_id: str, max_files: int = 100):
|
|
"""Get repository files from Git Integration Service API."""
|
|
|
|
# 1. Get file tree
|
|
response = await client.get(
|
|
f"{git_client.base_url}/api/github/repository/{repository_id}/ui-view?view_type=tree",
|
|
headers={'x-user-id': user_id}
|
|
)
|
|
|
|
data = response.json()
|
|
file_tree = data['data']['ui_data']['left_panel']['file_tree']
|
|
|
|
# 2. Extract file paths from tree
|
|
files_to_analyze = []
|
|
extract_files_from_tree(file_tree, files_to_analyze)
|
|
|
|
# 3. Fetch content for each file
|
|
files_with_content = []
|
|
for file_path, _ in files_to_analyze:
|
|
content_response = await client.get(
|
|
f"{git_client.base_url}/api/github/repository/{repository_id}/file-content?file_path={file_path}",
|
|
headers={'x-user-id': user_id}
|
|
)
|
|
|
|
content_data = content_response.json()
|
|
if content_data['success']:
|
|
content = content_data['content']
|
|
files_with_content.append((file_path, content))
|
|
|
|
return files_with_content
|
|
```
|
|
|
|
---
|
|
|
|
## Frontend Integration
|
|
|
|
### Configuration
|
|
|
|
**File**: `src/config/api-gateway.ts`
|
|
|
|
```typescript
|
|
export const API_GATEWAY_CONFIG = {
|
|
BASE_URL: process.env.NEXT_PUBLIC_API_GATEWAY_URL || 'http://localhost:8000',
|
|
|
|
ENDPOINTS: {
|
|
AI_ANALYSIS: {
|
|
BASE: '/api/ai-analysis',
|
|
ANALYZE: '/api/ai-analysis/analyze-repository',
|
|
REPORT: (filename: string) => `/api/ai-analysis/reports/${filename}`,
|
|
},
|
|
|
|
GIT_INTEGRATION: {
|
|
BASE: '/api/github',
|
|
REPOSITORIES: '/api/github/repositories',
|
|
REPOSITORY_INFO: (repositoryId: string) =>
|
|
`/api/github/repository/${repositoryId}/info`
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### React Hook for AI Analysis
|
|
|
|
**File**: `src/hooks/useAIAnalysis.ts`
|
|
|
|
```typescript
|
|
export const useAIAnalysis = () => {
|
|
const [isAnalyzing, setIsAnalyzing] = useState(false)
|
|
const [progress, setProgress] = useState<AnalysisProgress | null>(null)
|
|
const [result, setResult] = useState<AnalysisResult | null>(null)
|
|
const [error, setError] = useState<string | null>(null)
|
|
|
|
const startAnalysis = async (
|
|
repositoryId: string,
|
|
userId: string,
|
|
options = {}
|
|
) => {
|
|
try {
|
|
setIsAnalyzing(true)
|
|
|
|
// Start analysis via API Gateway
|
|
const response = await fetch(
|
|
buildApiUrl('/api/ai-analysis/analyze-repository'),
|
|
{
|
|
method: 'POST',
|
|
headers: getApiHeaders(),
|
|
body: JSON.stringify({
|
|
repository_id: repositoryId,
|
|
user_id: userId,
|
|
output_format: options.output_format || 'pdf',
|
|
max_files: options.max_files || 100
|
|
})
|
|
}
|
|
)
|
|
|
|
if (!response.ok) {
|
|
throw new Error('Analysis failed to start')
|
|
}
|
|
|
|
const data = await response.json()
|
|
|
|
if (data.success) {
|
|
setResult(data)
|
|
}
|
|
} catch (err) {
|
|
setError(err.message)
|
|
} finally {
|
|
setIsAnalyzing(false)
|
|
}
|
|
}
|
|
|
|
return { isAnalyzing, progress, result, error, startAnalysis }
|
|
}
|
|
```
|
|
|
|
### Usage Example
|
|
|
|
```typescript
|
|
import { useAIAnalysis } from '@/hooks/useAIAnalysis'
|
|
|
|
function RepositoryAnalysis({ repositoryId, userId }) {
|
|
const {
|
|
isAnalyzing,
|
|
result,
|
|
error,
|
|
startAnalysis
|
|
} = useAIAnalysis()
|
|
|
|
const handleAnalyze = async () => {
|
|
await startAnalysis(repositoryId, userId, {
|
|
output_format: 'pdf',
|
|
max_files: 100
|
|
})
|
|
}
|
|
|
|
return (
|
|
<div>
|
|
<button
|
|
onClick={handleAnalyze}
|
|
disabled={isAnalyzing}
|
|
>
|
|
{isAnalyzing ? 'Analyzing...' : 'Analyze Repository'}
|
|
</button>
|
|
|
|
{result && (
|
|
<div>
|
|
<h3>Analysis Complete</h3>
|
|
<p>Quality Score: {result.stats.code_quality_score}/10</p>
|
|
<p>Total Files: {result.stats.total_files}</p>
|
|
<p>Total Issues: {result.stats.total_issues}</p>
|
|
<a href={`/api/ai-analysis/reports/${result.analysis_id}_analysis.pdf`}>
|
|
Download Report
|
|
</a>
|
|
</div>
|
|
)}
|
|
|
|
{error && <div className="error">{error}</div>}
|
|
</div>
|
|
)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Request/Response Examples
|
|
|
|
### Example 1: Complete Analysis Flow
|
|
|
|
#### Step 1: Frontend initiates analysis
|
|
|
|
```http
|
|
POST http://localhost:8000/api/ai-analysis/analyze-repository
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"repository_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
"user_id": "660e8400-e29b-41d4-a716-446655440001",
|
|
"output_format": "pdf",
|
|
"max_files": 100
|
|
}
|
|
```
|
|
|
|
#### Step 2: API Gateway forwards to AI Analysis Service
|
|
|
|
```http
|
|
POST http://localhost:8022/analyze-repository
|
|
Content-Type: application/json
|
|
X-User-ID: 660e8400-e29b-41d4-a716-446655440001
|
|
|
|
{
|
|
"repository_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
"user_id": "660e8400-e29b-41d4-a716-446655440001",
|
|
"output_format": "pdf",
|
|
"max_files": 100
|
|
}
|
|
```
|
|
|
|
#### Step 3: AI Analysis Service gets repository info
|
|
|
|
```http
|
|
GET http://localhost:8012/api/github/repository/550e8400-e29b-41d4-a716-446655440000/ui-view?view_type=tree
|
|
x-user-id: 660e8400-e29b-41d4-a716-446655440001
|
|
```
|
|
|
|
#### Step 4: Git Integration returns repository data
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"data": {
|
|
"repository_info": {
|
|
"id": "550e8400-e29b-41d4-a716-446655440000",
|
|
"name": "my-project",
|
|
"owner": "johndoe",
|
|
"local_path": "/app/git-repos/johndoe__my-project__main"
|
|
},
|
|
"ui_data": {
|
|
"left_panel": {
|
|
"file_tree": [
|
|
{
|
|
"type": "file",
|
|
"path": "src/index.ts",
|
|
"name": "index.ts"
|
|
},
|
|
{
|
|
"type": "file",
|
|
"path": "src/utils.ts",
|
|
"name": "utils.ts"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Step 5: AI Analysis Service gets file content
|
|
|
|
```http
|
|
GET http://localhost:8012/api/github/repository/550e8400-e29b-41d4-a716-446655440000/file-content?file_path=src/index.ts
|
|
x-user-id: 660e8400-e29b-41d4-a716-446655440001
|
|
```
|
|
|
|
#### Step 6: Final response to frontend
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"message": "Repository analysis completed successfully",
|
|
"analysis_id": "repo_analysis_550e8400_20241216_143022",
|
|
"report_path": "/app/reports/repo_analysis_550e8400_20241216_143022_analysis.pdf",
|
|
"stats": {
|
|
"repository_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
"total_files": 85,
|
|
"total_lines": 15420,
|
|
"languages": ["typescript", "javascript"],
|
|
"code_quality_score": 7.8,
|
|
"high_quality_files": 45,
|
|
"medium_quality_files": 30,
|
|
"low_quality_files": 10,
|
|
"total_issues": 23
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
### Error Types and Responses
|
|
|
|
#### 1. Service Unavailable (502)
|
|
|
|
```json
|
|
{
|
|
"error": "AI Analysis service unavailable",
|
|
"message": "ECONNREFUSED",
|
|
"service": "ai-analysis"
|
|
}
|
|
```
|
|
|
|
**Causes**:
|
|
- AI Analysis service is down
|
|
- Network issues
|
|
- Service not responding
|
|
|
|
#### 2. Gateway Timeout (504)
|
|
|
|
```json
|
|
{
|
|
"error": "Gateway timeout",
|
|
"service": "ai-analysis"
|
|
}
|
|
```
|
|
|
|
**Causes**:
|
|
- Analysis taking longer than 4 minutes
|
|
- Service overloaded
|
|
- Large repository
|
|
|
|
#### 3. Authentication Required (401)
|
|
|
|
```json
|
|
{
|
|
"success": false,
|
|
"message": "GitHub authentication required for private repository",
|
|
"requires_auth": true,
|
|
"auth_url": "https://backend.codenuk.com/api/github/auth/github?..."
|
|
}
|
|
```
|
|
|
|
**Causes**:
|
|
- Private repository without OAuth token
|
|
- Expired OAuth token
|
|
|
|
#### 4. Repository Not Found (404)
|
|
|
|
```json
|
|
{
|
|
"success": false,
|
|
"message": "Repository not found"
|
|
}
|
|
```
|
|
|
|
**Causes**:
|
|
- Invalid repository ID
|
|
- Repository deleted
|
|
- User doesn't have access
|
|
|
|
#### 5. Rate Limit Exceeded (429)
|
|
|
|
```json
|
|
{
|
|
"success": false,
|
|
"message": "Claude API rate limit exceeded",
|
|
"retry_after": 60
|
|
}
|
|
```
|
|
|
|
**Causes**:
|
|
- Too many concurrent analyses
|
|
- Claude API limits hit
|
|
|
|
### Error Handling in Frontend
|
|
|
|
```typescript
|
|
try {
|
|
const result = await startAnalysis(repositoryId, userId)
|
|
// Handle success
|
|
} catch (error) {
|
|
if (error.status === 401) {
|
|
// Redirect to OAuth
|
|
window.location.href = error.auth_url
|
|
} else if (error.status === 504) {
|
|
// Show timeout message
|
|
setError('Analysis taking longer than expected. Please try again.')
|
|
} else if (error.status === 502) {
|
|
// Show service unavailable message
|
|
setError('AI Analysis service is currently unavailable.')
|
|
} else {
|
|
// Generic error
|
|
setError(error.message || 'An error occurred')
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Deployment Configuration
|
|
|
|
### Environment Variables
|
|
|
|
#### AI Analysis Service (.env)
|
|
|
|
```bash
|
|
# Service Configuration
|
|
PORT=8022
|
|
HOST=0.0.0.0
|
|
|
|
# Anthropic API
|
|
ANTHROPIC_API_KEY=sk-ant-api03-...
|
|
|
|
# Git Integration Service
|
|
GIT_INTEGRATION_SERVICE_URL=http://git-integration:8012
|
|
|
|
# Redis Cache
|
|
REDIS_HOST=redis
|
|
REDIS_PORT=6379
|
|
REDIS_PASSWORD=
|
|
|
|
# Rate Limiting
|
|
CLAUDE_REQUESTS_PER_MINUTE=90
|
|
|
|
# Analysis Configuration
|
|
MAX_FILES_DEFAULT=100
|
|
CACHE_TTL_SECONDS=86400
|
|
CONTENT_MAX_TOKENS=8000
|
|
```
|
|
|
|
#### API Gateway (.env)
|
|
|
|
```bash
|
|
# Service URLs
|
|
PORT=8000
|
|
NODE_ENV=production
|
|
|
|
AI_ANALYSIS_URL=http://localhost:8022
|
|
GIT_INTEGRATION_URL=http://localhost:8012
|
|
USER_AUTH_URL=http://localhost:8011
|
|
|
|
# CORS
|
|
CORS_ORIGIN=*
|
|
|
|
# Timeouts
|
|
PROXY_TIMEOUT=240000
|
|
```
|
|
|
|
#### Git Integration Service (.env)
|
|
|
|
```bash
|
|
# Service Configuration
|
|
PORT=8012
|
|
|
|
# Database
|
|
POSTGRES_HOST=postgres
|
|
POSTGRES_PORT=5432
|
|
POSTGRES_DB=git_integration
|
|
POSTGRES_USER=postgres
|
|
POSTGRES_PASSWORD=postgres123
|
|
|
|
# GitHub OAuth
|
|
GITHUB_CLIENT_ID=your_github_client_id
|
|
GITHUB_CLIENT_SECRET=your_github_client_secret
|
|
GITHUB_CALLBACK_URL=https://backend.codenuk.com/api/github/auth/github/callback
|
|
|
|
# Repository Storage
|
|
GIT_REPOS_PATH=/app/git-repos
|
|
|
|
# Public Base URL
|
|
PUBLIC_BASE_URL=https://backend.codenuk.com
|
|
API_GATEWAY_PUBLIC_URL=https://backend.codenuk.com
|
|
```
|
|
|
|
#### Frontend (.env.local)
|
|
|
|
```bash
|
|
NEXT_PUBLIC_API_GATEWAY_URL=http://localhost:8000
|
|
NEXT_PUBLIC_BACKEND_URL=http://localhost:8000
|
|
```
|
|
|
|
### Docker Compose Configuration
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
api-gateway:
|
|
build: ./services/api-gateway
|
|
ports:
|
|
- "8000:8000"
|
|
environment:
|
|
- AI_ANALYSIS_URL=http://ai-analysis:8022
|
|
- GIT_INTEGRATION_URL=http://git-integration:8012
|
|
depends_on:
|
|
- ai-analysis
|
|
- git-integration
|
|
networks:
|
|
- backend-network
|
|
|
|
ai-analysis:
|
|
build: ./services/ai-analysis-service
|
|
ports:
|
|
- "8022:8022"
|
|
environment:
|
|
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
|
|
- GIT_INTEGRATION_SERVICE_URL=http://git-integration:8012
|
|
- REDIS_HOST=redis
|
|
volumes:
|
|
- ./reports:/app/reports
|
|
depends_on:
|
|
- redis
|
|
- git-integration
|
|
networks:
|
|
- backend-network
|
|
|
|
git-integration:
|
|
build: ./services/git-integration
|
|
ports:
|
|
- "8012:8012"
|
|
environment:
|
|
- POSTGRES_HOST=postgres
|
|
- GITHUB_CLIENT_ID=${GITHUB_CLIENT_ID}
|
|
- GITHUB_CLIENT_SECRET=${GITHUB_CLIENT_SECRET}
|
|
- PUBLIC_BASE_URL=https://backend.codenuk.com
|
|
volumes:
|
|
- ./git-repos:/app/git-repos
|
|
depends_on:
|
|
- postgres
|
|
networks:
|
|
- backend-network
|
|
|
|
postgres:
|
|
image: postgres:15
|
|
environment:
|
|
- POSTGRES_DB=git_integration
|
|
- POSTGRES_USER=postgres
|
|
- POSTGRES_PASSWORD=postgres123
|
|
volumes:
|
|
- postgres-data:/var/lib/postgresql/data
|
|
networks:
|
|
- backend-network
|
|
|
|
redis:
|
|
image: redis:7-alpine
|
|
ports:
|
|
- "6379:6379"
|
|
volumes:
|
|
- redis-data:/data
|
|
networks:
|
|
- backend-network
|
|
|
|
frontend:
|
|
build: ./frontend
|
|
ports:
|
|
- "3000:3000"
|
|
environment:
|
|
- NEXT_PUBLIC_API_GATEWAY_URL=http://localhost:8000
|
|
depends_on:
|
|
- api-gateway
|
|
networks:
|
|
- backend-network
|
|
|
|
volumes:
|
|
postgres-data:
|
|
redis-data:
|
|
|
|
networks:
|
|
backend-network:
|
|
driver: bridge
|
|
```
|
|
|
|
### Network Flow in Docker
|
|
|
|
```
|
|
Internet
|
|
│
|
|
▼
|
|
[Nginx/Traefik Reverse Proxy]
|
|
│
|
|
├─> frontend:3000
|
|
│
|
|
└─> api-gateway:8000
|
|
│
|
|
├─> ai-analysis:8022 ──┐
|
|
│ ├─> redis:6379
|
|
└─> git-integration:8012 ─┘
|
|
│
|
|
└─> postgres:5432
|
|
```
|
|
|
|
---
|
|
|
|
## Performance Considerations
|
|
|
|
### 1. Rate Limiting
|
|
|
|
- Claude API: 90 requests/minute (sliding window)
|
|
- Wait times automatically calculated
|
|
- Queued requests processed sequentially
|
|
|
|
### 2. Caching
|
|
|
|
- Redis cache for analyzed files (24-hour TTL)
|
|
- SHA-256 hash-based cache keys
|
|
- Cache hit rate typically 40-60%
|
|
|
|
### 3. Content Optimization
|
|
|
|
- Files > 32KB tokens are truncated
|
|
- Preserves important code structures (imports, functions, classes)
|
|
- Reduces API costs by 70-80%
|
|
|
|
### 4. Timeouts
|
|
|
|
- API Gateway → AI Analysis: 240 seconds
|
|
- AI Analysis → Git Integration: 30 seconds
|
|
- AI Analysis → Claude API: 120 seconds per request
|
|
|
|
### 5. Concurrent Analysis
|
|
|
|
- Maximum 5 concurrent repository analyses
|
|
- File-level parallelization disabled (sequential processing)
|
|
- Prevents rate limit violations
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Issue 1: Service Cannot Connect to Git Integration
|
|
|
|
**Symptoms**:
|
|
- Error: "Failed to get repository info"
|
|
- 502 Bad Gateway errors
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Check if git-integration service is running
|
|
curl http://localhost:8012/health
|
|
|
|
# Check Docker network connectivity
|
|
docker network inspect backend-network
|
|
|
|
# Verify environment variable
|
|
docker exec ai-analysis env | grep GIT_INTEGRATION_SERVICE_URL
|
|
```
|
|
|
|
### Issue 2: Claude API Rate Limit Exceeded
|
|
|
|
**Symptoms**:
|
|
- Error: "Rate limit exceeded"
|
|
- Analysis fails midway
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Reduce max_files in request
|
|
{
|
|
"max_files": 50 // Instead of 100
|
|
}
|
|
|
|
# Check rate limiter configuration
|
|
CLAUDE_REQUESTS_PER_MINUTE=50 // Reduce from 90
|
|
```
|
|
|
|
### Issue 3: Redis Connection Failed
|
|
|
|
**Symptoms**:
|
|
- Warning: "Redis connection failed"
|
|
- No caching working
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Check Redis service
|
|
docker exec redis redis-cli ping
|
|
|
|
# Verify connection settings
|
|
REDIS_HOST=redis
|
|
REDIS_PORT=6379
|
|
```
|
|
|
|
### Issue 4: Authentication Errors
|
|
|
|
**Symptoms**:
|
|
- 401 Unauthorized
|
|
- "GitHub authentication required"
|
|
|
|
**Solution**:
|
|
- User needs to authenticate via OAuth
|
|
- Frontend should redirect to auth_url provided in error response
|
|
- Check GitHub OAuth credentials in git-integration service
|
|
|
|
---
|
|
|
|
## Monitoring and Logging
|
|
|
|
### Log Locations
|
|
|
|
```bash
|
|
# AI Analysis Service
|
|
docker logs ai-analysis -f
|
|
|
|
# API Gateway
|
|
docker logs api-gateway -f
|
|
|
|
# Git Integration
|
|
docker logs git-integration -f
|
|
```
|
|
|
|
### Key Log Patterns
|
|
|
|
#### Successful Analysis
|
|
```
|
|
✅ [AI ANALYSIS] Repository analysis completed successfully
|
|
📊 Stats: 85 files, 15420 lines, score: 7.8/10
|
|
```
|
|
|
|
#### Rate Limiting
|
|
```
|
|
⚠️ [RATE LIMITER] Waiting 5.2 seconds before next request
|
|
```
|
|
|
|
#### Cache Hit
|
|
```
|
|
📦 [CACHE] Using cached analysis for file: src/index.ts
|
|
```
|
|
|
|
#### Git Integration Communication
|
|
```
|
|
🔍 [GIT CLIENT] Getting repository info for: uuid-123
|
|
✅ [GIT CLIENT] Repository info retrieved: johndoe/my-repo
|
|
```
|
|
|
|
---
|
|
|
|
## Security Considerations
|
|
|
|
### 1. Authentication Flow
|
|
|
|
- Frontend passes user_id in requests
|
|
- API Gateway adds x-user-id header
|
|
- Services validate user ownership
|
|
|
|
### 2. Private Repository Access
|
|
|
|
- OAuth tokens stored in PostgreSQL
|
|
- Tokens encrypted at rest
|
|
- Token validation before file access
|
|
|
|
### 3. API Keys
|
|
|
|
- Claude API key in environment variable only
|
|
- Never exposed to frontend
|
|
- Rotated regularly
|
|
|
|
### 4. CORS Configuration
|
|
|
|
- Specific origins in production
|
|
- Credentials allowed for authenticated requests
|
|
|
|
---
|
|
|
|
## Additional Resources
|
|
|
|
- [AI Analysis Service Implementation](./ai-analyze.py)
|
|
- [API Gateway Configuration](../api-gateway/src/server.js)
|
|
- [Git Integration Routes](../git-integration/src/routes/github-integration.routes.js)
|
|
- [Frontend Hook](../../fronend/codenuk_frontend_mine/src/hooks/useAIAnalysis.ts)
|
|
|
|
---
|
|
|
|
## Support and Contact
|
|
|
|
For issues or questions:
|
|
- Check service logs first
|
|
- Verify environment variables
|
|
- Test service endpoints individually
|
|
- Check Docker network connectivity
|
|
|
|
---
|
|
|
|
**Last Updated**: December 2024
|
|
**Version**: 1.0.0
|
|
|