codenuk_backend_mine/services/multi-document-upload-service/TESTING_GUIDE.md

# Multi-Document Upload Service - Frontend Testing Guide

## Prerequisites

1. **Backend Services Running**:
   ```bash
   cd /home/tech4biz/Desktop/prakash/codenuk/backend_new1/codenuk_backend_mine
   docker-compose up -d
   ```

2. **Verify Services are Running**:
   - API Gateway: `http://localhost:8000/health`
   - Multi-Document Upload Service: `http://localhost:8024/health`
   - Neo4j: `http://localhost:7474` (Browser interface)
   - Frontend: `http://localhost:3001` (or your frontend port)

3. **Check Service Health**:
   ```bash
   # Check API Gateway
   curl http://localhost:8000/health

   # Check Multi-Document Upload Service directly
   curl http://localhost:8024/health

   # Check via API Gateway proxy
   curl http://localhost:8000/api/multi-docs/health
   ```

## Frontend Testing Steps

### Step 1: Navigate to Project Builder

1. Open your browser and go to: `http://localhost:3001` (or your frontend URL)
2. Log in if required
3. Click on **"Project Builder"** in the navigation

### Step 2: Go to Multi Docs Upload Step

1. In the Project Builder, you should see the workflow steps:
   - **Step 1**: Project Type
   - **Step 2**: Features
   - **Step 3**: Multi Docs Upload ← **This is the new step**
   - **Step 4**: Business Context
   - **Step 5**: Generate
   - **Step 6**: Architecture

2. Complete Steps 1 and 2 (Project Type and Features selection)
3. You will automatically be taken to **Step 3: Multi Docs Upload**

### Step 3: Upload Documents

1. **Click on the upload area** or **drag and drop files**
2. **Select multiple files** (you can mix different formats):
   - PDF files (`.pdf`)
   - Word documents (`.doc`, `.docx`)
   - PowerPoint (`.ppt`, `.pptx`)
   - Excel files (`.xls`, `.xlsx`)
   - JSON files (`.json`)
   - XML files (`.xml`)
   - Markdown files (`.md`)
   - Images (`.png`, `.jpg`, `.jpeg`) - will use OCR
   - Audio files (`.mp3`, `.wav`) - will be transcribed
   - Video files (`.mp4`, `.avi`) - will be transcribed

3. **View selected files**: You should see a list of all selected files with:
   - File icon
   - File name
   - Remove button for each file

4. **Click "Start Upload"** button

### Step 4: Monitor Upload Progress

After clicking "Start Upload", you should see:

1. **Upload Status**:
   - Button shows "Uploading..." with spinner
   - Progress bar appears
   - Stage messages appear:
     - "Job received"
     - "Saving files"
     - "Extracting document content"
     - "Calling Claude for causal relations"
     - "Writing to Neo4j knowledge graph"
     - "Completed"

2. **Progress Indicators**:
   - Progress percentage (0-100%)
   - Status message showing current stage
   - Processed files count vs total files count

3. **Polling**: The frontend automatically polls the job status every 4 seconds

### Step 5: Verify Results

Once the job is completed:

1. **Check Neo4j Graph**:
   - Open Neo4j Browser: `http://localhost:7474`
   - Login with:
     - Username: `neo4j`
     - Password: `password`
   - Run Cypher query to see the graph:
     ```cypher
     MATCH (n)-[r:CAUSES]->(m)
     RETURN n, r, m
     LIMIT 50
     ```

2. **Check Job Status via API**:
   ```bash
   # Replace {job_id} with the actual job ID from the frontend
   curl http://localhost:8000/api/multi-docs/jobs/{job_id}
   ```

3. **Get Graph Summary**:
   ```bash
   curl http://localhost:8000/api/multi-docs/jobs/{job_id}/graph
   ```

## Testing Different Scenarios

### Scenario 1: Single PDF File
- Upload one PDF file
- Verify it processes correctly
- Check Neo4j for causal relationships

### Scenario 2: Multiple Mixed Format Files
- Upload 3-5 files of different formats (PDF, DOCX, JSON, image)
- Verify all files are processed
- Check that progress updates correctly

### Scenario 3: Large Files
- Upload a large PDF (10+ MB)
- Verify it handles large files correctly
- Check processing time

### Scenario 4: Error Handling
- Try uploading an unsupported file type
- Verify error message appears
- Check that the error is displayed clearly

### Scenario 5: Skip Option
- Upload files
- Click "Skip" button before completion
- Verify you can proceed to the next step
- Job continues processing in the background

## Browser Developer Tools

### Check Network Requests

1. **Open Developer Tools** (F12)
2. **Go to Network tab**
3. **Filter by "multi-docs"**
4. **Monitor requests**:
   - `POST /api/multi-docs/jobs` - Upload files
   - `GET /api/multi-docs/jobs/{job_id}` - Poll job status
   - `GET /api/multi-docs/jobs/{job_id}/graph` - Get graph summary

### Check Console Logs

1. **Open Console tab**
2. **Look for**:
   - Upload progress logs
   - Job status updates
   - Any error messages

### Check Response Data

Verify the API responses:

```javascript
// Upload response should be:
{
  "job_id": "uuid-here",
  "stage": "received",
  "total_files": 3,
  "created_at": "2024-01-01T00:00:00Z"
}

// Status response should be:
{
  "job_id": "uuid-here",
  "stage": "extracting",
  "status_message": "Extracting document content",
  "total_files": 3,
  "processed_files": 1,
  "error": null,
  "created_at": "2024-01-01T00:00:00Z",
  "updated_at": "2024-01-01T00:01:00Z",
  "files": [...]
}
```

## Troubleshooting

### Issue: Upload fails with 502 Bad Gateway
**Solution**:
- Check if multi-document-upload-service is running:
  ```bash
  docker-compose ps multi-document-upload-service
  ```
- Check service logs:
  ```bash
  docker-compose logs multi-document-upload-service
  ```

### Issue: Upload fails with 413 Request Entity Too Large
**Solution**:
- Check file sizes (max 500MB total per job)
- Reduce number of files or file sizes
- Check API Gateway body size limits

### Issue: Status polling stops working
**Solution**:
- Check browser console for errors
- Verify job ID is correct
- Check if job completed or failed
- Check network tab for failed requests

### Issue: No causal relationships found
**Solution**:
- Check Claude API key is configured correctly
- Check service logs for Claude API errors
- Verify documents contain causal language
- Check Neo4j connection

### Issue: Frontend shows "Failed" status
**Solution**:
- Check the error message in the frontend
- Check backend service logs:
  ```bash
  docker-compose logs -f multi-document-upload-service
  ```
- Verify all dependencies are running (Neo4j, Redis, Postgres)

## Expected Behavior

### Successful Flow:
1. ✅ Files upload successfully
2. ✅ Job ID is returned
3. ✅ Status polling starts automatically
4. ✅ Progress updates every 4 seconds
5. ✅ Stage changes are displayed
6. ✅ Progress bar updates
7. ✅ Job completes successfully
8. ✅ Frontend automatically proceeds to next step
9. ✅ Neo4j contains causal relationships

### Error Flow:
1. ✅ Error message is displayed clearly
2. ✅ User can retry upload
3. ✅ User can skip and proceed
4. ✅ Error details are logged in console

## API Endpoints Reference

### Upload Files
```bash
POST /api/multi-docs/jobs
Content-Type: multipart/form-data

Form Data:
- files: File[] (multiple files)
- job_name: string (optional)
```

### Get Job Status
```bash
GET /api/multi-docs/jobs/{job_id}
```

### Get Graph Summary
```bash
GET /api/multi-docs/jobs/{job_id}/graph
```

### Health Check
```bash
GET /api/multi-docs/health
```

## Next Steps After Testing

1. **Verify Neo4j Graph**: Check that causal relationships are stored correctly
2. **Check Storage**: Verify files are stored in the persistent volume
3. **Monitor Performance**: Check processing times for different file types
4. **Test Error Scenarios**: Verify error handling works correctly
5. **Test Large Batches**: Upload 50+ files to test scalability

## Support

If you encounter issues:
1. Check service logs: `docker-compose logs multi-document-upload-service`
2. Check API Gateway logs: `docker-compose logs api-gateway`
3. Check Neo4j logs: `docker-compose logs neo4j`
4. Verify all environment variables are set correctly
5. Check network connectivity between services