codenuk_backend_mine/services/multi-document-upload-service/TESTING_GUIDE.md

Multi-Document Upload Service - Frontend Testing Guide

Prerequisites

1. Backend Services Running:

   cd /home/tech4biz/Desktop/prakash/codenuk/backend_new1/codenuk_backend_mine
   docker-compose up -d
   

2. Verify Services are Running:

   • API Gateway: http://localhost:8000/health
   • Multi-Document Upload Service: http://localhost:8024/health
   • Neo4j: http://localhost:7474 (Browser interface)
   • Frontend: http://localhost:3001 (or your frontend port)

3. Check Service Health:

   # Check API Gateway
   curl http://localhost:8000/health
   
   # Check Multi-Document Upload Service directly
   curl http://localhost:8024/health
   
   # Check via API Gateway proxy
   curl http://localhost:8000/api/multi-docs/health
   

Frontend Testing Steps

Step 1: Navigate to Project Builder

1. Open your browser and go to: http://localhost:3001 (or your frontend URL)
2. Log in if required
3. Click on "Project Builder" in the navigation

Step 2: Go to Multi Docs Upload Step

1. In the Project Builder, you should see the workflow steps:

   • Step 1: Project Type
   • Step 2: Features
   • Step 3: Multi Docs Upload ← This is the new step
   • Step 4: Business Context
   • Step 5: Generate
   • Step 6: Architecture

2. Complete Steps 1 and 2 (Project Type and Features selection)

3. You will automatically be taken to Step 3: Multi Docs Upload

Step 3: Upload Documents

1. Click on the upload area or drag and drop files

2. Select multiple files (you can mix different formats):

   • PDF files (.pdf)
   • Word documents (.doc, .docx)
   • PowerPoint (.ppt, .pptx)
   • Excel files (.xls, .xlsx)
   • JSON files (.json)
   • XML files (.xml)
   • Markdown files (.md)
   • Images (.png, .jpg, .jpeg) - will use OCR
   • Audio files (.mp3, .wav) - will be transcribed
   • Video files (.mp4, .avi) - will be transcribed

3. View selected files: You should see a list of all selected files with:

   • File icon
   • File name
   • Remove button for each file

4. Click "Start Upload" button

Step 4: Monitor Upload Progress

After clicking "Start Upload", you should see:

1. Upload Status:

   • Button shows "Uploading..." with spinner
   • Progress bar appears
   • Stage messages appear:
     • "Job received"
     • "Saving files"
     • "Extracting document content"
     • "Calling Claude for causal relations"
     • "Writing to Neo4j knowledge graph"
     • "Completed"

2. Progress Indicators:

   • Progress percentage (0-100%)
   • Status message showing current stage
   • Processed files count vs total files count

3. Polling: The frontend automatically polls the job status every 4 seconds

Step 5: Verify Results

Once the job is completed:

1. Check Neo4j Graph:

   • Open Neo4j Browser: http://localhost:7474
   • Login with:
     • Username: neo4j
     • Password: password
   • Run Cypher query to see the graph:

     MATCH (n)-[r:CAUSES]->(m)
     RETURN n, r, m
     LIMIT 50
     

2. Check Job Status via API:

   # Replace {job_id} with the actual job ID from the frontend
   curl http://localhost:8000/api/multi-docs/jobs/{job_id}
   

3. Get Graph Summary:

   curl http://localhost:8000/api/multi-docs/jobs/{job_id}/graph
   

Testing Different Scenarios

Scenario 1: Single PDF File

• Upload one PDF file
• Verify it processes correctly
• Check Neo4j for causal relationships

Scenario 2: Multiple Mixed Format Files

• Upload 3-5 files of different formats (PDF, DOCX, JSON, image)
• Verify all files are processed
• Check that progress updates correctly

Scenario 3: Large Files

• Upload a large PDF (10+ MB)
• Verify it handles large files correctly
• Check processing time

Scenario 4: Error Handling

• Try uploading an unsupported file type
• Verify error message appears
• Check that the error is displayed clearly

Scenario 5: Skip Option

• Upload files
• Click "Skip" button before completion
• Verify you can proceed to the next step
• Job continues processing in the background

Browser Developer Tools

Check Network Requests

1. Open Developer Tools (F12)
2. Go to Network tab
3. Filter by "multi-docs"
4. Monitor requests:
   • POST /api/multi-docs/jobs - Upload files
   • GET /api/multi-docs/jobs/{job_id} - Poll job status
   • GET /api/multi-docs/jobs/{job_id}/graph - Get graph summary

Check Console Logs

1. Open Console tab
2. Look for:
   • Upload progress logs
   • Job status updates
   • Any error messages

Check Response Data

Verify the API responses:

// Upload response should be:
{
  "job_id": "uuid-here",
  "stage": "received",
  "total_files": 3,
  "created_at": "2024-01-01T00:00:00Z"
}

// Status response should be:
{
  "job_id": "uuid-here",
  "stage": "extracting",
  "status_message": "Extracting document content",
  "total_files": 3,
  "processed_files": 1,
  "error": null,
  "created_at": "2024-01-01T00:00:00Z",
  "updated_at": "2024-01-01T00:01:00Z",
  "files": [...]
}

Troubleshooting

Issue: Upload fails with 502 Bad Gateway

Solution:

• Check if multi-document-upload-service is running:

  docker-compose ps multi-document-upload-service
  

• Check service logs:

  docker-compose logs multi-document-upload-service
  

Issue: Upload fails with 413 Request Entity Too Large

Solution:

• Check file sizes (max 500MB total per job)
• Reduce number of files or file sizes
• Check API Gateway body size limits

Issue: Status polling stops working

Solution:

• Check browser console for errors
• Verify job ID is correct
• Check if job completed or failed
• Check network tab for failed requests

Issue: No causal relationships found

Solution:

• Check Claude API key is configured correctly
• Check service logs for Claude API errors
• Verify documents contain causal language
• Check Neo4j connection

Issue: Frontend shows "Failed" status

Solution:

• Check the error message in the frontend
• Check backend service logs:

  docker-compose logs -f multi-document-upload-service
  

• Verify all dependencies are running (Neo4j, Redis, Postgres)

Expected Behavior

Successful Flow:

1. ✅ Files upload successfully
2. ✅ Job ID is returned
3. ✅ Status polling starts automatically
4. ✅ Progress updates every 4 seconds
5. ✅ Stage changes are displayed
6. ✅ Progress bar updates
7. ✅ Job completes successfully
8. ✅ Frontend automatically proceeds to next step
9. ✅ Neo4j contains causal relationships

Error Flow:

1. ✅ Error message is displayed clearly
2. ✅ User can retry upload
3. ✅ User can skip and proceed
4. ✅ Error details are logged in console

API Endpoints Reference

Upload Files

POST /api/multi-docs/jobs
Content-Type: multipart/form-data

Form Data:
- files: File[] (multiple files)
- job_name: string (optional)

Get Job Status

GET /api/multi-docs/jobs/{job_id}

Get Graph Summary

GET /api/multi-docs/jobs/{job_id}/graph

Health Check

GET /api/multi-docs/health

Next Steps After Testing

1. Verify Neo4j Graph: Check that causal relationships are stored correctly
2. Check Storage: Verify files are stored in the persistent volume
3. Monitor Performance: Check processing times for different file types
4. Test Error Scenarios: Verify error handling works correctly
5. Test Large Batches: Upload 50+ files to test scalability

Support

If you encounter issues:

1. Check service logs: docker-compose logs multi-document-upload-service
2. Check API Gateway logs: docker-compose logs api-gateway
3. Check Neo4j logs: docker-compose logs neo4j
4. Verify all environment variables are set correctly
5. Check network connectivity between services