dld_backend/POSTMAN_COLLECTION_GUIDE.md

# Dubai DLD Analytics API - Complete Postman Collection Guide

## 📋 Overview

This comprehensive Postman collection provides complete testing coverage for the Dubai DLD Analytics API. It includes natural language queries, predefined endpoints, context-aware follow-up queries, error handling, and performance testing scenarios.

## 🚀 Quick Start

### 1. Import the Collection
- Download `Dubai_DLD_Analytics_API_Complete.postman_collection.json`
- Open Postman and click "Import"
- Select the downloaded JSON file
- The collection will be imported with all folders and requests

### 2. Set Up Environment Variables
The collection uses these variables:
- `baseUrl`: `http://localhost:3000` (default)
- `sessionId`: Auto-generated random session ID

### 3. Start the API Server
```bash
cd /home/tech4biz/Desktop/dubai_dld
npm start
```

### 4. Run Tests
- Click on any folder to run all tests in that category
- Use "Run Collection" to execute all tests
- Individual requests can be run by clicking on them

## 📁 Collection Structure

### 🏥 Health & System
**Purpose**: Verify API health and system status
- **Health Check**: Basic API health verification
- **Database Info**: Check database tables and record counts
- **Available Queries**: List all available endpoints and examples

### 🔍 Natural Language Queries
**Purpose**: Test the core NLP functionality with various query types
- **Basic Rental Trend**: Standard rental price trend queries
- **Top Areas**: Find areas with most rental transactions
- **Project Summary**: General and specific project summaries
- **Fast Moving Projects**: Identify high-activity projects
- **Off-Plan Uptick**: Find areas with off-plan project growth
- **Commercial/Residential Leasing**: Test dynamic number extraction (Top 3, Top six, etc.)
- **3BHK Apartment Analysis**: Complex multi-area visualization queries

### 🔄 Context-Aware Follow-up Queries
**Purpose**: Test session-based context management
- **Initial Query**: Establish context with Business Bay trend
- **Follow-up Queries**: Test grouping changes (weekly/monthly) and filters (apartments/villas only)

### 📊 Predefined Query Endpoints
**Purpose**: Test direct API endpoints without NLP processing
- **Rental Trends**: Direct area-specific endpoints
- **Top Areas**: Predefined rental transaction queries
- **Project Summaries**: Direct project data endpoints
- **Leasing Queries**: Direct commercial/residential leasing endpoints

### 🧪 Advanced Test Scenarios
**Purpose**: Test edge cases and complex scenarios
- **Multi-Parameter Queries**: Complex queries with multiple filters
- **Edge Cases**: Invalid areas, long time periods, large numbers
- **Invalid Formats**: Test error handling and fallbacks

### ❌ Error Handling Tests
**Purpose**: Verify proper error handling and validation
- **Missing Parameters**: Test required field validation
- **Invalid JSON**: Test malformed request handling
- **Non-existent Endpoints**: Test 404 error responses
- **Wrong HTTP Methods**: Test method validation

### 🚀 Performance Tests
**Purpose**: Test API performance and concurrency
- **Concurrent Requests**: Same and different session IDs
- **Large Responses**: Queries returning substantial data
- **Response Time**: Verify performance benchmarks

### 📋 Sample Data Validation
**Purpose**: Verify data availability and quality
- **Business Bay Data**: Validate core area data
- **Commercial/Residential Data**: Verify leasing data
- **Project Data**: Check project information availability

## 🎯 Key Features Tested

### 1. Natural Language Processing
- **Intent Classification**: Trend, compare, summary, filter intents
- **Entity Extraction**: Areas, time periods, property types, room types
- **Number Extraction**: Both digits (3, 5) and words (three, six, ten)
- **Project Name Extraction**: Specific project identification

### 2. Dynamic Query Generation
- **Template Selection**: Automatic template matching
- **Parameter Building**: Dynamic SQL parameter construction
- **Context Awareness**: Session-based query refinement

### 3. Data Visualization
- **Chart Generation**: Line charts, bar charts, pie charts
- **Card Creation**: Metric cards with descriptions
- **Multi-Area Charts**: Comparative area analysis
- **Time Series**: Monthly and weekly grouping

### 4. Context Management
- **Session Persistence**: Maintain query context across requests
- **Follow-up Queries**: Refine existing queries with new parameters
- **Grouping Changes**: Switch between monthly/weekly grouping
- **Filter Application**: Add property type filters

## 📊 Expected Response Structure

### Successful Response
```json
{
  "success": true,
  "data": {
    "text": "Descriptive text about the query results",
    "visualizations": [
      {
        "type": "line|bar|pie",
        "title": "Chart title",
        "data": {
          "labels": ["Label1", "Label2"],
          "datasets": [...]
        },
        "options": {...}
      }
    ],
    "cards": [
      {
        "title": "Card title",
        "value": "Card value",
        "subtitle": "Card subtitle",
        "description": "Card description",
        "icon": "icon-name"
      }
    ],
    "sql_queries": ["SELECT ..."],
    "metadata": {
      "query": "Original query",
      "intent": "trend|compare|summary|filter",
      "time_period": {...},
      "areas": [...],
      "result_count": 5,
      "data_source": "rents|transactions",
      "topCount": 3,
      "fallback_reason": null
    }
  }
}
```

### Error Response
```json
{
  "success": false,
  "message": "Error description",
  "error": "Detailed error message (development only)"
}
```

## 🧪 Test Scenarios

### 1. Basic Functionality
- ✅ Health check passes
- ✅ Database connection works
- ✅ Basic queries return data
- ✅ Response structure is valid

### 2. Number Extraction
- ✅ "Top 3 areas" → `topCount: 3`
- ✅ "Top six areas" → `topCount: 6`
- ✅ "Top ten areas" → `topCount: 10`
- ✅ Invalid numbers fallback to default (5)

### 3. Context Management
- ✅ Initial query establishes context
- ✅ Follow-up queries use existing context
- ✅ Grouping changes work (weekly/monthly)
- ✅ Filters apply correctly (apartments/villas)

### 4. Error Handling
- ✅ Missing parameters return validation errors
- ✅ Invalid JSON returns parse errors
- ✅ Non-existent endpoints return 404
- ✅ Wrong HTTP methods return method errors

### 5. Performance
- ✅ Response time < 5000ms
- ✅ Concurrent requests handled properly
- ✅ Large responses processed efficiently

## 🔧 Customization

### Adding New Tests
1. **Create New Request**: Add to appropriate folder
2. **Set Variables**: Use `{{baseUrl}}` and `{{sessionId}}`
3. **Add Tests**: Include in the test script section
4. **Document**: Add description and expected behavior

### Modifying Variables
- **baseUrl**: Change to point to different environment
- **sessionId**: Modify for specific session testing
- **Custom Variables**: Add project-specific variables

### Environment Setup
Create different environments for:
- **Development**: `http://localhost:3000`
- **Staging**: `http://staging-api.example.com`
- **Production**: `http://api.example.com`

## 📈 Monitoring & Analytics

### Response Time Monitoring
All requests include response time tests:
```javascript
pm.test('Response time is less than 5000ms', function () {
    pm.expect(pm.response.responseTime).to.be.below(5000);
});
```

### Data Quality Validation
Key validation tests:
- Response structure validation
- Data completeness checks
- Error handling verification
- Performance benchmarks

### Success Rate Tracking
Monitor success rates across:
- Different query types
- Various parameter combinations
- Error scenarios
- Performance metrics

## 🚨 Troubleshooting

### Common Issues

1. **Connection Refused**
   - Verify API server is running (`npm start`)
   - Check port 3000 is available
   - Verify `baseUrl` variable

2. **Empty Responses**
   - Check database connection
   - Verify sample data is loaded
   - Review query parameters

3. **Context Errors**
   - Ensure session ID is consistent
   - Check follow-up query sequence
   - Verify context manager state

4. **Performance Issues**
   - Monitor response times
   - Check database query performance
   - Review server resources

### Debug Steps
1. **Check Health**: Run health check first
2. **Verify Database**: Check database info endpoint
3. **Test Basic Query**: Try simple rental trend query
4. **Review Logs**: Check server console output
5. **Validate Data**: Ensure sample data is loaded

## 📚 Additional Resources

- **API Documentation**: See `README.md` for detailed API docs
- **Database Schema**: Check `2.sql` for table structures
- **Sample Data**: Review JSON files for data examples
- **Implementation Guide**: See `IMPLEMENTATION_SUMMARY.md`

## 🎉 Success Criteria

A successful test run should show:
- ✅ All health checks pass
- ✅ Database connectivity confirmed
- ✅ Natural language queries work
- ✅ Context management functions
- ✅ Error handling works properly
- ✅ Performance meets benchmarks
- ✅ Data quality is maintained

---

**Happy Testing!** 🚀

This collection provides comprehensive coverage of the Dubai DLD Analytics API, ensuring all features work correctly and performance meets expectations.