Re_Backend/README.md

# Royal Enfield Workflow Management System - Backend

A comprehensive backend API for the Royal Enfield Workflow Management System built with Node.js, TypeScript, Express.js, and PostgreSQL.

## Features

- **Frontend SSO Integration**: Handles user authentication via frontend SSO
- **JWT Authentication**: Secure token-based authentication with refresh tokens
- **User Management**: Create and update users based on SSO data
- **Workflow Management**: Complete workflow request lifecycle
- **Approval System**: Multi-level approval workflow
- **Document Management**: File upload and management
- **Notification System**: Real-time notifications
- **TAT Tracking**: Turnaround time monitoring
- **Audit Logging**: Comprehensive activity tracking
- **RESTful API**: Well-structured API endpoints

## Technology Stack

- **Runtime**: Node.js 22 LTS
- **Language**: TypeScript 5.7
- **Framework**: Express.js 4.21
- **Database**: PostgreSQL 16
- **ORM**: Sequelize 6.37
- **Authentication**: JWT + Frontend SSO
- **Validation**: Zod
- **Logging**: Winston
- **Testing**: Jest + Supertest
- **Process Manager**: PM2

## Quick Start

### Prerequisites

- Node.js 22.x LTS
- PostgreSQL 16.x
- npm 10.x or higher

### Installation

1. **Clone the repository**
   ```bash
   git clone <repository-url>
   cd re-workflow-backend
   ```

2. **Install dependencies**
   ```bash
   npm install
   ```

3. **Setup environment**

   **Option A: Automated Setup (Recommended - Unix/Linux/Mac)**
   ```bash
   chmod +x setup-env.sh
   ./setup-env.sh
   # Follow the interactive prompts
   # The script will generate secure secrets automatically
   ```

   **Option B: Manual Setup (Windows or Custom Configuration)**
   ```bash
   cp env.example .env
   # Edit .env with your configuration
   # Generate JWT secrets manually:
   # openssl rand -base64 32 | tr -d "=+/" | cut -c1-32
   ```

4. **Setup database**
   ```bash
   # Create database
   createdb re_workflow_db
   
   # Run schema
   psql -U postgres -d re_workflow_db -f database/schema/schema.sql
   ```

5. **Start development server**
   ```bash
   npm run dev
   ```

The API will be available at `http://localhost:5000`

### Redis (for TAT and Pause-Resume jobs)

The backend uses **Redis** for TAT (turnaround time) alerts and Pause-Resume workflow jobs. The app runs without Redis, but those features need Redis on `localhost:6379`.

**Option 1 – Docker (easiest if you have Docker)**

```bash
# Start Redis in the background (port 6379)
npm run redis:start

# Stop when done
npm run redis:stop
```

**Option 2 – Windows (Memurai or WSL)**

- **Memurai** (Redis-compatible, native Windows): Download from [memurai.com](https://www.memurai.com/) and install. Default port 6379. You can install as a Windows service.
- **WSL2**: Install Ubuntu from Microsoft Store, then:
  ```bash
  sudo apt update && sudo apt install redis-server -y
  redis-server --daemonize yes
  ```

**Option 3 – macOS / Linux**

```bash
# macOS (Homebrew)
brew install redis && brew services start redis

# Ubuntu/Debian
sudo apt install redis-server -y && sudo systemctl start redis-server
```

**Verify:** `redis-cli ping` should return `PONG`. Then restart the backend so it connects to Redis.

### Docker Setup

```bash
# Setup environment (use automated script or manual)
# Option 1: Automated (if running on Unix/Linux/Mac host)
chmod +x setup-env.sh
./setup-env.sh

# Option 2: Manual
cp env.example .env
# Edit .env with your configuration

# Start services
docker-compose up --build -d

# Check logs
docker-compose logs -f
```

**Note:** Ensure your `.env` file is properly configured before starting Docker containers.

## API Endpoints

### Base URL
All API endpoints are prefixed with `/api/v1` unless otherwise specified.

### Authentication & Authorization
- `GET /health` - API health status (no auth required)
- `GET /api/v1/config` - Get public system configuration (no auth required)

### Authentication Endpoints (`/api/v1/auth`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `POST` | `/token-exchange` | Token exchange for localhost development | No |
| `POST` | `/sso-callback` | SSO callback from frontend | No |
| `POST` | `/refresh` | Refresh access token | No |
| `GET` | `/me` | Get current user profile | Yes |
| `GET` | `/validate` | Validate authentication token | Yes |
| `POST` | `/logout` | Logout user and clear cookies | Optional |

### Workflow Management (`/api/v1/workflows`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/` | List all workflows | Yes |
| `GET` | `/my` | List workflows created by current user | Yes |
| `GET` | `/open-for-me` | List workflows open for current user | Yes |
| `GET` | `/closed-by-me` | List workflows closed by current user | Yes |
| `POST` | `/` | Create new workflow | Yes |
| `POST` | `/multipart` | Create workflow with file uploads | Yes |
| `GET` | `/:id` | Get workflow by ID | Yes |
| `GET` | `/:id/details` | Get detailed workflow information | Yes |
| `PUT` | `/:id` | Update workflow | Yes |
| `PUT` | `/:id/multipart` | Update workflow with file uploads | Yes |
| `PATCH` | `/:id/submit` | Submit workflow for approval | Yes |

### Approval Management (`/api/v1/workflows`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/:id/approvals` | Get all approval levels for workflow | Yes |
| `GET` | `/:id/approvals/current` | Get current approval level | Yes |
| `PATCH` | `/:id/approvals/:levelId/approve` | Approve at specific level | Yes (Approver) |
| `PATCH` | `/:id/approvals/:levelId/reject` | Reject at specific level | Yes (Approver) |
| `POST` | `/:id/approvals/:levelId/skip` | Skip approver at level | Yes (Initiator/Approver) |
| `POST` | `/:id/approvers/at-level` | Add approver at specific level | Yes (Initiator/Approver) |

### Participants Management (`/api/v1/workflows`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `POST` | `/:id/participants/approver` | Add approver to workflow | Yes |
| `POST` | `/:id/participants/spectator` | Add spectator to workflow | Yes |

### Work Notes (`/api/v1/workflows`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/:id/work-notes` | Get all work notes for workflow | Yes |
| `POST` | `/:id/work-notes` | Create work note with attachments | Yes |
| `GET` | `/work-notes/attachments/:attachmentId/preview` | Preview work note attachment | Yes |
| `GET` | `/work-notes/attachments/:attachmentId/download` | Download work note attachment | Yes |

### Document Management (`/api/v1/workflows` & `/api/v1/documents`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/workflows/documents/:documentId/preview` | Preview workflow document | Yes |
| `GET` | `/workflows/documents/:documentId/download` | Download workflow document | Yes |
| `POST` | `/documents` | Upload document (multipart/form-data) | Yes |

### Activities & Notifications (`/api/v1/workflows`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/:id/activity` | Get activity log for workflow | Yes |
| `POST` | `/notifications/subscribe` | Subscribe to push notifications | Yes |
| `POST` | `/notifications/test` | Send test push notification | Yes |

### User Management (`/api/v1/users`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/search?q=<email or name>` | Search users by email or name | Yes |
| `POST` | `/ensure` | Ensure user exists (create if not) | Yes |

### Dashboard & Analytics (`/api/v1/dashboard`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/kpis` | Get KPI summary (all KPI cards) | Yes |
| `GET` | `/stats/requests` | Get detailed request statistics | Yes |
| `GET` | `/stats/tat-efficiency` | Get TAT efficiency metrics | Yes |
| `GET` | `/stats/approver-load` | Get approver load statistics | Yes |
| `GET` | `/stats/engagement` | Get engagement & quality metrics | Yes |
| `GET` | `/stats/ai-insights` | Get AI & closure insights | Yes |
| `GET` | `/stats/ai-remark-utilization` | Get AI remark utilization with trends | Yes |
| `GET` | `/stats/approver-performance` | Get approver performance metrics | Yes |
| `GET` | `/stats/by-department` | Get department-wise summary | Yes |
| `GET` | `/stats/priority-distribution` | Get priority distribution | Yes |
| `GET` | `/activity/recent` | Get recent activity feed | Yes |
| `GET` | `/requests/critical` | Get high priority/critical requests | Yes |
| `GET` | `/deadlines/upcoming` | Get upcoming deadlines | Yes |
| `GET` | `/reports/lifecycle` | Get Request Lifecycle Report | Yes |
| `GET` | `/reports/activity-log` | Get enhanced User Activity Log Report | Yes |
| `GET` | `/reports/workflow-aging` | Get Workflow Aging Report | Yes |

### Notifications (`/api/v1/notifications`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/?page=&limit=&unreadOnly=` | Get user's notifications (paginated) | Yes |
| `GET` | `/unread-count` | Get unread notification count | Yes |
| `PATCH` | `/:notificationId/read` | Mark notification as read | Yes |
| `POST` | `/mark-all-read` | Mark all notifications as read | Yes |
| `DELETE` | `/:notificationId` | Delete notification | Yes |

### TAT (Turnaround Time) Monitoring (`/api/v1/tat`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/alerts/request/:requestId` | Get all TAT alerts for a request | Yes |
| `GET` | `/alerts/level/:levelId` | Get TAT alerts for a specific level | Yes |
| `GET` | `/compliance/summary?startDate=&endDate=` | Get TAT compliance summary | Yes |
| `GET` | `/breaches` | Get TAT breach report | Yes |
| `GET` | `/performance/:approverId` | Get TAT performance for approver | Yes |

### AI & Conclusion Management (`/api/v1/conclusions` & `/api/v1/ai`)

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `POST` | `/conclusions/:requestId/generate` | Generate AI-powered conclusion remark | Yes (Initiator) |
| `PUT` | `/conclusions/:requestId` | Update conclusion remark | Yes (Initiator) |
| `POST` | `/conclusions/:requestId/finalize` | Finalize conclusion and close request | Yes (Initiator) |
| `GET` | `/conclusions/:requestId` | Get conclusion for a request | Yes |
| `GET` | `/ai/status` | Get AI service status | Yes (Admin) |
| `POST` | `/ai/reinitialize` | Reinitialize AI service | Yes (Admin) |

### Admin Management (`/api/v1/admin`)

**All admin routes require authentication and admin role.**

#### Holiday Management

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/holidays?year=` | Get all holidays (optional year filter) |
| `GET` | `/holidays/calendar/:year` | Get holiday calendar for specific year |
| `POST` | `/holidays` | Create new holiday |
| `PUT` | `/holidays/:holidayId` | Update holiday |
| `DELETE` | `/holidays/:holidayId` | Delete (deactivate) holiday |
| `POST` | `/holidays/bulk-import` | Bulk import holidays from CSV/JSON |

#### Configuration Management

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/configurations?category=` | Get all admin configurations |
| `PUT` | `/configurations/:configKey` | Update configuration value |
| `POST` | `/configurations/:configKey/reset` | Reset configuration to default |

#### User Role Management (RBAC)

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/users/assign-role` | Assign role to user by email |
| `PUT` | `/users/:userId/role` | Update user's role |
| `GET` | `/users/by-role?role=` | Get users filtered by role |
| `GET` | `/users/role-statistics` | Get count of users in each role |

### Debug Endpoints (`/api/v1/debug`)

**Note:** Debug endpoints are for development/testing purposes.

| Method | Endpoint | Description | Auth Required |
|--------|----------|-------------|---------------|
| `GET` | `/tat-jobs/:requestId` | Check scheduled TAT jobs for request | No |
| `GET` | `/tat-jobs` | Check all queued TAT jobs | No |
| `POST` | `/tat-calculate` | Calculate TAT times (debug) | No |
| `GET` | `/queue-status` | Check queue and worker status | No |
| `POST` | `/trigger-test-tat` | Manually trigger test TAT job | No |

## Environment Variables

### Quick Setup Options

1. **Automated Setup (Recommended)**
   ```bash
   chmod +x setup-env.sh
   ./setup-env.sh
   ```
   The script will:
   - Guide you through all required configuration
   - Automatically generate secure JWT and session secrets
   - Provide VAPID key generation instructions for web push notifications
   - Create a properly formatted `.env` file

2. **Manual Setup**
   - Copy `env.example` to `.env`
   - Fill in all required values
   - Generate secrets using:
     ```bash
     openssl rand -base64 32 | tr -d "=+/" | cut -c1-32
     ```

### Required Environment Variables

#### Application Configuration
- `NODE_ENV` - Environment (development/production)
- `PORT` - Server port (default: 5000)
- `BASE_URL` - Backend deployed URL
- `FRONTEND_URL` - Frontend URL for CORS

#### Database Configuration
- `DB_HOST` - PostgreSQL host (default: localhost)
- `DB_PORT` - PostgreSQL port (default: 5432)
- `DB_NAME` - Database name (default: re_workflow_db)
- `DB_USER` - Database user
- `DB_PASSWORD` - Database password

#### Authentication & Security
- `JWT_SECRET` - JWT signing secret (min 32 characters, auto-generated by setup script)
- `JWT_EXPIRY` - JWT expiration time (default: 24h)
- `REFRESH_TOKEN_SECRET` - Refresh token secret (auto-generated by setup script)
- `REFRESH_TOKEN_EXPIRY` - Refresh token expiration (default: 7d)
- `SESSION_SECRET` - Session secret (min 32 characters, auto-generated by setup script)

#### SSO Configuration (Okta)
- `OKTA_DOMAIN` - Okta domain URL
- `OKTA_CLIENT_ID` - Okta application client ID
- `OKTA_CLIENT_SECRET` - Okta application client secret
- `OKTA_API_TOKEN` - Okta API token (optional, for user management)

#### Web Push Notifications (VAPID)
- `VAPID_PUBLIC_KEY` - VAPID public key for web push
- `VAPID_PRIVATE_KEY` - VAPID private key for web push
- `VAPID_CONTACT` - Contact email (format: mailto:admin@example.com)

   **To generate VAPID keys:**
   ```bash
   npx web-push generate-vapid-keys
   ```
   The setup script provides detailed instructions, or run:
   ```bash
   ./setup-env.sh
   # Select option 2 for VAPID key generation instructions
   ```

#### Redis Configuration (TAT Queue)
- `REDIS_URL` - Redis connection URL (default: redis://localhost:6379)

#### Optional Services

**Email Service (SMTP)**
- `SMTP_HOST` - SMTP server host
- `SMTP_PORT` - SMTP port (default: 587)
- `SMTP_SECURE` - Use TLS (default: false)
- `SMTP_USER` - SMTP username
- `SMTP_PASSWORD` - SMTP password
- `EMAIL_FROM` - Sender email address

**Cloud Storage (GCP)**
- `GCP_PROJECT_ID` - Google Cloud Project ID
- `GCP_BUCKET_NAME` - GCS bucket name
- `GCP_KEY_FILE` - Path to GCP service account key file

**AI Service (Claude)**
- `CLAUDE_MODEL` - Claude model name (default: claude-sonnet-4-20250514)

### Environment Variables Reference

See `env.example` for the complete list with default values and descriptions.

## Development

### Prerequisites
- Backend API server running
- PostgreSQL database configured and accessible
- Environment variables set up (use `./setup-env.sh` or manually)
- Redis server (optional, for TAT queue)

### Available Scripts

```bash
# Run in development mode
npm run dev

# Run tests
npm test

# Run linting
npm run lint

# Run type checking
npm run type-check

# Build for production
npm run build
```

### Setup Checklist

- [ ] Node.js 22.x LTS installed
- [ ] PostgreSQL 16.x installed and running
- [ ] Environment variables configured (`.env` file created)
- [ ] Database created: `createdb re_workflow_db` (or your DB_NAME)
- [ ] Database schema applied (if applicable)
- [ ] Redis installed and running (for TAT monitoring)
- [ ] VAPID keys generated (for web push notifications)
- [ ] Optional: SMTP configured (for email notifications)
- [ ] Optional: GCP credentials configured (for cloud storage)

## Project Structure

```
src/
├── app.ts                 # Express app configuration
├── server.ts              # Server entry point
├── config/                # Configuration files
├── controllers/           # Request handlers
├── services/              # Business logic
├── models/                # Sequelize models
├── routes/                # API routes
├── middlewares/           # Express middlewares
├── validators/            # Request validation schemas
├── utils/                 # Utility functions
└── types/                 # TypeScript type definitions
```

## Database Schema

The database schema includes all tables from the ERD:
- `users` - User information
- `workflow_requests` - Main workflow requests
- `approval_levels` - Approval hierarchy
- `participants` - Workflow participants
- `documents` - Document metadata
- `work_notes` - Communication within workflow
- `activities` - Activity log
- `notifications` - User notifications
- `tat_tracking` - TAT monitoring
- And more...

## Authentication Flow

1. Frontend handles SSO authentication (Okta/Auth0)
2. Frontend sends user data to `/api/v1/auth/sso-callback`
3. Backend creates/updates user record
4. Backend generates JWT access and refresh tokens
5. Frontend uses tokens for subsequent API calls
6. Tokens are automatically refreshed when expired

## Web Push Notifications

The backend supports web push notifications via VAPID (Voluntary Application Server Identification). 

### Setup Instructions

1. **Generate VAPID Keys:**
   ```bash
   npx web-push generate-vapid-keys
   ```

2. **Add to Backend `.env`:**
   ```
   VAPID_PUBLIC_KEY=<your-public-key>
   VAPID_PRIVATE_KEY=<your-private-key>
   VAPID_CONTACT=mailto:admin@{{APP_DOMAIN}}
   ```

3. **Add to Frontend `.env`:**
   ```
   VITE_PUBLIC_VAPID_KEY=<same-public-key-as-backend>
   ```

4. **Important:** The VAPID public key must be the same in both backend and frontend `.env` files.

The `setup-env.sh` script provides detailed VAPID key generation instructions (select option 2).

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests and linting
5. Submit a pull request

## License

This project is proprietary to Royal Enfield.

## Support

For support and questions, please contact the development team.