213 lines
6.4 KiB
Markdown
213 lines
6.4 KiB
Markdown
# File Path Storage in Database - How It Works
|
|
|
|
This document explains how file paths and storage URLs are stored in the database for different storage scenarios (GCS vs Local Storage).
|
|
|
|
## Database Schema
|
|
|
|
### Documents Table
|
|
- **`file_path`** (VARCHAR(500), NOT NULL): Stores the relative path or GCS path
|
|
- **`storage_url`** (VARCHAR(500), NULLABLE): Stores the full URL for accessing the file
|
|
|
|
### Work Note Attachments Table
|
|
- **`file_path`** (VARCHAR(500), NOT NULL): Stores the relative path or GCS path
|
|
- **`storage_url`** (VARCHAR(500), NULLABLE): Stores the full URL for accessing the file
|
|
|
|
## Storage Scenarios
|
|
|
|
### Scenario 1: File Uploaded to GCS (Successfully)
|
|
|
|
When GCS is configured and the upload succeeds:
|
|
|
|
**Database Values:**
|
|
```sql
|
|
file_path = "requests/REQ-2025-12-0001/documents/1701234567890-abc123-proposal.pdf"
|
|
storage_url = "https://storage.googleapis.com/bucket-name/requests/REQ-2025-12-0001/documents/1701234567890-abc123-proposal.pdf"
|
|
```
|
|
|
|
**File Location:**
|
|
- Physical: Google Cloud Storage bucket
|
|
- Path Structure: `requests/{requestNumber}/{fileType}/{fileName}`
|
|
- Access: Public URL or signed URL (depending on bucket configuration)
|
|
|
|
---
|
|
|
|
### Scenario 2: File Saved to Local Storage (GCS Not Configured or Failed)
|
|
|
|
When GCS is not configured or upload fails, files are saved to local storage:
|
|
|
|
**Database Values:**
|
|
```sql
|
|
file_path = "requests/REQ-2025-12-0001/documents/1701234567890-abc123-proposal.pdf"
|
|
storage_url = "/uploads/requests/REQ-2025-12-0001/documents/1701234567890-abc123-proposal.pdf"
|
|
```
|
|
|
|
**File Location:**
|
|
- Physical: Local filesystem at `{UPLOAD_DIR}/requests/{requestNumber}/{fileType}/{fileName}`
|
|
- Path Structure: Same as GCS structure for consistency
|
|
- Access: Served via Express static middleware at `/uploads/*`
|
|
|
|
**Example:**
|
|
```
|
|
uploads/
|
|
└── requests/
|
|
└── REQ-2025-12-0001/
|
|
├── documents/
|
|
│ └── 1701234567890-abc123-proposal.pdf
|
|
└── attachments/
|
|
└── 1701234567890-xyz789-note.pdf
|
|
```
|
|
|
|
---
|
|
|
|
### Scenario 3: Legacy Files (Before This Implementation)
|
|
|
|
Older files may have different path formats:
|
|
|
|
**Possible Database Values:**
|
|
```sql
|
|
file_path = "/absolute/path/to/uploads/file.pdf" -- Absolute path
|
|
-- OR
|
|
file_path = "file.pdf" -- Simple filename (in root uploads folder)
|
|
storage_url = "/uploads/file.pdf" -- Simple URL
|
|
```
|
|
|
|
**File Location:**
|
|
- Physical: Various locations depending on when file was uploaded
|
|
- Access: Handled by legacy route logic
|
|
|
|
---
|
|
|
|
## How Download/Preview Routes Handle Different Storage Types
|
|
|
|
### Document Preview Route (`GET /workflows/documents/:documentId/preview`)
|
|
|
|
1. **Check if GCS URL:**
|
|
```typescript
|
|
const isGcsUrl = storageUrl && (
|
|
storageUrl.startsWith('https://storage.googleapis.com') ||
|
|
storageUrl.startsWith('gs://')
|
|
);
|
|
```
|
|
- If yes → Redirect to GCS URL
|
|
|
|
2. **Check if Local Storage URL:**
|
|
```typescript
|
|
if (storageUrl && storageUrl.startsWith('/uploads/')) {
|
|
res.redirect(storageUrl); // Express static serves it
|
|
return;
|
|
}
|
|
```
|
|
- If yes → Redirect to `/uploads/...` (served by Express static middleware)
|
|
|
|
3. **Legacy File Handling:**
|
|
```typescript
|
|
const absolutePath = filePath && !path.isAbsolute(filePath)
|
|
? path.join(UPLOAD_DIR, filePath)
|
|
: filePath;
|
|
```
|
|
- Resolve relative path to absolute
|
|
- Serve file directly using `res.sendFile()`
|
|
|
|
### Work Note Attachment Routes
|
|
|
|
Same logic as document routes:
|
|
- Preview: `/workflows/work-notes/attachments/:attachmentId/preview`
|
|
- Download: `/workflows/work-notes/attachments/:attachmentId/download`
|
|
|
|
---
|
|
|
|
## Key Points
|
|
|
|
### 1. Consistent Path Structure
|
|
- **Both GCS and local storage** use the same path structure: `requests/{requestNumber}/{fileType}/{fileName}`
|
|
- This makes migration seamless when moving from local to GCS
|
|
|
|
### 2. Storage URL Format
|
|
- **GCS:** Full HTTPS URL (`https://storage.googleapis.com/...`)
|
|
- **Local:** Relative URL (`/uploads/requests/...`)
|
|
- **Legacy:** May vary
|
|
|
|
### 3. File Path Format
|
|
- **GCS:** Relative path in bucket (`requests/REQ-.../documents/file.pdf`)
|
|
- **Local:** Same relative path format for consistency
|
|
- **Legacy:** May be absolute path or simple filename
|
|
|
|
### 4. Automatic Fallback
|
|
- When GCS fails, system automatically saves to local storage
|
|
- Same folder structure maintained
|
|
- No data loss
|
|
|
|
### 5. Serving Files
|
|
- **GCS files:** Redirect to public/signed URL
|
|
- **Local files (new):** Redirect to `/uploads/...` (Express static)
|
|
- **Local files (legacy):** Direct file serving with `res.sendFile()`
|
|
|
|
---
|
|
|
|
## Migration Path
|
|
|
|
When migrating from local storage to GCS:
|
|
|
|
1. **Files already follow same structure** - No path changes needed
|
|
2. **Upload new files** - They automatically go to GCS if configured
|
|
3. **Existing files** - Can remain in local storage until migrated
|
|
4. **Database** - Only `storage_url` field changes (from `/uploads/...` to `https://...`)
|
|
|
|
---
|
|
|
|
## Example Database Records
|
|
|
|
### GCS File (New Upload)
|
|
```json
|
|
{
|
|
"document_id": "uuid-123",
|
|
"file_path": "requests/REQ-2025-12-0001/documents/1701234567890-abc123-proposal.pdf",
|
|
"storage_url": "https://storage.googleapis.com/my-bucket/requests/REQ-2025-12-0001/documents/1701234567890-abc123-proposal.pdf",
|
|
"file_name": "1701234567890-abc123-proposal.pdf",
|
|
"original_file_name": "proposal.pdf"
|
|
}
|
|
```
|
|
|
|
### Local Storage File (Fallback)
|
|
```json
|
|
{
|
|
"document_id": "uuid-456",
|
|
"file_path": "requests/REQ-2025-12-0001/documents/1701234567891-def456-report.pdf",
|
|
"storage_url": "/uploads/requests/REQ-2025-12-0001/documents/1701234567891-def456-report.pdf",
|
|
"file_name": "1701234567891-def456-report.pdf",
|
|
"original_file_name": "report.pdf"
|
|
}
|
|
```
|
|
|
|
### Legacy File (Old Format)
|
|
```json
|
|
{
|
|
"document_id": "uuid-789",
|
|
"file_path": "/var/app/uploads/old-file.pdf",
|
|
"storage_url": "/uploads/old-file.pdf",
|
|
"file_name": "old-file.pdf",
|
|
"original_file_name": "old-file.pdf"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Issue: File not found when downloading
|
|
|
|
**Check:**
|
|
1. Verify `storage_url` format in database
|
|
2. Check if file exists at expected location:
|
|
- GCS: Check bucket and path
|
|
- Local: Check `{UPLOAD_DIR}/requests/...` path
|
|
3. Verify Express static middleware is mounted at `/uploads`
|
|
|
|
### Issue: Files not organizing correctly
|
|
|
|
**Check:**
|
|
1. Verify `requestNumber` is being passed correctly to upload functions
|
|
2. Check folder structure matches: `requests/{requestNumber}/{fileType}/`
|
|
3. Verify `fileType` is either `'documents'` or `'attachments'`
|
|
|