laxmanhalaki/Centralized_Reporting_Backend

Fork 0

yashwin-foxy c8eedf73ad n8n flow integrated fro sales force integration

2025-10-10 12:10:33 +05:30

9.3 KiB

Raw Blame History

Salesforce Pagination Guide

Overview

Salesforce uses a cursor-based pagination system with nextRecordsUrl instead of traditional offset/limit pagination.

Salesforce Response Structure

First Request Response

{
  "totalSize": 7530,
  "done": false,
  "nextRecordsUrl": "/services/data/v61.0/query/01gxx0000034XYZAAA-2000",
  "records": [
    {
      "attributes": {
        "type": "Lead",
        "url": "/services/data/v59.0/sobjects/Lead/00QdN00000AZGH4UAP"
      },
      "Id": "00QdN00000AZGH4UAP",
      "FirstName": "John",
      "LastName": "Steele (Sample)",
      "Company": "BigLife Inc.",
      "Email": "info@salesforce.com",
      "Status": "Working"
    }
    // ... more records
  ]
}

Key Fields

totalSize: Total number of records matching the query
done: Boolean indicating if there are more records
nextRecordsUrl: URL path to fetch the next batch (only present when done is false)
records: Array of record objects

Using Pagination in Your Backend

First Request - Get Initial Records

GET /api/v1/n8n/salesforce/crm/leads?limit=200
Authorization: Bearer <jwt_token>

Response:

{
  "status": "success",
  "message": "salesforce crm leads data fetched successfully",
  "data": {
    "success": true,
    "data": [
      {
        "Id": "00QdN00000AZGH4UAP",
        "FirstName": "John",
        "LastName": "Steele",
        ...
      }
    ],
    "count": 200,
    "metadata": {
      "totalSize": 7530,
      "done": false,
      "nextRecordsUrl": "/services/data/v61.0/query/01gxx0000034XYZAAA-2000"
    }
  },
  "timestamp": "2025-10-09T12:00:00.000Z"
}

Subsequent Requests - Get Next Pages

Use the nextRecordsUrl from the previous response:

GET /api/v1/n8n/salesforce/crm/leads?nextRecordsUrl=/services/data/v61.0/query/01gxx0000034XYZAAA-2000
Authorization: Bearer <jwt_token>

Response:

{
  "status": "success",
  "message": "salesforce crm leads data fetched successfully",
  "data": {
    "success": true,
    "data": [
      // Next 200 records
    ],
    "count": 200,
    "metadata": {
      "totalSize": 7530,
      "done": false,
      "nextRecordsUrl": "/services/data/v61.0/query/01gxx0000034XYZAAA-4000"
    }
  },
  "timestamp": "2025-10-09T12:00:00.000Z"
}

Last Page

When you reach the last page, done will be true and nextRecordsUrl will be null:

{
  "metadata": {
    "totalSize": 7530,
    "done": true,
    "nextRecordsUrl": null
  }
}

Frontend Implementation

JavaScript Example - Fetch All Pages

async function fetchAllSalesforceLeads(token) {
  const API_URL = 'http://localhost:3000/api/v1';
  let allRecords = [];
  let nextRecordsUrl = null;
  let hasMore = true;
  
  while (hasMore) {
    // Build URL
    const url = nextRecordsUrl
      ? `${API_URL}/n8n/salesforce/crm/leads?nextRecordsUrl=${encodeURIComponent(nextRecordsUrl)}`
      : `${API_URL}/n8n/salesforce/crm/leads?limit=200`;
    
    // Fetch data
    const response = await fetch(url, {
      headers: { 'Authorization': `Bearer ${token}` }
    });
    
    const result = await response.json();
    
    // Add records to collection
    allRecords = allRecords.concat(result.data.data);
    
    // Check if there are more pages
    hasMore = !result.data.metadata.done;
    nextRecordsUrl = result.data.metadata.nextRecordsUrl;
    
    console.log(`Fetched ${result.data.count} records. Total so far: ${allRecords.length}`);
  }
  
  console.log(`Completed! Total records: ${allRecords.length}`);
  return allRecords;
}

// Usage
const token = 'your_jwt_token';
const allLeads = await fetchAllSalesforceLeads(token);

React Example - Paginated Table

import React, { useState, useEffect } from 'react';
import axios from 'axios';

function SalesforceLeadsTable() {
  const [leads, setLeads] = useState([]);
  const [loading, setLoading] = useState(false);
  const [nextRecordsUrl, setNextRecordsUrl] = useState(null);
  const [hasMore, setHasMore] = useState(true);
  const [totalSize, setTotalSize] = useState(0);

  const API_URL = 'http://localhost:3000/api/v1';
  const token = localStorage.getItem('jwt_token');

  const fetchLeads = async (nextUrl = null) => {
    setLoading(true);
    try {
      const url = nextUrl
        ? `${API_URL}/n8n/salesforce/crm/leads?nextRecordsUrl=${encodeURIComponent(nextUrl)}`
        : `${API_URL}/n8n/salesforce/crm/leads?limit=50`;

      const response = await axios.get(url, {
        headers: { Authorization: `Bearer ${token}` }
      });

      const { data, metadata } = response.data.data;

      setLeads(prevLeads => [...prevLeads, ...data]);
      setNextRecordsUrl(metadata.nextRecordsUrl);
      setHasMore(!metadata.done);
      setTotalSize(metadata.totalSize);
    } catch (error) {
      console.error('Error fetching leads:', error);
    } finally {
      setLoading(false);
    }
  };

  useEffect(() => {
    fetchLeads();
  }, []);

  const loadMore = () => {
    if (nextRecordsUrl && hasMore) {
      fetchLeads(nextRecordsUrl);
    }
  };

  return (
    <div>
      <h2>Salesforce Leads ({leads.length} of {totalSize})</h2>
      
      <table>
        <thead>
          <tr>
            <th>Name</th>
            <th>Company</th>
            <th>Email</th>
            <th>Status</th>
          </tr>
        </thead>
        <tbody>
          {leads.map(lead => (
            <tr key={lead.Id}>
              <td>{lead.FirstName} {lead.LastName}</td>
              <td>{lead.Company}</td>
              <td>{lead.Email}</td>
              <td>{lead.Status}</td>
            </tr>
          ))}
        </tbody>
      </table>

      {hasMore && (
        <button onClick={loadMore} disabled={loading}>
          {loading ? 'Loading...' : 'Load More'}
        </button>
      )}
    </div>
  );
}

export default SalesforceLeadsTable;

Important Notes

1. URL Encoding

Always URL-encode the nextRecordsUrl when passing it as a query parameter:

const encodedUrl = encodeURIComponent(nextRecordsUrl);

2. Cursor Expiration

Salesforce cursors (nextRecordsUrl) expire after a certain time. If you get an error, start from the beginning.

3. Limit Parameter

The limit parameter only affects the first request. Subsequent requests using nextRecordsUrl return the same batch size as the first request.

4. Don't Mix Pagination Methods

Once you start using nextRecordsUrl, don't try to use offset for the same query sequence.

5. Performance

Fetching all records at once can be slow for large datasets
Consider implementing lazy loading or virtual scrolling
Use web workers for processing large datasets

Comparison with Zoho Pagination

Feature	Salesforce	Zoho
Method	Cursor-based	Page-based
Parameter	`nextRecordsUrl`	`page`
Total Count	`totalSize`	Varies
Has More	`done` boolean	Page calculation
URL	Changes each page	Increments page number

Troubleshooting

"Invalid nextRecordsUrl"

The cursor might have expired
Start a new query from the beginning

"Missing records"

Always use the nextRecordsUrl from the response
Don't manually construct the URL

"Slow performance"

Reduce the initial limit
Implement caching on the frontend
Use pagination instead of loading all records

API Reference

Endpoint

GET /api/v1/n8n/salesforce/:service/:module

Query Parameters

Parameter	Type	Required	Description
`limit`	number	No	Number of records (first request only, default: 200)
`nextRecordsUrl`	string	No	URL for next page (from previous response)

Response Format

{
  "status": "success",
  "message": "...",
  "data": {
    "success": true,
    "data": [...],
    "count": 200,
    "metadata": {
      "totalSize": 7530,
      "done": false,
      "nextRecordsUrl": "/services/data/v61.0/query/..."
    }
  },
  "timestamp": "..."
}

Examples

cURL - First Page

curl -X GET "http://localhost:3000/api/v1/n8n/salesforce/crm/leads?limit=100" \
  -H "Authorization: Bearer $TOKEN"

cURL - Next Page

NEXT_URL="/services/data/v61.0/query/01gxx0000034XYZAAA-2000"
curl -X GET "http://localhost:3000/api/v1/n8n/salesforce/crm/leads?nextRecordsUrl=$(echo $NEXT_URL | jq -sRr @uri)" \
  -H "Authorization: Bearer $TOKEN"

Python Example

import requests

API_URL = "http://localhost:3000/api/v1"
token = "your_jwt_token"
headers = {"Authorization": f"Bearer {token}"}

def fetch_all_leads():
    all_records = []
    url = f"{API_URL}/n8n/salesforce/crm/leads?limit=200"
    
    while url:
        response = requests.get(url, headers=headers)
        data = response.json()
        
        records = data['data']['data']
        metadata = data['data']['metadata']
        
        all_records.extend(records)
        
        # Check for next page
        if not metadata['done'] and metadata['nextRecordsUrl']:
            next_url = metadata['nextRecordsUrl']
            url = f"{API_URL}/n8n/salesforce/crm/leads?nextRecordsUrl={requests.utils.quote(next_url)}"
        else:
            url = None
        
        print(f"Fetched {len(records)} records. Total: {len(all_records)}")
    
    return all_records

leads = fetch_all_leads()
print(f"Total leads: {len(leads)}")

Last Updated: October 9, 2025
Version: 1.0.0

9.3 KiB Raw Blame History