Sync Queue

The Onboarded™ Sync Queue is a retry mechanism that ensures outbound sync operations complete successfully, even when Salesforce platform limits are reached or temporary failures occur.

When outbound sync operations (sending data from Salesforce to Onboarded™) cannot be processed immediately due to queueable job limits or other constraints, the requests are stored in the Onboarded_Sync_Queue__c object. These queued requests are then processed asynchronously, with automatic retry on failure.

• Salesforce queueable job limits are reached (max 50 per transaction)
• Multiple sync operations triggered in rapid succession
• Flow-triggered syncs that exceed asynchronous limits
• Bulk operations that generate many outbound requests
• The Onboarded™ API rate limit is exceeded (HTTP 429). The platform accepts up to 3,000 records per minute; requests beyond that are captured in the queue and retried automatically (see API Rate Limits and Transient Failures)

Each Sync Queue record progresses through a defined status lifecycle:

Success Behavior: When a sync operation completes successfully, the corresponding Sync Queue record is automatically deleted. You should only see records in the queue that are pending, processing, or have failed.

A scheduled job runs every 15 minutes to process failed records automatically. The system uses a backoff strategy to avoid overwhelming the API during outages:

Maximum Retries: After 3 failed retry attempts, records remain in Failed status and require manual intervention. Records marked as Do Not Retry are never automatically retried.

The Onboarded™ platform enforces an ingestion limit of 3,000 records per minute. When outbound sync traffic from Salesforce exceeds this threshold, the API responds with HTTP 429 (Too Many Requests).

A 429 is treated as a transient failure — the data itself is valid; the platform is simply asking Salesforce to slow down. Rather than failing the record permanently, the application captures the affected outbound request in the Sync Queue and feeds it back through the Automatic Retry Mechanism above. Other transient conditions — HTTP 5xx server errors and callout timeouts — are handled the same way.

Key behaviors:

• No premature error logging. A transient 429 does not create an Onboarded™ Error Log record on each attempt. An error is logged only if the record still fails after the retry attempts are exhausted; at that point the Error_Logged__c flag is set so the same record is not logged again on subsequent passes.
• Self-throttling recovery. The queue processor drains records in paced slices and chains itself between transactions, which naturally spreads retries back under the 3,000-per-minute ceiling instead of immediately re-hitting it.
• Only failed records are retried. When a batch partially succeeds, the successfully-sent records are dropped from the queue and only the failed record IDs are retried — minimizing repeat API calls and further rate-limit pressure.

Note: A 429 is the one 4xx response that is retried rather than marked Do Not Retry. All other 4xx client errors (400/401/403/404) indicate a problem the retry cycle cannot fix and are handled as Unrecoverable Errors below.

Certain errors are marked as Do Not Retry because retrying won't fix the underlying issue:

• HTTP 4xx errors: Bad Request (400), Unauthorized (401), Forbidden (403), Not Found (404). (HTTP 429 rate-limit responses are the exception — they are transient and retried; see API Rate Limits and Transient Failures above.)
• Data validation errors: Invalid JSON payload, missing required fields
• Encryption errors: Decryption failures due to corrupted data

These records require administrator review to fix the underlying data issue before triggering a new sync.

Error Log created, but not reprocessable. Every row that reaches Do Not Retry writes exactly one Onboarded_Error_Log__c entry (Class_Name__c = 'OnboardedSyncQueueProcessorQueueable', Impacted_Record__c = the Sync Queue row's own Id) — the Class_Name__c value identifies it as originating from the queue's own repeated-failure handling, distinct from a live single-record outbound push. Because the Reprocess capability is scoped to that live outbound-push path (see Error Handling), this Error Log entry is for audit visibility only — no Reprocess button will appear for it. Resolve the underlying issue and retry from the Sync Queue record itself (see Failed Record Retry above), not from the Error Log.

When outbound sync operations store data in the Sync Queue, the Payload__c field is encrypted using AES-256 encryption to protect sensitive employee and employer information.

• Algorithm: AES-256 (FIPS 140-2 compliant via Salesforce Crypto class)
• Key Management: Encryption key is automatically generated during package installation and stored in the application's protected configuration — not visible or editable by any admin, including you
• Randomization: Each encryption operation uses industry-standard randomization to ensure identical data produces unique encrypted output
• Automatic Encryption: Payloads are encrypted automatically before insert via trigger

To view the original (decrypted) payload data for troubleshooting:

1. Assign the Onboarded™ Encryption Access permission set to your user
2. Navigate to the Sync Queue record
3. Click the View Decrypted Payload button
4. The decrypted JSON will be displayed in a modal dialog

Security Note: Decrypted payloads may contain PII such as Social Security Numbers, dates of birth, and addresses. Only grant the Onboarded™ Encryption Access permission set to users with a legitimate need to troubleshoot sync failures.

Regularly monitor the Sync Queue for failed records to ensure data stays synchronized.

The package includes the following list views for monitoring:

• All: View all Sync Queue records
• Failed (Pending Retry): View records that will be automatically retried
• Do Not Retry: View records that require manual attention due to unrecoverable errors

You can also create custom list views:

1. Navigate to the Onboarded™ Sync Queue tab
2. Create a new list view with your desired filters
3. Add columns: Name, Status, Onboarded™ Object Type, Operation, Error Message, Retry Count, Last Attempt Time, Created Date
4. Sort by Created Date descending to see newest failures first

Create a workflow rule or Flow to send email notifications when Sync Queue records have Status = Failed:

1. Create a Flow that triggers on Sync Queue record update where Status changes to Failed
2. Send an email alert to your integration administrator
3. Include the Error_Message__c and record link in the notification

Add a report chart to your admin dashboard showing:

• Count of Failed Sync Queue records
• Failed records grouped by Onboarded™ Object Type
• Trend of failures over time

Automatic Retries: Most failed records will be retried automatically up to 3 times. You only need to intervene if records remain in Failed status after all retry attempts, or if records are marked as Do Not Retry.

1. Identify the Failure Cause. Review the Error_Message__c field to understand why the sync failed. Common causes include:
   • API authentication issues (token expired)
   • Network connectivity problems
   • Invalid data format in the payload
   • Onboarded™ API rate limiting
   • Missing required fields in the source record
2. Resolve the Underlying Issue. Address the root cause before retrying:
   • For auth issues: Re-authenticate in Onboarded™ Setup
   • For data issues: Correct the source record data
   • For API limits: Wait for the rate limit window to reset
3. Change Status to Pending. Once the issue is resolved, edit the failed Sync Queue record and change the Status__c field from Failed to Pending.
4. Monitor for Success. The system will automatically pick up the pending record and attempt to process it. If successful, the record will be deleted. If it fails again, review the updated error message.

Bulk Retry: To retry multiple failed records at once, use Data Loader or the Salesforce Inspector browser extension to mass-update the Status__c field from Failed to Pending.

The Sync Queue processor operates as follows:

1. When a new Sync Queue record is inserted with Pending status, the trigger handler checks if queueable capacity is available
2. If capacity is available, a bounded slice of pending records is selected and their status is updated to Processing (the slice is deliberately small so the transaction stays well within Salesforce governor limits)
3. A queueable job is enqueued to process that slice
4. Each record is processed individually — failures don't affect other records in the batch — and record write-backs are applied in bulk (grouped per object) rather than one record at a time
5. Successful records are deleted; failed records are updated with error details
6. If more Pending records remain, the processor chains itself to continue with the next slice — so one drain works through the entire backlog across multiple transactions, keeping even a large burst safely within Salesforce governor limits

Recovering a stranded queue. If a processor ever stops before finishing (or a backlog builds with no active job), the records simply wait as Pending/Processing — they are never lost. An admin can restart the drain at any time from Onboarded™ Setup → Data Operation Settings → Sync Queueable Operations, which reclaims any orphaned rows and drains the whole queue in safe slices with a single click.

The system prevents queue processing loops through these safeguards:

• A flag prevents new queueable jobs from being enqueued while the queue is already processing
• Records in Processing status are not picked up by subsequent trigger executions
• Inbound sync operations do not trigger outbound queue processing

Standard Sync: Uses record IDs to query Salesforce data and transform it for the Onboarded™ API. The Record_Ids__c field contains the Salesforce IDs to sync.

Direct Sync: Uses a pre-built JSON payload (encrypted in Payload__c) that is sent directly to the Onboarded™ API. This is typically used by Flow invocable actions that construct the payload programmatically.