Bulk API 2.0 and Large Data Volumes

Headless Salesforce & APIsBulk API 2.0 and Large Data VolumesMove millions of records in and out of Salesforce with Bulk API 2.0 ingest and query jobs, and design orgs that stay fast at large data volumes: skew, selectivity, indexes, skinny tables, archiving and deletion.Advanced · about 90 minutesWhat you will learn

Run a complete Bulk API 2.0 ingest job (create, upload, close, poll, fetch results) with curl, Python and Node.js

Run Bulk API 2.0 query jobs and page through results with Sforce-Locator and maxRecords

Explain when Bulk API 1.0 is still the right tool (serial mode, explicit PK chunking, batch control)

Diagnose lock contention and design loads that avoid UNABLE_TO_LOCK_ROW

Identify account, ownership and lookup skew and apply mitigations

Apply LDV techniques: selective queries, custom indexes, skinny tables, deferred sharing, big objects and archiving

Contents

When to use Bulk API, and which one

Bulk API 2.0 ingest jobs

A complete Python ingest script

Bulk API 2.0 query jobs

Limits and allocations

Serial vs parallel processing and lock contention

Command-line and desktop tools

Large Data Volumes: why big orgs behave differently

Data skew

Selective queries and indexes

Sharing at scale

Big Objects

Archiving strategies

Deleting large data volumes

Best practices

Common pitfalls

Hands-on exercises

Key takeaways