Scheduled integrations in OIC look simple on the surface — trigger at a time, pull data, push data. In practice, they’re where most production reliability issues live. Here’s what separates a scheduler that works once from one that runs cleanly for years.
The core challenges
- Volume management — what happens when the scheduled run picks up 50,000 records instead of the usual 500?
- Duplicate prevention — if the integration fails halfway through, does the next run re-process already-completed records?
- Failure visibility — if a scheduled run silently fails at 3am, how do you know?
Pagination and chunking
Never assume the data set is small. Always implement pagination on your source REST or DB query using limit and offset parameters. In OIC, this means a loop activity around your fetch, processing a chunk at a time until the API returns an empty page.
For very large volumes, consider splitting the scheduled job into two integrations: one that fetches and writes to a staging table, and a second that processes from that table in batches. This decoupling makes recovery much easier.
Watermark pattern for duplicate prevention
The watermark pattern is the most reliable way to prevent re-processing. Persist the timestamp or sequence number of the last successfully processed record (in an OIC lookup table or an ATP table), and use it as a filter on the next run.
On each run:
1. Read last_processed_timestamp from lookup
2. Fetch records WHERE updated_date > last_processed_timestamp
3. Process records
4. On success: update last_processed_timestamp = now()
Monitoring scheduled runs
Use OIC’s built-in monitoring but don’t rely on it exclusively. Set up email notifications for failed instances, and consider a heartbeat check — a lightweight integration that fires after each major scheduled run and posts a success signal to a monitoring endpoint.
A note on time zones
This trips up almost every team at least once. OIC scheduled triggers run in UTC. Fusion ERP data is often stored in the tenant’s local time zone. If you’re using timestamp-based filtering, make sure you’re comparing apples to apples.