Scheduled Data Sync

Periodically sync data from external sources using cron schedules and workflows. Grove schedules start workflows on a recurring basis, and activities handle the data fetching and processing.

Prerequisites: Schedules, Workflows, Activities. What you'll learn: How to set up periodic data synchronization.

The Pattern

  1. A schedule fires on a cron expression
  2. It starts a workflow that orchestrates the sync
  3. Activities fetch from external APIs and invoke Grove actions to update records

Example: Daily Inventory Sync

schedule daily_inventory_sync {
  when "0 2 * * *"
  start sync_inventory v1 {
    source: "warehouse-api"
  }
}

workflow sync_inventory v1 {
  node fetch = fetch_inventory v1
  node process = process_inventory_updates v1
  edge fetch -> process
}

activity fetch_inventory v1 {
  data items {
    fetch warehouse.list_items {
      updated_since: input.last_sync_time
    }
  }

  apply(input: SyncInput) -> InventoryData {
    return { items: items.results, count: items.results.len }
  }
}

activity process_inventory_updates v1 {
  data results {
    invoke inventory.bulk_update {
      items: input.items
    }
  }

  apply(input: InventoryData) -> SyncResult {
    return { updated: results.count, errors: results.errors }
  }
}

Cron Expression Reference

ExpressionMeaning
0 * * * *Every hour
0 2 * * *Daily at 2:00 AM
0 0 * * 1Every Monday at midnight
0 0 1 * *First day of every month
*/15 * * * *Every 15 minutes

Format: minute hour day-of-month month day-of-week

See Also