[2026-05-13T08:14:59.789878] Job 157442 queued by detail-backfill-auto
Connection: WBF NYC (meta_ads)
Sync template: Meta Ads - Detail Backfill
Queue class: historical
Procrastinate queue: kpiper-historical
Command: python meta_ads_pipeline.py
[2026-05-13T08:14:59.900922] Queued via Procrastinate as job 156293 (queue=kpiper-historical, priority=100).
[2026-05-13T08:14:59.956663] Starting job 157442 for connection WBF NYC (meta_ads) / sync template Meta Ads - Detail Backfill (type=dlt)
Resolved command: cd /srv/kpiper && python meta_ads_pipeline.py
Injected profile context env vars (tokens exported: 1).
Pipeline env overrides applied: AD_ACCOUNT_ID, BACKFILL_CHUNK_DAYS, ENABLE_ACCOUNT_INSIGHTS, ENABLE_ADS, ENABLE_AD_ACCOUNT_OBJECTS, ENABLE_AD_INSIGHTS, ENABLE_AD_SETS, ENABLE_AD_SET_INSIGHTS, ENABLE_CAMPAIGNS, ENABLE_CAMPAIGN_INSIGHTS, INSIGHTS_FIELDS, INSIGHTS_LOOKBACK_DAYS, INSIGHT_WINDOW_DAYS, OBJECT_LIMIT, PAGE_SIZE, SOURCES__META_ADS_REST_API_SOURCE__ACCESS_TOKEN, SYNC_MODE
Synced Meta system user token at 2026-05-13T08:14:59.936982.
Resuming historical backfill from 2025-08-02.
Forced unbuffered Python output for child process.
/usr/local/lib/python3.11/site-packages/dlt/pipeline/progress.py:27: UserWarning: psutil dependency is not installed and memory stats will not be available. Add psutil to your environment or pass dump_system_stats=False to disable this warning.
  return log()
----------------------- Extract meta_ads_rest_api_source -----------------------
Resources: 0/11 (0.0%) | Time: 0.00s | Rate: 0.00/s

Meta Ads rate limit on account 1829488277102363 (attempt 1/4); cooling down 30.0s. Error: {"error":{"message":"User request limit reached","type":"OAuthException","is_transient":false,"code":17,"error_subcode":2446079,"error_user_title":"Ad Account Has Too Many API Calls","error_user_msg":"There have been too many calls from this ad-account. Please wait a bit and try again.","fbtrace_id":"AgkiQHl0AzfC_a5uCrbRQpY"}}
Cooling down Meta Ads requests for account 1829488277102363 for 30.0s after rate limiting.
Meta Ads rate limit on account 1829488277102363 (attempt 2/4); cooling down 60.0s. Error: {"error":{"message":"User request limit reached","type":"OAuthException","is_transient":false,"code":17,"error_subcode":2446079,"error_user_title":"Ad Account Has Too Many API Calls","error_user_msg":"There have been too many calls from this ad-account. Please wait a bit and try again.","fbtrace_id":"AOdGBdQ7x3qUwHTQs40Av0j"}}
Cooling down Meta Ads requests for account 1829488277102363 for 60.0s after rate limiting.
Meta Ads rate limit on account 1829488277102363 (attempt 3/4); cooling down 120.0s. Error: {"error":{"message":"User request limit reached","type":"OAuthException","is_transient":false,"code":17,"error_subcode":2446079,"error_user_title":"Ad Account Has Too Many API Calls","error_user_msg":"There have been too many calls from this ad-account. Please wait a bit and try again.","fbtrace_id":"A3khJWrmz3UUPke7RN-fdE-"}}
Cooling down Meta Ads requests for account 1829488277102363 for 120.0s after rate limiting.
Meta Ads rate limit on account 1829488277102363 (attempt 4/4); cooling down 240.0s. Error: {"error":{"message":"User request limit reached","type":"OAuthException","is_transient":false,"code":17,"error_subcode":2446079,"error_user_title":"Ad Account Has Too Many API Calls","error_user_msg":"There have been too many calls from this ad-account. Please wait a bit and try again.","fbtrace_id":"AWG6l1fzUZWpRW5po3ZAcKN"}}
Cooling down Meta Ads requests for account 1829488277102363 for 240.0s after rate limiting.