[2026-05-12T22:24:01.868925] Job 156817 queued by schedule:2026-05-12T22:24:00
Connection: eatalyroma (instagram)
Sync template: Instagram Graph - Incremental Sync
Queue class: foreground
Procrastinate queue: kpiper
Command: python instagram_graph_pipeline.py
[2026-05-12T22:24:01.967953] Queued via Procrastinate as job 155668 (queue=kpiper, priority=50).
[2026-05-12T23:24:02.143979] Reused existing pending job without queue promotion (queue=kpiper, priority=50).
[2026-05-13T00:24:02.163986] Reused existing pending job without queue promotion (queue=kpiper, priority=50).
[2026-05-13T01:24:02.192920] Reused existing pending job without queue promotion (queue=kpiper, priority=50).
[2026-05-13T02:24:01.625480] Reused existing pending job without queue promotion (queue=kpiper, priority=50).
[2026-05-13T03:24:01.606368] Reused existing pending job without queue promotion (queue=kpiper, priority=50).
[2026-05-13T04:14:27.277607] Starting job 156817 for connection eatalyroma (instagram) / sync template Instagram Graph - Incremental Sync (type=dlt)
Resolved command: cd /srv/kpiper && python instagram_graph_pipeline.py
Injected profile context env vars (tokens exported: 2).
Pipeline env overrides applied: DOWNLOAD_ASSETS, ENABLE_ACCOUNT_OBJECTS, ENABLE_ASSETS, ENABLE_COMMENTS, ENABLE_CONTENT_PUBLISHING_LIMITS, ENABLE_MEDIA_CHILD_OBJECTS, ENABLE_MEDIA_INSIGHTS, ENABLE_MEDIA_OBJECTS, ENABLE_STORY_INSIGHTS, ENABLE_STORY_OBJECTS, ENABLE_USER_INSIGHTS, ENABLE_WEBHOOK_SUBSCRIBED_APPS, IG_PAGE_ID, IG_USER_ID, MEDIA_INCREMENTAL_LOOKBACK_DAYS, MEDIA_PAGINATION, SOURCES__INSTAGRAM_GRAPH_REST_API_SOURCE__ACCESS_TOKEN, SYNC_MODE
Refreshed Meta Page Token at 2026-05-13T04:14:27.256296.
Forced unbuffered Python output for child process.
/usr/local/lib/python3.11/site-packages/dlt/pipeline/progress.py:27: UserWarning: psutil dependency is not installed and memory stats will not be available. Add psutil to your environment or pass dump_system_stats=False to disable this warning.
  return log()
------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/6 (0.0%) | Time: 0.00s | Rate: 0.00/s

Retrying 17841407910334787 with a narrower field set after Graph error: {"error":{"message":"(#100) Tried accessing nonexisting field (account_type)","type":"OAuthException","code":100,"fbtrace_id":"AD_JhAMZVtpcbtoGFH1o9Is"}}
------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/6 (0.0%) | Time: 0.60s | Rate: 0.00/s
account_objects: 1  | Time: 0.00s | Rate: 79137.81/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 2/6 (33.3%) | Time: 0.61s | Rate: 3.28/s
account_objects: 1  | Time: 0.01s | Rate: 171.01/s
story_objects: 1  | Time: 0.00s | Rate: 71089.90/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 2/6 (33.3%) | Time: 0.61s | Rate: 3.25/s
account_objects: 1  | Time: 0.01s | Rate: 88.00/s
story_objects: 1  | Time: 0.01s | Rate: 180.84/s
content_assets: 1  | Time: 0.00s | Rate: 83886.08/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 2/6 (33.3%) | Time: 6.60s | Rate: 0.30/s
account_objects: 1  | Time: 6.00s | Rate: 0.17/s
story_objects: 1  | Time: 5.99s | Rate: 0.17/s
content_assets: 1  | Time: 5.99s | Rate: 0.17/s
story_insight_values: 1  | Time: 0.00s | Rate: 93206.76/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 3/6 (50.0%) | Time: 6.61s | Rate: 0.45/s
account_objects: 1  | Time: 6.01s | Rate: 0.17/s
story_objects: 3  | Time: 6.00s | Rate: 0.50/s
content_assets: 3  | Time: 5.99s | Rate: 0.50/s
story_insight_values: 2  | Time: 0.01s | Rate: 339.63/s
insight_fetch_skips: 1  | Time: 0.00s | Rate: 93206.76/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 3/6 (50.0%) | Time: 13.34s | Rate: 0.22/s
account_objects: 1  | Time: 12.74s | Rate: 0.08/s
story_objects: 7  | Time: 12.73s | Rate: 0.55/s
content_assets: 7  | Time: 12.73s | Rate: 0.55/s
story_insight_values: 2  | Time: 6.74s | Rate: 0.30/s
insight_fetch_skips: 5  | Time: 6.73s | Rate: 0.74/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 21.90s | Rate: 0.23/s
account_objects: 1  | Time: 21.30s | Rate: 0.05/s
story_objects: 7  | Time: 21.29s | Rate: 0.33/s
content_assets: 8  | Time: 21.28s | Rate: 0.38/s
story_insight_values: 2  | Time: 15.30s | Rate: 0.13/s
insight_fetch_skips: 11  | Time: 15.29s | Rate: 0.72/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 28.98s | Rate: 0.17/s
account_objects: 1  | Time: 28.38s | Rate: 0.04/s
story_objects: 7  | Time: 28.37s | Rate: 0.25/s
content_assets: 8  | Time: 28.36s | Rate: 0.28/s
story_insight_values: 2  | Time: 22.38s | Rate: 0.09/s
insight_fetch_skips: 17  | Time: 22.37s | Rate: 0.76/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 36.51s | Rate: 0.14/s
account_objects: 1  | Time: 35.90s | Rate: 0.03/s
story_objects: 7  | Time: 35.90s | Rate: 0.19/s
content_assets: 8  | Time: 35.89s | Rate: 0.22/s
story_insight_values: 2  | Time: 29.90s | Rate: 0.07/s
insight_fetch_skips: 23  | Time: 29.90s | Rate: 0.77/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 43.24s | Rate: 0.12/s
account_objects: 1  | Time: 42.63s | Rate: 0.02/s
story_objects: 7  | Time: 42.63s | Rate: 0.16/s
content_assets: 8  | Time: 42.62s | Rate: 0.19/s
story_insight_values: 2  | Time: 36.63s | Rate: 0.05/s
insight_fetch_skips: 29  | Time: 36.63s | Rate: 0.79/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 51.65s | Rate: 0.10/s
account_objects: 1  | Time: 51.05s | Rate: 0.02/s
story_objects: 7  | Time: 51.04s | Rate: 0.14/s
content_assets: 8  | Time: 51.03s | Rate: 0.16/s
story_insight_values: 2  | Time: 45.05s | Rate: 0.04/s
insight_fetch_skips: 35  | Time: 45.04s | Rate: 0.78/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 6/6 (100.0%) | Time: 51.68s | Rate: 0.12/s
account_objects: 1  | Time: 51.07s | Rate: 0.02/s
story_objects: 7  | Time: 51.07s | Rate: 0.14/s
content_assets: 8  | Time: 51.06s | Rate: 0.16/s
story_insight_values: 2  | Time: 45.07s | Rate: 0.04/s
insight_fetch_skips: 40  | Time: 45.07s | Rate: 0.89/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 0/5 (0.0%) | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 0/5 (0.0%) | Time: 0.02s | Rate: 0.00/s
account_objects: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 1/5 (20.0%) | Time: 0.03s | Rate: 36.22/s
account_objects: 1  | Time: 0.01s | Rate: 169.54/s
story_objects: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 2/5 (40.0%) | Time: 0.03s | Rate: 61.17/s
account_objects: 1  | Time: 0.01s | Rate: 91.04/s
story_objects: 7  | Time: 0.01s | Rate: 1374.54/s
content_assets: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 3/5 (60.0%) | Time: 0.04s | Rate: 74.45/s
account_objects: 1  | Time: 0.02s | Rate: 53.81/s
story_objects: 7  | Time: 0.01s | Rate: 551.51/s
content_assets: 8  | Time: 0.01s | Rate: 1051.50/s
story_insight_values: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 4/5 (80.0%) | Time: 0.05s | Rate: 87.95/s
account_objects: 1  | Time: 0.02s | Rate: 42.07/s
story_objects: 7  | Time: 0.02s | Rate: 391.48/s
content_assets: 8  | Time: 0.01s | Rate: 625.16/s
story_insight_values: 2  | Time: 0.01s | Rate: 384.92/s
insight_fetch_skips: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778645672.8465455 --------
Files: 5/5 (100.0%) | Time: 0.06s | Rate: 83.58/s
account_objects: 1  | Time: 0.04s | Rate: 26.24/s
story_objects: 7  | Time: 0.03s | Rate: 217.24/s
content_assets: 8  | Time: 0.03s | Rate: 294.79/s
story_insight_values: 2  | Time: 0.02s | Rate: 102.37/s
insight_fetch_skips: 40  | Time: 0.01s | Rate: 2787.65/s

---------- Load instagram_graph_rest_api_source in 1778645672.8465455 ----------
Jobs: 0/5 (0.0%) | Time: 0.00s | Rate: 0.00/s

---------- Load instagram_graph_rest_api_source in 1778645672.8465455 ----------
Jobs: 9/9 (100.0%) | Time: 0.67s | Rate: 13.41/s

Pipeline instagram_graph_pipeline load step completed in 0.67 seconds
1 load package(s) were loaded to destination postgres and into dataset instagram_graph
The postgres destination used postgresql://kpiper_dlt_app:***@[/cloudsql/pacelab-487114:europe-west4:pacelab-pg18-ew4]:5432/kpiper_data location to store data
Load package 1778645672.8465455 is LOADED and contains no failed jobs
[2026-05-13T04:15:26.022827] Command completed successfully.
[2026-05-13T04:15:26.023110] Pipeline status payload: {"backfill_complete": true, "backfill_progress_mode": "incremental", "ig_user_id": "17841407910334787", "sync_mode": "incremental"}