[2026-05-13T07:32:01.739661] Job 157379 queued by schedule:2026-05-13T07:32:00
Connection: pietrocoricelli (instagram)
Sync template: Instagram Graph - Incremental Sync
Queue class: foreground
Procrastinate queue: kpiper
Command: python instagram_graph_pipeline.py
[2026-05-13T07:32:01.829615] Queued via Procrastinate as job 156230 (queue=kpiper, priority=50).
[2026-05-13T07:38:37.494192] Starting job 157379 for connection pietrocoricelli (instagram) / sync template Instagram Graph - Incremental Sync (type=dlt)
Resolved command: cd /srv/kpiper && python instagram_graph_pipeline.py
Injected profile context env vars (tokens exported: 1).
Pipeline env overrides applied: DOWNLOAD_ASSETS, ENABLE_ACCOUNT_OBJECTS, ENABLE_ASSETS, ENABLE_COMMENTS, ENABLE_CONTENT_PUBLISHING_LIMITS, ENABLE_MEDIA_CHILD_OBJECTS, ENABLE_MEDIA_INSIGHTS, ENABLE_MEDIA_OBJECTS, ENABLE_STORY_INSIGHTS, ENABLE_STORY_OBJECTS, ENABLE_USER_INSIGHTS, ENABLE_WEBHOOK_SUBSCRIBED_APPS, IG_PAGE_ID, IG_USER_ID, MEDIA_INCREMENTAL_LOOKBACK_DAYS, MEDIA_PAGINATION, SOURCES__INSTAGRAM_GRAPH_REST_API_SOURCE__ACCESS_TOKEN, SYNC_MODE
Refreshed Meta Page Token at 2026-05-13T07:38:37.471329.
Forced unbuffered Python output for child process.
/usr/local/lib/python3.11/site-packages/dlt/pipeline/progress.py:27: UserWarning: psutil dependency is not installed and memory stats will not be available. Add psutil to your environment or pass dump_system_stats=False to disable this warning.
  return log()
------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/6 (0.0%) | Time: 0.00s | Rate: 0.00/s

Retrying 17841400043394895 with a narrower field set after Graph error: {"error":{"message":"(#100) Tried accessing nonexisting field (account_type)","type":"OAuthException","code":100,"fbtrace_id":"A2hPMwHIiq8I0WupWZirE5D"}}
------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/6 (0.0%) | Time: 0.67s | Rate: 0.00/s
account_objects: 1  | Time: 0.00s | Rate: 95325.09/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/6 (0.0%) | Time: 0.68s | Rate: 0.00/s
account_objects: 1  | Time: 0.00s | Rate: 266.41/s
media_objects: 1  | Time: 0.00s | Rate: 95325.09/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 1/6 (16.7%) | Time: 0.68s | Rate: 1.46/s
account_objects: 1  | Time: 0.01s | Rate: 108.48/s
media_objects: 1  | Time: 0.01s | Rate: 182.65/s
story_objects: 1  | Time: 0.00s | Rate: 93206.76/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 1/6 (16.7%) | Time: 0.69s | Rate: 1.45/s
account_objects: 1  | Time: 0.01s | Rate: 69.74/s
media_objects: 1  | Time: 0.01s | Rate: 94.39/s
story_objects: 1  | Time: 0.01s | Rate: 194.90/s
content_assets: 1  | Time: 0.00s | Rate: 97541.95/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 1/6 (16.7%) | Time: 5.98s | Rate: 0.17/s
account_objects: 1  | Time: 5.31s | Rate: 0.19/s
media_objects: 1  | Time: 5.30s | Rate: 0.19/s
story_objects: 1  | Time: 5.30s | Rate: 0.19/s
content_assets: 1  | Time: 5.29s | Rate: 0.19/s
story_insight_values: 1  | Time: 0.00s | Rate: 83886.08/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 2/6 (33.3%) | Time: 5.99s | Rate: 0.33/s
account_objects: 1  | Time: 5.31s | Rate: 0.19/s
media_objects: 3  | Time: 5.31s | Rate: 0.57/s
story_objects: 3  | Time: 5.30s | Rate: 0.57/s
content_assets: 3  | Time: 5.30s | Rate: 0.57/s
story_insight_values: 2  | Time: 0.01s | Rate: 299.16/s
insight_fetch_skips: 1  | Time: 0.00s | Rate: 76260.07/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 3/6 (50.0%) | Time: 12.53s | Rate: 0.24/s
account_objects: 1  | Time: 11.85s | Rate: 0.08/s
media_objects: 3  | Time: 11.85s | Rate: 0.25/s
story_objects: 7  | Time: 11.84s | Rate: 0.59/s
content_assets: 7  | Time: 11.84s | Rate: 0.59/s
story_insight_values: 2  | Time: 6.55s | Rate: 0.31/s
insight_fetch_skips: 5  | Time: 6.54s | Rate: 0.76/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 4/6 (66.7%) | Time: 20.31s | Rate: 0.20/s
account_objects: 1  | Time: 19.64s | Rate: 0.05/s
media_objects: 3  | Time: 19.63s | Rate: 0.15/s
story_objects: 11  | Time: 19.63s | Rate: 0.56/s
content_assets: 13  | Time: 19.62s | Rate: 0.66/s
story_insight_values: 2  | Time: 14.33s | Rate: 0.14/s
insight_fetch_skips: 11  | Time: 14.32s | Rate: 0.77/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 4/6 (66.7%) | Time: 26.14s | Rate: 0.15/s
account_objects: 1  | Time: 25.47s | Rate: 0.04/s
media_objects: 3  | Time: 25.46s | Rate: 0.12/s
story_objects: 11  | Time: 25.46s | Rate: 0.43/s
content_assets: 19  | Time: 25.45s | Rate: 0.75/s
story_insight_values: 2  | Time: 20.16s | Rate: 0.10/s
insight_fetch_skips: 17  | Time: 20.15s | Rate: 0.84/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 4/6 (66.7%) | Time: 31.83s | Rate: 0.13/s
account_objects: 1  | Time: 31.16s | Rate: 0.03/s
media_objects: 3  | Time: 31.15s | Rate: 0.10/s
story_objects: 11  | Time: 31.15s | Rate: 0.35/s
content_assets: 25  | Time: 31.14s | Rate: 0.80/s
story_insight_values: 2  | Time: 25.85s | Rate: 0.08/s
insight_fetch_skips: 23  | Time: 25.84s | Rate: 0.89/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 38.70s | Rate: 0.13/s
account_objects: 1  | Time: 38.02s | Rate: 0.03/s
media_objects: 3  | Time: 38.02s | Rate: 0.08/s
story_objects: 11  | Time: 38.01s | Rate: 0.29/s
content_assets: 26  | Time: 38.01s | Rate: 0.68/s
story_insight_values: 2  | Time: 32.72s | Rate: 0.06/s
insight_fetch_skips: 29  | Time: 32.71s | Rate: 0.89/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 44.39s | Rate: 0.11/s
account_objects: 1  | Time: 43.72s | Rate: 0.02/s
media_objects: 3  | Time: 43.71s | Rate: 0.07/s
story_objects: 11  | Time: 43.71s | Rate: 0.25/s
content_assets: 26  | Time: 43.70s | Rate: 0.59/s
story_insight_values: 2  | Time: 38.41s | Rate: 0.05/s
insight_fetch_skips: 35  | Time: 38.40s | Rate: 0.91/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 49.27s | Rate: 0.10/s
account_objects: 1  | Time: 48.60s | Rate: 0.02/s
media_objects: 3  | Time: 48.60s | Rate: 0.06/s
story_objects: 11  | Time: 48.59s | Rate: 0.23/s
content_assets: 26  | Time: 48.59s | Rate: 0.54/s
story_insight_values: 2  | Time: 43.29s | Rate: 0.05/s
insight_fetch_skips: 41  | Time: 43.29s | Rate: 0.95/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 54.87s | Rate: 0.09/s
account_objects: 1  | Time: 54.19s | Rate: 0.02/s
media_objects: 3  | Time: 54.19s | Rate: 0.06/s
story_objects: 11  | Time: 54.19s | Rate: 0.20/s
content_assets: 26  | Time: 54.18s | Rate: 0.48/s
story_insight_values: 2  | Time: 48.89s | Rate: 0.04/s
insight_fetch_skips: 47  | Time: 48.88s | Rate: 0.96/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 59.12s | Rate: 0.08/s
account_objects: 1  | Time: 58.45s | Rate: 0.02/s
media_objects: 3  | Time: 58.44s | Rate: 0.05/s
story_objects: 11  | Time: 58.44s | Rate: 0.19/s
content_assets: 26  | Time: 58.43s | Rate: 0.44/s
story_insight_values: 2  | Time: 53.14s | Rate: 0.04/s
insight_fetch_skips: 53  | Time: 53.14s | Rate: 1.00/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 5/6 (83.3%) | Time: 65.47s | Rate: 0.08/s
account_objects: 1  | Time: 64.80s | Rate: 0.02/s
media_objects: 3  | Time: 64.79s | Rate: 0.05/s
story_objects: 11  | Time: 64.79s | Rate: 0.17/s
content_assets: 26  | Time: 64.78s | Rate: 0.40/s
story_insight_values: 2  | Time: 59.49s | Rate: 0.03/s
insight_fetch_skips: 59  | Time: 59.48s | Rate: 0.99/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 6/6 (100.0%) | Time: 65.50s | Rate: 0.09/s
account_objects: 1  | Time: 64.82s | Rate: 0.02/s
media_objects: 3  | Time: 64.82s | Rate: 0.05/s
story_objects: 11  | Time: 64.82s | Rate: 0.17/s
content_assets: 26  | Time: 64.81s | Rate: 0.40/s
story_insight_values: 2  | Time: 59.52s | Rate: 0.03/s
insight_fetch_skips: 64  | Time: 59.51s | Rate: 1.08/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s
_dlt_pipeline_state: 1  | Time: 0.00s | Rate: 116508.44/s

------------------- Extract instagram_graph_rest_api_source --------------------
Resources: 1/1 (100.0%) | Time: 0.02s | Rate: 53.92/s
_dlt_pipeline_state: 1  | Time: 0.01s | Rate: 71.61/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 0/7 (0.0%) | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 0/7 (0.0%) | Time: 0.02s | Rate: 0.00/s
account_objects: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 1/7 (14.3%) | Time: 0.03s | Rate: 36.38/s
account_objects: 1  | Time: 0.01s | Rate: 157.35/s
media_objects: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 2/7 (28.6%) | Time: 0.03s | Rate: 60.92/s
account_objects: 1  | Time: 0.01s | Rate: 85.49/s
media_objects: 3  | Time: 0.01s | Rate: 560.81/s
story_objects: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 3/7 (42.9%) | Time: 0.06s | Rate: 50.50/s
account_objects: 1  | Time: 0.04s | Rate: 26.12/s
media_objects: 3  | Time: 0.03s | Rate: 93.95/s
story_objects: 11  | Time: 0.03s | Rate: 413.65/s
content_assets: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 4/7 (57.1%) | Time: 0.07s | Rate: 59.67/s
account_objects: 1  | Time: 0.05s | Rate: 21.78/s
media_objects: 3  | Time: 0.04s | Rate: 75.83/s
story_objects: 11  | Time: 0.03s | Rate: 321.45/s
content_assets: 26  | Time: 0.01s | Rate: 3404.36/s
story_insight_values: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 5/7 (71.4%) | Time: 0.07s | Rate: 69.89/s
account_objects: 1  | Time: 0.05s | Rate: 19.84/s
media_objects: 3  | Time: 0.04s | Rate: 68.08/s
story_objects: 11  | Time: 0.04s | Rate: 284.05/s
content_assets: 26  | Time: 0.01s | Rate: 2141.04/s
story_insight_values: 2  | Time: 0.00s | Rate: 442.95/s
insight_fetch_skips: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 6/7 (85.7%) | Time: 0.08s | Rate: 73.70/s
account_objects: 1  | Time: 0.06s | Rate: 16.59/s
media_objects: 3  | Time: 0.05s | Rate: 55.62/s
story_objects: 11  | Time: 0.05s | Rate: 226.37/s
content_assets: 26  | Time: 0.02s | Rate: 1181.25/s
story_insight_values: 2  | Time: 0.01s | Rate: 139.06/s
insight_fetch_skips: 64  | Time: 0.01s | Rate: 6480.50/s
_dlt_pipeline_state: 0  | Time: 0.00s | Rate: 0.00/s

------- Normalize instagram_graph_rest_api_source in 1778657927.3840208 --------
Files: 7/7 (100.0%) | Time: 0.10s | Rate: 71.89/s
account_objects: 1  | Time: 0.08s | Rate: 13.12/s
media_objects: 3  | Time: 0.07s | Rate: 42.92/s
story_objects: 11  | Time: 0.06s | Rate: 170.41/s
content_assets: 26  | Time: 0.04s | Rate: 684.81/s
story_insight_values: 2  | Time: 0.03s | Rate: 65.92/s
insight_fetch_skips: 64  | Time: 0.03s | Rate: 2477.55/s
_dlt_pipeline_state: 1  | Time: 0.02s | Rate: 62.64/s

---------- Load instagram_graph_rest_api_source in 1778657927.3840208 ----------
Jobs: 0/7 (0.0%) | Time: 0.00s | Rate: 0.00/s

---------- Load instagram_graph_rest_api_source in 1778657927.3840208 ----------
Jobs: 12/12 (100.0%) | Time: 0.60s | Rate: 20.15/s

Pipeline instagram_graph_pipeline load step completed in 0.60 seconds
1 load package(s) were loaded to destination postgres and into dataset instagram_graph
The postgres destination used postgresql://kpiper_dlt_app:***@[/cloudsql/pacelab-487114:europe-west4:pacelab-pg18-ew4]:5432/kpiper_data location to store data
Load package 1778657927.3840208 is LOADED and contains no failed jobs
[2026-05-13T07:39:54.408784] Command completed successfully.
[2026-05-13T07:39:54.409146] Pipeline status payload: {"backfill_complete": true, "backfill_progress_mode": "incremental", "ig_user_id": "17841400043394895", "sync_mode": "incremental"}