[2026-05-12T21:21:01.908366] Job 156720 queued by schedule:2026-05-12T21:21:00 Connection: netflixit (instagram) Sync template: Instagram Graph - Incremental Metrics Refresh Queue class: background Procrastinate queue: kpiper-background Command: python instagram_graph_pipeline.py [2026-05-12T21:21:02.020303] Queued via Procrastinate as job 155571 (queue=kpiper-background, priority=10). [2026-05-12T21:21:02.894348] Starting job 156720 for connection netflixit (instagram) / sync template Instagram Graph - Incremental Metrics Refresh (type=dlt) Resolved command: cd /srv/kpiper && python instagram_graph_pipeline.py Injected profile context env vars (tokens exported: 1). Pipeline env overrides applied: DOWNLOAD_ASSETS, ENABLE_ACCOUNT_OBJECTS, ENABLE_ASSETS, ENABLE_COMMENTS, ENABLE_CONTENT_PUBLISHING_LIMITS, ENABLE_MEDIA_CHILD_OBJECTS, ENABLE_MEDIA_INSIGHTS, ENABLE_MEDIA_OBJECTS, ENABLE_STORY_INSIGHTS, ENABLE_STORY_OBJECTS, ENABLE_USER_INSIGHTS, ENABLE_WEBHOOK_SUBSCRIBED_APPS, IG_PAGE_ID, IG_USER_ID, MEDIA_INSIGHTS_LOOKBACK_DAYS, MEDIA_PAGINATION, SOURCES__INSTAGRAM_GRAPH_REST_API_SOURCE__ACCESS_TOKEN, SYNC_MODE Refreshed Meta Page Token at 2026-05-12T21:21:02.872812. Forced unbuffered Python output for child process. /usr/local/lib/python3.11/site-packages/dlt/pipeline/progress.py:27: UserWarning: psutil dependency is not installed and memory stats will not be available. Add psutil to your environment or pass dump_system_stats=False to disable this warning. return log() ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 0.71s | Rate: 0.00/s media_insight_values: 1 | Time: 0.00s | Rate: 104857.60/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 2.14s | Rate: 0.00/s media_insight_values: 13 | Time: 1.43s | Rate: 9.12/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 3.81s | Rate: 0.00/s media_insight_values: 28 | Time: 3.10s | Rate: 9.03/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 5.33s | Rate: 0.00/s media_insight_values: 40 | Time: 4.62s | Rate: 8.66/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 6.76s | Rate: 0.00/s media_insight_values: 52 | Time: 6.05s | Rate: 8.59/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 8.29s | Rate: 0.00/s media_insight_values: 70 | Time: 7.58s | Rate: 9.24/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 9.75s | Rate: 0.00/s media_insight_values: 85 | Time: 9.04s | Rate: 9.40/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 11.58s | Rate: 0.00/s media_insight_values: 97 | Time: 10.87s | Rate: 8.92/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 12.69s | Rate: 0.00/s media_insight_values: 106 | Time: 11.98s | Rate: 8.85/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 13.97s | Rate: 0.00/s media_insight_values: 115 | Time: 13.26s | Rate: 8.67/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 15.60s | Rate: 0.00/s media_insight_values: 133 | Time: 14.89s | Rate: 8.93/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 17.04s | Rate: 0.00/s media_insight_values: 145 | Time: 16.33s | Rate: 8.88/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 18.76s | Rate: 0.00/s media_insight_values: 160 | Time: 18.04s | Rate: 8.87/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 19.88s | Rate: 0.00/s media_insight_values: 169 | Time: 19.17s | Rate: 8.82/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 22.62s | Rate: 0.00/s media_insight_values: 178 | Time: 21.91s | Rate: 8.13/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 24.14s | Rate: 0.00/s media_insight_values: 184 | Time: 23.43s | Rate: 7.85/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 25.70s | Rate: 0.00/s media_insight_values: 196 | Time: 24.98s | Rate: 7.85/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 27.09s | Rate: 0.00/s media_insight_values: 211 | Time: 26.37s | Rate: 8.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 28.33s | Rate: 0.00/s media_insight_values: 217 | Time: 27.62s | Rate: 7.86/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 29.73s | Rate: 0.00/s media_insight_values: 235 | Time: 29.02s | Rate: 8.10/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 31.48s | Rate: 0.00/s media_insight_values: 250 | Time: 30.77s | Rate: 8.13/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 33.37s | Rate: 0.00/s media_insight_values: 265 | Time: 32.66s | Rate: 8.11/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 35.02s | Rate: 0.00/s media_insight_values: 283 | Time: 34.31s | Rate: 8.25/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 36.16s | Rate: 0.00/s media_insight_values: 289 | Time: 35.45s | Rate: 8.15/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 37.18s | Rate: 0.00/s media_insight_values: 298 | Time: 36.47s | Rate: 8.17/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 38.55s | Rate: 0.00/s media_insight_values: 310 | Time: 37.84s | Rate: 8.19/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 40.00s | Rate: 0.00/s media_insight_values: 322 | Time: 39.28s | Rate: 8.20/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 41.34s | Rate: 0.00/s media_insight_values: 337 | Time: 40.63s | Rate: 8.29/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 42.84s | Rate: 0.00/s media_insight_values: 352 | Time: 42.13s | Rate: 8.36/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 44.33s | Rate: 0.00/s media_insight_values: 367 | Time: 43.62s | Rate: 8.41/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 45.86s | Rate: 0.00/s media_insight_values: 373 | Time: 45.15s | Rate: 8.26/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 47.53s | Rate: 0.00/s media_insight_values: 385 | Time: 46.81s | Rate: 8.22/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 49.07s | Rate: 0.00/s media_insight_values: 400 | Time: 48.36s | Rate: 8.27/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 50.72s | Rate: 0.00/s media_insight_values: 415 | Time: 50.01s | Rate: 8.30/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 52.48s | Rate: 0.00/s media_insight_values: 427 | Time: 51.76s | Rate: 8.25/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 53.92s | Rate: 0.00/s media_insight_values: 442 | Time: 53.21s | Rate: 8.31/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 55.25s | Rate: 0.00/s media_insight_values: 457 | Time: 54.54s | Rate: 8.38/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 56.96s | Rate: 0.00/s media_insight_values: 472 | Time: 56.25s | Rate: 8.39/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 58.83s | Rate: 0.00/s media_insight_values: 487 | Time: 58.12s | Rate: 8.38/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 59.88s | Rate: 0.00/s media_insight_values: 496 | Time: 59.17s | Rate: 8.38/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 61.38s | Rate: 0.00/s media_insight_values: 514 | Time: 60.67s | Rate: 8.47/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 63.10s | Rate: 0.00/s media_insight_values: 529 | Time: 62.39s | Rate: 8.48/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 64.43s | Rate: 0.00/s media_insight_values: 544 | Time: 63.72s | Rate: 8.54/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 66.03s | Rate: 0.00/s media_insight_values: 556 | Time: 65.32s | Rate: 8.51/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 67.15s | Rate: 0.00/s media_insight_values: 565 | Time: 66.44s | Rate: 8.50/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 68.63s | Rate: 0.00/s media_insight_values: 583 | Time: 67.92s | Rate: 8.58/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 70.07s | Rate: 0.00/s media_insight_values: 595 | Time: 69.36s | Rate: 8.58/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 71.50s | Rate: 0.00/s media_insight_values: 601 | Time: 70.78s | Rate: 8.49/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 72.99s | Rate: 0.00/s media_insight_values: 616 | Time: 72.27s | Rate: 8.52/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 76.17s | Rate: 0.00/s media_insight_values: 628 | Time: 75.46s | Rate: 8.32/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 77.91s | Rate: 0.00/s media_insight_values: 643 | Time: 77.20s | Rate: 8.33/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 80.27s | Rate: 0.00/s media_insight_values: 661 | Time: 79.56s | Rate: 8.31/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 81.84s | Rate: 0.00/s media_insight_values: 676 | Time: 81.12s | Rate: 8.33/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 83.29s | Rate: 0.00/s media_insight_values: 694 | Time: 82.57s | Rate: 8.40/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 85.67s | Rate: 0.00/s media_insight_values: 700 | Time: 84.96s | Rate: 8.24/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 87.60s | Rate: 0.00/s media_insight_values: 718 | Time: 86.89s | Rate: 8.26/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 89.47s | Rate: 0.00/s media_insight_values: 736 | Time: 88.76s | Rate: 8.29/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 90.49s | Rate: 0.00/s media_insight_values: 745 | Time: 89.78s | Rate: 8.30/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 92.24s | Rate: 0.00/s media_insight_values: 760 | Time: 91.53s | Rate: 8.30/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 93.65s | Rate: 0.00/s media_insight_values: 775 | Time: 92.94s | Rate: 8.34/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 1/1 (100.0%) | Time: 93.68s | Rate: 0.01/s media_insight_values: 780 | Time: 92.97s | Rate: 8.39/s ------- Normalize instagram_graph_rest_api_source in 1778620898.9425063 -------- Files: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s ------- Normalize instagram_graph_rest_api_source in 1778620898.9425063 -------- Files: 0/1 (0.0%) | Time: 0.02s | Rate: 0.00/s media_insight_values: 0 | Time: 0.00s | Rate: 0.00/s ------- Normalize instagram_graph_rest_api_source in 1778620898.9425063 -------- Files: 1/1 (100.0%) | Time: 0.18s | Rate: 5.41/s media_insight_values: 780 | Time: 0.17s | Rate: 4659.46/s ---------- Load instagram_graph_rest_api_source in 1778620898.9425063 ---------- Jobs: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s ---------- Load instagram_graph_rest_api_source in 1778620898.9425063 ---------- Jobs: 2/2 (100.0%) | Time: 0.39s | Rate: 5.19/s Pipeline instagram_graph_pipeline load step completed in 0.39 seconds 1 load package(s) were loaded to destination postgres and into dataset instagram_graph The postgres destination used postgresql://kpiper_dlt_app:***@[/cloudsql/pacelab-487114:europe-west4:pacelab-pg18-ew4]:5432/kpiper_data location to store data Load package 1778620898.9425063 is LOADED and contains no failed jobs [2026-05-12T21:23:13.825336] Command completed successfully. [2026-05-12T21:23:13.825627] Pipeline status payload: {"backfill_complete": true, "backfill_progress_mode": "incremental", "ig_user_id": "17841401286038439", "sync_mode": "incremental"}