[2026-05-13T03:21:01.607633] Job 157089 queued by schedule:2026-05-13T03:21:00 Connection: netflixit (instagram) Sync template: Instagram Graph - Incremental Metrics Refresh Queue class: background Procrastinate queue: kpiper-background Command: python instagram_graph_pipeline.py [2026-05-13T03:21:01.703550] Queued via Procrastinate as job 155940 (queue=kpiper-background, priority=10). [2026-05-13T03:21:03.108823] Starting job 157089 for connection netflixit (instagram) / sync template Instagram Graph - Incremental Metrics Refresh (type=dlt) Resolved command: cd /srv/kpiper && python instagram_graph_pipeline.py Injected profile context env vars (tokens exported: 1). Pipeline env overrides applied: DOWNLOAD_ASSETS, ENABLE_ACCOUNT_OBJECTS, ENABLE_ASSETS, ENABLE_COMMENTS, ENABLE_CONTENT_PUBLISHING_LIMITS, ENABLE_MEDIA_CHILD_OBJECTS, ENABLE_MEDIA_INSIGHTS, ENABLE_MEDIA_OBJECTS, ENABLE_STORY_INSIGHTS, ENABLE_STORY_OBJECTS, ENABLE_USER_INSIGHTS, ENABLE_WEBHOOK_SUBSCRIBED_APPS, IG_PAGE_ID, IG_USER_ID, MEDIA_INSIGHTS_LOOKBACK_DAYS, MEDIA_PAGINATION, SOURCES__INSTAGRAM_GRAPH_REST_API_SOURCE__ACCESS_TOKEN, SYNC_MODE Refreshed Meta Page Token at 2026-05-13T03:21:03.086259. Forced unbuffered Python output for child process. /usr/local/lib/python3.11/site-packages/dlt/pipeline/progress.py:27: UserWarning: psutil dependency is not installed and memory stats will not be available. Add psutil to your environment or pass dump_system_stats=False to disable this warning. return log() ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 0.78s | Rate: 0.00/s media_insight_values: 1 | Time: 0.00s | Rate: 99864.38/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 2.44s | Rate: 0.00/s media_insight_values: 13 | Time: 1.66s | Rate: 7.84/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 3.57s | Rate: 0.00/s media_insight_values: 19 | Time: 2.79s | Rate: 6.81/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 5.86s | Rate: 0.00/s media_insight_values: 34 | Time: 5.08s | Rate: 6.69/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 7.87s | Rate: 0.00/s media_insight_values: 40 | Time: 7.09s | Rate: 5.64/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 9.45s | Rate: 0.00/s media_insight_values: 52 | Time: 8.67s | Rate: 6.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 10.55s | Rate: 0.00/s media_insight_values: 61 | Time: 9.77s | Rate: 6.24/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 12.58s | Rate: 0.00/s media_insight_values: 76 | Time: 11.80s | Rate: 6.44/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 14.49s | Rate: 0.00/s media_insight_values: 91 | Time: 13.71s | Rate: 6.64/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 16.31s | Rate: 0.00/s media_insight_values: 106 | Time: 15.53s | Rate: 6.82/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 17.42s | Rate: 0.00/s media_insight_values: 115 | Time: 16.64s | Rate: 6.91/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 19.15s | Rate: 0.00/s media_insight_values: 133 | Time: 18.36s | Rate: 7.24/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 21.05s | Rate: 0.00/s media_insight_values: 145 | Time: 20.27s | Rate: 7.15/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 23.39s | Rate: 0.00/s media_insight_values: 160 | Time: 22.61s | Rate: 7.08/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 24.51s | Rate: 0.00/s media_insight_values: 169 | Time: 23.73s | Rate: 7.12/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 26.55s | Rate: 0.00/s media_insight_values: 184 | Time: 25.77s | Rate: 7.14/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 27.97s | Rate: 0.00/s media_insight_values: 190 | Time: 27.19s | Rate: 6.99/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 29.03s | Rate: 0.00/s media_insight_values: 196 | Time: 28.25s | Rate: 6.94/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 30.68s | Rate: 0.00/s media_insight_values: 211 | Time: 29.90s | Rate: 7.06/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 33.06s | Rate: 0.00/s media_insight_values: 226 | Time: 32.28s | Rate: 7.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 35.17s | Rate: 0.00/s media_insight_values: 241 | Time: 34.39s | Rate: 7.01/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 36.40s | Rate: 0.00/s media_insight_values: 250 | Time: 35.62s | Rate: 7.02/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 37.56s | Rate: 0.00/s media_insight_values: 256 | Time: 36.78s | Rate: 6.96/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 39.58s | Rate: 0.00/s media_insight_values: 274 | Time: 38.80s | Rate: 7.06/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 40.88s | Rate: 0.00/s media_insight_values: 283 | Time: 40.10s | Rate: 7.06/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 42.10s | Rate: 0.00/s media_insight_values: 289 | Time: 41.32s | Rate: 6.99/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 43.60s | Rate: 0.00/s media_insight_values: 298 | Time: 42.82s | Rate: 6.96/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 44.84s | Rate: 0.00/s media_insight_values: 304 | Time: 44.06s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 46.59s | Rate: 0.00/s media_insight_values: 316 | Time: 45.81s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 47.71s | Rate: 0.00/s media_insight_values: 322 | Time: 46.92s | Rate: 6.86/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 48.75s | Rate: 0.00/s media_insight_values: 331 | Time: 47.97s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 50.37s | Rate: 0.00/s media_insight_values: 346 | Time: 49.59s | Rate: 6.98/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 51.40s | Rate: 0.00/s media_insight_values: 352 | Time: 50.62s | Rate: 6.95/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 53.20s | Rate: 0.00/s media_insight_values: 367 | Time: 52.42s | Rate: 7.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 55.30s | Rate: 0.00/s media_insight_values: 379 | Time: 54.52s | Rate: 6.95/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 56.32s | Rate: 0.00/s media_insight_values: 385 | Time: 55.54s | Rate: 6.93/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 57.47s | Rate: 0.00/s media_insight_values: 394 | Time: 56.69s | Rate: 6.95/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 59.08s | Rate: 0.00/s media_insight_values: 406 | Time: 58.30s | Rate: 6.96/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 60.42s | Rate: 0.00/s media_insight_values: 421 | Time: 59.64s | Rate: 7.06/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 61.42s | Rate: 0.00/s media_insight_values: 427 | Time: 60.64s | Rate: 7.04/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 63.88s | Rate: 0.00/s media_insight_values: 442 | Time: 63.10s | Rate: 7.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 65.02s | Rate: 0.00/s media_insight_values: 451 | Time: 64.24s | Rate: 7.02/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 66.06s | Rate: 0.00/s media_insight_values: 457 | Time: 65.28s | Rate: 7.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 67.89s | Rate: 0.00/s media_insight_values: 472 | Time: 67.11s | Rate: 7.03/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 68.93s | Rate: 0.00/s media_insight_values: 478 | Time: 68.15s | Rate: 7.01/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 70.54s | Rate: 0.00/s media_insight_values: 496 | Time: 69.76s | Rate: 7.11/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 71.95s | Rate: 0.00/s media_insight_values: 505 | Time: 71.17s | Rate: 7.10/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 73.15s | Rate: 0.00/s media_insight_values: 514 | Time: 72.36s | Rate: 7.10/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 74.46s | Rate: 0.00/s media_insight_values: 523 | Time: 73.68s | Rate: 7.10/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 75.71s | Rate: 0.00/s media_insight_values: 529 | Time: 74.93s | Rate: 7.06/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 76.73s | Rate: 0.00/s media_insight_values: 538 | Time: 75.95s | Rate: 7.08/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 78.52s | Rate: 0.00/s media_insight_values: 550 | Time: 77.73s | Rate: 7.08/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 80.22s | Rate: 0.00/s media_insight_values: 556 | Time: 79.44s | Rate: 7.00/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 82.22s | Rate: 0.00/s media_insight_values: 574 | Time: 81.44s | Rate: 7.05/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 84.23s | Rate: 0.00/s media_insight_values: 583 | Time: 83.45s | Rate: 6.99/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 85.25s | Rate: 0.00/s media_insight_values: 589 | Time: 84.46s | Rate: 6.97/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 86.29s | Rate: 0.00/s media_insight_values: 595 | Time: 85.50s | Rate: 6.96/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 87.37s | Rate: 0.00/s media_insight_values: 601 | Time: 86.59s | Rate: 6.94/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 88.37s | Rate: 0.00/s media_insight_values: 607 | Time: 87.59s | Rate: 6.93/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 89.63s | Rate: 0.00/s media_insight_values: 616 | Time: 88.85s | Rate: 6.93/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 90.71s | Rate: 0.00/s media_insight_values: 622 | Time: 89.93s | Rate: 6.92/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 91.90s | Rate: 0.00/s media_insight_values: 628 | Time: 91.12s | Rate: 6.89/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 93.14s | Rate: 0.00/s media_insight_values: 634 | Time: 92.35s | Rate: 6.86/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 95.26s | Rate: 0.00/s media_insight_values: 652 | Time: 94.48s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 96.78s | Rate: 0.00/s media_insight_values: 661 | Time: 96.00s | Rate: 6.89/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 97.85s | Rate: 0.00/s media_insight_values: 670 | Time: 97.06s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 98.92s | Rate: 0.00/s media_insight_values: 676 | Time: 98.13s | Rate: 6.89/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 100.02s | Rate: 0.00/s media_insight_values: 685 | Time: 99.24s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 101.84s | Rate: 0.00/s media_insight_values: 694 | Time: 101.05s | Rate: 6.87/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 102.95s | Rate: 0.00/s media_insight_values: 700 | Time: 102.17s | Rate: 6.85/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 105.01s | Rate: 0.00/s media_insight_values: 718 | Time: 104.23s | Rate: 6.89/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 106.14s | Rate: 0.00/s media_insight_values: 727 | Time: 105.36s | Rate: 6.90/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 107.92s | Rate: 0.00/s media_insight_values: 745 | Time: 107.14s | Rate: 6.95/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 110.26s | Rate: 0.00/s media_insight_values: 760 | Time: 109.48s | Rate: 6.94/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 111.38s | Rate: 0.00/s media_insight_values: 766 | Time: 110.60s | Rate: 6.93/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 0/1 (0.0%) | Time: 112.69s | Rate: 0.00/s media_insight_values: 775 | Time: 111.90s | Rate: 6.93/s ------------------- Extract instagram_graph_rest_api_source -------------------- Resources: 1/1 (100.0%) | Time: 112.72s | Rate: 0.01/s media_insight_values: 780 | Time: 111.94s | Rate: 6.97/s ------- Normalize instagram_graph_rest_api_source in 1778642488.7718878 -------- Files: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s ------- Normalize instagram_graph_rest_api_source in 1778642488.7718878 -------- Files: 0/1 (0.0%) | Time: 0.02s | Rate: 0.00/s media_insight_values: 0 | Time: 0.00s | Rate: 0.00/s ------- Normalize instagram_graph_rest_api_source in 1778642488.7718878 -------- Files: 1/1 (100.0%) | Time: 0.19s | Rate: 5.16/s media_insight_values: 780 | Time: 0.17s | Rate: 4568.42/s ---------- Load instagram_graph_rest_api_source in 1778642488.7718878 ---------- Jobs: 0/1 (0.0%) | Time: 0.00s | Rate: 0.00/s ---------- Load instagram_graph_rest_api_source in 1778642488.7718878 ---------- Jobs: 2/2 (100.0%) | Time: 0.39s | Rate: 5.06/s Pipeline instagram_graph_pipeline load step completed in 0.40 seconds 1 load package(s) were loaded to destination postgres and into dataset instagram_graph The postgres destination used postgresql://kpiper_dlt_app:***@[/cloudsql/pacelab-487114:europe-west4:pacelab-pg18-ew4]:5432/kpiper_data location to store data Load package 1778642488.7718878 is LOADED and contains no failed jobs [2026-05-13T03:23:22.825406] Command completed successfully. [2026-05-13T03:23:22.825693] Pipeline status payload: {"backfill_complete": true, "backfill_progress_mode": "incremental", "ig_user_id": "17841401286038439", "sync_mode": "incremental"}