>Our system handles hundreds of worker nodes constantly polling our PostgreSQL-b... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

gloosx on Jan 6, 2025 | parent | context | favorite | on: Back to basics: Why we chose long-polling over web...

>Our system handles hundreds of worker nodes constantly polling our PostgreSQL-backed control plane for new jobs

Does everybody poll their PosgreSQL to get new rows in real-time? This is really weird, there are trigger functions and notifications.

lunarcave on Jan 7, 2025 [–]

In pg, each unique `listen(channel)` takes up server resources, and if you don't reliably clean them up, everything comes to a screeching halt.

There's also `max_notify_queue_pages`

>Specifies the maximum amount of allocated pages for NOTIFY / LISTEN queue. The default value is 1048576.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact