> My understanding is that if I still want to ensure that the Beam VM can continue to schedule all processes efficiently
It's worse than that, overlong NIFs can lead to scheduler collapse with your schedulers being put to sleep and stuck there.
> these NIFs shouldn't calculate forever, but return quickly to avoid blocking all other processes.
Recent BEAM (R17 and later) has the experimental feature of dirty schedulers which allows running long NIFs without risking scheduler collapse. There is one set of dirty schedulers for IO-bound NIFs and one for CPU-bound NIFs, they should be marked accordingly.
> I've no idea how big the overhead actually is, I'd be interested in a rough estimate. Is it small enough that I can just ignore it entirely?
It's worse than that, overlong NIFs can lead to scheduler collapse with your schedulers being put to sleep and stuck there.
> these NIFs shouldn't calculate forever, but return quickly to avoid blocking all other processes.
Recent BEAM (R17 and later) has the experimental feature of dirty schedulers which allows running long NIFs without risking scheduler collapse. There is one set of dirty schedulers for IO-bound NIFs and one for CPU-bound NIFs, they should be marked accordingly.
> I've no idea how big the overhead actually is, I'd be interested in a rough estimate. Is it small enough that I can just ignore it entirely?
https://news.ycombinator.com/item?id=10141945 and the linked article https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea... measure regular nifs at ~1.2µs and dirty nifs at ~10µs by default though they can be as low as ~3µs for core-bound schedulers and tuned wakeup options.