If I'm understanding their comments correctly it's because the downside isn't just a possible not-increase/decrease in performance it's instability and unpredictable behavior. I worry that it could translate into those vague and difficult to reproduce "the application is weird/slow" reports.

Of course you could profile and measure performance to determine if the warning is applicable but is that something I should be doing for every part of the stack? I should but should I prioritize that over x, y or z?