But it only applies to AI controlling critical infrastructure, you think this is an issue in practice?
I would think if a power plant deploys some AI model to optimize something or other, it would be on the plant operator to perform the reviews, regardless of who they get the AI from.
In practice, there will only be one or two "safe" AI vendors approved for such infrastructure. On one hand, that's probably a good thing. On the other hand, it's deeply anti-competitive and it's pretty much a recipe for indefinitely renewable contracts at arbitrary high prices that get passed on to taxpayers.
I would think if a power plant deploys some AI model to optimize something or other, it would be on the plant operator to perform the reviews, regardless of who they get the AI from.