The point is that if you have only x86 and ARM binaries, you can't run them directly on a new architecture. It isn't a matter of changing an implementation.
It's like trying to run a C64 binary today. The only practical way is emulation, which is slow - but thankfully fast enough in this case. In general though, it means new architectures will run more slowly than existing ones. That's not a good thing.
Do you think we still have the source code to all C64 binaries out there? We don't. And in 5 years, we won't have the source to all the stuff we are running now.
It's like trying to run a C64 binary today. The only practical way is emulation, which is slow - but thankfully fast enough in this case. In general though, it means new architectures will run more slowly than existing ones. That's not a good thing.