It seems like "tracing JIT" is really just "advanced compiler hints from runtime analysis". There's no reason this couldn't be gathered by tools and saved in a file to be used by compilers. Combine this with very advanced symbolic debugging and very rapid compilation, there would no longer be any need for late-binding dynamic languages. (Says the lover of Smalltalk, no less! It would have to be with debugging on the level of Smalltalk or IPython and then some.)
And true to form, on the JRuby project we're looking into ways to optimize around where the JVM doesn't quite serve us well. I've recently been experimenting with doing my own dynamic optimization passes, and that let me to think about how to save off this dynopt information to disk for instant gratification on future runs.
I think part of the problem with the JVM is that its requirements have too long been driven by the big EE server folks, who have almost completely different requirements from day-to-day developers, client app developers, RIA developers, and so on. The key here might be making a good business case for those other domains, to help drive the JVM in the direction those domains need. I actually have hope that the (perhaps misguided) push for JavaFX at Sun/Oracle will bear fruit in the form of client-side and non-giant-server-app domains, since I know the JavaFX team have butted their heads up against a lot of the same problems we've faced in JRuby.
