I used Delphi for more tan 10 years, so this is what I believe it went wrong:
* They lost Anders Hejlsberg
* Free compilers got aceptable.
* Lack of back compatibility, new VCL components from one version weren't compatible with older ones, every new versión require to biy the new components.
* To expensive.
* No new books to learn Delphi, they are to old.
For me (Delphi programmer until around 2001), there were several more important factors in the eventual failure of Delphi.
First, at the time Borland (or, briefly, Inprise) tried to move into the "enterprise" market, with expensive, nebulously defined "middleware" products like Midas. This is why they aggressively pursued failed tech like CORBA. It was a disaster, and had nothing to offer those who wanted to get stuff done. They also clung to BDE despite the fact that ODBC was much more mature and fast. Serious users used some ODBC components that someone built.
They forgot that Delphi was a tool to build great apps in, and didn't figure out that Delphi was great on the back end. It was all about GUI apps, but it turns out that it was incredibly productive for non-visual apps, too. With a better strategy, Delphi might have competed with Java, but they missed a lot of what made Java great in the beginning. Support for web development, even sockets, was near-non-existent.
One problem with backend apps in Delphi was that it was painful to work with C libraries. One huge thing they could have done was to make it trivial to generate C bindings, but the best they had was a rather terrible tool to convert header files to Pascal files. (I wrote a much better tool that understood macros and could even translate complex C++ headers like Microsoft's MAPI, but I'm sure Borland could have done even better.)
Delphi also tried to pursue COM (in addition to CORBA) as a distributed programming glue. Delphi 4 even had a typelib editor (which was buggy and horrible), which you needed to use to get any performance at all; the automagic "OLE Automation" support built into the language was awesome but really slow. It turns out that distributed programming with COM was not very mature, and trying to use it with Delphi was painful (though possible).
Borland also got distracted by Linux (not a good fit for a closed-source company) and by C++ (C++Builder could never be as successful as Delphi since they didn't control the language and had to extend the C++ compiler with custom directives to make it work the way Delphi did).
In the end, it's best summarized as: Lost focus, didn't realize what they had, pursued the wrong markets. Being stuck with a proprietary language didn't help, of course.
I agree with everything you're saying here except for one thing: Delphi was amazing for COM programming. In fact, it took me years of reading complaints about how complicated COM programming was (and thinking everybody is nuts) before I first saw an example of COM done with C++ and realized what people meant. We were doing a ton of COM / COM+ programming in Delphi 5 and we've never had any problems.
Having done a ton of COM programming in Delphi, the problem wasn't that it didn't work that well — the language integration was fantastic, and things like QueryInterface and AddRef/Release were done for you — but the support wasn't quite good enough.
The typelib editor, for example, was atrocious in Delphi 4 (maybe it got better in later versions), and if you wanted to use a COM-based library (MAPI, TAPI, OLE DB, etc.) you couldn't do it without the headers. Since so much of COM came from Microsoft, it felt like fighting a losing battle against the official way of doing COM programming (Visual C++, at the time). Ironically COM programming seemed much worse in VC++, given the lack of language integration.
Funny thing is that now, around 20 years later we have .NET going fully AOT with .NET Native (kind of Delphi experience) and C++/CX with XAML (kind of C++ Builder experience).
Delphi was so powerful when Win32 rich clients were king. Pre 1998 the web was not a serious development target for applications for most people. Mac OS was on a pretty large downward spiral. Linux was still under the radar (and changing too rapidly for serious closed source development).
But the web started developing, OS X started to win back the crowd and Linux showed potential (I stopped doing Delphi development when I moved to a linux box).
Since then they have added a lot of the functionality to do cross development, however a lot of it was too late, and worse a lot of it is ugly. I have never found a rich client building experience as good as Delphi. I still think it is one of the best IDE / GUI Builders packages ever assembled. However when I went for a look back at in a few years ago, the price was eye watering, and unfortunately it seems if I want that style of development my best bet now is C#.
A pet theory for why OSX and the web seems to be such friendly is that the early web was considered an extension of print (and later video/audio) media.
And as best i can tell, media production has been long held bastion of Apple.
It likely didn't hurt that OSX is deep down a BSD, and so you could switch out the L in LAMP and be on your way.
Certainly media companies have always worked on Macs, but the platform was actually quite bad for web development in the early days. The colors were wrong, the fonts were wrong, the entire scaling was wrong. Mac users had a defective version of Netscape and a uniquely defective version of IE.
It wasn't until Mac browsers adopted "standards" which matched the Windows way of doing things (96dpi, sRGB, MS fonts) that the Mac was really a viable web development platform.
Still using D7 nearly everyday, alongside XE7 at work. The impressive, yet sad, thing is 2002's Delphi 7 still runs circles around XE7. Fast, robust, sound documentation. Even Visual Studio and IntelliJ feel weird compared to the simplicity of that old IDE. Sure, lots of parts are missing, no refactoring, no reformatting, whatever. For me it's more fun to use D7 than anything else at the moment.
That's not really about the IDE side of Delphi, but the compiler and runtime side of Delphi. It was an absolutely awesome product, targeting the individual developer or small team, writing a custom application. Nothing I've ever worked since compares for that. But it's limitations are the same as Ruby on Rails limitations: Wonderful at one thing, only a headache anywhere else, and as we ask developers to do other things, the tools just stop being quite so good.
Looking at what we do on websites today, and how much it costs to build it, it sure feels like we've been taking steps backwards for quite a while.
Pretty much my view too. Anyone who knew Delphi immediately went "OMG that's Delphi/VCL with C-like syntax!" when they saw C#/.NET ca ~2000. And once Microsoft started giving away VS Express for free in 2005, Borland's original market - IDEs and compilers for the masses - was well and truly dead.
It's easy to blame management for shifting focus to enterprise software, but they had to try doing something new.
1997: "Microsoft also offered Mr. Hejlsberg a $1.5 million signing bonus, a base salary of $150,000 to $200,000 and options for 75,000 shares of Microsoft stock. After Borland's counteroffer last October, Microsoft offered another $1.5 million bonus, the complaint says."
Your description of C#/.NET sounds like it might get me to take another look at C# in the future.
I learned Windows programming on-the-job using Delphi and then thought I should investigate a "more common" C++ environment (I already knew console-based C++). Learning that _Visual_ C++ had no similar RAD environment was really shocking. My resource editor isn't attached to an an automatic code generator? Say what?
After that experience with Visual C++, I'd assumed Microsoft did roughly the same thing with "Visual C#" and never gave it a more than cursory look. I also had the fear that this new .NET thing would be yet another technology they hype then deprecate. What's the official Windows GUI framework this month? Is it Win32 Or maybe MFC? Or was it WPF or WinForms or some new XAML thing? (Sorry, went off on a tangent there...)
Yes, until C#/.NET came out, Delphi was much nicer to work with than either VB or Java. Delphi was originally conceived as a 'VB killer' and thats exactly what it was.
Not sure about the exact timeline, but I would also say that they didn't even try to stay in the "for the masses" market. Delphi 8 and later IMHO didn't offer anything compelling, then there was this confusing new thing called Delphi.NET which was Delphi but different, ... So this space stayed on Delphi 7
Even after not using it for 10 years I miss Delphi a lot. You could click a UI together in a very short time, add some code and you had a running standalone .exe without the need of any runtime dlls. Yes, Delphi it was limited in some ways and the language was not the best, but is was (and in my opinion still is) the best way to easily create a "simple" standalone gui applications.
I remember Delphi and its Microsoft counterpart Visual Basic 1-6. They were the best RAD tools to develop GUI applications. Frontpage/Dreamweaver/GoLive were RAD tools for HTML3-4 and very good too.
I currently miss the rapid development tools we had 15 years ago. There is definitely a need for a HTML5 RAD tool - ideally with the combined functionality of JetBrains WebStorm and Adobes Dreamweaver.
I'd be interested in how that'd be better, today, than C#/WinForms (WPF is nice, but not for quick things). I only used Delphi a little, but C# and WinForms are super nice for clicky things and I don't think you'll find a machine in any sort of common use without at least .NET 2.0 on it, being at least effectively standalone.
I think the biggest problem was that between delphi 7 and delphi 2010 there wasn't a version worth paying for, even if you were an enterprise shop at the core of their delphi target market. All they did was bundle components which you could have bought yourself, and tack new features on the side, while letting the core platform go basically unmaintained. After embarcadero took over, delphi releases actually improved the core again.
I don't buy the argument that the IDE market was dead. If properly managed to cater to enterprise developer's needs, delphi could have still been big. A lot of commercial software got built with delphi not because it was affordable, but because it was better than the alternatives.
I have fond memories of Delphi 3 during my high school years.
I did a marketing internship at Borland France around 2004 and Delphi was still the cash cow but a great deal of effort was put toward "enterprise" softwares like StartTeam, Together etc that were from acquired companies IIRC.
It's easy to judge now but back then the shift toward the web apps was slowly happening, C#/.Net was not especially a success ...
Sadly I always wanted to get into Delphi, but it being unaffordable (even as a student - which JetBrain and MS give me their IDE's for free...) as well as the lack of books or online documentation are a factor. I try using FreePascal with Lazarus, but Lazarus / FP still lack the proper documentation as well sadly.
That's strange, I remember myself buying an Italian dev magazine back at the end of the 90s that included a full copy of Delphi (ok, 16bit version) with application redist rights.
I think I probably paid it the equivalent of 10euro..
One thing that I liked a lot about Delphi (compared to VB for example) that there were plenty of free (sometimes with sources too) VCL components that could be downloaded and used for your own project.
I was also introduced to Delphi from a computer magazine in Denmark, they even used it as a theme in the next magazines where you would follow articles on how to code Delphi.
Same here - in Poland. We had Delphi on school computers, but obviously I couldn't "take it home". So it was really great (and surprising, too) to find it in a magazine.
I remember a few books coming out lately, Coding in Delphi from Nick Hodges comes to mind if you want to look at the latest for what Delphi has to offer.