Or perhaps, just perhaps, the true higher-dimensional move is realizing that choice of programming language isn’t usually the critical factor in whether a project, system, or business succeeds or fails, and that obsessing over the One True Way is a trap.
It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world.
The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.
The reality of modern software development is that most people focus on languages they use for work, and developers are statistically likely to be employed at companies with large numbers of other developers.
The technical merits of languages just aren't relevant to choosing them for most developers, unless they're helping solve a people problem.
"Artisanal" languages like Lisp, and Forth can be fantastic at solving problems elegantly, but that's not the most important thing to optimize for in big organizations where a large portion of your time is spent reading code written by people you've never met who may not have known what they were doing.
Many of the tools that come from big tech are designed to ease the challenges of organizational scale. Golang enforces uniform styles so that you don't have idiosyncratic teams doing their own things. Bazel is a largely language agnostic build system, with amazing build farm support. Apple and Google have both contributed heavily to sanitizers and standard library hardening in order to detect/eliminate issues without reading the code. Facebook has poured vast resources into automatic static analysis. AWS built an entire organization around treating all their internal interfaces the same as external ones.
> "Artisanal" languages like Lisp, and Forth can be fantastic at solving problems elegantly, but that's not the most important thing to optimize for in big organizations ... Many of the tools that come from big tech are designed to ease the challenges of organizational scale.
I think the field of programming languages has grown enough that we have to start acknowledging the future of programming largely won't be in the context of what it means for devs working at large corporations. One of my favorite talks is from Amy J. Ko called A Human View of Programming [1], which argues there are many other ways to look at programming than "tool for generating business activity" and "mathematical construct", which heretofore have been the dominant views of programming languages.
Because there are so many other forms and purposes programming languages can and will take (she goes through them in the talk), so evaluating them and creating them solely on how well they are able to fit into a corporate R&D pipeline is a very narrow and short-term view of the field.
Indeed, it's been the case for a long time now that most people who write programs are not in fact professional software developers. The most used language in the world is Excel, by several orders of magnitude, and it's the opposite of everything devs say a "proper" language must be. There's something we as a field still need to learn from that.
I have very mixed feelings on this topic, starting with how you quantify and weigh something like "most used" for a programming language. To me, the claim feels almost as much a non sequitur as saying the most used building material in the western world is Legos blocks or Play-Doh...
Is the most used bridge-building technique a plank over a small culvert, or the properly engineered bridge that carries constant, multi-lane highway traffic for a century? How do we weigh the usage of resulting products into the usage of a design and production method? Should we consider the number of program users? The users X hours of usage?
Fundamentally, the software field is still just so young and we haven't teased apart the "obvious" different domains and domain rules that we have for production of different material goods. In some sense, the domains and domain rules for material goods emerge out of the connection to culture, economic roles, health, and safety aspects. Whether it falls into civil engineering, building codes, transporation rules, consumer product safety, food and drug, ...
The self-similar way that software can be composed into systems also makes it confusing to categorize. Imagine if we talked about other crafts the same way, and conflated textile manufacturing, clothing design, tailoring, costume making, wardrobe management, scripting, choreography, acting, and dancing as a single field that coordinates the visual movement of fabric on a stage.
> how you quantify and weigh something like "most used" for a programming language.
Define it as # of people who possess the knowledge and resources to effectively use said language to solve a problem they have in their actual lives.
> Fundamentally, the software field is still just so young and we haven't teased apart the "obvious" different domains and domain rules that we have for production of different material goods.
I think we're saying the same thing here from different angles. I said it's developed enough that we can see there are very different ways of doing things. You said it's young enough that we don't know all the different things there are.
As a member of the handmade community, I certainly hope that corporate constraints aren't the main future of the field. I just think it's a major part of the answer as it stands today.
no way? excel? and corporate programmers are not the majority of programmers? -- i mean im a non-corp programmer but i thought i was a special snowflake
Clojure is built on dynamic typing. This is pain. I wrote enough Python (pre-mypy), Javascript, and elisp to say this. Past certain size a dynamically typed codebase becomes needlessly hard to wrangle because of that. Hence the success of Python type annotations and Typescript.
Instead, the world should have seen the light of Hindley-Milner type systems, ML-inspired languages, immutability, or at least not sharing mutable state. Did Haskell fail? Hmm, let's look at Typescript and Rust.
Don't get me wrong, a Lisp is always a great and fun language, and you can write whatever DSL you might like on top of it. But the old joke that "a Lisp programmer knows the value of everything, and the cost of nothing" still has quite a bit of truth to it.
The reason I switched from Scheme to Common Lisp was because I wanted type checking more than I wanted hygienic macros or case-sensitive (by default) symbols.
If we approach the question as engineers, scientifically, with numbers and studies, not anecdotes and hand-waving, then Clojure is hands down the best language in terms of productivity and bug reduction.
To this day, I know of no study that was able to demonstrate superiority of statically-typed languages - [1].
What studies clearly show, is that both in terms of productivity [2] and bug reduction [3], expressivity reigns supreme.
And Clojure is the most expressive [4] out of languages that can leverage huge ecosystems (Java and JS, soon C++ through Jank dialect).
Plenty of ways to define complex data shapes in Clojure
Spec is definitely underrated here considering it's built into the language and has a wider scope but for most people they want the intellisense experience which you can get with clj-kondo + mailli but is not built in so most teams don't use it, fair enough
I'd like to move the goal posts though and say I want flowstorm in every (any other?!) language
I can just run the program and scrub backwards and forwards through the execution and look at all the immutable values frame by frame with a high level UI with plenty of search/autocomplete options
For program understanding there's nothing better
The fact I can program against the timeline of values of my program and create custom UI on top is crazy
One of the most mind blowing demos to me was Bret Victor's inventing on principle and having a programmable reverse debugger for your language makes those demos viable
I built an emulator recently for work that replays what happens on live locally, combined with flowstorm I can go line by line and tell you exactly what happened and why, no print statements no reruns with my own custom UI customised to our apps interesting parts
This is my appeal to anyone outside of Clojure please build flowstorm for JavaScript and or Python
The design of flowstorm is definitely helped by the fact that 95% of Clojure programs are immutable but I don't think it's impossible to replicate just very difficult
On the other hand, it would be easier to add type checking to a Lisp than it was to Python or JavaScript, and I don’t know any technical reason you couldn’t. A little Googling shows it’s been experimented with several times.
But the real strength of Lisp is in the macros, the metaprogramming system. And I suspect that typing most macros properly would be a bit less trivial than even typing of complex generic types, like lenses. Not typing a macro, and only typechecking the macroexpansion would formally work, but, usability-wise, could be on par with C++ template error reporting.
My point was that you could implement type checking with macros, not that you could type check macros. (Though that would be cool!) As opposed to having to change the language definition first (Python) or implement an entirely new compiler (TypeScript).
Certainly you can implement the typechecker with macros, but it should also work on macros, before expansion. That is, you likely want (-> ...) typechecked as written, not (only) as expanded, and typing errors reported on the non-expanded form.
"It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world."
This is me to a T — even when I'm building hobby projects. The point of writing any code, for me, is most of all to see a certain idea to fruition, so I choose what will make me most productive getting where I want to go. And while I still worship at the altar of Common Lisp as an incredibly good language, the language matters much less than the libraries, ecosystem, and documentation for productivity (or even effective DSL style abstraction level!), so eventually I have had to make my peace with Python, TypeScript, and Rust.
Tacking on, part of seeing it to fruition, and continued lifetime, is to ensure you can communicate the intent and operation to a large group of potential successors and co-workers.
An incredible epiphany that you can't transmit may not be as useful as a a moderately clever idea you can.
I think the missing piece is that "more expressive" languages do not automatically create more value at the team or company level.
Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions, but they also increase cognitive load and ramp up cost. In most orgs with churn and constant hiring, you do not get to enjoy "we built great abstractions and now we are fast". You live in "someone new is trying to understand what the last person did".
That is why hand holding and guard rails win. Not because Python or similar are technically superior, but because they support a commoditised, fungible workforce. Even if a wizard in a high dimension language is 2x more productive, that does not necessarily beat a slightly larger team in a mainstream language once you factor in turnover and ramp up. Companies mostly optimise for business impact, predictable delivery, and ease of staffing, not for maximising the ceiling of the top few programmers.
That said, at the individual level, as a programmer, you definitely can benefit from learning and mastering those added dimensions, even if you are to never use them again professionally, they expand your mindset.
> Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions ... You live in "someone new is trying to understand what the last person did".
In a language like C++, Rust or even Haskell, those "powerful abstractions" are mostly about building libraries, not application code. You still benefit from powerful libraries that were built as part of the ecosystem, even while keeping your own application code simple and intuitive to ease onboarding of new coders. Part of the "powerful abstractions" is to enable a "hand-holding and guard rails" approach for application code that's going to interface with that full-featured library.
There are several languages that I could use and be economically successful with, but I refuse to use because I consider them to be poorly designed.
Using a bad language for 8 hours a day makes me irritable and it's impossible to prevent that irritability from overflowing into my interactions with other people. I'd rather that my conversations with the computer be joyful ones.
Most of the time when someone adds these fancy languages what happens is that they leave and the ones left are the ones that have to deal with the shit that was produced.
I'm going through this now, having to deal with code nobody wants to touch because it is overly complex, has no documentation, and is in a language no one else knows. Now, whenever i see an effort like this, to bring an exoteric language for absolutely no good reason, i try to kill it as fast as possible.
I don't want to be the victim of this code in the future or have my team bear the cost of maintaining stuff they don't understand.
Sadly, much as I love Forth, it's kind of the same thing. It's an awesome language and it's a great way to bring up bare metal to a functional state, but who does that these days?
I could probably include Forth as a scripting language in a bigger app, but that app is probably going to want more complex variables than machine word size ints, and fixed-length strings. So, oh dear, Forth's not a great fit for that, and everyone just uses Lua anyway, so Lua it is.
Which is a pity, because I like Forth, and I used to to create possibly the nerdiest project on Github. I like Forth a lot, and I'd encourage anyone curious about how you get from "chunk of thinking sand and copper" to "thing I can type commands in" to have a crack at it - it's easy enough to implement your own, just to see how it's done.
But I don't expect anyone else to jump up and like it too, just because I said it's cool.
While what you say is true (I’ve used Lisps for 40 years and here I am writing Rust), the people who consciously make that choice are a tiny niche. There are vastly more people who don’t and can’t make that choice because they don’t have 1-3. So the empirical evidence for what’s actually critical is pretty slim.
yes. and as a long time lisper, i don't think that it's the macros.
i think lisp's magic is a lot more cultural than most people think. i.e. how lispnicks implement lisps and the ecosystem around it. how easy it is to walk the entire ladder of abstractions from machine code to project specific DSL's. how pluggable its parsing pipeline is -- something that is not even exposed in most languages, let alone customizable.
the language, the foundation, of course matters. but i think to a lesser extent than what people think. (hence the trend of trying to hire lispnicks to hard, but non-lisp positions?)
and it's not even an obviously good culture... (just how abrasive common lispers are? need to have a thick skin if you ask a stupid question... or that grumpy, pervasive spirit of the lone wolf...?)
maybe it's just a peculiar filter that gets together peculiar people who think and write code in peculiar ways.
maybe it's not the macros, but the patterns in personality traits of the people who end up at lisp?
> The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.
The talk I posted from Alan Kay is the steel man. I think you've missed the essence of TFA because it's not really about Clojure or lisp.
You may need to explain more? I don’t think I missed the big idea - the metaphor of a separate plane or higher dimension that contains ideas not expressible in the ordinary one is a nice metaphor, and does apply well to some things (Kuhn’s paradigms in history of science come to mind, e.g. Newtonian Mechanics versus Relativity). I just don’t think it really applies well here. What business concepts or thoughts can you express in Clojure that you can’t express in Python or Rust?
Not GP, but … because the overwhelming majority of programming is done in support of businesses selling things?
I’m not just talking about people who program for a living. The majority of academic CS chooses its research directions because of what limits people are running into for business; even privacy-focused software has been commoditized by many business; a large amount of OSS development is by (and often paid for by the employers of) people working for money; heck, after Linus’s initial “just a hobby OS” period even Linux’s contribution base was primarily driven by business needs (even if those needs influenced it via “contributor had a problem at work and committed a solution for it upstream in spare time” as often as “paid contributor upstreamed a change on behalf of their employer”).
The post isn't about Clojure or Lisp, it's about the author's journey as a programmer, and the mind-bending effect learning a Lisp had on their development. They're still in the midst of figuring it out, but a lot of people have been on this path before them. TFA has been written over the years by various authors about Prolog, or Haskell, or Smalltalk. In my case I would have written it about Lucid.
The interesting bit here isn't related to Clojure or Lisp, that's what people are chewing on because it's the surface level topic of the essay. The thing that interests me about this post is how it touches on the psychedelic nature of learning programming languages and what that does to one's perspective as a programmer.
So when you ask "what business thoughts can you express in the language", my response is it's not about what you can express, but more about "what new thoughts / ways of thinking has the experience of learning the language caused you to become aware of?".
Few people can go their whole lives writing Python and think all the possible thoughts there are to think about the shapes and forms programming can take. It's hard to develop a good sense for your own practice of programming if you never step outside and see it from other perspectives. It often takes exposure to completely new languages with different design points and abstractions to really give one perspective about their own practice.
The easiest one related to lisp is just the form of the syntax, which is surprising to many students. Most programmers don't even consider you can write (+ 1 1) and that's the same thing as (1 + 1). They don't think about the pros, or the cons, or why one might be better than the other, because every language they've used and ever will use writes it as (1 + 1). But as soon as they see Lisp, they immediately see something that changes their perspective about everything they have previous learned, and therefore will reshape how they approach programming in the future. It doesn't have to mean they will use Lisp going forward, but it does mean they will program in their language of choice with greater purpose. That's how we each hone our craft.
Add on homoiconicity, read/eval, programming as manipulating an AST, and meta programming, and you've got yourself a righteous trip.
Now, can people learn about those things from other sources, without encountering Lisp? Of course. Can you express those ideas in mainstream languages? Yes. But the point is many devs don't think those thoughts until they've had a psychedelic experience, it's very common, that we need to consider that it's part of one's growth journey as a programmer to have these experiences, so we should encourage it. The author of TFA doesn't have experience enough to make that point, as they are on the journey, which is why I brought Alan Kay's talk into context, since his perspective is from the other end of the journey.
That's a bit of an ad feminam attack, isn't it? Just because I used the phrase "business concepts", somehow money is the only thing I care about when it comes to language choice? And yet, in my top-level post I said I went and learned lisp and clojure and read SCIP, and I will add that I did both of those things for fun. So no, I don't only think of programming languages as a way to make money. Elegance and expressiveness are interesting for their own sake. I trained as a mathematician; of course I think that.
But TFA was riffing on Paul Graham's old essay Beating the Averages, which argued precisely that the expressiveness of Lisp gave his startup a business edge. That was the context of my comment. I'd add that most of what most of us do in our day jobs is to use programming languages to make money, and there's no shame in that at all. And if you want to talk about why certain languages get widespread adoption and others not, you have to talk about the corporate context: there is no way around it.
But I'll rephrase my question, just for you: "what abstract problems can you solve or thoughts can you express in Clojure that you can’t express in Python or Rust?"
I’m sympathetic to looking down on the obsession with money. But there’s something deep and important about the monetary element. Engineering is about solving real-world, practical problems. The cost is a real factor in whether a potential solution is a useful one.
I think the money question is a red herring here. I’d phrase it more like: what problem in a user’s problem space is expressible only like this? And if the only user is the programmer, that’s alright, but feels more aligned with pure academia. That’s important, too! But has a much smaller audience than engineering at large.
some people only think about life as a way to make money. unfortunately coding was best-in-slot career for too long and these kinds of people hijacked the culture.
Big lisp guy here. Have written tens of thousands of lines of scheme, at least, and common lisp.
But I don't get this "Lisp is so much better than everything else," thing. It feels very jejune to me.
Most lisp programmers barely use macros and most programming languages these days have most of the features of Lisp that originally made it useful (automatic memory management, repls, dynamic typing*, and even meta-programming if you really want it).
I do think that most common languages are mediocre but mediocrity is just how humans are.
--
If I had one thing I want fixed about Scheme it would be the dynamic typing, especially since many Schemes compile aggressively. Finding bugs is much harder when your apparently dynamic language has compiled out everything useful for understanding an error condition. Most of those mistakes could be caught at compile time.
> None of the macro systems in those languages are really equivalent to lisp macros.
Sure, but we’d hardly expect equivalence in such different languages.
I’ve written macros in all the languages I’ve mentioned, as well as Scheme’s syntax-rules and syntax-case, and in practice I don’t see some big advantage of any of them.
What actual useful macros do you write in Lisp, that would also be useful in one of those other languages, but can’t be expressed by their macro systems? Not saying it’s not possible to come up with theoretical examples, but what’s going to actually impact a programmer in those languages?
> If you really want static typing, Lisp macros are powerful enough to implement a full ML type system in userspace as a library.
On the other side of the fence, using Camlp5 you can implement Lisp or Scheme syntax and have OCaml run it directly. I did a toy implementation of that for Scheme years ago using Camlp4. You could easily implement syntax-rules macros in that environment. I enjoy games like that, but I don’t see much impact for real-world programming.
I think Coalton is cool, but I think you start to run into the limitations of Lisp syntax. I’d rather use Haskell to write that kind of code, it’s more concise. And to quote Paul Graham, succinctness is power. (PG does not endorse this message!)
> The ability to manipulate compile-time so effortlessly is a new dimension of programming. This new dimension enables you to write fundamentally better code that you’ll never be able to achieve in a lower dimension.
Show me. Specifically, material outcomes that I will care about.
There are quite a few programmers who say lisp led to early retirement. That was a pretty interesting idea to me. I like going to the beach a lot.
I am not so sure about people who don’t want to get done: if you like doing what the ticket says instead of the other way around lisp probably isn’t going to be something you’re interested in.
"Lisp makes people rich, and I love being rich. Using Lisp actually can't help but make you rich. But I can't actually provide any examples of that happening or how they might translate to anyone else. Get so rich with Lisp. Lisp."
Serialization & deserialization, for instance. Macros are great for generating ser/de hooks automatically.
Thing is, other languages do this with metaprogramming or explicit codegen. Everyone needs metaprogramming sometimes—that's why everything supports it, actually.
Lisp has been around for 65 years (not 50 as in the author believes), and is one of the very first high-level programming languages. If it was as great as its advocates say, surely it would have taken over the world by now. But it hasn't, and advocates like PG and this article author don't understand why or take any lessons from that.
If something is marginally better, it's not guaranteed to win out because markets aren't perfectly rational. However if something is 10x better than its competitors it will almost always win.
The sketch here would be that Lisps used to be exceptionally resource-intensive, allowing closer-to-metal languages to proliferate and become the default. But nowadays even Common Lisp is a simple and lightweight language next to say Python or C++. Still it's hard to overcome the inertia of the past's massive investments in education in less abstraction-friendly languages.
I take Lisp more like artisanal work. It actually requires more skill and attention to use, but in good hands it can let someone really deliver a lot quickly.
That said, like in anything else, this kind of craftsmanship doesn't translate to monetization and scale the markets demands. What markets want is to lower barrier for entry, templatize, cheapen things, and so on.
It's normal then that languages optimized for the lowest common denominator, with less expressive power and more hand holding have won in popularity in enterprise and such, where making money is the goal, but that Lisp remains a strong and popular language for the enthousiasts looking to level up their craft or just geek out.
the silent assumption in both of your perspectives is that the current monetary system is an even playing field when it comes to this context (corporations and their programmers)
In the pro-macro camp, if languages like JS had macros, the language could be kept much simpler, leaving things like pipeline operators, async / await, etc to developers.
In the anti-macro camp, they’re hard to write, reason about, and debug stack traces. They are also tempting to use when you shouldn’t, and I think a lot of software shops would run into trouble with them.
Regarding Clojure, I wouldn’t call Clojure a write-only language, but I did find that my Clojure code was more inscrutable than my code in other languages— roughly as inscrutable as my Haskell code. Something about it makes me want to code-golf my way into tiny little clever solutions.
Also, I’ve been burnt by various pitfalls of dynamically typed languages— upgrading dependencies in large dynamically typed projects, etc. I’ll take static types over macros any time.
Also, Clojure’s start up time was off-putting, and would probably be even more so today, coming from Bun and Go.
These days, most of my work is in TypeScript, and it’s just fine. Not perfect, but fine. I haven’t missed macros much.
All that said, I do like Clojure. I miss the baked in immutability, the ability to omit commas in arrays / lists / maps / etc, keywords, and the threading macros.
In summary, some of us have given it a shot, and ended up choosing a different path, and that’s ok.
> Also, Clojure’s start up time was off-putting, and would probably be even more so today, coming from Bun and Go.
Why is that? Never understood the complaint about slow Clojure startup time. Usually the context is either your local development environment, where you start the process once until you're done for the day, or you're deploying on a server and 5 seconds vs 10 seconds doesn't make that big of an impact. Short-lived CLIs aren't really suitable for Clojure in the first place, you'd use something like Babashka for that.
So why is the "slow startup" actually a problem? I don't seem to hit that issue ever myself, wondering what kind of situation people find themselves at where this hurts.
I also understand this is an issue for a lot of people but it’s never been an issue for me! I wouldn’t want it for serverless stuff I guess. But I would never think to do that lol
I've been using emacs and have written a a few thousand lines of elisp. I like elisp. I generally like any language that I become proficient in. But lisp isn't some sort of magical hammer that turns everyone into 10x programmers.
Maybe I still haven't had my epiphany, but I'm not a huge fan of macros in lisps and DSLs (like what ruby is known for). It makes code harder to understand.
> Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
Macros can be very powerful. But! They are like DSLs, in that they create their own mini language that you have to learn. Arguably worse than DSLs, macros modify the context of the host language in which they are invoked. That is their power, but it's also what distinguishes them from regular library functions, whose interface semantics are generally simpler to reason about.
Macros are preferable to runtime reflection and monkey patching, but the compile-time reflection and monkey-patching represented by macros still incurs a complexity tax that needs to be weighed against the alternative of non-macro code.
Homoiconicity is overrated. Python is an acceptable lisp: higher order functions, dynamic types, generators, decorators.
If you really need syntactic transformation you can use the ast module.
"Lisp/Clojure macros derive from the uniformity of the language to enable composing the language back on itself. Logic can be run at compile-time no differently than at runtime using all the same functions and techniques. The syntax tree of the language can be manipulated and transformed at will, enabling control over the semantics of code itself. "
If you are a smaller consultancy solving hard problems, then you might need this.
The problem sometimes is: "I don't want this level of complication, especially when I am going to hand it off to other people to maintain it."
In the business world, you are not gated by your intelligence, but by the average IQ of the people who are going to maintain it over the years.
A sadly typical flavor of essay: a lisp enthusiast who believes that learning lisp has made them into a uniquely Very Smart Boy who can think thoughts denied from programmers who use other languages. The "blub" paper asserts that there exists a linear hierarchy of goodness and expressiveness in languages, where lisp, by virtue of its shapelessness, exemplifies the pinnacle of expressiveness.
This is a profound misapprehension of the nature of language design. Languages exist within contexts, and embody tradeoffs. It is possible- common, even- to fully grasp the capabilities of a language like lisp and still find it inappropriate or undesirable for a given task. Pick any given context- safety-critical medical applications, constrained programming for microcontrollers or GPUs, livecoding environments where saving keystrokes is king- and you can find specialized languages with novel tools, execution models, and affordances. Perhaps it never crossed Paul Graham's mind that lisp itself might be a "blub" to others, in other situations.
The idea of a linear hierarchy in languages is the true flatlander mindset.
> The idea of a linear hierarchy in languages is the true flatlander mindset.
100% this. I think you can replace "languages" in that sentence with many things (employee levels is another big one that is relevant to this forum - employee value comes in many, many shapes). Reducing complicated things to one dimension can be a useful shortcut in a pinch, but it's rarely the best way to make complicated choices among things.
It would also be a lot more persuasive if the article provided even a single example of how Lisp enables superior solutions.
Instead, it's just an ad-hominem attack based on the idea that non-Lisp programmers are too limited in their thinking to appreciate Lisp.
Show me a convincing example of something that's simple/clear/elegant/superior in Lisp, and how difficult/complicated/ugly/impossible it would be to do the same thing in Java/C++/Ruby/Python.
In the absence of that, the entire article can be refuted by quoting The Big Lebowski: "Yeah, well, you know, that's just, like, your opinion, man."
It’s amazing how people are reading this to say the opposite of what it says. The end of the essay literally tells the reader they can appreciate Lisp if they just take the time to understand it, and they should make the effort. Not “if you don’t already know this, you must be stupid.”
If someone writes code based on an algorithm out of a 1985 textbook, and I tell them that they could make it go 20X faster if they learned more about processor architecture (out-of-order execution, cache coherency, NUMA, etc.) — a new dimension of programming to them — am I making an ad hominem attack?
Once I made somebody’s SQL query 100X faster by explaining what an index was. Fortunately they didn’t think I was attacking their intelligence.
For me macros are a power user tool that are useful in libraries to seriously upgrade UX of your public functions. Kinda like how crazy complex typescript generics can seriously upgrade the UX of your lib.
It’s hard to explain without explaining a bunch of what a specific library does. But many clojure libraries expose macros that let you interact with them in much more readable/terse ways.
I’m on my phone but a good one to google is core.logic. It exposes a macro to define a “logic function”, which lets you define logical expressions using pattern matching. You can come up with rules that would be many more LOC and less readable as a normal function. You can of course define them as a normal function too, which is useful when the pattern matching doesn’t express the problem well!
> Show me a convincing example of something that's simple/clear/elegant/superior in Lisp, and how difficult/complicated/ugly/impossible it would be to do the same thing in Java/C++/Ruby/Python.
Serialize and deserialize data. You're currently using something like XML or JSON for a human readable data serialization format in those languages. JSON and XML are not first class components of those languages. S-expressions are a better version of JSON and are first class components of Lisp.
I am not so sure how it works, but you can define your own evaluation handler for `eval` which, I assume, can be as restrictive as you need if you're dealing with untrusted data.
I don't think the article argues for superior solutions, but I understand how it can feel as such.
I think it's just trying to say there's another dimension, the meta-level enabled by macros and Lisp's syntax that opens up the possibility of new solutions, which may or may not be better, as that's so context dependent.
But what I feel it's saying is you can't even begin to imagine solutions that leverage this new dimension without learning about it and getting to grip with it mentally.
In that sense, it's saying when you don't know, you can't even explore the space of solutions at that higher dimension, not necessarily that they're better for all problems.
“Common, even”? Citation needed. I’ve worked closely with hundreds of developers over the years and maybe two of them made a conscious, knowledgeable choice whether to use Lisp for something.
You’re even sort of making the same point. Specialized problems need specialized tools. How do you write those specialized tools? Start from scratch, or just make a Lisp package?
Robust macros allow you to create domain-specific abstractions. That's cool, but there are plenty of other ways. Even functions are a way to create abstractions. And with anonymous functions, you can easily create higher-order abstractions.
The only thing AST-level macros help with is creating custom syntax to cut down on boilerplate. That's very cool, but it comes with a cost: now you have to learn new syntax.
I love Lisp. I've written tiny Lisp interpreters for most of my games (Chron X, Transcendence) and even GridWhale started out with a Lisp-like language.
In my experience, Lisp is great when you have a single programmer who understands and controls the whole source tree. Once a program exceeds the capacity of a single programmer, more conventional languages work better.
I’m writing a lot of Rust lately, which is rapidly becoming regarded as a conventional language, and I sure do appreciate all those things I use every day that end in exclamation points.
I'm curious here, because I don't know Rust. What's the difference between a macro and a function call from the caller's perspective? Do I (as the caller) need to know I'm calling a macro? Why?
Why is println! a macro when it's a function in almost all other languages?
GCC can type-check printf (matching format string to arguments) because the compiler doesn’t just treat it like a function. But that requires special-case code in the C compiler itself that is basically opaque magic.
Rust doesn’t need that, it’s mostly Rust code in the standard library, with only a small bit of compiler magic triggered by the macro. (Println! isn’t the best example because it does have that small bit of magic; most macros are just plain Rust code.)
Here’s a very impressive set of macros that I use daily. [0] This lets you do “printf logging” on an embedded device, with the human readable strings automatically pulled out into a separate section of the ELF file so the actual log stream data is tiny.
I did a similar thing for C a while ago, as a pre- and post- build step. It worked, but much less well, and was a maintenance nightmare.
Edit: and yeah, I think you do need to know you’re calling a macro, because macros aren’t limited to “normal” syntax or semantics. The ! is a signal that you’re escaping the usual bounds of the language. Like this. [1]
I think another macro, `json!()` works better as an example for that: inside the `json!()` you write something very similar to actual JSON. So when you see a `!` you know that there might be something out-of-the-ordinary following: https://docs.rs/serde_json/latest/serde_json/macro.json.html
Incidentally it also means that formatters like `rustfmt` won't apply the usual rules. For the macros that don't really deviate from ordinary Rust syntax, that can be a bit annoying.
> Many point to “ecosystems” as the barrier, an argument that’s valid for Common Lisp but not for Clojure, which interops easily with one of the largest ecosystems in existence. So many misperceptions dominate, especially the reflexive reaction that the parentheses are “weird”. Most importantly, you almost never see these perceived costs weighed against Clojure’s huge benefits. Macros are the focus of this post, but Clojure’s approach to state and identity is also transformative. The scale of the advantages of Clojure dwarfs the scale of adoption.
> In that essay Paul Graham introduced the “blub paradox” as an explanation for this disconnect. It’s a great metaphor I’ve referenced many times over the years. This post is my take on explaining this disconnect from another angle that complements the blub paradox.
The blub paradox, and the author's "flatland" methaphors, function as thought-terminating cliches. They provide the author (and Lisp proponents) with a simple explanation ("Everyone else is stupid") that doesn't force them to reconcile with more difficult questions ("Is it possible that other intelligent people have considered Lisp and rejected it for good reasons?")
And, honestly, it's just an annoying line of reasoning to hear that the only reason <you> don't use <favorite technology> is because you're just not perceptive enough.
For instance, the suggestion that "ecosystem" problems are "misconceptions" that critics fail to reconcile seems inaccurate to me. Does Clojure have a package manager as simple and straightforward as npm/cargo? Does it have a type system as well-maintained as TypeScript? Does it have a UI library as good as (choose your favorite web UI library)? These are all ecosystem problems. Do you think these problems meant nothing to everyone who decided against Clojure? Or do they all live in Flatland?
> The ability to manipulate compile-time so effortlessly is a new dimension of programming. This new dimension enables you to write fundamentally better code that you’ll never be able to achieve in a lower dimension.
There are many such "new dimensions of programming". Macros are cool, don't get me wrong. But given the choice between a proper macro system or a proper type system, I know which one I'm choosing every time.
In reality, most people, intelligent though they may be, don’t consider and reject Lisp, so that argument doesn’t really work. I know it irritates people who actually do consider and reject Lisp, but those people don’t realize that they’re a tiny elite who are not the target of these essays.
There are plenty of reasons it might be better not to use Lisp, but very few people actually get as far as considering them.
Quite a lot of people have given Lisp a shot and determined, for one reason or another, that it doesn't work for them. Why wouldn't that be the case? There are no special forces that prevent people from giving Lisp a shot when every other popular language in the world was at one point in time was at Lisp's level of popularity, and overcome the barriers that Lisp could not.
The essay’s thesis is “most people don’t consider Lisp because they don’t know what’s different and special about it”. I think that’s unarguably the case. You equated that with “everyone else is stupid”, which is uncharitable and not at all what the essay says. Why would you even bother to write an essay if your audience is too stupid to understand what you’re saying?
Is it really that uncharitable? Yes, it's slightly hyperbolic, but I argue only slightly. Whether intended or not, the tone of the article is patronizing. Here are some examples.
> The programmers who live in Flatland
> Likewise, you cannot comprehend a new programming dimension because you don’t know how to think in that dimension
> the sphere is unable to get the square to comprehend what “up” and “down” mean.
All of this is patronizing. It implies that I am incapable of understanding the benefits of Lisp. If only I were able to lift myself out of the dull swamp I find myself in! But I am capable, and I do understand them, and I still don't like it! And I think most Lisp detractors do as well! I would argue that it is the Lisp proponents that live in Flatland - they need to understand that there's another dimension to criticisms of Lisp that aren't just "I don't like parentheses" and that there is substantive feedback to be gleaned.
> they need to understand that there's another dimension to criticisms of Lisp that aren't just "I don't like parentheses" and that there is substantive feedback to be gleaned.
I'm also somewhat reminded of the decline of Perl and how some people who love and still frequently use Perl don't really seem to even acknowledge the complaints people have about it, which seems to prove the claim about the decline being cultural. According to that kind of attitude, the lack of popularity is inexplicable, and we might actually be lucky that they're not resorting to conspiracy theories to "explain" the mismatch between their preferences and observable reality.
The Haskell motto of "avoid success at all costs" seems a lot healthier, as in, they know they might need to choose between going mainstream and getting to keep a language that suits them personally.
Lots of the Lisp advocacy also comes off as either entirely too vague, like this blog post, or stuck in the age of the `worse-is-better` talk (1989, so predates WWW and nearly all the programming languages in widespread general use). I don't care about comparisons to C, because the only places C is seriously considered for new projects these days are in places where a GC is unacceptable (and purposes Rust isn't certified for or whatever).
>Does Clojure have a package manager as simple and straightforward as npm/cargo? Does it have a type system as well-maintained as TypeScript? Does it have a UI library as good as (choose your favorite web UI library)? These are all ecosystem problems.
The irony is Clojure(script) has all those things. By virtue of being hosted on the JVM and Javacript and having first class interop with both. ClojureCLR even gives you access to all of C# etc.
Being hosted was a great play in terms of ecosystem.
I use both approaches. One thing is that Clojure code bases are comically hard for anyone to mentally parse if they didn't write it. At least the bulk of programmers... like you'll find on an actual team. Great to write, sure, but not useful in terms of onboarding new team members. Clojure programmers are typically great thinkers. And veterans. But if you are actually trying to build a company, then beware. Your handful of expensive brilliant programmers will build something that you can't bring people in to expand or maintain. Also watch out for the fact that the companies making the awesome tools that COULD be used by noobs often keep them closed source (Datomic and, I think here, Rama). They intend for you to hire them as consultants and pay licenses. Which is all fine... except the 2D languages have real open source libraries with huge adoption and ecosystems.
I'm not sure I'd call a programmer "brilliant" if they cannot A) make a codebase simple enough for people to contribute to and B) handle the social parts of training someone to get good enough to contribute to the codebase.
Agreed. And this is the real miss of much of the Clojure community. There is a handful of amazing people in there pulling 90% of the weight of bringing noobs in to the language. And the rest don't even seem to notice their efforts. Little awards and grants here and there, yes. But the majority don't even care if the language has wider adoption or not. It works for them and that's enough. But many of the successful projects are toy or side project ones. A large number have comically minimal UIs... sub useful in today's world. Quite a few Clojure programmers use 2D languages in their day jobs, only bringing Clojure in for small parts if at all. All of this is a top down vibe. The core team has never meaningfully addressed the terrible error messages the language spits out because they are able to decode the problem themselves. Empathy or concern for noobs or wanting to grow the language seems a far priority for them. The same for the cryptic documentation, seemingly written for themselves at best. Very talented people, mind you... just not concerned with the things that would have caused adoption of their entirely unadopted language (percentagewise).
> And this is the real miss of much of the Clojure community
I wasn't pointing to the Clojure community at large, the Clojure community is very welcoming to beginners and newbies, like I was once.
It's true that the focus on the language is improving it for everyone, not just beginners, and that can appear like a bad idea if you're a beginner, but I personally agree with this goal.
The problem mentioned by parent is something I haven't personally experienced, but seems exclusive to their job/position/experience, and I don't understand calling that specific person/developer brilliant.
it'd be nice if there was an attempt to give an example of what kind of powers moving into 3d-lispland allows instead of just saying that it's beyond the comprehension of the 2d-planar programmers.
because, i guarantee that it's not beyond our comprehension. at some point the author was a 2d-er that read/did something and had their understanding expanded. so... do that for us
Let say.. I remember quite well when I learned Lisp, and differently to any other language I learned, it gave me profound insights and even more profound frustration trying to understand some concepts.
I remember playing with call with cc, or creating a flow programming language, thinking in higher order, etc..
I clearly do not want to work with lisp, and many of those concepts can be used in other languages without too much effort now (lambdas, map, filter, reduce,... Among the most common and useful).
I think learning lisp is nice as it helps explore interesting areas of programming on a mental level. I can't stress enough how it can wrap your mind sometimes.
Will it help you program faster and bug free? Probably not, will it improve your mental model of programming languages, probably. Will you enjoy learning abstract things, if you like solving math puzzles, probably, otherwise probably not.
It's hard to express the 'powers' it gives you, it's like spending much time thinking about simple things that usually you just use without thinking about.
As a crude example, the scope of a variable shared between two lambda function that is not shared with the global space.
Sorry that I don't have a specific example in mind, I feel like the 'power' is just spending enough time thinking on complex things.
I had heard before that Lisp had something called "macros", but I didn't know exactly what they were or how they differed from C macros. This blog post kind of explains that, but not in a way that couldn't also apply to C macros if you tried hard enough.
I want to know more, but I didn't have any examples here to look at. I may look them up now that I have an idea.
I agree. Especially as someone that likes LISP-like languages and uses Janet and Fennel quite a bit (and some elisp, in the past also Clojure) but never used a macro for anything. Would love to hear more about that third dimension I am missing out on.
Marcos are only very appealing to tyros. Most old salt Lispers avoid them. I would argue that a macro is only appropriate if you are adding a genuine syntactic feature to a language (one hint that this is the case is if your macro involves binding variables).
"You can get in touch with us at consult@redplanetlabs.com to schedule a free consultation to talk about your application and/or pair program on it. Rama is free for production clusters for up to two nodes and can be downloaded at this page."
> A big chunk of our code was doing things that are very hard to do in other languages. The resulting software did things our competitors’ software couldn’t do.
I've never seen a general purpose programming language that couldn't do everything the underlying hardware is capable of. It could only be unperformant enough that you could call it unfeasible at worst. What's so hard to do in languages other than Lisp? Spam parentheses?
> Learning new dimensions
> You can’t persuade someone in 2D with 3D arguments. This is exactly like how in Flatland the sphere is unable to get the square to comprehend what “up” and “down” mean.
Just like today's UX designers who need a 3D surface to draw a lousy rectangle.
As others have said, the lack of any examples makes this post fall flat.
Also, consider that good work - particularly in art but also in engineering - requires constraints. Knowing what you cannot do adds guard rails and a base set of axioms around which you can build. Perhaps the power of LISP macros and AST manipulation is not “powerful and thus good”, but rather “too powerful and thus complicated”. Needing to write out a boring old function/class/module instead might leave you with code that is simpler to read and design around.
Alan Kay said something similar back in the 90s, framing it as ants living on the "pink plane" and the "blue plane". What he said at the time was an ant crawling on the pink plane can do a lot of pink plane things, but he won't think about the kinds of things he can do on the orthogonal blue plane, because he's never been there so he doesn't have those thoughts. Innovation in PLs comes when someone from the pink plane travels to the blue plane and brings back all they have seen, which causes people in the pink plane to start thinking new (bluer) thoughts.
I'm going to use a metaphor for this talk which is drawn from a wonderful book called The Act of Creation by Arthur Koestler. Koestler was a novelist who became a cognitive scientist in his later years. One of the great books he wrote was about what might creativity be.—Learning.—He realized that learning, of course, is an act of creation itself, because something happens in you that wasn't there before. He used a metaphor of thoughts as ants crawling on a plane. In this case it's a pink plane, and there's a lot of things you can do on a pink plane. You can have goals. You can choose directions. You can move along. But you're basically in the pink context. It means that progress, in a fixed context, is almost always a form of optimization, because if you're actually coming up with something new, it wouldn't have been part of the rules or the context for what the pink plane is all about. Creative acts, generally, are ones that don't stay in the same context that they're in. He says, every once in a while, even though you have been taught carefully by parents and by school for many years, you have a blue idea. Maybe when you're taking a shower. Maybe when you're out jogging. Maybe when you're resting in an unguarded moment, suddenly, that thing that you were puzzling about, wondering about, looking at, appears to you in a completely different light, as though it were something else.
That's the upshot of what he's saying. But how the cross pollination happens -- crossing over to another plane of thought -- is I think the real point. That is: Cross-pollination between programming languages is a good thing, and therefore we must often force ourself to think outside of our narrow way of thinking in order to find new ways, or else we will dig ourselves into a trench (or find ourselves at a local maxima in the design space of languages).
It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world.
The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.
reply