The Projectification of Programming
Really what is it that happened in the 2010s that made programming in modern languages so infuriating? I am not talking about quite the languages themselves (though you'll understand why the terminology is convoluted), but moreso the ecosystems surrounding modern languages.
You learn about new language X that you want to try out, so you install a suspiciously large package to get started and the first thing on the guide is how to "initialize a project" (???) You mean create a new directory and edit a program file?
Not quite. Apparently, it's this init command that you're never quite sure what it does but once finished you see a "manifest" file, a "lockfile", and "identity" file, random directories and hidden cache folders, tool configs and whatever else that you apparently need to write a hello world
I'm sorry -- what is this stuff? Why do I need this stuff? Why was this mess added to my filesystem? What happened to lang_x program.x?
The reality is that modern programming languages are more so complete development environments. Each language essentially attempts to replace your entire operating system with an arguably worse alternative. Gone are the days of the lexer and the parser: modern languages are more about build systems, development pipelines, dependency management, formatters, tests, workspaces etc.
Why use a distro's package manager when you can create a separate package manager with a different interface that's not meaningfully better and possibly causes conflicts with the operating system? Why trust the user to a create a file structure that matches their own needs and not just create useless boilerplate directories for no reason? What's next? Language specific version control?
This is what turned me off from rust before seeing a single line of code. This is what has turned me off from multiple languages, and it's even worse when a language presents itself as minimal and no-BS (Zig is a great example, though like Golang, it's more sane). For some reason I do not understand, infiltrating your workflow seems appropriate, and presuming complex project needs for every would-be program is, apparently, not insane. "Would-be program", because this all happens before a single line of code written. The program has been "projectified".
By projectification I mean the process by which languages and their environments become project-first (as opposed to program-first): they treat the project -- not the program -- as the fundamental unit of programming.
I'd like to clarify a few things. Of course, these developer environments are (usually) not literally necessary to use the language. The issue is that they seem to become more and more assumed and expected. Likewise, manifest files, standardized integration testing, pipelines, build tools etc. aren't inherently bad. There are complex projects with complex needs. The issue arises when these things are assumed necessary for every 100-line script written in the language.
The "Reasons"
At this point, like me, you should probably be wondering what exactly causes this trend. The answer is complex, but the justification given is mostly that general purpose programming tools are not semantically aware, and therefore aren't appropriate for these languages.
I understand that the lack of semantic-awareness can be a limiting factor of some common general programming tools, however (and excuse my low wits), I still don't quite understand what these languages do that is so advanced and so unique that other languages that integrate fine with common toolchains don't do? And, perhaps more importantly, why the jump? Why do we jump from "Make can't quite handle the orchestation of this build" to "we need to create a completely separate developing environment"?
And have we considered that, if the language does not integrate well with existing toolkits, maybe it isn't that good of a language? Maybe if common tools cannot operate well on your language without knowing the project's context and structure, that's a sign that your language is placing too much semantic weight on those things? I understand there are domain-specific, or otherwise unique languages that will inevitably benefit from specialized toolchains, but isn't integration with existing environments a goal of PL design?
The cultural domino
Let's look at the effects of this language ecosystem design on your language's userbase:
- You drive experienced programmers away: why should they bother to learn your own unique toolchain unless they absolutely have to? Why not prioritize a language that doesn't require them to change their habits? Expertise goes with them.
- Toolchain expertise does not transfer: the experienced programmers that do stick will inevitably become beginners again with regards to the environment, as they cannot just bring their toolchain expertise with them.
- You prepare every program to become a large project with your inits.
That last one is particularly important, I feel. Having a complex directory structure mentally guides the developer to comply with it (and the language's canonical workflow), even when that's completely unnecessary. Do not assume that every program will need large build scripts, do not assume every program needs dependencies. I believe this corporate-style project-mindset leads to more corporate-style bad software.
Any hope?
What I'm arguing for is not difficult or complex. In fact, it's probably substantially easier than developing (or learning anew) a separate development environment: Let the programmer decide when a program needs to become a structured project, and let the programmer define what that means. The compiler or the interpreter shouldn't know or care about that structure, and the toolchain needs to be separate and reasonably modular.
There are some modern languages that seem to have slipped through the cracks of forced project-design. Hare, Odin, Janet, and some modern-ish ones like D and Lua. And, of course, there's a whole tradition of older program-first languages still around, including C, Sh, Scheme, Tcl, Perl, awk, Forth etc., that you probably already know about. Languages like Go, Zig, Nim etc. appear to be somehwere in the middle but I've only seriously used Go (it works fine with just files, though dependencies can project-creep your programs). All of these languages are signs that project-madness isn't the only direction for language ecosystems.