TIL that #Rust, while amazing, isn't so amazing for quick-and-dirty coding. It's like flying the Concorde for a 1-hour car trip.

@JonYoder There are reasons I often reach for the shell for quick-and-dirty things. But for anything serious, it quickly becomes a problem. In the days of Python 2, it was a good replacement, but with Python 3's inability to correctly handle POSIX filenames, environment variables, and parameters, I wouldn't bother. Rust is actually pretty decent for a lot of quick stuff once you get used to it.

@jgoerzen What kinds of problems have you run into with Python 3?

@JonYoder I wrote a series of articles on this subject culminating in changelog.complete.org/archive . In short, I was so burned by my effort to port to 3 -- and the utter crappiness and inconsistency of handling non-UTF8 in even the library bundled with Python -- that I consider it unsuitable for any purpose involving filenames, command-line parameters, or environment variables. One lovely tidbit is the zipfile.py tries to decode non-UTF8 sequences as cp437 in ALL cases!

@JonYoder If you want a coffee with your reading, after that first link, you can check out changelog.complete.org/archive and changelog.complete.org/archive . The upshot of it is, as far as I can tell, it is impossible to write cross-platform code that handles filenames correctly on both POSIX and Windows. gets this right, and Python's attempt to assume the whole world has used since the beginning of time is a real pain.

@JonYoder You got me thinking in more detail why I reflexively avoid now, despite the fact that I wrote two large programs ( and ) in it, and published a book about it. 1/


@JonYoder Avoiding - Besides the absurd inconsistencies in changelog.complete.org/archive and the extreme difficulty verging on the impossibility of properly handling filenames in POSIX (see changelog.complete.org/archive and changelog.complete.org/archive ), there is more that makes me shy away. 2/

@JonYoder It is astonishing to me that still has a Global Interpreter Lock in 2022. wiki.python.org/moin/GlobalInt Multithreading in Python is mostly a fiction. There are kludges like docs.python.org/3/library/mult which use fork, pipes, pickling, and message passing to simulate threads. But there are so many dragons down that path -- performance and platform-specific ones (different things can be pickled on Windows vs. Linux) that it is a poor substitute. 3/

@JonYoder Sure, people use for things like work. In this case, Python is merely a shell; the real multithreaded code is in a different language (often C). The way to get performant multithreading out of Python is to not use Python at all. 4/

@JonYoder When I started using more than 20 years ago now, it was an attractive alternative to Perl: like Perl, you don't have to worry about memory management as with C, but Python code was more maintainable. By now, though, even writing a Unix-style cat command in Python is extraordinarily complicated lucumr.pocoo.org/2014/5/12/eve . All the "foo-like objects" are an interesting abstraction until they break horribly, and the lack of strong types makes it hard to scale code size. 5/

@JonYoder These days, we have credible alternatives to : , , and (among many others). All three of these are performant, avoid all the manual legwork of or the boilerplate of , and provide easy ways to do simple things. 6/

@JonYoder The one place I still see being used is situations where the is valuable. (Note, also has this). is an example of this too. People use for rapid testing of things and interactive prototyping. For a time, when I had date arithmetic problems, I'd open up the Python CLI and write stuff there. Nowadays it's simpler to just write a Rust program to do it for me, really. 7/

@JonYoder So that leaves me thinking: We're thinking about wrong these days. Its greatest utility is as a shell, not a language to write large programs in. As a shell, it is decent, especially for scientific work. Like other shells, most of the serious work is farmed out to code not written in Python, but there is utility in having it as a shell anyhow. And like a shell, once your requirements get to a certain point, you reach for something more serious. end/

@jgoerzen @JonYoder fascinating stuff, thanks for sharing this insight. You mentioned Haskell, I was wondering if you had much experience with Lisp dialects such as CL or Clojure? I imagine Clojure may have similar concerns as Java. CL is as old as the hills but it has my curiosity

@trevdev @JonYoder I used CL some back in college, and dabble in elisp from time to time, but I'm not expert in either. I found Haskell particularly fascinating as I think it takes the FP paradigm further than even those do (via laziness and segmented side-effects). But Lisps could also serve the REPL need. Maybe someone else would know more ( is in this universe too, but I'm not fluent in it)

@trevdev @jgoerzen @JonYoder

I'm into it mostly to train my brain on something that is not inspired by C. I discovered the Lambda Calculus and that made the exercise even more interesting.

I really like the syntax, it's so simple that other languages seem complicated to read for me now haha
Can feel weird at first but you get used to it : everything is expression (no statement), parentheses is the single expression delimiter, …

@trevdev @jgoerzen @JonYoder

But you can taste Scheme without parens, because Scheme lets you to tweak it if you want, it's still Scheme (draketo.de/software/wisp 👋 @ArneBab)

I don't leverage the REPL very far but I do use it pretty often.
Imagine a Python program running a Python REPL you can use to load new code to modify/extend the behavior of the Python program while it's running (more powerful than hot reload, gnu.org/software/guile/manual/ 👋 @dthompson )

@jeko @jgoerzen @JonYoder I am not a stranger to the parens and am in no hurry to get rid of them myself :) I am also trying to expand my mind a bit here. If that expansion can be practical, all the better.

This is part of why I am looking at Clojure(Script?) closer that Scheme or CL, but I would rather get lower level if I can.

@trevdev @jgoerzen @JonYoder If you want to toy around in Clojure, take a look at babashka, a "Native, fast starting Clojure interpreter for scripting" -- it's basically Clojure built via Graal to produce a very small native executable which is great to use Clojure instead of bash.

@trevdev I went from a CL background to Clojure. IMHO both languages are fine and have their merits and issues. CL, being standardized on older Lisps has certainly more warts, but then again you have Clojure(script) based on Java or Javascript, resp. @jgoerzen @JonYoder

@jgoerzen Read your article. I can see why you avoid #Python, especially 3. In this instance, I think *both* Python and POSIX are stupid. By permitting basically everything, the standard makes it *much* harder for developers to do it without breaking on *something*. And Python 2 definitely had a much better thing going on for string type consistency.

@JonYoder Oh you get no argument from me on it being ill-advised for \n, \t, \a, and '-' being valid filenames in POSIX (among many other characters that shouldn't be valid in filenames). Fully in agreement; this has been the cause of many bugs over the years and no doubt will be for many years to come.

@JonYoder I have more sympathy for the lack of a declared encoding in POSIX, since Unix predated Unicode by decades. And historically, mostly it was not possible to have a binary sequence that was somehow "invalid" in a given encoding as it is now with UTF-8 (basically, we used to just select how our terminals would render certain characters; it was more of a font selection than anything). So now with UTF-8, some binary sequences are invalid in UTF-8 but still valid in POSIX.

@jgoerzen @JonYoder > We're thinking about #Python wrong these days. Its greatest utility is as a shell, not a language to write large programs in.

What an alien thing to say. All you have to do is open up github.com or any job listing website to be proven wrong.

I write this using Qutebrowser on Qtile started from a Xonsh shell - Python is not fit for writing large programs! /s

@Wraptile @JonYoder 1/ I have several responses to that. First of all, popularity does not equal suitability. I would say that, today, C isn't really suitable for writing large programs in. I am perfectly aware that many large programs are written in C; I have worked on them personally and professionally. But factors other than suitability come into play (eg, the high cost of rewriting an existing code base in a new language, institutional inertia, dev team skillset, etc.)

@Wraptile @JonYoder 2/ So I looked into Qutebrowser - thinking "could it really be possible to write a performant browser in ?" The answer is, of course not. Qutebrowser falls into exactly the sort of thing I described: serious work is done in a different language (QtWebEngine).

@Wraptile @JonYoder 3/ "Not fit for" != "never used for". Returning to the example, I think C is not fit for modern development, now that we have . That doesn't mean it's impossible to use C, or that nobody does; just that, from a purely technical view, having to manage memory manually and deal with all the other numerous pitfalls of C isn't worth it TODAY. Nobody's going to rewrite Apache in Rust tomorrow, but if somebody were starting a new web server tomorrow, C shouldn't be chosen.

@Wraptile @JonYoder 4/ And so it is here. I don't begrudge people the choices they made in the days before Haskell, Go, and Rust were ready. I made similar choices. I also pointed out some cases where Python was helpful (REPL and interactive prototyping, shell-like uses). I said the GIL was a huge problem in modern computing. Your reply confirmed what I said on both counts, so I'm at a loss as to where the substantive disagreement is. /end

@jgoerzen @JonYoder nah, popularity is influenced by suitability - that's economics 101, isn't it?

Python doesn't did shop with any system and still doesn't ship with most popular OS. Python is jot available in the browser like Javascript is and yet it's extremely popular and behind billions of dollars of turnover every year.

I think python is opposite - it's made to address the human problem of engineering and that's why it's great for "big programs". :blobcatthinksmart:

@Wraptile @JonYoder You think that economic activity is the measure of quality? Well then, I guess we should have just given up on Linux in 1995 then, because everything from Windows to AIX had more economic activity than Linux.

I absolutely reject the notion that cash equals quality. That notion excludes the possibility of new and better things, because the new will necessarily have less economic activity - that doesn't make it worse. 1/

@Wraptile @JonYoder There's this thing called "inertia". is 30 years old; C more like 50; Rust and Go not nearly that.

I can't help but notice that none of your arguments are technical. I'm sorry, but arguments that boil down to "X makes more money than Y" or "X is more popular than Y" are utterly unconvincing in a technical conversation. I have spent most of my life seeing the less-popular solutions be better than the popular ones in material ways.

@Wraptile @JonYoder I am happy to have a conversation with you about the technical merits of different approaches, but if your arguments are all going to be about popularity, I find that irrelevant, unconvincing, and not worth the time discussing. I'm happy to have a technical conversation but would rather spend my time at a keyboard doing something other than arguing about things that don't matter.

@jgoerzen @JonYoder I think you greatly misunderstand what makes a good programming language good. It's a tool not a religion or a physical property and tools need to deal with people.

You list couple of edge scenarios like newlines in filenames being allowed as some sort of stink against language when it's not going to be encountered in 99.999% of programs and can be fixed with a single line of code. Your second question is impossible to answer - how do you prove tech merit of a language?

@Wraptile @JonYoder This is the kind of attitude that gave us thousands of security bugs over the years. I *directly* addressed this oft-repeated argument in changelog.complete.org/archive . It's not just a matter of one or two lines of code; it's an utter inability to, eg, extract a ZIP file correctly. You're also ignoring the very real reality of the GIL, which makes a language unsuitable for a wide range of tasks in today's multi-core-is-standard world.

@Wraptile @JonYoder The fundamental point I was trying to make is not that Python is bad for all tasks. Just that it makes simple things (dealing with filenames and stdin) extremely difficult, has semantics that lead to vastly counterintuitive results with comparisons, and with more modern type inference, the weakly-typed nature of it isn't holding up well. This is why I personally would not reach for Python, given a choice.

@Wraptile @JonYoder I have long since passed having an emotional attachment to languages. I have had books published (by O'Reilly, APress, and others) on C, Python, and Haskell. I've contributed significant libraries to OCaml and Haskell, including a popular database layer for Haskell. And here I am writing code in Rust. Use what makes you happy, I don't care. But don't try to tell me Python's filename handling is correct, because it objectively isn't.

@jgoerzen @JonYoder I'm not saying it's correct I'm saying it's statistically insignificant. Having more people being able to work and solve problems in an elegant, sustainable fashion is more important than handling 1 edge case that will be encountered by one program and that can be patched with a line of code or a community package.

If you want to play this game of edge cases then you can put any language down even Rust - it's an absurdist, pointless, argument.

@Wraptile @JonYoder Except none of your premises are correct. The problem is pervasive in the Python ecosystem. Community packages get it wrong, the standard library gets it wrong, and it has significant real-world consequences (eg, gpodder crashing), and it is not a matter of a one-line fix. Using the wrong data type for a filename is a pretty fundamental problem. And it's not the only problem; the GIL is another fundamental problem; multithreading in Python is a fiction.

Show newer

@jgoerzen > We're thinking about #Python wrong these days. Its greatest utility is as a shell, not a language to write large programs in.

Oh how I wish that Tcl could have kept that niche; as a shell, it's actually fantastic

@tfb @JonYoder That is a really interesting point. I remember two programming languages, and , that were really hot for awhile, for various reasons. Both went through a sort of boom and then a decline. I wonder what it is about these interpreted languages that caused that effect? I also wonder if we will see it effect more interpreted languages.

@tfb @JonYoder I remember reading the Perl Camel Book with actual excitement - "wow, this is so much easier than C!" And similar writing a GUI in Tcl/Tk nearly 30 years ago now. Today I wouldn't touch either of them. I moved from both of them to Python in the early 2000s, and then from Python to Haskell, OCaml, Rust, etc. after experiencing some issues with scaling Python even in the v2 era. (Now, people are, eg, rewriting server in non-Python languages for performance)

@tfb @JonYoder One of the things that hit me starting with and continuing through is that rapid prototyping is overrated, and there is a lot to be said for "if it compiles, it is (more) likely to work." I think this has significantly raised my expectations for code quality and reordered how I think about productivity (invest a bit more upfront and spend less time debugging later) C, C++, Java, etc. didn't have that property really.

@jgoerzen I think Test Driven Development provides that assurance, but for real instead of kinda if you squint.

@mike TDD definitely helps, but I guess my experience is that tests don't catch all bugs, compilers don't catch all bugs, and the more bug-catching tools you can have in your toolbox the better.

@jgoerzen There’s also lint available for most interpreted languages and sophisticated configuration environments

@jgoerzen @tfb @JonYoder I felt the same way; I think in both of these cases it's that the languages do some small (but important) things well, but then lack bottom. Tk is still the nicest toolkit I've ever used in some ways (lack of modern widgets notwithstanding), but it's freaking hard to use it for large programs because Tcl's data types suck. Ditto for perl and text slicing.

@jgoerzen @tfb @JonYoder

I think you could argue that Ruby and now JS is starting to hit that cycle now too. That's not to say either is dead or really dying, just that they're not the hype language any more (Now it seems to be Rust.)

That being said, Perl is enjoying a nice renaissance at the moment. It's nowhere near its glory days but it has an active community that are working on it regularly and adding new features.

@splatt9990 @tfb @JonYoder Interesting. I could have guessed Ruby but JS surprises me. Also that Rust is the hype language; am I actually doing the popular thing at the right time for once? 🙂 Also I'm glad to hear that Perl is having a renaissance.

@jgoerzen @tfb @JonYoder IME/O sentiment is beginning to turn against JS yes. It's still early on the decline side but the first few signs of it are starting to appear.

Re: Rust yeah that seems to be the "hot new thing" at the moment. We're at the "everything must be re/written in it" stage of the hype cycle right now.

@jgoerzen Now that I hear #Ruby mentioned, how do you think it compares to Python overall? Better choice? Not so much?

@JonYoder I don't really have enough experience to comment. Maybe someone else has done a lot of work in both and

@jgoerzen @JonYoder i've used both quite a bit. languages are pretty similar, but communities feel quite different (both healthy IMO), while ruby seems very front end focused and python back and front

@sckottie @JonYoder I do pretty much zero web development in the usual sense, so I may be coming at this from a different angle. Systems programming, integration, working with radios, asynchronous communication, machine learning, and global-scale caching have been my personal and professional areas of interest. Either not web-related at all or, if you have to stuff it into some category, backend I guess.

@jgoerzen @JonYoder I refuse to learn a new language unless it has some sort of multithreading built into it from the start. Even C / C++ multithreading is a nightmare hack. Julia and Go are my favorites these days

@jdarnold @JonYoder Right now, I've been enjoying quite a bit, but both of those are also interesting and I'd love to find time to dive in more. Sadly time is limited for me right now...

Sign in to participate in the conversation

For people who care about, support, or build Free, Libre, and Open Source Software (FLOSS).