Or maybe keep things the way they are.If you really need performance python is not the language you should be looking for.

Instead of breaking decades of code, maybe use a language like Go or Rust for performance instead.

Python is also the dominant language for machine learning which does care for performance. The person who made recent nogil work is one of the core maintainers of key ML library. The standard workaround is ML libraries, the performance sensitive stuff is written in C/C++ (either manually or with cython) and then uses python bindings. But it would be much friendlier if we could just use python directly.

It's also commonly used language for numerical work in general. Most of the time numpy is enough and then occasionally you'll need something not already implemented and then have to do your own bindings.

> Python is also the dominant language for machine learning which does care for performance. The person who made recent nogil work is one of the core maintainers of key ML library. The standard workaround is ML libraries, the performance sensitive stuff is written in C/C++ (either manually or with cython) and then uses python bindings. But it would be much friendlier if we could just use python directly.

Multithreading is not really the reason why things get written in cython etc., you can easily see 100x improvements in single threaded performance (compared to maybe a factor of 2-8x for multithreading). If you care about performance you'd definitely write the performance critical stuff in cython/pythran/c.

Nope, C++ and Fortran are.

The bindings available in Python can also be used from other languages.

I would have thought convincing people they’ll just have to use Go or Rust or Elixir would have been an easy sell around here.

Turns out they just want a better Python.

>Turns out they just want a better Python.

That's Go. It gives actual types[1] and structs (so you dont have to wonder about dict, class, class with slots, dataclasses, pydantic, attrs, cattrs, marshmallow, etc). It removes exceptions and monkeypatching. It's async-first (a bit like gevent). It's inherently multicore. And you can package and distribute it without marking a pentagram on the ground and sacrificing an intern to Apep.

You just need to stop being friends with C FFI. Which is fine for web and sysadmin tools. For data science and ML/AI/DL, it's less ok.

[1] And the types are actually checked! I think a lot of people aren't really using python type checking given how slow it is and no one seems to complain. Or maybe everyone is using pyright and pyre and are satisfied with how slow these are.

Going to the very unexpressive Go from the expressivity of python is a goddamn huge jump though.

Going to JS or even TS for performance would be saner, and it has a same-ish object model even.

The expressivity in Python is a problem that needs to be solved though. Moving to JS goes the wrong way.

It already exists, but they don't want to learn other languages.

> Instead of breaking decades of code

Pin your version.

Concurrency is not solely required for performance. I'm designing a small tool (a file ingester/processor) for myself, which gonna need several, really concurrent threads. I love Python, but I can't use that, so I'm learning Go.

Why put Go and Rust in the same category? I never really understood that.

Either include like almost every language from JS, Java, C# to Haskell, or just list C++ and Rust. But Go is in the former category.

Python has basically already done exactly that with 2.7 to 3 and we came out of that relatively fine.

I say bring it.

> As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language.

Hmm, how is that so?

As a python dev as well, I don't have much complaint with multiprocessing.

The API is simple, it works OK, the overall paradigm is simple to grok, you can share transparently with pickle, etc.

Multiprocessing is fine, but the cost of starting another interpreter is pretty visible, so you need to create a pool, and it may not be an overall speedup if the run time is short.

It takes more careful planning than async in JS?, say, or goroutines.

Yeah but for JS style async you'd use probably use an event loop in Python, not multi processing.

Yes. But, frankly, async is also simpler in JS than in Python: e.g. no need to start a reactor loop.

Starting the event loop is no worse than any setup of a main function, it’s a oneliner: asyncio.get_event_loop().run_until_completion(my_async_main)

Errr no, that has been replaced with asyncio.run quite some time ago.

Pickle isn't transparent though, custom objects that wrap files or database sessions need to override serialization.

The ProcessPoolExecutor is nice but shouldn't be necessary.

On the flip side, if your workload can be parallelized across thousands of cores, python has about the best CUDA support anywhere.

That would be C++ and Fortran actually.

But the python bindings are great and useful to many, so Python gets to be added to the list.

Just like any language with FFI capabilities to call the same libraries.

As another python dev, it has basically never been a painpoint for me.

Your anecdote adds little.