Re: venvs vs. package management

There are several independent problems here:

1. Very short release cycle. This is independent of the Python venv
module but is indirectly influenced by Python's own release cycle.
Package maintainers don't have time for proper testing, they are
encouraged to release a bunch of new (and poorly tested) versions, and
they never get a break. So, when you install the latest, there will be
something else broken.  There's never a window to properly test
anything.

2. Python made a very short-sighted decision about how imports work.
Python doesn't have a concept of "application", and therefore there's
no way to specify dependencies per application (and imports import
anything that's available, not versioned). That's why every Python
application ends up carrying its own Python, with the version of its
own dependencies around. Python's venv module is just an
acknowledgement of this design flaw.  I.e. the proper solution
would've been a concept of application and per-application dependency
specification, but instead we got this thing that doesn't really work
(esp. when native modules and shared libraries are considered), but it
"works" often enough to be useful.

3. The Python community grew to be very similar to what PHP 4 was,
where there were several "poisonous" examples, which were very popular
on the Web, which popularized a way of working with MySQL databases
that was very conducive to SQL injections. Python has spread very bad
ideas about project management. Similar to how PHP came up with
mysql_real_escape() and mysql_this_time_promise_for_real_escape() and
so on functions, Python came up with bad solutions to the problems
that had to be fixed by removing bad functionality (or, perhaps,
education). So, for example, it's very common to use requirements.txt,
which is generated by running pip freeze (both practices are bad
ideas). Then PyPA came up with a bunch of bad ideas in response to
problems like this, eg. pyproject.toml.  In an absurd way very much
mirroring the situation between makefiles and makefiles generated by
autotools, today Python developers are very afraid of doing simple
things when it comes to project infrastructure (it absolutely has to
be a lot of configuration fed into another configuration, processed by
a bunch of programs to generate even more configuration...) And most
Python programmers don't really know how the essential components of
all of this infrastructure work. They rely on a popular / established
pattern of insane multi-step configuration generation to do simple
things. And the tradition thus developed is so strong, that it became
really cultish. This, of course, negatively contributes to the overall
quality of Python packages and tools to work with them.

Unfortunately, the landscape of Python today is very diverse.  There's
no universally good solution to package management because it's broken
in the place where nobody is allowed to fix it.  Commercial and
non-commercial bodies alike rely on people with a lot of experience
and knowledge of particular Python gotchas to get things done. (Hey,
that's me!) And in different cases, the answer to the problem will be
different. Sometimes venv is good enough. Other times you may want a
container or a vm image. Yet in a different situation you may want a
PyPA or conda package... and there's more.

On Sun, May 19, 2024 at 4:05 PM Piergiorgio Sartor via Python-list
<python-list@python.org> wrote:

>
On 19/05/2024 08.49, Peter J. Holzer wrote:
[...]

That's what package management on Linux is for. Sure, it means that you
won't have the newest version of anything and some packages not at all,
but you don't have to care about dependencies. Or updates.

>
Well, that doesn't work as well.
Distributions do not pack everything, this
also depending on licenses.
>
Sometimes, or often, you need to use the
*latest* version of something, due to some
bugfix or similar.
>
The distribution does not always keep up
to date everything, so you're stuck.
>
The only solution is a venv, with all
needed packages for the given task.
>
Typical problem with PyTorch / TensorFlow.
>
In case of trouble, the first answer is:
"Check with the latest (nightly) release".
>
Which means installing something *outside*
the Linux distribution support.
And this impossible, because this will pull
in dependencies like crazy, which are not
(yet) in the Linux distribution path.
>
Saying it differently, the latest greatest
update is not a wish, it's a must...
>
So, long story short, the only solution I
know are venvs...
>
Of course, other solutions are welcome!
>
bye,
>
--
>
piergiorgio
>
--
https://mail.python.org/mailman/listinfo/python-list