I have a couple of systems which used to have python2 as well as
python3 but as Ubuntu and Debian verions have moved on they have
finally eliminated all dependencies on python2.

So they now have only python3 and there is no python executable in
PATH.

There's still both /usr/bin/pip and /usr/bin/pip3 but they're
identical so presuably I can now simply use pip and it will be a
python3 pip.

So, going on from this, how do I do the equivalent of "apt update; apt
upgrade" for my globally installed pip packages?

--
Chris Green
·

On 2023-11-02, Chris Green <cl@isbd.net> wrote:
> I have a couple of systems which used to have python2 as well as
> python3 but as Ubuntu and Debian verions have moved on they have
> finally eliminated all dependencies on python2.
>
> So they now have only python3 and there is no python executable in
> PATH.
>
> There's still both /usr/bin/pip and /usr/bin/pip3 but they're
> identical so presuably I can now simply use pip and it will be a
> python3 pip.
>
>
> So, going on from this, how do I do the equivalent of "apt update; apt
> upgrade" for my globally installed pip packages?

I'm not sure what that question has to do with everything that preceded
it, but you don't want to install python packages globally using pip.
Either install them with 'apt', or install them in a virtual environment.

Chris Green wrote at 2023-11-2 10:58 +0000:
> ...
>So, going on from this, how do I do the equivalent of "apt update; apt
>upgrade" for my globally installed pip packages?

`pip list -o` will tell you for which packages there are upgrades
available.
`pip install -U ...` will upgrade packages.

Be careful, though.
With `apt`, you usually have (`apt`) sources representing a consistent
package universe. Someone tests that package upgrades in this
universe do not break other packages (in this universe).
Because of this, upgrading poses low risk.

`PyPI` does not guarantes consistency. A new package version
may be incompatible to a previous one -- and with other
package you have installed.

I do not think that you would want to auto-upgrade all installed
packages.

On 2023-11-02, Dieter Maurer <dieter@handshake.de> wrote:
> Chris Green wrote at 2023-11-2 10:58 +0000:
>> ...
>>So, going on from this, how do I do the equivalent of "apt update; apt
>>upgrade" for my globally installed pip packages?
>
> `pip list -o` will tell you for which packages there are upgrades
> available.
> `pip install -U ...` will upgrade packages.
>
> Be careful, though.
> With `apt`, you usually have (`apt`) sources representing a consistent
> package universe. Someone tests that package upgrades in this
> universe do not break other packages (in this universe).
> Because of this, upgrading poses low risk.
>
> `PyPI` does not guarantes consistency. A new package version
> may be incompatible to a previous one -- and with other
> package you have installed.
>
> I do not think that you would want to auto-upgrade all installed
> packages.

Indeed. What you're describing is a very unfortunate failing of pip.
'Upgrade' doesn't even follow requirements when you tell it what to
upgrade - e.g. if you do "pip install foo" and foo requires "bar<2"
so you end up with:

Package Version
---------------------- ---------
foo 1.0.0
bar 1.2.0

and then a new version 1.3.0 of bar comes out and you do
"pip install -U foo", pip will not upgrade bar even though it could
and should, because foo is already at the latest version so pip won't
even look at its dependencies.

Indeed there is no way of knowing that you should upgrade bar without
manually following all the dependency graphs. ("pip list -o" will tell
you there's a newer version, but that isn't the same - e.g. if the new
version of bar was 2.0.0 then "pip list -o" will list it, but you should
not upgrade to it.)

You can do "pip install -I foo", which will pointlessly reinstall foo
and then presumably upgrade bar as well, thus probably getting to the
right result via a rather roundabout route, but I'm not sure if that
does indeed work properly and if it is a reliable and recommended way
of doing things.

Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
> On 2023-11-02, Chris Green <cl@isbd.net> wrote:
> > I have a couple of systems which used to have python2 as well as
> > python3 but as Ubuntu and Debian verions have moved on they have
> > finally eliminated all dependencies on python2.
> >
> > So they now have only python3 and there is no python executable in
> > PATH.
> >
> > There's still both /usr/bin/pip and /usr/bin/pip3 but they're
> > identical so presuably I can now simply use pip and it will be a
> > python3 pip.
> >
> >
> > So, going on from this, how do I do the equivalent of "apt update; apt
> > upgrade" for my globally installed pip packages?
>
> I'm not sure what that question has to do with everything that preceded
> it, but you don't want to install python packages globally using pip.
> Either install them with 'apt', or install them in a virtual environment.

Why in a virtual environment? When I install a package whether from
apt or from pip I want everyone/everything on my system to be able to
use it.

I do only install a few things using pip.

--
Chris Green
·

Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
> On 2023-11-02, Dieter Maurer <dieter@handshake.de> wrote:
> > Chris Green wrote at 2023-11-2 10:58 +0000:
> >> ...
> >>So, going on from this, how do I do the equivalent of "apt update; apt
> >>upgrade" for my globally installed pip packages?
> >
> > `pip list -o` will tell you for which packages there are upgrades
> > available.
> > `pip install -U ...` will upgrade packages.
> >
> > Be careful, though.
> > With `apt`, you usually have (`apt`) sources representing a consistent
> > package universe. Someone tests that package upgrades in this
> > universe do not break other packages (in this universe).
> > Because of this, upgrading poses low risk.
> >
> > `PyPI` does not guarantes consistency. A new package version
> > may be incompatible to a previous one -- and with other
> > package you have installed.
> >
> > I do not think that you would want to auto-upgrade all installed
> > packages.
>
> Indeed. What you're describing is a very unfortunate failing of pip.
> 'Upgrade' doesn't even follow requirements when you tell it what to
> upgrade - e.g. if you do "pip install foo" and foo requires "bar<2"
> so you end up with:
>
> Package Version
> ---------------------- ---------
> foo 1.0.0
> bar 1.2.0
>
> and then a new version 1.3.0 of bar comes out and you do
> "pip install -U foo", pip will not upgrade bar even though it could
> and should, because foo is already at the latest version so pip won't
> even look at its dependencies.
>
> Indeed there is no way of knowing that you should upgrade bar without
> manually following all the dependency graphs. ("pip list -o" will tell
> you there's a newer version, but that isn't the same - e.g. if the new
> version of bar was 2.0.0 then "pip list -o" will list it, but you should
> not upgrade to it.)
>
> You can do "pip install -I foo", which will pointlessly reinstall foo
> and then presumably upgrade bar as well, thus probably getting to the
> right result via a rather roundabout route, but I'm not sure if that
> does indeed work properly and if it is a reliable and recommended way
> of doing things.

It is a bit of a minefield isn't it. I try to minimise my use of
packages installed using pip for this very reason. Maybe the safest
route would simply be to uninstall everything and then re-install it.
·

On 2023-11-02, Chris Green <cl@isbd.net> wrote:
> Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
>> On 2023-11-02, Chris Green <cl@isbd.net> wrote:
>> > I have a couple of systems which used to have python2 as well as
>> > python3 but as Ubuntu and Debian verions have moved on they have
>> > finally eliminated all dependencies on python2.
>> >
>> > So they now have only python3 and there is no python executable in
>> > PATH.
>> >
>> > There's still both /usr/bin/pip and /usr/bin/pip3 but they're
>> > identical so presuably I can now simply use pip and it will be a
>> > python3 pip.
>> >
>> >
>> > So, going on from this, how do I do the equivalent of "apt update; apt
>> > upgrade" for my globally installed pip packages?
>>
>> I'm not sure what that question has to do with everything that preceded
>> it, but you don't want to install python packages globally using pip.
>> Either install them with 'apt', or install them in a virtual environment.
>
> Why in a virtual environment? When I install a package whether from
> apt or from pip I want everyone/everything on my system to be able to
> use it.

Because pip barely plays well by itself, let alone with other package
managers at the same time.

> I do only install a few things using pip.

Are they not available in your system's package manager?
I guess you might get away with "sudo -H pip install -U foo"
for a couple of things, if they don't have many dependencies.

On 2023-11-02, Chris Green <cl@isbd.net> wrote:
> Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
>> On 2023-11-02, Dieter Maurer <dieter@handshake.de> wrote:
>> > Chris Green wrote at 2023-11-2 10:58 +0000:
>> >> ...
>> >>So, going on from this, how do I do the equivalent of "apt update; apt
>> >>upgrade" for my globally installed pip packages?
>> >
>> > `pip list -o` will tell you for which packages there are upgrades
>> > available.
>> > `pip install -U ...` will upgrade packages.
>> >
>> > Be careful, though.
>> > With `apt`, you usually have (`apt`) sources representing a consistent
>> > package universe. Someone tests that package upgrades in this
>> > universe do not break other packages (in this universe).
>> > Because of this, upgrading poses low risk.
>> >
>> > `PyPI` does not guarantes consistency. A new package version
>> > may be incompatible to a previous one -- and with other
>> > package you have installed.
>> >
>> > I do not think that you would want to auto-upgrade all installed
>> > packages.
>>
>> Indeed. What you're describing is a very unfortunate failing of pip.
>> 'Upgrade' doesn't even follow requirements when you tell it what to
>> upgrade - e.g. if you do "pip install foo" and foo requires "bar<2"
>> so you end up with:
>>
>> Package Version
>> ---------------------- ---------
>> foo 1.0.0
>> bar 1.2.0
>>
>> and then a new version 1.3.0 of bar comes out and you do
>> "pip install -U foo", pip will not upgrade bar even though it could
>> and should, because foo is already at the latest version so pip won't
>> even look at its dependencies.
>>
>> Indeed there is no way of knowing that you should upgrade bar without
>> manually following all the dependency graphs. ("pip list -o" will tell
>> you there's a newer version, but that isn't the same - e.g. if the new
>> version of bar was 2.0.0 then "pip list -o" will list it, but you should
>> not upgrade to it.)
>>
>> You can do "pip install -I foo", which will pointlessly reinstall foo
>> and then presumably upgrade bar as well, thus probably getting to the
>> right result via a rather roundabout route, but I'm not sure if that
>> does indeed work properly and if it is a reliable and recommended way
>> of doing things.
>
> It is a bit of a minefield isn't it. I try to minimise my use of
> packages installed using pip for this very reason. Maybe the safest
> route would simply be to uninstall everything and then re-install it.

That is literally what I do quite often - completely erase the
virtual env and then re-create it from scratch - because it seems
to be the only / easiest way to upgrade the packages to the latest
versions consistent with given dependencies.

On 11/2/23 04:58, Chris Green via Python-list wrote:
> I have a couple of systems which used to have python2 as well as
> python3 but as Ubuntu and Debian verions have moved on they have
> finally eliminated all dependencies on python2.
>
> So they now have only python3 and there is no python executable in
> PATH.

FWIW, for this you install the little stub package python-is-python3.
Especially if you want to keep a python2 installation around - "python"
will still be python3 in this case.

> So, going on from this, how do I do the equivalent of "apt update; apt
> upgrade" for my globally installed pip packages

Odds are you don't want to. The internet is full of surprises about
dependency problems when stuff is blindly updated; the set of Python
packages in the apt repositories is carefully curated to avoid these
problems - and this is part of the reason why sometimes certain such
packages are irritatingly down-rev.

Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
> On 2023-11-02, Chris Green <cl@isbd.net> wrote:
> > Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
> >> On 2023-11-02, Chris Green <cl@isbd.net> wrote:
> >> > I have a couple of systems which used to have python2 as well as
> >> > python3 but as Ubuntu and Debian verions have moved on they have
> >> > finally eliminated all dependencies on python2.
> >> >
> >> > So they now have only python3 and there is no python executable in
> >> > PATH.
> >> >
> >> > There's still both /usr/bin/pip and /usr/bin/pip3 but they're
> >> > identical so presuably I can now simply use pip and it will be a
> >> > python3 pip.
> >> >
> >> >
> >> > So, going on from this, how do I do the equivalent of "apt update; apt
> >> > upgrade" for my globally installed pip packages?
> >>
> >> I'm not sure what that question has to do with everything that preceded
> >> it, but you don't want to install python packages globally using pip.
> >> Either install them with 'apt', or install them in a virtual environment.
> >
> > Why in a virtual environment? When I install a package whether from
> > apt or from pip I want everyone/everything on my system to be able to
> > use it.
>
> Because pip barely plays well by itself, let alone with other package
> managers at the same time.
>
Well that's a criticism of pip rather than of how I use it! :-)

OK, there are risks.

> > I do only install a few things using pip.
>
> Are they not available in your system's package manager?
> I guess you might get away with "sudo -H pip install -U foo"
> for a couple of things, if they don't have many dependencies.

I obviously check the package manager and also other sources which
work through apt before resorting to using pip.

The sort of thing I have to use pip for is software for odd I2C sensor
devices and such. These rarely have any dependencies or they are all
the same dependencies as the other I2C device software. So,
hopefully, I won't hit any big problems. I have to say that so far I
haven't been bitten.

I would point out that this is mostly for a headless Beaglebone Black
single board computer (comparable with a Raspberry Pi) with only a
minimal 'console' installation of Debian so it's a pretty minimal
system.

--
Chris Green
·

Am Thu, Nov 02, 2023 at 09:35:43PM -0000 schrieb Jon Ribbens via Python-list:

Regardless of ...

> Because pip barely plays well by itself, let alone with other package
> managers at the same time.

.... being true ...

> > I do only install a few things using pip.
>
> Are they not available in your system's package manager?

.... this clearly often answers to "no" for applications of
any complexity.

Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

On 2023-11-03, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
> Am Thu, Nov 02, 2023 at 09:35:43PM -0000 schrieb Jon Ribbens via Python-list:
>
> Regardless of ...
>
>> Because pip barely plays well by itself, let alone with other package
>> managers at the same time.
>
> ... being true ...
>
>> > I do only install a few things using pip.
>>
>> Are they not available in your system's package manager?
>
> ... this clearly often answers to "no" for applications of
> any complexity.
>
> Is there a suggested proper path to deal with that (Debian is
> of interest to me here) ?

Yes, as previously mentioned, use virtual environments.

These days they don't even need to be "activated". For package 'foo'
for example you could create /usr/local/lib/foo, under which you would
create a virtual environment and install the 'foo' package inside it,
and then you could do:

ln -s /usr/local/lib/foo/env/bin/foo /usr/local/bin/foo

and then you could just type 'foo' to run it.

Karsten Hilbert wrote at 2023-11-3 14:47 +0100:
> ...
>> Are they not available in your system's package manager?
>
>... this clearly often answers to "no" for applications of
>any complexity.
>
>Is there a suggested proper path to deal with that (Debian is
>of interest to me here) ?

Complex applications may maintain a set of "known workable versions"
associated with the application's releases.
They may describe those "known workable versions" in a `pip` constraint file.
In this case, you can upgrade to a new application release
by using this constraint file.

Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:

> >So they now have only python3 and there is no python executable in
> >PATH.
>
> FWIW, for this you install the little stub package python-is-python3. Especially if you
> want to keep a python2 installation around - "python" will still be python3 in this
> case.

Since you seem knowledgeable in this area: Do you know of a
resource for learning the *canonical* way of packaging a
Python application for installation via apt which

- needs some packages available via apt
- needs some packages only available via pip
- needs some packages newer than what is available via apt

?

Thanks,
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

On 2023-11-03, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
> Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:
>> >So they now have only python3 and there is no python executable in
>> >PATH.
>>
>> FWIW, for this you install the little stub package python-is-python3.
>> Especially if you want to keep a python2 installation around -
>> "python" will still be python3 in this case.
>
> Since you seem knowledgeable in this area: Do you know of a
> resource for learning the *canonical* way of packaging a
> Python application for installation via apt which
>
> - needs some packages available via apt
> - needs some packages only available via pip
> - needs some packages newer than what is available via apt
>
> ?

I suspect the answer to that is that you would have to:

* create packages yourself for the unpackaged dependencies
* create a dependency graph of *every* Python package in the package
repository (whether or not the package is relevant to what you're doing)
* work out what versions of every Python package are required in order
to have a dependency graph that can be successfully resolved, taking
into account the requirements of your new package also
* contact every single maintainer of every single one of the packages
that needs updating and persuade them to update their packages and
reassure them that you are getting all the other package maintainers
to update their packages accordingly and that you have a plan and
that you know what you're doing

... screen fades to black, title card "3 years later", fade in to ...

* publish your package

Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
> On 2023-11-03, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
> > Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:
> >> >So they now have only python3 and there is no python executable in
> >> >PATH.
> >>
> >> FWIW, for this you install the little stub package python-is-python3.
> >> Especially if you want to keep a python2 installation around -
> >> "python" will still be python3 in this case.
> >
> > Since you seem knowledgeable in this area: Do you know of a
> > resource for learning the *canonical* way of packaging a
> > Python application for installation via apt which
> >
> > - needs some packages available via apt
> > - needs some packages only available via pip
> > - needs some packages newer than what is available via apt
> >
> > ?
>
> I suspect the answer to that is that you would have to:
>
> * create packages yourself for the unpackaged dependencies
> * create a dependency graph of *every* Python package in the package
> repository (whether or not the package is relevant to what you're doing)
> * work out what versions of every Python package are required in order
> to have a dependency graph that can be successfully resolved, taking
> into account the requirements of your new package also
> * contact every single maintainer of every single one of the packages
> that needs updating and persuade them to update their packages and
> reassure them that you are getting all the other package maintainers
> to update their packages accordingly and that you have a plan and
> that you know what you're doing
>
> ... screen fades to black, title card "3 years later", fade in to ...
>
> * publish your package
>
Surely it's not that bad, the vast bulk of Debian, Ubuntu and other
distributions are installed via systems that sort out dependencies once
given a particular package's requirements. Python is surely not
unique in its dependency requirements.

--
Chris Green
·

On 2023-11-05, Chris Green <cl@isbd.net> wrote:
> Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
>> On 2023-11-03, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
>> > Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:
>> >> >So they now have only python3 and there is no python executable in
>> >> >PATH.
>> >>
>> >> FWIW, for this you install the little stub package python-is-python3.
>> >> Especially if you want to keep a python2 installation around -
>> >> "python" will still be python3 in this case.
>> >
>> > Since you seem knowledgeable in this area: Do you know of a
>> > resource for learning the *canonical* way of packaging a
>> > Python application for installation via apt which
>> >
>> > - needs some packages available via apt
>> > - needs some packages only available via pip
>> > - needs some packages newer than what is available via apt
>> >
>> > ?
>>
>> I suspect the answer to that is that you would have to:
>>
>> * create packages yourself for the unpackaged dependencies
>> * create a dependency graph of *every* Python package in the package
>> repository (whether or not the package is relevant to what you're doing)
>> * work out what versions of every Python package are required in order
>> to have a dependency graph that can be successfully resolved, taking
>> into account the requirements of your new package also
>> * contact every single maintainer of every single one of the packages
>> that needs updating and persuade them to update their packages and
>> reassure them that you are getting all the other package maintainers
>> to update their packages accordingly and that you have a plan and
>> that you know what you're doing
>>
>> ... screen fades to black, title card "3 years later", fade in to ...
>>
>> * publish your package
>>
> Surely it's not that bad, the vast bulk of Debian, Ubuntu and other
> distributions are installed via systems that sort out dependencies once
> given a particular package's requirements. Python is surely not
> unique in its dependency requirements.

I think there's a lot of work that goes on behind the scenes to keep the
entire package set consistent so that you don't end up in the situation
where, e.g. package A depends on D<2.0 and package B requires D>=2.0 and
therefore you can't install A and B at the same time (and trying to
avoid as much as possible the hacky situation where you have two
packages for D, one for <2.0 and another called D2 for >=2.0).

Am Fri, Nov 03, 2023 at 05:26:06PM +0100 schrieb Dieter Maurer:

> Karsten Hilbert wrote at 2023-11-3 14:47 +0100:
> > ...
> >> Are they not available in your system's package manager?
> >
> >... this clearly often answers to "no" for applications of
> >any complexity.
> >
> >Is there a suggested proper path to deal with that (Debian is
> >of interest to me here) ?
>
> Complex applications may maintain a set of "known workable versions"
> associated with the application's releases.
> They may describe those "known workable versions" in a `pip` constraint file.
> In this case, you can upgrade to a new application release
> by using this constraint file.

Hello Dieter,

do you happen to know where to read up on how to fit a pip
constraint file into a Debian package creation workflow ?

Thanks,
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

Am Fri, Nov 03, 2023 at 01:53:32PM -0000 schrieb Jon Ribbens via Python-list:

> >> Are they not available in your system's package manager?
> >
> > ... this clearly often answers to "no" for applications of
> > any complexity.
> >
> > Is there a suggested proper path to deal with that (Debian is
> > of interest to me here) ?
>
> Yes, as previously mentioned, use virtual environments.
>
> These days they don't even need to be "activated". For package 'foo'
> for example you could create /usr/local/lib/foo, under which you would
> create a virtual environment and install the 'foo' package inside it,
> and then you could do:
>
> ln -s /usr/local/lib/foo/env/bin/foo /usr/local/bin/foo
>
> and then you could just type 'foo' to run it.

This all being nice and well, but:

How does one "fill" that venv with packages from pip during

apt-get install python3-app-of-interest

?

Is the suggested way really to pip-install into this venv
during apt-get install ?

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

Am Sun, Nov 05, 2023 at 03:00:41PM +0000 schrieb Chris Green via Python-list:

> > * contact every single maintainer of every single one of the packages
> > that needs updating and persuade them to update their packages and
> > reassure them that you are getting all the other package maintainers
> > to update their packages accordingly and that you have a plan and
> > that you know what you're doing
> >
> > ... screen fades to black, title card "3 years later", fade in to ....
> >
> > * publish your package
> >
> Surely it's not that bad, the vast bulk of Debian, Ubuntu and other
> distributions are installed via systems that sort out dependencies once
> given a particular package's requirements. Python is surely not
> unique in its dependency requirements.

Oh, Debian does just fine in resolving dependencies it knows
about within its .deb package universe. The problem arises
when there's unpackaged modules required. The only answer
seems to be to package such modules oneself.

If that's been conquered a venv isn't even needed anymore.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

On 2023-11-05, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
> Am Fri, Nov 03, 2023 at 01:53:32PM -0000 schrieb Jon Ribbens via Python-list:
>
>> >> Are they not available in your system's package manager?
>> >
>> > ... this clearly often answers to "no" for applications of
>> > any complexity.
>> >
>> > Is there a suggested proper path to deal with that (Debian is
>> > of interest to me here) ?
>>
>> Yes, as previously mentioned, use virtual environments.
>>
>> These days they don't even need to be "activated". For package 'foo'
>> for example you could create /usr/local/lib/foo, under which you would
>> create a virtual environment and install the 'foo' package inside it,
>> and then you could do:
>>
>> ln -s /usr/local/lib/foo/env/bin/foo /usr/local/bin/foo
>>
>> and then you could just type 'foo' to run it.
>
> This all being nice and well, but:
>
> How does one "fill" that venv with packages from pip during
>
> apt-get install python3-app-of-interest
>
> ?
>
> Is the suggested way really to pip-install into this venv
> during apt-get install ?

I don't know what you mean by that. But if you install the apt packages
and then create your venv with --system-site-packages then I believe
your venv should be able to see the apt packages and import them.

Karsten Hilbert wrote at 2023-11-5 23:19 +0100:
> ...
>do you happen to know where to read up on how to fit a pip
>constraint file into a Debian package creation workflow ?

I have only rudimentary `apt` knowledge.

I know it is quite flexible, e.g. it used to handle `flash`
in a special way. I expect it flexible enough to load from `PyPI`.

`apt` is essentially controlled by a set of `sources`.
Each "source" describes a list of package distributions.
A description contains metadata about the distribution
including name, version, description, dependencies, incompatibilities, etc.

Any integration would require you to define one or more
"source"s, listing the distributions which should come from `PyPI`.
This would allow you to specify potential upgrades at a central
place, the "source"s, you control.
It would not solve the more fundamental problem: to determine
which versions are compatible with the universe of (all the
"sources" described) package versions.

I know that debian packagers create debian packages
from Python distributions not using the approach sketched above
and likely they have their reasons.

You might want to discuss this on an `apt` related mailing list.

Am Mon, Nov 06, 2023 at 01:17:11AM -0000 schrieb Jon Ribbens via Python-list:

> >> >> Are they not available in your system's package manager?
> >> >
> >> > ... this clearly often answers to "no" for applications of
> >> > any complexity.
> >> >
> >> > Is there a suggested proper path to deal with that (Debian is
> >> > of interest to me here) ?
> >>
> >> Yes, as previously mentioned, use virtual environments.
> >>
> > How does one "fill" that venv with packages from pip during
> >
> > apt-get install python3-app-of-interest
> >
> > ?
> >
> > Is the suggested way really to pip-install into this venv
> > during apt-get install ?
>
> I don't know what you mean by that. But if you install the apt packages
> and then create your venv with --system-site-packages then I believe
> your venv should be able to see the apt packages and import them.

The problem is when there's modules not available via apt as
packages. In that case on needs to either package them or pip
them into the venv suggested above.

When they are apt-gettable no venv is needed in the first
place. One would just install the application script like any
other binary, and which loads apt-installed modules just so.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

Am Mon, Nov 06, 2023 at 08:58:00AM +0100 schrieb Dieter Maurer:

> I know that debian packagers create debian packages
> from Python distributions not using the approach sketched above
> and likely they have their reasons.
>
> You might want to discuss this on an `apt` related mailing list.

Yeah, I guess. I know all this stuff about python modules
being packaged for Debian and how apt handles dependencies
etc etc.

I had just hoped someone here might have a handy pointer for
how to deal with modules having to be installed from pip for
use with an apt-installed python-based application.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

On 11/6/23 14:28, Karsten Hilbert via Python-list wrote:

> I had just hoped someone here might have a handy pointer for
> how to deal with modules having to be installed from pip for
> use with an apt-installed python-based application.

That just shouldn't happen - such packages are supposed to be
dependency-complete within the packaging universe in question.