Talk:Python

Improve the recommendations on the "Package management" section

I'd like to sugest some edits in this section, speciall concerning user installed python packages (not system wide installed) and available tools to do it.

Much as happened lately, and some new, solid and idiotproof tools are available today, that were not before. One good example is pipx, https://pipxproject.github.io/pipx/ . I noticed there's a surge in github reccomending this.

I cannot recommend enough, to read their README/FAQ page, as it helps understand not only their tool compared to others, but the recent panorama of similar python tools

This is in my opinion worth the effort, as python is a very popular language, and as much as a lot of stuff is available officially and/or the AUR, there will always be some interesting app, that isnt there.

This is especially important for end users (not programmers/developers) who just want to try out, in a clean and safe way, a (cli) python app, not available in the AUR. They are not focused on installing libraries or dependencies for programming or scripting their own app. It's important to stress and instruct the less informed users, as to not bork their system doing this.

As of now we have this super important opening tip:

   Official repositories and AUR — A large number of popular packages are available in the Arch repositories. This is the preferred way to install system-wide packages.

Then comes pip:

   pip — The official package installer for Python. You can use pip to install packages from the Python Package Index and other indexes.

This was for some time the only way to do it. Still many beginners are not aware of the difference between "pip install XXX" and "pip install --user XXX". And because the Internet is full of bad advice, many end up doing "sudo pip install"

This cannot stop being stressed: - warning about the "sudo pip install XXX" - mention the --user flag

I would suggest to move up here, this note that was at the end

   When installing packages from sources other than the official repositories and AUR, it is recommended to use a virtual environment (or Conda environment management) to prevent conflicts with system packages in /usr. Alternatively, pip install --user can be used to install packages into the user scheme instead of /usr.

Additionally, since it's so dead simple, with tools already available in the system, suggest right here that:

"pip install XXX" can be done in a virtual environment for quick testing, with the super simple:

   $ python -m venv envname
   $ source envname/bin/activate
   (envname) $

and then run the pip install inside it

   $ pip install XXX

with a link to, https://wiki.archlinux.org/index.php/Python/Virtual_environment

Now for the new kid on the block. Add an entry with a recommendation to pipx, https://pipxproject.github.io/pipx/. Maybe copy and reuse some of their documentation, especially:

- How is it Different from pip? - Comparison to Other Tools

Then comes this old entry for Conda, which isnt't even in the official repos, only AUR. So in my opinion should be at the end or removed from here

This should be moved to the end:

   Historically, easy_install (part of python-setuptools) was used to 

And almost hidden at the end:

  Tip: pipenv provides a single CLI for Pipfile, pip and virtualenv. It is available as python-pipenv.

pipenv should should be listed right after pipx.

-- M040601 (talk) 02:48, 1 October 2020 (UTC)

We won't make any recommendations based just on "surges" in Github recommendations. -- Lahwaacz (talk) 17:08, 1 October 2020 (UTC)

The easy_install mention is a holdover from an older version of the page that suggested using it. I added that note to make it clear that easy_install should no longer be used. I like the suggestion on making it more obvious why sudo pip is a bad idea. I also don't see harm in adding pipx to the list. Dharmab (talk) 23:52, 1 October 2020 (UTC)

Hello, I share most of what M040601 said and think the Package management section and Python/Virtual environment page need a (probably urgent) rework because of:

* Recent development of Python distribution. Specifically the rollout of PEP668, which now prevents system-wide pip package installations, even with --user; unless the --break-system-packages option is given.

* The recent popularity of AI tools, most of which are written in Python and have lots of dependencies on the AUR or not even there; leads many new non-python-developer users to try and install them not knowing how without breaking something or relying on not always updated/compatible AUR packages.

For starters I agree that the note:

   When installing packages using pip, it is recommended to use a virtual environment to prevent conflicts with system packages in /usr.

should be moved up somewhere or be made more obvious.

I also agree that we need a simple snippet of code detailing: create a virtual environment (any tool), activate it, then pip install (or poetry add if mentioning poetry). It will make it so much easier for someone to redirect to this wiki page without explaining additional things. Even many snippets of all the tools still commonly used in subsection examples would be fine: venv+pip, pipx, pipenv, poetry, conda

Then we need to remove the pip install --user recommendation and include a reference to PEP668. Also add a red warning for the --break-system-packages flag, and again redirect to instead opt for a virtual environment. Possibly a one sentence explanation of PEP668.

Also, clarify that easy_install is deprecated, possibly with a yellow warning.

Other things I think are worth considering:

* Clarify that after installation in a virtual env, the user can exit it (not even really mentioned in the Python/Virtual environment page), but then to use the app again it is recommended (sometimes necessary) to activate it again.

* I also think pipenv should be in the list.

* I haven't used conda recently but someone pointed me that after PEP668, the base conda environment is broken. Here is the issue. I don't know if this applies to non-base conda environments the user has created.

* For packages that require an old Python version, maybe also include a quick note for installing with pacman with a reference to the "Other versions" section, or installing with pyenv.

Xijang (talk) 04:05, 25 June 2023 (UTC)

Here are some thoughts on the stylistic concerns. The pipenv, easy_install, and conda stuff sound like accuracy/expansion concerns that needn't be discussed further if you have sources to include with the edits.

the [pip virtual-environment] note…should be moved up somewhere or be made more obvious.

+1. I would personally move it to right above the bulleted list, and wrap it inside the warning (and edit the contents a little to make sense there).

we need a simple snippet of code…

To provide per-tool CLI snippets, I think we would need to commit to having a subsection for each package manager in that case. If done right (without providing too much detail for each package manager), I could see that working out.

Clarify that after installation in a virtual env, the user can exit

This concern might be better confined to Python/Virtual environment. Thanks, -- CodingKoopa (talk) 18:20, 16 July 2023 (UTC)

Adding site-packages permissions workaround

A common problem when using secondary package managers such as pip are incorrect permissions given the system configuration. In pip's case it usually boils down to missing world-readable and directory-executable permissions in the site-packages directory. A quick and effective (but a bit crude) workaround is:

# SITE_PACKAGES="$(python -c 'import site; print(site.getsitepackages()[0])')" && chmod -R a+rX "$SITE_PACKAGES"

Possible unintended permission changes can be checked with paccheck --quiet --file-properties (and manually reverted), but all of this is a shotgun approach.

The solution is still better than most of what can be found around the 'nets. Is it safe enough to be added to the Troubleshooting section?

Drws (talk) 17:19, 22 July 2023 (UTC)

pip and other secondary package managers should NOT be used to install packages globally. Use a virtual environment or make a PKGBUILD. -- andreymal (talk) 18:09, 22 July 2023 (UTC)

I do use both approaches you mentioned, but was asked to make it work with pip in one case. Noticed there are many even worse solutions laying around and since many people seek information in Arch Wiki this one could be included with all the appropriate caveats and warnings. Advice against using secondary global package managers is already listed in applicable places and this section would be just one of them. Arch Wiki covers many advanced topics and offers other advice with caveats so I'm still not sure if this one needs to be swept under the rug, even if advised against. Drws (talk) 14:20, 28 July 2023 (UTC)

Advice against using a secondary global package manager is completely pointless when it is immediately followed by an example of using a secondary global package manager. — Lahwaacz (talk) 08:52, 29 July 2023 (UTC)

Fair point but still this is not an example of using a secondary package manager. While it does effectively encourage its usage a bit by offering one of the commands needed to make it work, this is dampened by warnings. What's more important is that it also offers working advice should the user need it for some reason. I believe I've seen similar cases in the Wiki, but if you all think it's best to hide this one away, that's OK too. Drws (talk) 21:46, 1 August 2023 (UTC)