Alternative

Python on MacOS BigSur

Date: 20.10.2021

Author: Patrick Rottländer

When you want to use Python as a programming language on MacOS you will first be confused about it because Python is preinstalled in an outdated version by Mac OS BigSur. It is not advisable to touch or even delete the so-called System Python version. Additional Python versions have to be managed and that is exactly what this article is about.

MacOS comes with Python 2 pre-installed. Python 2 is required by the Operating System and managed by MacOS. This System Python should never be touched.

System Python is installed in /usr/bin directory which is owned by root. This mean all files and symlinks in /usr/bin are system owned which means managed by the system and not managed by the user. In /usr/bin you find python symlinks pointing to the Python binaries in /System/Library/Frameworks/Python.framework/Versions/2.7/bin directory.

patrick@PatrickMBNeu ~ % ls -l /usr/bin/python* 

lrwxr-xr-x  1 root  wheel      75  1 Jan  2020 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7

lrwxr-xr-x  1 root  wheel      75  1 Jan  2020 /usr/bin/python2 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7

lrwxr-xr-x  1 root  wheel      75  1 Jan  2020 /usr/bin/python2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7

....

patrick@PatrickMBNeu ~ %

The symlinks python, python2 and python2.7 in /usr/bin each point to the same binary in /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7. So when you type one of these as shell command in your shell terminal the binary in /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 will be executed.

patrick@PatrickMBNeu ~ % python --version
Python 2.7.16

patrick@PatrickMBNeu ~ % python2 --version
Python 2.7.16

patrick@PatrickMBNeu ~ % python2.7 --version
Python 2.7.16

patrick@PatrickMBNeu

Type in the command without using the path like /usr/bin/python works because /usr/bin is configured in the shell PATH variable.

patrick@PatrickMBNeu ~ % echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

patrick@PatrickMBNeu ~ %  

The shell lookup for a command in the paths defined in the PATH variable and starts on the left side of the PATH. So python, python2 and python2.7 do not exist in /usr/local/bin but in /usr/bin.

When you came from Catalina and upgraded to BigSur you realized that System Python has not been upgraded from Python 2 to Python 3 for example. So obviously BigSur still relies on Python 2 which is ok for your Mac but maybe not for you as a Developer. You might need newer or different versions of Python on your system. You must install this them separately.

Many people use Homebrew to install software packages on their MacOS. Homebrew can also be used to install a Python version in addition to the System Python. You can check if Python is installed via Homebrew on your system using the command brew info \.

patrick@PatrickMBNeu ~ % brew info python
python@3.9: stable 3.9.7 (bottled)
Interpreted, interactive, object-oriented programming language
https://www.python.org/
/usr/local/Cellar/python@3.9/3.9.7_1 (3,191 files, 56MB) *
  Poured from bottle on 2021-10-15 at 07:09:27
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/python@3.9.rb
License: Python-2.0
==> Dependencies
Build: pkg-config ✔
Required: gdbm ✔, mpdecimal ✔, openssl@1.1 ✔, readline ✔, sqlite ✔, xz ✔
==> Caveats
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python@3.9/libexec/bin

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.9/site-packages

tkinter is no longer included with this formula, but it is available separately:
  brew install python-tk@3.9

See: https://docs.brew.sh/Homebrew-and-Python
==> Analytics
install: 717,857 (30 days), 1,988,212 (90 days), 8,421,529 (365 days)
install-on-request: 322,364 (30 days), 784,390 (90 days), 2,811,664 (365 days)
build-error: 0 (30 days)

patrick@PatrickMBNeu ~ %

Here on my system Python has been installed with Homebrew in the directory /usr/local/bin which is owned by the user. That means all files and symlinks in this directory can be managed by the user. In /usr/local/bin you find the python symlinks python3 and python3.9 pointing to further symlinks in the Homebrew Cellar in /usr/local/Cellar.

patrick@PatrickMBNeu ~ % ls -l /usr/local/bin/python*

lrwxr-xr-x  1 patrick  admin  40 15 Okt 07:09 /usr/local/bin/python3 -> ../Cellar/python@3.9/3.9.7_1/bin/python3

lrwxr-xr-x  1 patrick  admin  42 15 Okt 07:09 /usr/local/bin/python3.9 -> ../Cellar/python@3.9/3.9.7_1/bin/python3.9

...

patrick@PatrickMBNeu ~ %

The python3 symlink in the Homebrew Cellar point again to another python3 symlink in /usr/local/Frameworks/Python.framework/Versions/3.9/bin and this symlink point finally to python3.9 binary /usr/local/Frameworks/Python.framework/Versions/3.9/bin. The python3.9 symlink from the Homebrew Cellar point directly to the python3.9 binary in /usr/local/Frameworks/Python.framework/Versions/3.9/bin.

patrick@PatrickMBNeu ~ % ls -l /usr/local/Frameworks/Python.framework/Versions/3.9/bin
total 136

lrwxr-xr-x  1 patrick  admin      9 30 Aug 21:19 python3 -> python3.9

-rwxr-xr-x  1 patrick  admin  50472 15 Okt 07:09 python3.9

...

patrick@PatrickMBNeu ~ % 

So If you enter python3 or python3.9 as a command in the terminal the shell lookup these symlinks in directory /usr/local/bin and find a match. In any case you always end up with the python3.9 binary in /usr/local/Frameworks/Python.framework/Versions/3.9/bin and execute the same binary.

patrick@PatrickMBNeu ~ % python3 --version
Python 3.9.7

patrick@PatrickMBNeu ~ % python3.9 --version
Python 3.9.7

patrick@PatrickMBNeu ~ %

It is not a problem to install other Python versions with Homebrew in addition to the System Python, but the Homebrew ecosystem has a special feature: dependencies. Python versions that are installed in the Homebrew perimeter can also represent a dependency on other applications. node, for example, needs Python 3.9. If we uninstall Python 3.9, we have a problem with node. Therefore, Python can never be seen completely isolated in the Homebrew ecosystem and this is exactly the problem we solve with pyenv.

patrick@PatrickMBNeu ~ % brew info node   

node: stable 16.11.1 (bottled), HEAD
Platform built on V8 to build network applications
https://nodejs.org/
/usr/local/Cellar/node/16.11.1 (1,980 files, 45.4MB) *
  Poured from bottle on 2021-10-15 at 07:09:45
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/node.rb
License: MIT
==> Dependencies
Build: pkg-config ✔
Required: brotli ✔, c-ares ✔, icu4c ✔, libnghttp2 ✔, libuv ✔, openssl@1.1 ✔, python@3.9 ✔
==> Options
--HEAD
    Install HEAD version
==> Analytics
install: 501,912 (30 days), 1,345,537 (90 days), 4,731,822 (365 days)
install-on-request: 404,509 (30 days), 1,070,393 (90 days), 3,650,272 (365 days)
build-error: 0 (30 days)

patrick@PatrickMBNeu ~ %

pyenv create a complete isolated environment for your Python versions. You are able to install, use and uninstall Python versions and you can switch between them. pyenv can be easily installed with Homebrew. pyenv is therefore installed in the /usr/local/bin directory.

patrick@PatrickMBNeu ~ % which pyenv
/usr/local/bin/pyenv

patrick@PatrickMBNeu ~ % pyenv --version
pyenv 2.1.0

patrick@PatrickMBNeu ~ % 

We must adapt the PATH variable for the standard shell. Here in my example this is configured in the .zshrc file in my Home-Directory.

patrick@PatrickMBNeu ~ % cat .zshrc

# ~/.pyenv provides Python before others of the same name
export PYENV_ROOT=$(pyenv root)
export PATH="$PYENV_ROOT/shims:$PATH"

patrick@PatrickMBNeu ~ %

You check the versions managed by pyenv with the following command. As you see pyenv realize the System Python. With the command pyenv install 3.9.7 you install Python 3.9.7 in your pyenv environment. You can switch to your desired Python version using the command pyenv global \ .

When you then run the command python you are directly connected to global python version you selected.

patrick@PatrickMBNeu ~ % pyenv versions 
*  system

patrick@PatrickMBNeu ~ % pyenv install 3.9.7

.....

patrick@PatrickMBNeu ~ % pyenv versions 
* system
  3.9.7 (set by /Users/patrick/.pyenv/version)

patrick@PatrickMBNeu ~ % pyenv global 3.9.7

patrick@PatrickMBNeu ~ % pyenv versions 
  system
* 3.9.7 (set by /Users/patrick/.pyenv/version)

patrick@PatrickMBNeu ~ % python
Python 3.9.7 (default, Oct 19 2021, 16:02:07) 
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

patrick@PatrickMBNeu ~ % pyenv global system

patrick@PatrickMBNeu ~ % pyenv versions     
* system (set by /Users/patrick/.pyenv/version)
  3.9.7

patrick@PatrickMBNeu ~ % python             

WARNING: Python 2.7 is not recommended. 
This version is included in macOS for compatibility with legacy software. 
Future versions of macOS will not include Python 2.7. 
Instead, it is recommended that you transition to using 'python3' from within Terminal.

Python 2.7.16 (default, Aug 30 2021, 14:43:11) 
[GCC Apple LLVM 12.0.5 (clang-1205.0.19.59.6) [+internal-os, ptrauth-isa=deploy on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

patrick@PatrickMBNeu ~ %

Finally: To avoid a warning message from brew when you run brew doctor you must create a brew alias in your .zshrc file.

patrick@PatrickMBNeu ~ % cat .zshrc

alias brew='PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin brew'

# ~/.pyenv provides Python before others of the same name
export PYENV_ROOT=$(pyenv root)
export PATH="$PYENV_ROOT/shims:$PATH"

patrick@PatrickMBNeu ~ %