Zlib, where are you? Geriatric Mac users, beware.
Zlib, can you hear me?
Are you a MacOs user trying to use psycopg2 in VSCode? Is your Mac a little outdated and is sending you error after error when trying to use databases?
You're not alone and you're not silly. I spent 3 days trying to figure out what was wrong with my code, convinced it was something silly like a typo.
I re-read every file; others read my code. No one could figure out why was I getting this error??
Behold.
Collecting psycopg2
Using cached psycopg2-2.9.5.tar.gz (384 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [25 lines of output]
/Users/gracemckenna/Code/test_postgres/venv/lib/python3.10/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
warnings.warn(msg, warning_class)
running egg_info
creating /private/var/folders/rw/dtl9c56x0r5dm6l4qbxmgm9w0000gn/T/pip-pip-egg-info-_pa_g8u5/psycopg2.egg-info
writing /private/var/folders/rw/dtl9c56x0r5dm6l4qbxmgm9w0000gn/T/pip-pip-egg-info-_pa_g8u5/psycopg2.egg-info/PKG-INFO
writing dependency_links to /private/var/folders/rw/dtl9c56x0r5dm6l4qbxmgm9w0000gn/T/pip-pip-egg-info-_pa_g8u5/psycopg2.egg-info/dependency_links.txt
writing top-level names to /private/var/folders/rw/dtl9c56x0r5dm6l4qbxmgm9w0000gn/T/pip-pip-egg-info-_pa_g8u5/psycopg2.egg-info/top_level.txt
writing manifest file '/private/var/folders/rw/dtl9c56x0r5dm6l4qbxmgm9w0000gn/T/pip-pip-egg-info-_pa_g8u5/psycopg2.egg-info/SOURCES.txt'
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
Huh. Interesting error, right? One would assume it's a fairly simple fix considering the error has quite plainly told me what the problem is. Either I'm missing the pg_config file or the path to it is mixed up.
Simple, right?
Nope.
Down the rabbit hole I went.
So what is the pg_config?
pg_config is a command line utility usually included when you install PostgresSQL and is needed for Postgres extensions like pyscopg2. By using pg_config to retrieve installation details, you can ensure that the extensions are compiled against the correct version of PostgreSQL and using the correct header files and libraries.
Well, what is pyscopg2? It allows you to write Python code to interact with PostgreSQL databases and perform operations such as querying, inserting, updating, and deleting data. It's pretty essential, and if you're a Code Institute student, you'll be using it at some stage along with ElephantSQL.
This should be an easy fix, right?
Not only was this not an easy fix, it was also incredibly specific. For Mac, you usually install things via homebrew. One of the many things I tried was homebrew installing PostgresSQL, in hope that this would add the pg_config that was apparently missing.
Surprise! Another error; this one preventing me installing PostgreSQL!
Running `brew update --auto-update`... ==> Auto-updated Homebrew! Updated 2 taps (homebrew/core and homebrew/cask). ==> New Formulae aws-sso-cli clifm gmssl joern nixpacks ==> New Casks iconchamp tastytrade You have 2 outdated formulae installed. You can upgrade them with brew upgrade or list them with brew outdated. Warning: Use postgresql@14 instead of deprecated postgresql Warning: Use postgresql@14 instead of deprecated postgresql Warning: You are using macOS 10.14. We (and Apple) do not provide support for this old version. It is expected behaviour that some formulae will fail to build in this old version. It is expected behaviour that Homebrew will be buggy and slow. Do not create any issues about this on Homebrew's GitHub repositories. Do not create any issues even if you think this message is unrelated. Any opened issues will be immediately closed without response. Do not ask for help from Homebrew or its maintainers on social media. You may ask for help in Homebrew's discussions but are unlikely to receive a response. Try to figure out the problem yourself and submit a fix as a pull request. We will review it but may or may not accept it. ==> Fetching postgresql@14 ==> Downloading https://ftp.postgresql.org/pub/source/v14.7/postgresql-14.7.tar. ######################################################################## 100.0% Warning: A newer Command Line Tools release is available. Update them from Software Update in System Preferences. If that doesn't show you any updates, run: sudo rm -rf /Library/Developer/CommandLineTools sudo xcode-select --install Alternatively, manually download them from: https://developer.apple.com/download/all/. You should download the Command Line Tools for Xcode 11.3.1. ==> ./configure --prefix=/usr/local/Cellar/postgresql@14/14.7 --datadir=/usr/loc Last 15 lines from /Users/gracemckenna/Library/Logs/Homebrew/postgresql@14/01.configure: checking for library containing getopt_long... none required checking for library containing shm_open... none required checking for library containing shm_unlink... none required checking for library containing clock_gettime... none required checking for library containing fdatasync... none required checking for library containing shmget... none required checking for library containing backtrace_symbols... none required checking for library containing gethostbyname_r... no checking for library containing pthread_barrier_wait... no checking for library containing readline... -lreadline checking for inflate in -lz... no configure: error: zlib library not found If you have zlib already installed, see config.log for details on the failure. It is possible the compiler isn't looking in the proper directory. Use --without-zlib to disable zlib support. Do not report this issue to Homebrew/brew or Homebrew/homebrew-core! These open issues may also help: postgresql: enable gettext support https://github.com/Homebrew/homebrew-core/pull/124120 postgresql@14: enable gettext support https://github.com/Homebrew/homebrew-core/pull/124252 brew postgresql-upgrade-database fails with initb https://github.com/Homebrew/homebrew-core/issues/60686 Error: You are using macOS 10.14. We (and Apple) do not provide support for this old version. It is expected behaviour that some formulae will fail to build in this old version. It is expected behaviour that Homebrew will be buggy and slow. Do not create any issues about this on Homebrew's GitHub repositories. Do not create any issues even if you think this message is unrelated. Any opened issues will be immediately closed without response. Do not ask for help from Homebrew or its maintainers on social media. You may ask for help in Homebrew's discussions but are unlikely to receive a response. Try to figure out the problem yourself and submit a fix as a pull request. We will review it but may or may not accept it. Error: A newer Command Line Tools release is available. Update them from Software Update in System Preferences. If that doesn't show you any updates, run: sudo rm -rf /Library/Developer/CommandLineTools sudo xcode-select --install Alternatively, manually download them from: https://developer.apple.com/download/all/. You should download the Command Line Tools for Xcode 11.3.1.
What on earth? Zlib? I'm missing another thing?
Okay, let's install Zlib then!
Well... what now? When checked, homebrew lists Zlib as installed.
Yet when I tried to install PostgreSQL, it still said it was missing.
This genuinely felt like a never end cycle of error upon error. I thought I'd found a solution, then apparently I was missing something else. After 3 days of googling, crying and multiple other people trying to figure it out, I found this post on Stack Overflow.
It was 3am; I had been trying to fix this one issue for over 3 days. Honestly, my heart skipped a beat when I tried this solution because part of me was terrified it wouldn't work like everything else and I'd be back to square one! I was on the verge of restarting my entire project and wiping my Mac.
HomeBrew will says it's installing Zlib; will show it as installed, but trust me, it's not.
Basically, I had to manually download Zlib and install it. At this point I barely took in the reasoning behind it, but essentially because my Mac wasn't up to date, homebrew didn't function correctly. Even though I had run the command to install it, and it even showed it as installed in my homebrew list, Zlib was nowhere to be found.
Once I had it manually downloaded and installed, psycopg2 worked, and I finally got my database migrated.
I realise this probably isn't the most informative article on a coding issue, however my main reason for writing on this topic is so other people know this could be why you're having trouble.
It took me days to finally find the solution; if this article helps even just one person narrow down what could be wrong a little quicker, then my goal has been achieved.
