Solved: “error: cannot open Packages database in /var/lib/rpm” and “Error: rpmdb open failed”

My Problem

After a failed yum update I was no longer able to run yum. I received errors that culminated in:

Error: rpmdb open failed

My Solution

Back up the rpm directory with something like tar -czvf /var/lib/rpm.bak /var/lib/rpm/

Then use the Berkeley DB tools db_dump and db_load to reload the Packages database:

$ mv /var/lib/rpm/Packages /var/lib/rpm/Packages.borked
$ db_dump -f /var/lib/rpm/Packages.borked /var/lib/rpm/Packages.dump
$ db_load -f /var/lib/rpm/Packages.dump /var/lib/rpm/Packages

The Long Story (and alternative fixes)

When attempting to do pretty much anything with yum, such as yum update or yum-complete-transaction, I would receive a rather nasty looking error:

error: rpmdb: BDB0113 Thread/process 8630/140300564338496 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 - (-30973)
error: cannot open Packages database in /var/lib/rpm

Error: rpmdb open failed

These errors were preceded by an attempt to simply yum update install -y a CentOS 7 machine, however the update failed because the kernel oomkilled the yum process. As a result, the rpm database was a bit unhappy.

The Packages file is a Berkeley DB database, so the first thing I did was checked the file with db_verify /var/lib/rpm/Packages. Oddly, it appeared to be fine.

$ db_verify /var/lib/rpm/Packages
BDB5105 Verification of /var/lib/rpm/Packages succeeded.

Attempting to recover the database was of no use since there were apprently no log file to replay:

$ db_recover -fv
BDB2525 No log files found

Another trick is to dump and reload the database. db_dump has a few options to consider. Most notably -r and -R — both used for recovery, one being more aggressive than the other. Check man pages for more info.

$ mv /var/lib/rpm/Packages /var/lib/rpm/Packages.borked
$ db_dump -f /var/lib/rpm/Packages.borked /var/lib/rpm/Packages.dump
$ db_load -f /var/lib/rpm/Packages.dump /var/lib/rpm/Packages

In my specific case, that did the trick, however there were some possible options that remained for me:

Most notably, db_dump itself has -r and -R options for recovery from a corrupted database file. I’d strongly suggest 1) Having a backup of the Packages file (and a backup of the backup), and 2) Reading the man page for db_dump a few times to have a basic idea of what each flag is going to do.

Another option would be to completely rebuild the RPM database with rpm --rebuilddb.