My Problem
After a failed yum update I was no longer able to run yum. I received errors that culminated in:
Error: rpmdb open failed
My Solution
Back up the rpm directory with something like tar -czvf /var/lib/rpm.bak /var/lib/rpm/
Then use the Berkeley DB tools db_dump and db_load to reload the Packages database:
$ mv /var/lib/rpm/Packages /var/lib/rpm/Packages.borked
$ db_dump -f /var/lib/rpm/Packages.borked /var/lib/rpm/Packages.dump
$ db_load -f /var/lib/rpm/Packages.dump /var/lib/rpm/Packages
The Long Story (and alternative fixes)
When attempting to do pretty much anything with yum, such as yum update or yum-complete-transaction, I would receive a rather nasty looking error:
error: rpmdb: BDB0113 Thread/process 8630/140300564338496 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db5 - (-30973) error: cannot open Packages database in /var/lib/rpm CRITICAL:yum.main: Error: rpmdb open failed
These errors were preceded by an attempt to simply yum update install -y a CentOS 7 machine, however the update failed because the kernel oomkilled the yum process. As a result, the rpm database was a bit unhappy.
The Packages file is a Berkeley DB database, so the first thing I did was checked the file with db_verify /var/lib/rpm/Packages. Oddly, it appeared to be fine.
$ db_verify /var/lib/rpm/Packages BDB5105 Verification of /var/lib/rpm/Packages succeeded.
Attempting to recover the database was of no use since there were apprently no log file to replay:
$ db_recover -fv
BDB2525 No log files found
Another trick is to dump and reload the database. db_dump has a few options to consider. Most notably -r and -R — both used for recovery, one being more aggressive than the other. Check man pages for more info.
$ mv /var/lib/rpm/Packages /var/lib/rpm/Packages.borked $ db_dump -f /var/lib/rpm/Packages.borked /var/lib/rpm/Packages.dump $ db_load -f /var/lib/rpm/Packages.dump /var/lib/rpm/Packages
In my specific case, that did the trick, however there were some possible options that remained for me:
Most notably, db_dump itself has -r and -R options for recovery from a corrupted database file. I’d strongly suggest 1) Having a backup of the Packages file (and a backup of the backup), and 2) Reading the man page for db_dump a few times to have a basic idea of what each flag is going to do.
Another option would be to completely rebuild the RPM database with rpm --rebuilddb.