In RSA public-key cryptography the private and the public key paar are methematically related by sharing the same modulus. The length of the modulus, express in bits, is the key length and should also be the same.

To check that a private key matches the public key contained in a certificate signing request (CSR) and a certificate we have to check that the moduli of both keys are identical. This can be done with OpenSSL on Linux as follows:

check the MD5 hash of the private key:

openssl rsa -noout -modulus -in private.key | openssl md5

check the MD5 hash of the CSR:

openssl req -noout -modulus -in cert.csr | openssl md5

check the MD5 hash of the certificate:

openssl x509 -noout -modulus -in cert.crt | openssl md5

If all three hashes match, the CSR, certificate and private key are compatible.

Also it is a nice thing to backup these things if you have to regenerate a CSR the private key is really essential.