I disagree about the copying. You should protect your private key wherever you take it, but copying it is certainly a reasonable thing to do under some circumstances. With an appropriate passphrase, storing your private key in a PKCS#12 file is just as secure as storing the private key in software-based OS or application keystore. In fact it may be identical depending on the OS or application. You are subject to the same attacks (password jacking, in memory key copying) in both cases. I keep some of my lower-value private keys on a USB stick for exactly this scenario.
The only way (IMHO) to get around those attacks is to never decrypt the key on a machine with untrusted software running and accessible memory. The only device that comes close to this is a smartcard or TPM type scenario, which uses a separate CPU and protected memory to do the RSA operations.
It depends on what you are doing. You can do public key operations without the private key (for example signature verification). You can do private key operations without the public key (for example signing). In practice the certificate is always available near the private key, but the reverse is usually not true -- there are many cases where you will have the certificate but not the private key.
Not to be too pedantic here -- but what do you mean by "PRIVATE CRYPTOGRAPHIC CERTIFICATES"? Certificates are intended to be public. Private keys are intended to be private. They are not equivalent although they are certainly related.
Except that in most "real" two-factor systems, you can't duplicate the second factor easily. The problem here is if the private key is in main memory, malware can copy it to another computer that will be indistinguishable (for the purposes of authentication) from the first. With something like a smartcard that is made much more difficult. Not impossible, but beyond the reach of most folks.
The only way (IMHO) to get around those attacks is to never decrypt the key on a machine with untrusted software running and accessible memory. The only device that comes close to this is a smartcard or TPM type scenario, which uses a separate CPU and protected memory to do the RSA operations.