Public Key Certificate
Note: Read on if you wish to understand and apply the concepts of certificates. It is not necessary in order to use the network, with the assumption that you can trust certificates signed by a particular network -- this check will be automatic unless you turn it off and you'll only notice if something is wrong with the certificate.
A public key comes with a name-tag which identifies the owner of the key. A certificate is a digital 'document' providing evidence that the entity on the tag for the key is who they say they are. It is a certificate of identification which can be distributed along with the public key. It is important to have this as there are two sides to every key -- the digital side which exists in an abstract and virtual space where the operations are performed, and the other side which consist of the organisation and the tangible people behind the key. There is no way to tell that the key belongs to the entity from the key alone, it requires evidence from others vouching that the owner is the same as that claimed by the key. A bit like a name tag on some luggage provides no proof that the person behind that tag is really that person.
When you retrieve or receive a Certificate, it will come with a list of one or more signatures. These signatures are Digital Signatures made by other private keys. The signatures are effectively saying that the people behind those signatures -- the owners of the private keys -- vouch for the identity as claimed by the key's name-tag. Now if we trust those signatures then we can trust that the organisation is the same as claimed.
When you have another organisation's public key and associated Certificate, and you know that the organisation is the same as that claimed on the public key, then you can also use your private key to sign the certificate thus adding more weight to the evidence provided by the certificate.
Integrity of Signatures
You don't want someone signing certificates pretending to be you so, as with the normal digital signing process, the signatures on a certificate can be verified as being done by a particular private key by checking them against their complimentary public key.
Web of Trust
The question arises -- how do you know that the public key used for verifying the signature on the certificate actually belongs to the claimed organisation? The answer is you check that signature's public key certificate which should have other signatures vouching that the organisation behind the public key is the same as claimed. This expands out into what is known as a Web of Trust. It is distributed model of trust, where if you trust a particular signature then it means you can trust that the key is valid.
Every day usage
A more narrow implementation is used for securing web site identification in HTTPS and is used every day across the Internet. A Certificate Authority (CA) signs a web site's certificate using their private key to vouch that the website is who they say they are. The CAs public key is held by web browsers (like Firefox or Chrome). When you visit a website on a secure connection, the signature on the website's certificate is verified using the public key of the CA held in the browser. In this special case, the browsers automatically trust the public key of the certificate authorities.