Off-the-Record (OTR) Messaging is an amazing cryptographic protocol granting neat properties to your conversations: encryption, authentication, deniability as well as perfect forward secrecy. It was created by Nikita Borisov, Eric Brewer and Ian Goldberg, around 2004, with a version 2 in 2005, and a v3 in 2012.
I've written an article in French about its inner working and properties. If you don't want to take this opportunity to learn to read French, but are still interested in learning more about OTR, you should check this slide deck. It was an amazing cryptographic construction at the time.
Unfortunately, the hardcoded modulus it use is the 1536-bit MODP Group, which isn't safe according to today's standards, and attacks against it are believed to be both practical, and practised. Moreover it's using 1024-bit DSA keys for signatures, which are also dangerously low. This means that the effective security level of OTRv3 is around 80bits for signatures, and ~90 bits for the key exchange, which are now way too small to be comfortable.
Nowadays, while we'll all be using waiting on OTRv4 to be completed and deployed, I would recommend using the Signal protocol, inspired by OTR but improved in every way: stronger crypto, asynchronous messages, post-compromise security, group chats, … if you're using XMPP, you can also use OMEMO as well.