So if you manage to compromise a system you need to make sure that you do not lose the shell. If you have used an exploit that messes with the machine the user might want to reboot, and if the user reboots you will lose your shell.
Or, maybe the way to compromise the machine is really complicated or noisy and you don't want to go through the hassle of doing it all again. So instead you just create a backdoor that you can enter fast and easy.
Create a new user
The most obvious, but not so subtle way is to just create a new user (if you are root, or someone with that privilege) .
adduser pelle
adduser pelle sudo
Now if the machine has ssh you will be able to ssh into the machine.
On some machines, older Linux I think, you have to do
useradd pelle
passwd pelle
echo "pelle ALL=(ALL) ALL" >> /etc/sudoers
Crack the password of existing user
Get the /etc/shadow file and crack the passwords. This is of course only persistent until the user decides to change his/her password. So not so good.
SSH key
Add key to existing ssh-account.
Cronjob NC
Create cronjob that connects to your machine every 10 minutes. Here is an example using a bash-reverse-shell. You also need to set up a netcat listener.