Using the "magic SysRq key" to deal with an otherwise unresponsive system

Though relatively obscure there is a set of key combinations that can be used to communicate and send commands to a GNU/Linux kernel even when the system has become unresponsive (frozen, hung, crashed). I have found it beneficial to keep these commands in mind because I have come across a couple of situations where their use proved valuable. If nothing else, the 1 thing to remember is the Alt+SysRq+H command to get help on the other combinations.

If Alt+SysRq+H is pressed it should display a very short help message on the console. If not it’s worth trying Alt+SysRq+R which takes the keyboard out of raw more and trying again. Regardless of the outcome at this point other commands can be entered in order to minimize data loss on the system. Mnemonics are often suggested as a way to remember the keys and their order to be entered to maximize the safety of the data. “Raising Elephants Is So Utterly Boring” and “Raising Skinny Elephants Is Utterly Boring” are 2 alternatives. In each case the first letter is combined with Alt and SysRq to issue a command. The only difference between the 2 is when to sync disks with the Alt+SysRq+S command. I’ve also combined the 2 and synced the disks both before and after terminating processes with Alt+SysRq+E and Alt+SysRq+I.

Following the mnemonic should result in a rebooted system with minimal data loss. However, there may be other ways to get the system running again. Some commands such as those issued by keys ‘t’, ‘m’, and ‘p’ can be used to dump diagnostic information. The keys 0-9 control the level of verbosity in kernel logging to the console (0 being the most quiet and 9 being the most verbose). If a rogue process has frozen the system by consuming too much memory using the Alt+SysRq+F may be successful in killing that 1 process.

Keeping these keys in mind can be invaluable when faced with a system that has stopped responding. It is not guaranteed to work but it may at least allow the administrator to minimize data loss by syncing data to the disks and avoiding a hard reset.

For more information:
* Magic SysRq key (Wikipedia)
* The magic sysreq options introduced