Linux bash cheat sheet

Some definitions

• Linux is an open-source operating system. It comes under different “flavours” or distributions such as Ubuntu, Arch Linux, Fedora, etc…
• Bash - for Bourne Again SHell - is Linux’ command line interpreter (or shell) that provides a command line interface inside a terminal emulator.

Cheat sheet

You can download a printable Linux command line cheat sheet here from Cheatography.

1. Find files

• Find files match a pattern

find -name "query"

• Find files match a pattern, case insensitive

find -iname "query" 

• Find all files that don’t contain the pattern

find -not -name "query_to_avoid"

• Find all files in current directory

find . -type f

• Find all directories in current directory

find . -type d

• Find all symbolic links in current directory

find . -type l 

• Find all items ending in “.conf” in current directory

find . -name *.conf

• Find all files in the current directory starting with “pro”

find . -name pro* 

• Find all items containing “yolo” in current directory

find . -name *yolo*

• Find pattern ‘tensorflow’ in python files in the current directory

find . -name *.py | grep -rnw tensorflow 

Option explained:

• -r or -R : recursive
• -n : show line number
• -w : match the whole word

• -l (lower-case L) : give the file name of matching files (optional).

• Find pattern in file recursively

grep -rnw '/path/to/somewhere/' -e 'pattern'

grep -r 'pattern'

• Move file with extension recursively to a directory

find /thisdir -type f -name '*.ogg' -exec mv -i {} /somedir \;

2. Backup installed packages

• Write list of packages

sudo dpkg --get-selections "*" > my_packages.txt

• Install list of packages from txt file

sudo dpkg --set-selections < my_packages.txt
sudo apt-get -u dselect-upgrade

• 3. Command line history

• Search command line history (shortcut)

CTRL + r

• View command line history

history

4. Copy to/from SSH server

• When logged in locally, copy from local to remote

scp /file/to/send username@remote:/where/to/put
# OR
rsync -avzh /file/to/send username@remote:/where/to/put

where the rsync options are:

• -a : archive mode, which preserves permissions, ownership, and modification times
• -v : enables verbose
• -z : enables compression during transfer

• -h : outputs numbers in human-readable format

• When logged in locally, from remote to local

scp username@remote:/file/to/send /where/to/put
# OR
rsync -avzh username@remote:/file/to/send /where/to/put

• When logged in remotely, copy from remote to local

sftp username@remote
get -r "remote/dir/path" "local/dir/path"

• When logged in remotely, copy from local to remote

sftp username@remote
put -r "local/dir/path" "remote/dir/path"

(note: in Ubuntu, you may encounter this error: “Couldn’t canonicalise: No such file or directory”. Solution: create a directory on the remote first using mkdir)

• Launch a job on an SSH server and be able to log off without killing the job

nohup python my_script.py >& log.run &

5. File ownership / make executable

• Change directory ownership

chown -R user:group /directory/of/interest

• Make script executable

chmod +x "file name"

6. System specs

• Disk usage of the entire filesystem

df -h --total
df -h

• Disk usage of current directory

du -sh .

• Show disk usage of the 40 biggest subfolders in the current directory (ranked)

du -hsx * | sort -rh | head -n 40

• Show disk usage of all the subfolders in the current directory one by one

du -hsx *

• Show disk usage of the biggest subfolders in the current directory (ranked), with hidden directories

du -sch .[!.]* * | sort -rh

• In Ubuntu, this nice tool can explore the disk usage of your entire system quickly

sudo apt-get install ncdu
ncdu

• View system specs (and export them to a html file)

sudo lshw
sudo lshw -html > specs.html

• Check Linux Kernel

uname -a

• Check Ubuntu version

lsb_release -a

• Print total, free and used RAM memory

free -h

• List CPU and processor info

lscpu

• Print generic hardware information

hwinfo

• Print PCI buses, including graphics card, network adapter

lspci

• List block devices (storage and partitions)

lsblk

• List devices connected via USB

cd /dev
ls

• View running processes

top  
htop

• Print the process ID of a running task

pidof firefox

7. Kill a running job

kill 'process ID'

Note, you can find process ID using the ‘top’ command or using pidof.

8. Use SLURM (job scheduler for computing clusters)

• Submit job

sbatch job.sh

• View queue

squeue -u "user name"  
squeue

9. Resolve merging conflicts with Git

git mergetool --tool=emerge

It will open 3 windows: version a on top left, version b on top right and final version at the bottom.

• press n for next change
• press a or b to choose which version I want to keep
• press q to quit and save

10. Manipulate PDF documents with pdftk

• Cut pdf pages

pdftk full-pdf.pdf cat 12-15 output outfile_p12-15.pdf

• Merge pdf pages

pdftk file1.pdf file2.pdf file3.pdf cat output newfile.pdf

11. Make Gifs and videos from images using ffmpeg

• Create video from images

ffmpeg -i input_0%d.png -vcodec libx264 ouput.mp4
ffmpeg -i input_%d.png -filter:v "setpts=10.0*PTS" -vcodec libx264 ouput.mp4
ffmpeg -start_number 1 -i input%d.png -filter:v "setpts=2*PTS" -filter:v "crop=490:360:15:15" -r 48 -c:v libx264 -crf 0 -profile:v high444 -preset slow -b:v 15M ouput.mp4

• Create gif from video

ffmpeg -t 30 -i input.mp4 -vf "fps=10,scale=1280:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 output.gif
ffmpeg -t 30 -i input.mp4 -vf "fps=24,scale=300:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 output.gif

Create gif from images using Convert (from ImageMagik)

convert -resize 50% -delay 20 -loop 0 input_{0..99}.jpg output.gif
convert -resize 50% -delay 20 -loop 0 *.jpg output.gif
convert -resize 50% -delay 20 -loop 0 `ls -v` output.gif

Create gif from images using Gimp (GUI, potentially more robust than Convert)

1. Select File Menu > Open as Layers > Select all images you want to be in the GIF > Open

2. Select Filters from main Menu > Animation > Click Optimize for GIF

3. Save GIF Select File > click Export as > Select File Type as gif > Select ‘As Animation’ > Select ‘Loop Forever’

12. Useful shortcuts

• Open a terminal new widow

CTRL + ALT +T

• Open a new terminal tab

CTRL + SHIFT + T

• Close the current terminal tab

CTRL + SHIFT + W

• Kill the current running process

CTRL + C

• Exit Python interpreter / log out from SSH

CTRL + D 

• Copy-paste in terminal with a mouse
  1. select text
  2. move cursor where you want to paste
  3. paste with a middle mouse click
• Insert the euro sign (€)

CTRL+SHIFT+u
20ac
"enter"

• Put task in background

CTRL + C 

• Bring it back to the foreground

fg

• Nicely formated ls command

ls -lrt

• Download from website

wget http://www.website-name.com
curl http://www.website-name.com

• Execute the previous command with sudo

  sudo !!
  

• Login as root

  sudo -i
  

• Logout as root

  CTRL + D