The Missing Semester of Your CS Education

Topic 1: The Shell

The $ tells you that you are not the root user.

If you want to provide an argument that contains spaces or other special characters (e.g., a directory named “My Photos”), you can either quote the argument with ' or " ("My Photos"), or escape just the relevant characters with \ (My\ Photos).

One thing you need to be root in order to do is writing to the sysfs file system mounted under /sys. sysfs exposes a number of kernel parameters as files, so that you can easily reconfigure the kernel on the fly without specialized tools. Note that sysfs does not exist on Windows or macOS.

Operations like |, >, and < are done by the shell, not by the individual program. echo and friends do not “know” about |.

1
2

#!/bin/sh
curl --head --silent https://missing.csail.mit.edu

It’s helpful to know that # starts a comment in Bash, and ! has a special meaning even within double-quoted (") strings. Bash treats single-quoted strings (') differently: they will do the trick in this case.

Shebang (Unix)

Shell Scripting

To assign variables in bash, use the syntax foo=bar and access the value of the variable with $foo. Note that foo = bar will not work since it is interpreted as calling the foo program with arguments = and bar.

Strings in bash can be defined with ' and " delimiters, but they are not equivalent. Strings delimited with ' are literal strings and will not substitute variable values whereas " delimited strings will.

1
2
3
4
5

foo=bar
echo "$foo"
# prints bar
echo '$foo'
# prints $foo

Unlike other scripting languages, bash uses a variety of special variables to refer to arguments, error codes, and other relevant variables.

  • $0 - Name of the script

  • $1 to $9 - Arguments to the script. $1 is the first argument and so on.

  • $@ - All the arguments

  • $# - Number of arguments

  • $? - Return code of the previous command

  • $$ - Process identification number (PID) for the current script

  • !! - Entire last command, including arguments. A common pattern is to execute a command only for it to fail due to missing permissions; you can quickly re-execute the command with sudo by doing sudo !!

    学CS学了这么久，才知道这个快速恢复上一个命令的快捷方式。遇到很多次permissions denied都是手动敲sudo和剩下命令，的确是够蠢🙃。

  • $_ - Last argument from the last command. If you are in an interactive shell, you can also quickly get this value by typing Esc followed by . or Alt+.

diff <(ls foo) <(ls bar) will show differences between files in dirs foo and bar.

shell globbing

  • Wildcards - Whenever you want to perform some sort of wildcard matching, you can use ? and * to match one or any amount of characters respectively.
  • Curly braces {} - Whenever you have a common substring in a series of commands, you can use curly braces for bash to expand this automatically. This comes in very handy when moving or converting files.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

convert image.{png,jpg}
# Will expand to
convert image.png image.jpg

cp /path/to/project/{foo,bar,baz}.sh /newpath
# Will expand to
cp /path/to/project/foo.sh /path/to/project/bar.sh /path/to/project/baz.sh /newpath

# Globbing techniques can also be combined
mv *{.py,.sh} folder
# Will move all *.py and *.sh files


mkdir foo bar
# This creates files foo/a, foo/b, ... foo/h, bar/a, bar/b, ... bar/h
touch {foo,bar}/{a..h}
touch foo/x bar/y
# Show differences between files in foo and bar
diff <(ls foo) <(ls bar)
# Outputs
# < x
# ---
# > y

shell Tools

Finding how to use commands

Sometimes manpages can provide overly detailed descriptions of the commands, making it hard to decipher what flags/syntax to use for common use cases. TLDR pages are a nifty complementary solution that focuses on giving example use cases of a command so you can quickly figure out which options to use. For instance, I find myself referring back to the tldr pages for tar and ffmpeg way more often than the manpages.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

$ tldr tar

  tar

  Archiving utility.
  Often combined with a compression method, such as gzip or bzip2.
  More information: https://www.gnu.org/software/tar.

  - [c]reate an archive and write it to a [f]ile:
    tar cf target.tar file1 file2 file3

  - [c]reate a g[z]ipped archive and write it to a [f]ile:
    tar czf target.tar.gz file1 file2 file3

  - [c]reate a g[z]ipped archive from a directory using relative paths:
    tar czf target.tar.gz --directory=path/to/directory .

  - E[x]tract a (compressed) archive [f]ile into the current directory [v]erbosely:
    tar xvf source.tar[.gz|.bz2|.xz]

  - E[x]tract a (compressed) archive [f]ile into the target directory:
    tar xf source.tar[.gz|.bz2|.xz] --directory=directory

  - [c]reate a compressed archive and write it to a [f]ile, using [a]rchive suffix to determine the compression program:
    tar caf target.tar.xz file1 file2 file3

  - Lis[t] the contents of a tar [f]ile [v]erbosely:
    tar tvf source.tar

  - E[x]tract files matching a pattern from an archive [f]ile:
    tar xf source.tar --wildcards "*.html"

TLDR 展示也太清晰了！

Finding files

1
2
3
4
5
6
7
8

# Find all directories named src
find . -name src -type d
# Find all python files that have a folder named test in their path
find . -path '*/test/*.py' -type f
# Find all files modified in the last day
find . -mtime -1
# Find all zip files with size in range 500k to 10M
find . -size +500k -size -10M -name '*.tar.gz'

1
2
3
4

# Delete all files with .tmp extension
find . -name '*.tmp' -exec rm {} \;
# Find all PNG files and convert them to JPG
find . -name '*.png' -exec convert {} {}.jpg \

Finding shell commands

If we want to search there we can pipe that output to grep and search for patterns. history | grep find will print commands that contain the substring “find”.

In most shells, you can make use of Ctrl+R to perform backwards search through your history. After pressing Ctrl+R, you can type a substring you want to match for commands in your history. As you keep pressing it, you will cycle through the matches in your history. This can also be enabled with the UP/DOWN arrows in zsh. A nice addition on top of Ctrl+R comes with using fzf bindings. fzf is a general-purpose fuzzy finder that can be used with many commands. Here it is used to fuzzily match through your history and present results in a convenient and visually pleasing manner.

Editors (Vim)

Vim has multiple operating modes.

  • Normal: for moving around a file and making edits
  • Insert: for inserting text
  • Replace: for replacing text
  • Visual (plain, line, or block): for selecting blocks of text
  • Command-line: for running a command

You change modes by pressing <ESC> (the escape key) to switch from any mode back to Normal mode. From Normal mode, enter Insert mode with i, Replace mode with R, Visual mode with v, Visual Line mode with V, Visual Block mode with <C-v> (Ctrl-V, sometimes also written ^V), and Command-line mode with :.

Basics

Buffers, tabs, and windows

Vim maintains a set of open files, called “buffers”. A Vim session has a number of tabs, each of which has a number of windows (split panes). Each window shows a single buffer. Unlike other programs you are familiar with, like web browsers, there is not a 1-to-1 correspondence between buffers and windows; windows are merely views. A given buffer may be open in multiple windows, even within the same tab. This can be quite handy, for example, to view two different parts of a file at the same time.

By default, Vim opens with a single tab, which contains a single window.

Movement

You should spend most of your time in Normal mode, using movement commands to navigate the buffer. Movements in Vim are also called “nouns”, because they refer to chunks of text.

  • Basic movement: hjkl (left, down, up, right)

  • Words: w (next word), b (beginning of word), e (end of word)

  • Lines: 0 (beginning of line), ^ (first non-blank character), $ (end of line)

  • Screen: H (top of screen), M (middle of screen), L (bottom of screen)

  • Scroll: Ctrl-u (up), Ctrl-d (down)

  • File: gg (beginning of file), G (end of file)

  • Line numbers: :{number}<CR> or {number}G (line {number})

  • Misc: % (corresponding item)

  • Search: /{regex}, n / N for navigating matches

Edits

  • ienter Insert mode

    • but for manipulating/deleting text, want to use something more than backspace

  • o / O insert line below / above

  • d{motion}delete {motion}

    • e.g. dw is delete word, d$ is delete to end of line, d0 is delete to beginning of line

  • c{motion}change {motion}

    • e.g. cw is change word
    • like d{motion} followed by i

  • x delete character (equal do dl)

  • s substitute character (equal to cl)

  • Visual mode + manipulation

    • select text, d to delete it or c to change it

  • u to undo, <C-r> to redo

  • y to copy / “yank” (some other commands like d also copy)

  • p to paste

  • Lots more to learn: e.g. ~ flips the case of a character