Title: CS544 / 01-foundations
Tags: cs544-foundations

Q: What do systems primarily manage?
A: Resources
Major resource types: compute, memory, storage, and network

Q: what does a compiler do?
A: translate high-level code (something a programmer would write) to low-level code, such as machine code that a CPU can directly execute

Q: what does an interpreter do?
A: loop over high-level code and execute corresponding operations on the CPU, as the program executes

Q: What is a language (other than Java) that compiles to bytecode for the JVM?
A: Scala is the best answer (important for us, because we'll learn Spark, which was written in Scala).
Python could technically be an answer too due to the Jython project, but this probably isn't used much in practice.

Q: Consider high level code (like Python), machine code, and byte code -- what can run directly on a CPU?
A: machine code

Q: What does JVM stand for?
A: Java Virtual Machine

Q: What is a common extension for files containing Java bytecode?
A: .class

Q: What is a common extension for files containing Python bytecode?
A: .pyc

Q: What does this Python code do?
import dis
dis.dis("????")
A: Shows you the Python bytecode corresponding to the Python code in the string.

Q: If you start using more machines, what kind of scaling is it?
A: scaling out

Q: If you start using a more powerful machine, what kind of scaling is it?
A: scale up

Q: What does CPU stand for?
A: Central Processing Unit

Q: What does GPU stand for?
A: Graphics Processing Unit

Q: What does RAM stand for?
A: Random Access Memory

Q: What does HDD stand for?
A: Hard Disk Drive

Q: What does SSD stand for?
A: Solid State Disk

Q: What does NIC stand for?
A: Network interface card/controller

Q: What does it mean for a device to provide random access?
A: You can jump around and read from different locations efficiently.

Q: What values can a bit hold?
A: 0 or 1

Q: How many different values can a byte hold?
A: A byte has 8 bits, so there are 2^8 different possible combinations of bits.  So 256 different values.

Q: How much is 1 KB?
A: 1024 (or 1000) bytes

Q: How much is 1 MB?
A: 1024 KB (or 1000 KB)

Q: How much is 1 GB?
A: 1024 (or 1000) MB

Q: How much is 1 TB?
A: 1024 (or 1000) GB

Q: What does it mean for a device to be "byte addressible"?
A: Each byte has it's own address, and the CPU can use that address to get the value.
Extracting a single bit from a byte actually involves more steps than using the whole byte.

Q: What is generally faster, storage or memory?
A: Memory

Q: What does volatile mean, and what is an example of hardware that is volatile?
A: Volatile means you lose the contents upon reboot.  Memory (e.g., RAM) is volatile, storage (e.g., SSD) is non-volatile.

Q: What do we mean with we call HDDs and SSDs "block devices"?
A: Data is read/written in blocks of many bytes (for example, 0.5 KB).
Accessing 1 byte in the block takes the same time as accessing the whole block.

Q: A device can store 1000 GB, can transfer 100 MB/s, and serve one request in 10 ms.  What kinds of metrics are these, respectively?
A: 1000 GB: capacity
100 MB/s: throughput
10 ms: latency

Q: If throughput can be expressed as X MB/s or Y Mbps, what is the relationship between X and Y?
A: Y = X * 8
(because the "b" in Mbps stands for bits, and there are 8 bits in a byte)

Q: often a bunch of server machines are stacked on top of each other in a metal box called a ______ and connected with a network switch
A: rack

Q: a large facility containing many racks of computers is called a ______
A: data center

Q: a cluster is a ______
A: bunch of computers working together

Q: what is a program you can use to run shell programs on another machine?
A: ssh

Q: what is the technical term for "folder"?
A: "directory" -- "d" shows up in many acronyms/commands for this reason (pwd, cd, etc)

Q: What Linux command tells you see what directory you're in?
A: pwd

Q: What Linux command shows you all the contents of a directory?
A: ls

Q: How do you create a new empty file, file.txt?
A: touch file.txt

Q: What Linux command do you use to install software on Ubuntu?
A: apt

Q: What is a Linux command you can use to download from a URL to a file on your computer?
A: wget SOME_URL
(curl is a similar tool)

Q: What Linux command lets you rename a file?
A: mv name1.txt name2.txt
It also lets you move a file, like this:
mv name.txt somedir

Q: How can you copy from file A to file B?
A: cp A B

Q: What Linux command lets you copy files between computers?
A: scp

Q: What Linux command lets you see everything in a file?
A: cat

Q: What Linux commands let you see just the start or end of a file?
A: head file.txt
tail file.txt

Q: What Linux command creates a new directory?
A: mkdir somedir

Q: What command lets you see entries in the Linux manual?
A: man SOMECOMMAND

Q: How do you navitage to a new directory?
A: cd SOMEDIR

Q: How do you run commands as the root user?
A: sudo COMMAND

Q: What Linux command lets you change the access permissions on a file?
A: chmod

Q: Given a program name, how can you find where it is on your computer?
A: which PROGRAMNAME

Q: How can you see an environment variable (for example, PATH)?
A: echo $PATH

Q: How can you pipe the standard output of A to the standard input of B?
A: A | B

Q: How do you redirect the standard output of A to file.txt, creating file.txt (or replacing any prior content)?
A: A > file.txt

Q: How do you redirect the standard output of A to file.txt, creating file.txt (or appending to the end of any prior content)?
A: A >> file.txt

Q: How do you send BOTH standard output and standard error to a file?
A: myprogram &> file.txt

Q: What program counts lines, words, and characters?
A: wc

Q: What program filters standard input, writing matching lines to standard output?
A: grep

Q: What program recursively finds directories and files?
A: find

Q: How can you run a program asyncronously?  (meaning you can type other things on the prompt while waiting for it to finish)
A: longprogram &

Q: How can you see running programs and their PIDs?  (process IDs)
A: ps
To see more:
ps ax
a=other users
x=not just processes attached to a terminal

Q: How do you kill a process by its PID number?
A: kill PID

Q: How do you kill a process by its name?
A: pkill NAME

Q: How can you see CPU and memory usage on your machine?
A: htop

Q: What Linux command shows you disk utilization?
A: df
("df -h" shows human readable)

Q: What command shows you disk usage by everything in the current directory?
A: du -hs .

Q: How can you see most port numbers being used by processes on your machine?
A: sudo ss -tlpn

Q: If a file is continually growing (for example, a program is logging to it), how can you run the "tail" command so that you keep seeing additional content as it is written to the file?
A: tail -f somefile.txt
Without following (-f), tail will print the current end of the file and exit, without waiting for future data to be written.

Q: What keyboard shortcut can you use to run the previous shell command again?
A: Up arrow key

Q: What keyboard shortcut can you use to kill a program running in a shell
A: Control C
(it's not copy!)

Q: What keyboard shortcut can you use to indicate you are done typing, and want to exit the shell (or other program taking input)?
A: Control D

Q: What keyboard shortcut can you use to search over previous shell commands?
A: Control R
(reverse search)

Q: In the shell, if you start typing a few characters and want the shell to complete it, what key should you hit?
A: TAB

Q: How can you set an environment variable MYVAR to value xyz, in a way that's visible to programs you will later launch in the shell session?
A: export MYVAR=xyz

Q: If you want to download an image X from DockerHub to your VM, what command do you use?
A: docker pull X

Q: How can you see all the Docker images on your VM?
A: docker images

Q: How can you give a Docker image "a" an additional second name, "b"?
A: docker tag a b

Q: How can you launch a container from image X?
A: docker run X

Q: How can you see existing containers, including those exited?
A: For running containers:
docker ps
For exited ones too, add "-a":
docker ps -a

Q: How can you delete a container X?
A: docker rm X

Q: How can you delete an image X?
A: docker rmi X

Q: How can you see Docker's disk utilization?
A: docker system df

Q: How can you cleanup unnecessary Docker data to free up space?
A: docker system prune

Q: How can you see the output (for example prints) for an application running inside container X?
A: docker logs X

Q: How do you run a command (lets say "ls") inside a running container X?
A: docker exec X ls

Q: How do you see resource consumption of running containers?
A: docker stats

Q: How do you get a bash shell inside an existing container X?
A: docker exec -it X bash

Q: How do you cause a container X to immediately quit?
A: docker kill X
This sends a kill signal, which the container cannot ignore.

Q: How do you use the Dockerfile in the current directory to build an image X?
A: docker build . -t X

Q: What is the difference between "docker rm" and "docker kill"
A: After "docker kill", the container won't be running, but it will still exist (you can see it with "docker ps -a").  "docker rm" actually cleans up the space that was used by that container.

Q: In a Dockerfile, how do you specify a base image X?
A: FROM X

Q: In a Dockerfile, how do you execute a command, say "apt update"?
A: RUN apt update

Q: In a Dockerfile, how do you copy a file X.txt (in the same directory on the VM) to /Y.txt (in the image)?
A: COPY X.txt /Y.txt

Q: In a Dockerfile, why is a single install line preferred?
RUN apt-get update && apt-get install -y ...
In contrast to two lines:
RUN apt-get update
RUN apt-get install -y ...
A: The "update" refreshes the VM's knowledge of what packages are in the apt repositories, and "install" actuall installs something from a repository.  If you wait too long between an update and install, apt's view of the repositories will be stale, and the install won't work.
Docker builds cache results on a line-by-line basis to avoid running repeat commands.  That means that it might not run "apt-get update" again if it has been run previously.  Usually that's ok, but in this case we want it to be recent.

Q: How do you cause a container X to quit gracefully?
A: docker stop X
This sends a terminate signal to the container, which the application can catch and then handle by cleaning up before stopping.  If it takes too long, a kill signal will be sent, though.