Let me tell you why you're here.
You're here because you know something, but you can't explain it.
You've felt it your entire career.
There's something wrong in your front end developer skillset -- you don't know what it is, but it's there.
Like a splinter in your mind. Driving you mad.
It is this feeling that has brought you to me. Do you know what I'm talking about?
Of course you do. It's Unix shell scripting ?
You never learned this world of scripting -- bash commands and .sh files just looked like something other devs did. You, much like me, just copied and pasted commands and hoped for the best.
It's good to take a little time to learn shell scripting, because it's helpful and it's actually not that hard to learn. The basic commands will take you far.
Knowing how to use shell commands will also make you a more valuable software developer. Plus, you will be just that much closer to seeing the matrix like Neo ?
In this first post of my Unix shell scripting series, I will try to explain shell scripting in a simple way, and I'll show you the foundational commands that will make you productive right away.
With that said, pick the red pill, stay in wonderland, and let's see how far the rabbit hole goes...
The MacOS user interface is hiding from you a lot of what happens behind the scenes. The reality is, everything you do on a Mac is being controlled by Unix commands -- moving files around, copying/saving, searching for files, downloading -- the list goes on and on. The visual layer that we interact with, is just a pretty looking design that lets us control the lower things.
Unix is the core of the Mac operating system. It is what runs "under the hood" of the pretty UI we are used to and interact with. You don't need to know much about Unix to get started using it, but here are some key points that are helpful:
Let's get you ready for our ride. Sit still. This won't hurt...
OK, you may have heard these before, but never really took time to know what they were.
Let's get these terms out of the way. You will see they are very easy to understand.
You control Unix through a command line interface, as I said earlier. Unix commands need to be interpreted into something the innermost/lower-level components of Unix can understand. Unix requires a "shell" to do this.
The shell is the program that sits between us and the low level Unix things, that manages a lot of technical details, and interprets our commands to Unix in a way Unix can understand and execute our commands. Any time we write a command on the command line, we are doing it through a shell, therefore we never write commands outside of a shell program.
Unix is not married to one specific shell. This means you can actually pick a shell you like to work with, and use that. One popular shell is called Bash.
By default, MacOS uses the Bash shell. After some years of using Bash I thought I'd try a different shell that seemed to have some additional features. So, right now I personally use a shell called "Zsh". If you're interested in the differences between these two shells you might find this article intertesting.
That's pretty much all the background info you probably need about the Unix shell. Let's dive deeper.
You'll be suprised what you can do on the command-line to speed up your workflow. You can automate moving and processing files, downloading data, replacing text in files, creating files and directories, searching files, and much much more.
The main idea with scripting is to make your work life easier. You can create commands that will automate repetitive tasks, and in the end make you work faster.
In front end development, this usually means we will automate some part of our project's build process, but there are many other reasons to use shell commands.
Yes you can. This is a big reason for automation. Most programs are built to allow you to use them in the command line, through a CLI (command line interface).
An example of a CLI can be seen when running Node.js. Node.js doesn't ship with a visual interface. You can only interact with it through commands.
Other examples are Java, or the Android SDK, or Xcode. They all have a CLI that lets you interact with these programs. These are just a small list of programs. Like I said, most programs out there that you will use in front end development will probably have a CLI.
Note: It's important for programs to be this flexible, because automations are rarely done on a user interface. Without the command line access, automating program tasks would be really difficult to do.
So, lets start with basic Unix commands you should know, but first, I want to make sure you know exactly where to write the commands.
Before we start, I'll show you how to actually open the terminal on MacOS, so you can write commands into it.
To open it, simply open your finder, and open
You might want to add the terminal program to your dock, so it's easy for you to open later.
By default, MacOS will use the Bash shell. So when you open the terminal, you will be logged into a Bash shell instance.
As I said earlier, you interact with Unix by executing commands. To execute a command, simply type a command in the terminal, and hit enter.
There are commands that allow you to do probably more than you will ever need. Since we are talking about front end developers here, we will stick to what is most useful to front end developers, and I will touch on some other commands that are good to know as well.
Let's get into it. First, I'll show you a useful command, that helps you get info on any command you want to learn about. There is a handy command in Unix, called
man, that does this.
man = The command manual that will tell you anything you want to know about any Unix command.
This example would output the manual for the
ls command (the
ls command just lists files on a directory) and would tell you the options associated with it. This is really helpful when you are not sure what options a command has.
The manual on how to use the
ls command, separated in pages.
The next command you typically learn is
echo = Lets you print values on to the console. Similar to
echo Hello World
This will output a series of paths, tied to the $PATH variable:
Note: More on the $PATH variable later, but it is basically a very commonly used global variable in Unix that stores paths
Exiting is odd to me, it is not consistent. It is annoying enough that I had to make sure I got in all the possible ways to exit here.
Typically, you can hit
q when you are in a paginated output (such as the output the
man command gives you)
You can also try
ctrl + q,
ctrl + x, or
The more common ways to exit would be force quit with
ctrl + c, or type
We haven't really done much with commands that would require and exit yet, but I have written this and will be here for you when you need it.
When you have a lot of commands you've written, you don't want to write them over again later. You can press the ⬆️ or ⬇️ arrows in your keyboard, to go back/forward to the commands you have already used.
Sometimes you want to edit a command you wrote already, this will help you go to the exact spot you need to edit.
ctrl + a = Move cursor to beginning of the line
ctrl + e = Move cursor to the end of the line
option + click = Moves cursor to where you clicked
You can save some time and have the shell help you autocomplete commands of file names for you
tab = Auto complete command or file name
tab + tab = Whenever auto complete doesn't work, it will show a list of possible matches
Sometimes you have too much output in your Terminal. Run
cmd + k to clear your output.
** Note: This will not exit/force quit your running script. It will just clear the output. **
You don't need to run a command one by one. You can separate commands with semicolons.
ls; echo $PATH; echo complete!
That concludes the absolute basics. Are you ready for more?
Cool! Let's continue by navigating the file system!
Unix lets you manipulate files in any way possible. You can view files in the command line, you can create new files, you can list files/directories, or launch an app that shows the file you want to view.
To do this, it's first important to know that you need to target the directory the file is located in first, before you can actually do anything with a file. To do this, you typically will need to change directory from where you are to the directory the file is located in.
When you open the Terminal, the default directory you are targeting at that time is typically your user directory (the directory with your name). To switch to another directory, you will use the
cd = Change directories
Note: If you know the name of the directory, you can type some of the name, then hit tab to autocomplete the rest of the name. If you are not sure what the name of the directory is, you can hit tab twice to get a list of options.
You can type
cd and press the up/down arrow key to get the last directories you changed to.
To go back to the parent folder, just use
To go back multiple parent folders, just use
cd ../../../, adding as many
../ as you need to go to the right parent folder
To go to the top most folder in your Mac, type
To go to your user folder (the folder with your name on it), type
To toggle between current and previous directory, type
Now that you can change directories, you might want to know what files are in a directory. To do this just use the
This command will list all files and directories in the current directory you are in.
There are a few options you can add to the command, which are explained below:
ls = List files/folders located in the current directory
ls -l = List files vertically, and display details about each file/directory
ls -la = List all files, including hidden files/directories
ls -lah = List all files as -la, but also showing file sizes
If you are ever confused on what directory you are targeting commands into, simply execute the
pwd command. This returns the present working directory.
Now that you know how to change to directories and list files, let me show you how you can read, modify, create and delete files.
There's a few useful commands for reading files:
cat = Outputs the contents of the file.
less = Outputs contents of files, with pagination.
cat -n = Will output the contents of multiple files.
cat -n file.text file2.txt
Note: Doesn't have to be .txt, can be any file type that can be read.
mv = Move file -- also commonly used for renaming files
mv myFile.txt my-directory-name
mv myFile.txt my-renamed-file.txt
cp = Copies a file. You can't copy a directory with this command.
cp -R = Use this to copy directories. The
-R option tells the
cp command to copy recursively. This means it will copy a directory and its children.
In this example, "some-file.txt" is copied as "some-file-copy.txt":
cp some-file.txt some-file-copy.txt
In this example, "some-directory" is copied as "some-directory-copy". All files and directories inside of "some-directory" will also be copied into "some-directory-copy":
cp -R some-directory some-directory-copy
mkdir = Creates a directory
touch = Updates access time of a file, or if the file doesn't exist, will create it with default permissions.
rm = Removes a file (does not send to trash!)
rm -rf = Deletes a file or directory recursively (does not send to trash!)
I tend to always use
rm -rf even on single files. This command will remove "some-directory" and any files/folders inside of it.
rm -rf some-directory
Note: Worth noting again that using these commands will not send the files to the trash. You will delete them immediately. So make sure you really want to delete the file before you run this command!
I have shown you a good amount of commands so far. Play with those commands and make sure you are comfortable with them.
Once you are ready lets continue our trip through the file system and see what else we can do.
In software development, you will run into issues where you cannot run a command because the folder you want to execute a command on doesn't allow you to do it.
This happens a lot when running build scripts on your machine and things like that.
Here are some tools that will let you make a Matrix move around all those pesky directory and file ownership issues.
When you use a shell, you are typically defaulted to the MacOS logged in user. There are folders that have different owners, and some commands may not be executable if the user you are logged in as doesn't have permission. This is the reason for the sometimes frustrating situation where you try to modify a file, or run an installation script, but it won't let you because you don't have permission.
It means the user type you are logged into in your Mac, does not allow you to perform the operation.
There are three types of users you should be aware of.
The first is the root user, second is an admin user, and the other is a non-admin type of user.
The root user refers to the top user who can modify and execute anything in the OS, outside of admin users. You will not ever need to log in as the root user, and in fact it is a security issue to do so.
The root user is disabled by default on MacOS.
This is a user that has elevated privileges -- more so than regular users, but less privileges than the root user.
It will be common to have an admin user in your Mac to have freedom of making changes to the system and files.
This is a more restricted user type, which is necessary to disallow unathorized users from modifying your computer in ways they should not be allowed to do so.
As I said, login as root user is disabled, but it does have some execution privileges you may need. If that's the case, this is where the
sudo command comes into play.
sudo Stands for "substitute user and do". It executes the provided command as root, which is able to do anything in the computer.
Note: Notice I said it executes AS root, but it does not log you in as root. It is the recommended way to execute root commands, in a secure manner.
sudo mv my-file.txt my-renamed-file.txt
When you run the
sudo command, you may be prompted for a password. This should be the password you established as your admin user.
sudo you can also take on the privileges of other users, not just
sudo -u someotheruser commandtoexecute
Not everyone can do
sudo, only admins. This is setup when creating a user account.
If you are unsure of what user you are logged in as. Try these commands below:
whoami = Tells you the user you are logged in as.
groups = The groups that your user belongs to.
Unix lets you control permissions on a file pretty easily. Before I show you how to update permissions on a file, let me show you how to determine what permissions are applied to a file before you actually modify a permission.
Let's go back to the
ls command, using the
This is going to give you a list of files for your current directory. To the left of that list, in the first column, you will see some odd-looking characters next to each item, that look something like this:
These characters are telling you whether the item is a file or directory, and what permissions are applied to that item.
Directories will always start with a 'd' as the first character:
Items that are not directories will start with a '-':
After this initial character, you will have a series of characters, 'r', 'w', 'x', and '-'. These characters define the permissions the item has:
'r' = This file/directory is allowed to be read
'w' = This file/directory is allowed to be written to
'x' = This file/directory is allowed to be executed
'-' = This file/directory does not have this permission. The permission affected depends on where the '-' is set.
You'll notice these characters can be repeated many times. This is because each file or directory has 3 "user" categories permissions can be set to.
The categories, from left to right, are "user", "group", and "other".
Unix gives you full control for setting permissions for each category. Since there are 3 permissions you can set, each category will have 3 slots you can activate a permission onto it.
For example, if a file had read, write, and executing permissions for a user, but not for groups or other, you would see a permission that looks like this:
If the file had only read permissions for a user it would look like this:
Notice how the '-' character works. For permissions that are denied, a '-' is used.
Note: Remember the first '-' in the line of my example just tells you this item is a file. It is not telling you about a denied permission.
If a file only had group permissions you would see this:
If the permissions were set for the "other" category of users. You would see this:
If we allowed all permissions for all 3 categories, you would see this:
Now you should be able to read the permissions like a kung fu masta :)
Now that you know how to read permissions. Setting them will be just as easy.
The command you will use is
chmod = Stands for "change mode"
You can write the command in two different ways.
1. The less-common way
You will write
chmod, then as options you will target each category I talked about earlier ("user", "group", "other") and set the permissions to it.
The below example sets permissions to each category, replacing whatever permissions where previously set:
chmod u=rwx,g=rwx,o=rwx someFile.ext
Slightly quicker way to write. This will update user and group ("ug") to add (+) the write permission. Existing permissions are not affected:
chmod ug+w someFile.ext
You can also remove a permission by changing the + into a -:
chmod ug-w someFile.ext
So far we have only modified permissions for one file or folder.
If you modify a folder, its subfolders are actually not modified with the permission.
To make sure all sub folders and files are modified as well, use the
chmod -R a+x somedirectory
There is an even quicker way to set permissions. And this one is the most common way to do it. You write numbers that equal to the permissions you want to set.
With octal notation, read = 4, write = 2, execute = 1, when we add up the 3 numbers when setting for each category, it represents the state of the permissions for a file.
The highest number (777), adds permissions for users, groups, other, and gives read, write, execute permissions:
chmod 777 filename.ext
This would set read, write, execute. Everyone else just read:
chmod 755 filename.ext
This would give no permissions to anyone:
chmod 000 filename.ext
In Unix, you can take ownership of a file away from the one who owns it. In my experience as a software developer, I have used this typically to gain read, write, and execute permissions on a file. This is what you will most likely use it for as well.
chown = Stands for change ownership
chown kevin:somegroup somefile.txt
chown kevin somfile.txt = Changes owner to the kevin user
chown :somegroup somefile.text = Changes owner to the group provided
chown -R kevin:group somedirectory = Changes owner and everything in it. -R stands for recursive. You will need
sudo to make this work.
Congrats Neo, you are well on your way to confronting agent Smith.
You now know a whole lot about shell scripting and can begin to read the Matrix, sorta.
This trip goes deeper. I can only show you the door, you're the one that has to walk through it.
The journey continues on the next part, coming soon...