Unix Shell Scripting for Front End Developers

Let me tell you why you're here.

You're here because you know something, but you can't explain it.

You've felt it your entire career.

There's something wrong in your front end developer skillset -- you don't know what it is, but it's there.

Like a splinter in your mind. Driving you mad.

It is this feeling that has brought you to me. Do you know what I'm talking about?

Of course you do. It's Unix shell scripting ?

You never learned this world of scripting -- bash commands and .sh files just looked like something other devs did. You, much like me, just copied and pasted commands and hoped for the best.

It's good to take a little time to learn shell scripting, because it's helpful and it's actually not that hard to learn. The basic commands will take you far.

Knowing how to use shell commands will also make you a more valuable software developer. Plus, you will be just that much closer to seeing the matrix like Neo ?

In this first post of my Unix shell scripting series, I will try to explain shell scripting in a simple way, and I'll show you the foundational commands that will make you productive right away.

With that said, pick the red pill, stay in wonderland, and let's see how far the rabbit hole goes...

Unix on MacOs

You've been living in a dream world, Neo

The MacOS user interface is hiding from you a lot of what happens behind the scenes. The reality is, everything you do on a Mac is being controlled by Unix commands -- moving files around, copying/saving, searching for files, downloading -- the list goes on and on. The visual layer that we interact with, is just a pretty looking design that lets us control the lower things.

OK, So What is Unix exactly?

Unix is the core of the Mac operating system. It is what runs "under the hood" of the pretty UI we are used to and interact with. You don't need to know much about Unix to get started using it, but here are some key points that are helpful:

• It is a mainframe OS. This just means it controls low level things like the device's memory, processor, and file system, to name a few things.
• Not all Unix distributions are the same.
• Unix on MacOS is based on The BSD Unix (Berkley Software Distribution)
• The MacOS version of Unix is called Darwin, Apple's proprietary improvements built on top of BSD Unix
• Unix has no pretty UI, just a blank terminal screen (a command line interface) that takes commands
• You can interact directly with Unix through the MacOS terminal

Let's get you ready for our ride. Sit still. This won't hurt...

That helps, but what is bash scripting, and what is the Unix shell?

OK, you may have heard these before, but never really took time to know what they were.

Let's get these terms out of the way. You will see they are very easy to understand.

You control Unix through a command line interface, as I said earlier. Unix commands need to be interpreted into something the innermost/lower-level components of Unix can understand. Unix requires a "shell" to do this.

The shell is the program that sits between us and the low level Unix things, that manages a lot of technical details, and interprets our commands to Unix in a way Unix can understand and execute our commands. Any time we write a command on the command line, we are doing it through a shell, therefore we never write commands outside of a shell program.

Unix is not married to one specific shell. This means you can actually pick a shell you like to work with, and use that. One popular shell is called Bash.

By default, MacOS uses the Bash shell. After some years of using Bash I thought I'd try a different shell that seemed to have some additional features. So, right now I personally use a shell called "Zsh". If you're interested in the differences between these two shells you might find this article intertesting.

That's pretty much all the background info you probably need about the Unix shell. Let's dive deeper.

What Are You Waiting for Neo? You are faster than this

You'll be suprised what you can do on the command-line to speed up your workflow. You can automate moving and processing files, downloading data, replacing text in files, creating files and directories, searching files, and much much more.

The main idea with scripting is to make your work life easier. You can create commands that will automate repetitive tasks, and in the end make you work faster.

In front end development, this usually means we will automate some part of our project's build process, but there are many other reasons to use shell commands.

Can I use programs I have installed on my machine to help automate things?

Yes you can. This is a big reason for automation. Most programs are built to allow you to use them in the command line, through a CLI (command line interface).

An example of a CLI can be seen when running Node.js. Node.js doesn't ship with a visual interface. You can only interact with it through commands.

Other examples are Java, or the Android SDK, or Xcode. They all have a CLI that lets you interact with these programs. These are just a small list of programs. Like I said, most programs out there that you will use in front end development will probably have a CLI.

Note: It's important for programs to be this flexible, because automations are rarely done on a user interface. Without the command line access, automating program tasks would be really difficult to do.

So, lets start with basic Unix commands you should know, but first, I want to make sure you know exactly where to write the commands.

How to open the terminal

Before we start, I'll show you how to actually open the terminal on MacOS, so you can write commands into it.

To open it, simply open your finder, and open Applications/utilities/Terminal

You might want to add the terminal program to your dock, so it's easy for you to open later.

By default, MacOS will use the Bash shell. So when you open the terminal, you will be logged into a Bash shell instance.

Basic Shell Commands

As I said earlier, you interact with Unix by executing commands. To execute a command, simply type a command in the terminal, and hit enter.

There are commands that allow you to do probably more than you will ever need. Since we are talking about front end developers here, we will stick to what is most useful to front end developers, and I will touch on some other commands that are good to know as well.

Let's get into it. First, I'll show you a useful command, that helps you get info on any command you want to learn about. There is a handy command in Unix, called man, that does this.

man

man = The command manual that will tell you anything you want to know about any Unix command.

Bash Script Example

This example would output the manual for the ls command (the ls command just lists files on a directory) and would tell you the options associated with it. This is really helpful when you are not sure what options a command has.

man ls

Output
The manual on how to use the ls command, separated in pages.

While viewing a manual:

• Hit the spacebar to go to the next page
• Hit f to scroll the page (basically the same as spacebar)
• Hit b to go to previous page, which basically scrolls the page back
• Hit q to close the manual

The next command you typically learn is echo

echo

echo = Lets you print values on to the console. Similar to console.log in Javascript. You can print messages or print variable names to see their value.

Bash Script Example: Printing a message

echo Hello World

Output
Hello World

Bash Script Example: Printing a variable

This will output a series of paths, tied to the $PATH variable:

echo $PATH

Output
/Users/embp/.nvm/versions/node/v10.9.0/bin:/Users/embp/.fastlane/bin:/Users/embp/android-sdk-macosx/platform-tools...

Note: More on the $PATH variable later, but it is basically a very commonly used global variable in Unix that stores paths

Exiting

Exiting is odd to me, it is not consistent. It is annoying enough that I had to make sure I got in all the possible ways to exit here.

Typically, you can hit q when you are in a paginated output (such as the output the man command gives you)

You can also try x, ctrl + q, ctrl + x, or ESC

The more common ways to exit would be force quit with ctrl + c, or type exit.

We haven't really done much with commands that would require and exit yet, but I have written this and will be here for you when you need it.

Going to a Previous Command

When you have a lot of commands you've written, you don't want to write them over again later. You can press the ⬆️ or ⬇️ arrows in your keyboard, to go back/forward to the commands you have already used.

Going to the beginning or end of a command, or to a specific character in a command

Sometimes you want to edit a command you wrote already, this will help you go to the exact spot you need to edit.

ctrl + a = Move cursor to beginning of the line
ctrl + e = Move cursor to the end of the line
option + click = Moves cursor to where you clicked

Autocomplete commands or file names

You can save some time and have the shell help you autocomplete commands of file names for you

tab = Auto complete command or file name
tab + tab = Whenever auto complete doesn't work, it will show a list of possible matches

Clearing your screen of output

Sometimes you have too much output in your Terminal. Run cmd + k to clear your output.

** Note: This will not exit/force quit your running script. It will just clear the output. **

Multiple Commands

You don't need to run a command one by one. You can separate commands with semicolons.

Bash Script Example

ls; echo $PATH; echo complete!

That concludes the absolute basics. Are you ready for more?

Cool! Let's continue by navigating the file system!

Navigating the File System

Unix lets you manipulate files in any way possible. You can view files in the command line, you can create new files, you can list files/directories, or launch an app that shows the file you want to view.

To do this, it's first important to know that you need to target the directory the file is located in first, before you can actually do anything with a file. To do this, you typically will need to change directory from where you are to the directory the file is located in.

When you open the Terminal, the default directory you are targeting at that time is typically your user directory (the directory with your name). To switch to another directory, you will use the cd command.

cd

cd = Change directories

Bash Script Example

cd to/some/other/directory

Note: If you know the name of the directory, you can type some of the name, then hit tab to autocomplete the rest of the name. If you are not sure what the name of the directory is, you can hit tab twice to get a list of options.

Other cd tricks

You can type cd and press the up/down arrow key to get the last directories you changed to.

To go back to the parent folder, just use cd ..

To go back multiple parent folders, just use cd ../../../, adding as many ../ as you need to go to the right parent folder

To go to the top most folder in your Mac, type cd /

To go to your user folder (the folder with your name on it), type cd ~

To toggle between current and previous directory, type cd -

ls

Now that you can change directories, you might want to know what files are in a directory. To do this just use the ls command.

This command will list all files and directories in the current directory you are in.

There are a few options you can add to the command, which are explained below:

ls = List files/folders located in the current directory
ls -l = List files vertically, and display details about each file/directory
ls -la = List all files, including hidden files/directories
ls -lah = List all files as -la, but also showing file sizes

pwd

If you are ever confused on what directory you are targeting commands into, simply execute the pwd command. This returns the present working directory.

Manipulating the File System

Now that you know how to change to directories and list files, let me show you how you can read, modify, create and delete files.

How to Read Files

There's a few useful commands for reading files:

cat = Outputs the contents of the file.

Bash Script Example:

cat filename.txt

less = Outputs contents of files, with pagination.

Bash Script Example:

less filename.txt

cat -n = Will output the contents of multiple files.

Bash SCript Example:

cat -n file.text file2.txt

Note: Doesn't have to be .txt, can be any file type that can be read.

How to Modify Files

mv = Move file -- also commonly used for renaming files

Bash Script Example: Move a File

mv myFile.txt my-directory-name

Bash Script Example: Rename a File

mv myFile.txt my-renamed-file.txt

cp = Copies a file. You can't copy a directory with this command.
cp -R = Use this to copy directories. The -R option tells the cp command to copy recursively. This means it will copy a directory and its children.

Bash Script Example

In this example, "some-file.txt" is copied as "some-file-copy.txt":

cp some-file.txt some-file-copy.txt

In this example, "some-directory" is copied as "some-directory-copy". All files and directories inside of "some-directory" will also be copied into "some-directory-copy":

cp -R some-directory some-directory-copy

How to Create New Files/Directories

mkdir = Creates a directory

Bash Script Example

mkdir myNewDirectory

touch = Updates access time of a file, or if the file doesn't exist, will create it with default permissions.

Bash Script Example

touch someFileName.txt

How to Delete Files

rm = Removes a file (does not send to trash!)
rm -rf = Deletes a file or directory recursively (does not send to trash!)

Bash Script Example

I tend to always use rm -rf even on single files. This command will remove "some-directory" and any files/folders inside of it.

rm -rf some-directory

Note: Worth noting again that using these commands will not send the files to the trash. You will delete them immediately. So make sure you really want to delete the file before you run this command!

I have shown you a good amount of commands so far. Play with those commands and make sure you are comfortable with them.

Once you are ready lets continue our trip through the file system and see what else we can do.

User Types And Their Privileges

In software development, you will run into issues where you cannot run a command because the folder you want to execute a command on doesn't allow you to do it.

This happens a lot when running build scripts on your machine and things like that.

Here are some tools that will let you make a Matrix move around all those pesky directory and file ownership issues.

User Types

When you use a shell, you are typically defaulted to the MacOS logged in user. There are folders that have different owners, and some commands may not be executable if the user you are logged in as doesn't have permission. This is the reason for the sometimes frustrating situation where you try to modify a file, or run an installation script, but it won't let you because you don't have permission.

It means the user type you are logged into in your Mac, does not allow you to perform the operation.

There are three types of users you should be aware of.

The first is the root user, second is an admin user, and the other is a non-admin type of user.

The Root User

The root user refers to the top user who can modify and execute anything in the OS, outside of admin users. You will not ever need to log in as the root user, and in fact it is a security issue to do so.

The root user is disabled by default on MacOS.

Admin User

This is a user that has elevated privileges -- more so than regular users, but less privileges than the root user.

It will be common to have an admin user in your Mac to have freedom of making changes to the system and files.

Non-admin User

This is a more restricted user type, which is necessary to disallow unathorized users from modifying your computer in ways they should not be allowed to do so.

Take on Privileges, With the sudo Command

As I said, login as root user is disabled, but it does have some execution privileges you may need. If that's the case, this is where the sudo command comes into play.

sudo Stands for "substitute user and do". It executes the provided command as root, which is able to do anything in the computer.

Note: Notice I said it executes AS root, but it does not log you in as root. It is the recommended way to execute root commands, in a secure manner.

Bash Script Example

sudo mv my-file.txt my-renamed-file.txt

When you run the sudo command, you may be prompted for a password. This should be the password you established as your admin user.

With sudo you can also take on the privileges of other users, not just root:

sudo -u someotheruser commandtoexecute

Not everyone can do sudo, only admins. This is setup when creating a user account.

Determine Who You Are Logged in as

If you are unsure of what user you are logged in as. Try these commands below:

whoami = Tells you the user you are logged in as.
groups = The groups that your user belongs to.

File/directory permissions

Unix lets you control permissions on a file pretty easily. Before I show you how to update permissions on a file, let me show you how to determine what permissions are applied to a file before you actually modify a permission.

Determine existing permissions first

Let's go back to the ls command, using the -l option:

ls -l

This is going to give you a list of files for your current directory. To the left of that list, in the first column, you will see some odd-looking characters next to each item, that look something like this:

drwxr-xr-x
-rw-r--r--
drwxrwxr-x

These characters are telling you whether the item is a file or directory, and what permissions are applied to that item.

Directories will always start with a 'd' as the first character:

drwxr-xr-x

Items that are not directories will start with a '-':

-rw-r--r--

After this initial character, you will have a series of characters, 'r', 'w', 'x', and '-'. These characters define the permissions the item has:

'r' = This file/directory is allowed to be read
'w' = This file/directory is allowed to be written to
'x' = This file/directory is allowed to be executed
'-' = This file/directory does not have this permission. The permission affected depends on where the '-' is set.

You'll notice these characters can be repeated many times. This is because each file or directory has 3 "user" categories permissions can be set to.

The categories, from left to right, are "user", "group", and "other".

Unix gives you full control for setting permissions for each category. Since there are 3 permissions you can set, each category will have 3 slots you can activate a permission onto it.

Bash Script Example: User permissions

For example, if a file had read, write, and executing permissions for a user, but not for groups or other, you would see a permission that looks like this:

-rwx------

If the file had only read permissions for a user it would look like this:

-r--------

Notice how the '-' character works. For permissions that are denied, a '-' is used.

Note: Remember the first '-' in the line of my example just tells you this item is a file. It is not telling you about a denied permission.

Bash Script Example: Group and other permissions

If a file only had group permissions you would see this:

----rwx---

If the permissions were set for the "other" category of users. You would see this:

-------rwx

Bash Script Example: Permissions on all categories

If we allowed all permissions for all 3 categories, you would see this:

-rwxrwxrwx

Now you should be able to read the permissions like a kung fu masta :)

Setting permissions with chmod

Now that you know how to read permissions. Setting them will be just as easy.

The command you will use is chmod.

chmod = Stands for "change mode"

You can write the command in two different ways.

1. The less-common way

You will write chmod, then as options you will target each category I talked about earlier ("user", "group", "other") and set the permissions to it.

Bash Script Example

The below example sets permissions to each category, replacing whatever permissions where previously set:

chmod u=rwx,g=rwx,o=rwx someFile.ext

Bash Script Example: Modify permission

Slightly quicker way to write. This will update user and group ("ug") to add (+) the write permission. Existing permissions are not affected:

chmod ug+w someFile.ext

You can also remove a permission by changing the + into a -:

chmod ug-w someFile.ext

Bash Script Example: Modify permissions recursively

So far we have only modified permissions for one file or folder.

If you modify a folder, its subfolders are actually not modified with the permission.

To make sure all sub folders and files are modified as well, use the -R option:

chmod -R a+x somedirectory

Octal notation permissions

There is an even quicker way to set permissions. And this one is the most common way to do it. You write numbers that equal to the permissions you want to set.

With octal notation, read = 4, write = 2, execute = 1, when we add up the 3 numbers when setting for each category, it represents the state of the permissions for a file.

Bash Script Example

The highest number (777), adds permissions for users, groups, other, and gives read, write, execute permissions:

chmod 777 filename.ext

This would set read, write, execute. Everyone else just read:

chmod 755 filename.ext

This would give no permissions to anyone:

chmod 000 filename.ext

Changing File/Directory Ownership

In Unix, you can take ownership of a file away from the one who owns it. In my experience as a software developer, I have used this typically to gain read, write, and execute permissions on a file. This is what you will most likely use it for as well.

chown = Stands for change ownership

Bash Script Example

chown kevin:somegroup somefile.txt

chown kevin somfile.txt = Changes owner to the kevin user
chown :somegroup somefile.text = Changes owner to the group provided
chown -R kevin:group somedirectory = Changes owner and everything in it. -R stands for recursive. You will need sudo to make this work.

I know what you're trying to do...

Congrats Neo, you are well on your way to confronting agent Smith.

You now know a whole lot about shell scripting and can begin to read the Matrix, sorta.

This trip goes deeper. I can only show you the door, you're the one that has to walk through it.

The journey continues on the next part, coming soon...