LPI 010-160 – Scripting Basics

1. Text Files and Text Editors

By now, we’ve learned some basic commands to work with files and directories, but let’s start focusing on a specific type of file known as a text file. There are lots of files in a computer system, but the most basic and flexible of all of these is the text file. In fact, all your configuration files and shell scripts, these are all going to be text files on your system. Now, because a Linux user will often modify configuration files and create shell scripts, you really need to know how to edit text files. To edit a text file, the user has to be able to start up the editor either on an existing document or creating a new document, and then be able to edit it and save that file. There are lots of different editors out there. For example, there’s a Pico editor and a nano editor, and both of these are quite similar, so their operation will be described together as we go through.

And then there’s another one out there called VI, which is probably the most popular one used by people in the Linux and Unix community. Personally, I really like the nano editor. It’s the one that I use most of the time. Now, when you’re dealing with a text editor, it’s going to let you edit documents that are stored in plain text format text files cannot include any graphics, any kind of fonts, or any kind of emphasis for words like italicizing them or bolding them like you could in Microsoft Word or other GUI based programs. Text files are going to consist of lines that can vary in length from zero characters to the file’s entire size, and can hold any number of different data types.

Users may want to create or edit some of these as an ordinary user, while others are really important for administering a Linux system, and therefore they have special permissions so only Root can edit them. There are lots of different file types out there, but the main file types we’re going to talk about are human language files, program language files, formatted text files, program and system configuration files, and program log files. All Linux distributions ship with some text editors built into them. The four most popular text editors include VI, Emacs, Pico, and Nano.

VI is an editor that is a Linux and Unix staple. It’s very small and it’s installed by default, but it’s a little more difficult to use. Emacs is an editor that’s another Linux staple, but it’s a big editor with a lot more features than VI, but it’s also less likely to be installed by default on lightweight distributions. There’s also going to be a Guy version of Emacs as well, if you want to install that. Pico is modeled after Emacs by omitting many of its advanced features, though it’s more lightweight and it’s suited better to light weight distributions, so it’s often included by default.

And finally, nano. Nano is an editor that’s a clone of Pico, but adds a couple of extra features as well. Even with those additional features though, it’s much more lightweight than Emacs, and so a lot of times you’ll see nano found on Linux distributions that are small and lightweight as well. Now, each of these text editors has their own conventions for displaying the information on the screen, their own ways of manipulating the text, and other features like that. Most text mode text editors are similar up to a point. For example, one or more lines at the top or the bottom of the display is going to be used to show summary information and brief command inputs that you might be able to use. You’re going to have to refer to the man pages for each of these text editors or launch them and play with them yourself to really see the differences between them.

Now, after you launch a text editor and you start making changes to a file, don’t forget to save the file or make a backup copy. It’s also important to take note that some of these programs do not feature auto save functionality, like Microsoft Word might, and so it’s best to regularly save your changes to prevent any loss of data due to interruptions in your work. You’re going to have to play with these text editors and get comfortable with them, because you’re going to do a lot of that as you start playing with scripts, reading and editing files, and other things from the command line.

2. Using Vi and Nano

There are many text based editors within Linux and these are helpful when you need to edit a configuration file or just access a quick file and make some changes. Now, the first one I’m going to show you is VI, and the second one is Nano. Now, I’m going to warn you, VI is a little bit more complicated to use than Nano, but stick with me and I’m going to show it to you. The reason is a lot of system administrators really swear by VI. They love it. It’s a relic of the old 1970s and 1980s. But some of the old timers just really, really love VI. It’s kind of the sign of being a true Linux guy is that you love VI. Personally, I’m not really a big fan of VI. I much prefer Pico or nano. But I’m going to show it to you because the exam wants you to know it. And a lot of people that you’re going to experience in the field will say that if you don’t know VI, you just don’t know Linux. So we’re going to go into it. All right? So the first one we’re going to use is VI. And if we’re going to use VI, we need to have a file to use. So I’m just going to call this test one.

And by doing this, I’m going to open up VI and open up a file now known as test one. And this file is going to be blank. So when I do this, you’re going to see that we have all these blue till days going down the left side of the screen. This signifies that those lines have nothing on them. In fact, the whole file is just one line with one character, this blank blinking cursor. Because it was an empty file, nothing has been done to it yet. So when we open up a file, by default we are in edit mode. For us to be able to type in information, we have to go into Insert mode and we do that by pressing the I key. So in this case, I’m going to press I and you see no, I came on the screen. That’s because it just put me into Insert mode and now I can start typing. This is a test file. This is only one line. That’s fine. You can enter whatever you want here. Now, if I wanted to enter more data, I can hit Enter. I could say this is the second line and I’ll say this is the third line.

That’s enough data for us to play with right now. Now, if I wanted to go up to the second line and be able to get to it, I actually can’t use my arrow keys. That won’t work. Instead, I have to hit the escape key, which goes back to Edit mode. By doing that, I can now arrow my way around to where I want to be able to touch things. And so if I want to be able to change where it says second and maybe put that as a second instead, I can then go ahead and press the X key to delete. And so as I hit X, it’s deleting them one at a time. Now if I wanted to type in the second, I have to go back to insert mode. So I press the I key and now I could put second and I’m back there.

Now again, if I want to move around again, I’d have to hit the escape key. And now I can move around with my cursor or I can even use some shortcuts. There’s a couple of shortcuts. For instance, if I use the carrot key, which is the shift and the six, that will bring me to the beginning of a line. Now, if I want to move forward a certain number of words, I can actually do that by typing in the number and the w. So if I want to go to the third word of this line, I would type three W. And you can see I went past the three words. And now I’m at the fourth word, ready to start typing because I moved forward three words. This is the those are the three words. Now if I want to move back, I can do that by typing a number and B. So if say I want to go back two words, I would do two B.

And that takes me back from the and back to Is. And so that’s why a lot of system administrators like it because they can move around pretty quickly this way. For example, if you wanted to go to the first line of the file, you could just type one capital G. If you want to go to the third line, you type three capital G. If you want to go to the 50th line, you do 50 capital G. And so they can jump around really quickly. Another thing you can do when you’re in this edit mode is you can issue commands.

So for instance, if I wanted to show line numbers, if this was a configuration file, for instance, I can do that by typing in the colon, which gives me the command here at the bottom, and then set space NU which stands for set numbers. If I hit enter, you’ll now see that I can see what line numbers there are and be able to identify things a little bit easier. Again, if you want to learn more about VI, you can use the man pages or there’s a lot of great cheat sheets for VI that are one page PDFs you can download online. Just search VI cheat sheet and you’ll find them. They’re everywhere.

Now that’s the idea of using VI. We can take it a little bit further if we wanted to, and undo actions which would be something like a U. If you type in U, that will undo an action which in this case took away my second that I added in. If I want to insert that back, I hit the I and type in my second and other things of that nature. But again, I’m going to leave that to you to play with a little bit more, because, again, VI is pretty complicated because there’s no mouse and it has all these weird commands you have to use. So instead, I really want to focus on a program called Nano instead. But before we do that, we have to save this file. Now, how do we do that? Well, if we hit Escape, we are back into our edit mode. We can then hit the colon key, which brings up our command issuing.

And here I’m going to do w and hit enter. That stands for Right, which will save my changes, but it won’t exit the file. So there we go. I wrote that three lines and those 90 characters to disk. Now, if I wanted to quit and save, I can do that by hitting the colon, then WQ, which stands for Right and Quit. Or if I wanted to quit without saving my changes, I can do Q exclamation, which stands for discard all my changes since I saved last and then Exit. In my case, I do want to save. So I’m going to go ahead and backspace out here and do WQ. So I’m going to write and quit. And that’ll bring me back to the command line. Now, if I want to see the contents of that file, we can again use cat. We can type in test one. And you’ll see, I have those three lines to the screen. All right, so you can see how complicated VI was. Let me show you a much easier way to edit your configuration files.

And this is using a program called Nano. So you just type in Nano and the name of the file you want to open and up. It shows up. Now, you can see already this looks more like a command program. You can see that there’s some information here. For instance, the name of the program in the top left. This is Nano two nine three, the name of the file in the top center, test one, the contents of the file. And then at the bottom, it shows me some information and it even shows me some commands. All of those things like the carrot x, carrot garret o that you see at the bottom, that means Control. So if I wanted to exit out of here, I can just hit Control X. If I want to get help, I hit Control G, and there is some help, just like a man file. And I can then go next page Control and read through as I need to to find the information if I want to close that again. ctrlX.

Now, you’re already in edit mode here. You can already go in and change files and delete files you can arrow around just like a regular word processor. So instead of maybe I want to backspace that and then hit this is the text file, right? And you can do a lot of this information here. Just like in Nano, you can go to a certain line number. So if you do the control and then the underscore character, it’ll say enter the line number or the column number. So the line is how many rows up and down? Maybe I want to be on the third row and the column is how many characters over from left to right and it goes from zero over to 80. So you could do that by doing maybe three comma five, which should get me there to about the I and is on the third line. And you can see right there, it put me into the space, right, because Th I s, which is the fifth character, is that space right before the I and is. So this is the idea of being able to move around here. And again, this is much easier to use than something like VI because you can simply go around and be able to access things.

The other thing is you can actually cut text and uncut text, which means you can actually bring things around and copy and paste as you go. There’s a lot of nice features like that inside of Nano. For example, Nano even has a spell checker there, as you can see with Control T. And for example, if you want to take away your justification and make it all so there’s no single lines, everything goes in one big line together. We can do that by hitting Control J. If you want to undo that, you hit Control U and it goes back to normal. So you can see there’s a lot more features here inside of Nano than using something like VI.

So the reason why I want to bring this up is because as you’re trying to make test files so you can play with things as you want to edit configuration files, nano is a great tool for that. Now, when you’re done using this file and you want to save your changes, for example, let’s go up here and just change the second back to a written out second. We can save that by using the WriteOut command that’s Control O. It’ll ask you where do you want to save this file? If you want to use the same name, is hit Enter and that is going to write those three lines back to disk for you. At that point you can hit Control X, exit out of here, get back to your command line and go back to doing whatever else it is you need to do on your Linux system.

3. Beginning a Shell Script

Let’s talk about scripts. A script is a program written in an interpreted language typically associated with a shell or a compiled program. In Linux, many scripts are going to be written as shell scripts. These are associated with the bash or another. Shell users can write shell scripts to help automate tedious, repetitive tasks or to perform new and complex tasks. Scripts perform many of the Linux startup functions as well, so mastering scripting is going to help you manage that startup process for your distribution.

Shell scripts are plain text files, so they’re created in a text editor like VI, emacs, Nano, or Pico. A shell script is going to begin with a line that identifies the shell that’s going to be used to run it. For example, if you see something like hashtag exclamation, slash, bin, slash bash, you know it’s a bash shell. The first two characters, the hashtag and the exclamation, are a special code that tells Linux that this is a script and the rest of the line is going to tell it what program is going to be run to interpret that script. In our case, slash bin bash scripting languages use a hash mark as a comment character as well, so the script utility is going to ignore this line throughout the script, even though your kernel isn’t going to ignore it on most systems.

Bin Sh is a symbolic link that points to bin bash, but it can also point to a different shell as well. If you decide to set up your distro that way, when you specify the script as using bin Sh, this is going to guarantee that any Linux system will have a shell program to run that script. Simple bash scripts can run in other shells, but more complex scripts may be bash specific and they won’t run other places. Now, after writing the shell script, it should be modified to become executable.

This can be accomplished using the Change mod permissions command setting. The ownership and permissions on files will be discussed in a later video when we start talking about permissions and the Change mod command in depth. As you’re creating shell scripts and they start getting longer and lengthier and more complex, you’re going to want to have a good text editing program available for use. If you’re using a Gui based desktop like KDE or Gnome, there’s programs like Krite and G Edit that you can use that have excellent features for creating text shell scripts. However, if you’re working from something like the command line only, you’re going to be using VI or Nano or Pico. But even the are pretty easy to use, as you saw in the last lesson.

4. Commands

So now that we’ve created the start of a shell script, we need to add some commands to it. And that’s one of the most basic features of a shell script, is its ability to run commands. Commands are those things that are built into the shell and those external commands, and they both can be used from a script. This means other programs can be run as commands as well. Most of the commands that are entered into the shell prompt are going to be external commands. These are programs that are located in your slash bin directory or your user bin directory or even some other directory altogether on your file system. These programs are also going to be using internal commands, and these can be run by using their names inside the script. Parameters can also be specified to all these different programs inside the script. For example, for a script that launches two external windows and the Kmail mail reader program, it might look something like this hashtag exclamation bin bash.

This says this is a bash script. Then we’re going to run a program slash user bin xterm ampersand. This says launch the external program in a new window, then slash user bin xterm ampersand, launch another copy of the exterm in a new window, and then slash user binkmail ampersand. This says launch Kmail in its own window as well. Now, aside from the first line that identifies it as a script, this script looks just as if I entered all these commands individually in the command line. And that’s the idea of a script. The only exception here that tells me this is a script is that first part that tells me that hashtag exclamation bin bash. After that, each list of each command is going to be given the complete path to each program and it’s going to launch it.

This is usually not strictly necessary, but the reason why we use the full path here is to make sure the program will find the right program regardless of where you start the script from. Otherwise, it may use a relative path and that can be messed up depending on where you launch the script from. Now, if a script produces something like a no such file or directory error for a command, this means that you’re going to have an issue where you have to start troubleshooting your scripts. This is going to be able to help you figure out where those programs are and figure out how to launch them. Now, if you noticed in the script, I pointed out that there was an ampersand at the end of each line. This character tells the shell that it can go on and continue processing the script without waiting for the results from the first command to finish executing. If we didn’t have that ampersand at the end of each of those, the first x term would have opened, but it would have paused the script until I closed that x term, and then the second one would have opened, and then when I closed, that the third thing the Kmail would open. Now, by using the ampersand, I’m able to launch several programs at once from the single script. And this saves me time as I’m starting up scripts and in other situations. And so it’s important to remember that ampersand allows you to do that.

Now, such scripts are typically not going to be done to include ampersands at the end of every command, because a lot of times your script is waiting for one command to finish and taking the output of that as input into the next command. In that case, you’d leave the ampersand out. Now, a few commands that commonly are used in scripts are going to be things like LS for listing files, MV for moving files, CP for copying files, and RM for removing files. All of these commands can be used to help automate repetitive file maintenance tasks like backups, moving logs, consolidating files, and deleting files. Another command that we find often in scripts is the Grep command. We use that to search for things right? And so if we use Grep, we can locate files containing specific strings and display the lines that contain those strings into a file or to the screen.

We also can use things like Find. Now, while Grep is going to search for patterns within the contents of files, find does this based on file names, ownerships, and other permissions at the file system level. And so if you’re looking for particular files, find can be useful, and scripts will use that a lot. Another thing that’s used is cut. Cut is a command that will extract text from a field within a file. And so you might be able to pull information from a file and then copy at other places. Another thing we’ll use is echo. Echo is heavily used inside scripts because it’s going to display something as a message to the user in your command line terminal. So, for instance, I might echo, please wait while I start up the computer, and then I run a bunch of commands after that. You get the idea. Any command can really be used inside of a script, and we’re going to play with that as we go through some scripts later on in this course.

5. Arguments

As we’ve already talked about, scripts are great and they really are useful. But to really expand the utility of a script, we need to use variables and arguments. Now, a variable is a placeholder for a value that will be determined when the script runs. The values of variables can be passed as parameters to a script, generated internally to that script, or extracted from within the scripts environment. An environment is just a set of variables, including in things such as the current directory, the search path for running programs, or anything else like that which a program can access.

Variables that are passed to the script are frequently called arguments or parameters. They’re represented in the script by a dollar sign followed by a number from zero upwards. So for instance, dollar sign zero stands for the name of the script. Dollar sign one is the first parameter to the script, dollar sign two is the second parameter from the script, and so on. These parameters when you run your script, act just like the arguments we passed to other commands as we were using the commands before. For instance, when I did LS and a file name, that file name was really a variable and that’s a parameter in that case. This is the same idea with our scripts. Let’s take a look at an example script to illustrate the convenience of using variables as we input a parameter to a script.

Notice in the script you see dollar sign one several different times. So whatever parameter I give to the script is going to be used and inputted wherever you see dollar sign one. This script is going to create an account and change the account’s password and then create a directory in the shared directory tree corresponding to this account. It’s also going to set up a symbolic link to that directory from the new user’s home directory and adjust the ownership and permissions depending on the system’s ownership and permission policies. To use this script, let’s type the script name and the desired username and the new password twice.

For example, if the script is called make user, let’s go ahead and call it MK user space JD on the next prompt will then ask us to input our password and then enter it again to confirm. Now, do take note the fact that this scripts program is going to operate silently unless they encounter problems or need some input from the user. So during this interaction, including the typing of the passwords, this is all just a result of the password command. In effect, the script is going to replace all seven lines of the commands with just one MK user. Every one of those lines is going to use the same value stored for the username JD and then put that in place of the dollar sign one inside the script.