Python Institute PCAP – File IO and Exception Handling in Python Part 2

3. File IO with Exception Handling

What we learned about with regards to exception handling and what you would need to do to surround your operations, such as opening files, reading and writing them. You want to surround them with a try and accept block so that you can prevent any program crashes and you can gracefully handle situations where the file doesn’t exist or you may not have permissions to that file or whatever the exception is. You can capture all those exceptions.

So let’s try this here. So recall that on my desktop I’ve got two files that we worked on the previous lesson. There’s Sample and sample two text. All right? Sample just has this is my sentence, and I think the first line is empty. And then sample two text just has this yoyo written in there. Okay? So we’re going to use these two files. Remember, these are the only two files that I have on my desktop. So if I, for example, try to open a file that isn’t there, let’s do that. If I do with make sure you spell that correctly with Open and I give the directory location of where that follow so that’s in users MyHome directory, which is empty as a mod desktop. Notice that the IntelliSense here is smart enough to tell where we’re going here.

That’s the beauty of PyCharm and many other ides in integrated development environments. And so now when I do a slash and I choose, if I type in Sam, notice sample and sample two exist. But let’s try to open a file that doesn’t exist. So if I do sample three text that we know doesn’t exist and let’s open it in our mode mode. So we’re going to reopen it in read only mode, and we’ll say that as my file. This is the variable that will be assigned to the object, to the file object.

 And now if I try to print, we can say my file read. Okay? And I would just like to print the contents of whatever this returns. And so I also want to do one more thing outside of this width block. Moving on to bigger and better code. Let’s say somewhere down here we have some line where it says this line was run. Okay? Now, if we have an exception, our program crashes, meaning this line never gets run.

That’s the whole idea behind exception handling. You need to be able to have your program continue to run if there are exceptions or crashes. So this is actually going to cause a crash. We don’t have any exception handling going on here. We didn’t add the try or the Accept anywhere here. So let me give you an example. Let’s run this and you’ll see. Notice it says file not found. Error no such file or directory, and it cannot find this file. Now, did this line run notice in the output? It did not run. Okay, so this program crashed right at this line, or it crashed right here when It Tried To Open That File. That file doesn’t even exist. Okay? And right there, you can see that on line one. So let’s catch this exception. So what we need to do first is we need to surround this in a try. Catch block. So I’m going to move this indentation one in and out here. It will say try colon. So this is the code that we want to try. And then in the case that it doesn’t work, we want to make sure we have an accept clause here and we can print that file does not exist.

Okay? And I can just copy this exact error name, which is file not found error. Okay, so let’s copy this exact error. This is actually an object file not found error. So I’m going to copy that and paste it as part of the print line print statement that gets printed in the case that an exception occurs and this is what is going to happen. So now you’ll see that this handles the exception in a much more graceful manner. And the program doesn’t just crash wherever it sees a problem.

It tries this code. And if there’s an error, it moves on to the accept clause, and then it moves on to bigger and better things. And then you’ll see that this line does get printed. So let’s run this. And there we go. It says file does not exist. File not found error. This line was run. Okay? So if you handle exceptions correctly, the rest of your program will continue to run. Otherwise, you will experience an interruption in your program and it will just crash. Okay? Now, you can handle specific kinds of exceptions. So we can have a type error exception, right? We saw that if I have a type if I do type error.

That’s also an object in Python, right? This accept clause will only run if there is a type error. We know that the kind of exception that we were seeing was a file not found error. So now, if I try to run this, you’ll see that we get the error. It did not see a type error. And so this exception did not even accept a block. Did not run because the type error doesn’t exist. The error is actually file not found error. Okay? So you can have a general, except where it catches all errors. If you just do accept like that without giving the error name or you can capture specific errors. So in our case, if we capture specifically the file not found error. I could put that here. Okay.

And now, if we run this, it’ll be captured correctly. All Right? The file not found error. But if we have some kind of type error in the code above let’s say, for example, we try to add a string and an integer, right? So we do two plus the word, the text to notice the ID is smart enough to say, hey, what are you trying to do here? In it and string cannot be added, but let’s just go ahead and do that anyway. I’m going to save it in result variable and let’s run it and you’ll see that? Boom, we get it. Well, we didn’t get the error.

I’m really glad that this happened because this really shows that, hey, even though we have this error, this code did not even cross this particular line. As soon as the interpreter got to this line, it experienced a file not found error. And so the rest of the code didn’t even run. It just jumped from here to here. Okay, so hopefully that makes sense. This code is not going to run because the first line right there triggered the acceptance and the interpreter jumped to the accept clause and it found that there was this file not found error exception area.

And so it printed whatever it did, whatever was instructed in that block of code. Now let’s say that the file was there. We know sample two does exist, okay? This file does exist. Now look what happens. Let’s run this and boom, we get the other error. And what is that error? That’s this type error, right? Notice it says right here, type error, unsupported operands. Of course, we can’t add two plus string so I can have another accept clause that specifically looks for the type error, which is right here. So let’s paste that type error. And in the case where there’s a type error, we’ll say that there was a type error. Okay? So now this code is able to capture multiple kinds of exceptions.

It can handle file not found error. It can handle a type error when we try to do things with incompatible types. And so now let’s run this and you’ll see that no matter what the error is, it’s handled gracefully and we move on to bigger and better things, which is right here.

Okay? So let’s hit run and boom. There we go. Everything ran correctly. We got to this line. It printed that content. There was an exception here. Notice it says there was a type error. So it jumped to this section right here, right? And since there was no file not found error, we didn’t have to worry about triggering this line because sample two dot text does exist, okay? And so it moved on to bigger and better things and ran this line as well.

Now, what some people do, and this is common practice, is have multiple accept sections like this, errors that you can predict, right? So if we’re performing operations that have to do with files, you might want to include a file not found error. Or if you’re trying to access the Internet, you might make sure that the Internet connection is there. So you might have a capture error for that, and we’ll talk about those later. But what happens is after a couple of hours that you can predict. There’s so many other errors that could take place that you might not predict, like disk out of space, memory issues, whatever.

So in those situations, you can have a general catch all accept statement right here, all right? For errors that you can’t really just a catch all so that we can run this program in somewhat of a graceful manner. So you can say error occurred, logging to the system, all right? And then you’ll be able to log to the system whatever the error was. I think there needs to be another R for the occurred, but it doesn’t really matter. All right?

So now if we run this, notice this line didn’t run, but if we had some kind of error in the code that didn’t hit any of these, neither the file not found error or type error, maybe some other kind of error, then this line would have, of course, been triggered. You also have something called finally, okay? And finally is typically the last thing in a try accept block, all right? And that looks like this. You have a finally, okay? And in here, this will always run. No matter whether the code here succeeded correctly and no exceptions were hit or any exception was hit, this code is still the final block must always run. So we can let me just say that this will always run regardless of whether we have an exception or not.

So let’s run this now, and you’ll see that this will always run. Notice there was a type error. So this was executed, and then this was not executed, of course, because there’s no other error that we were able to trigger from the code that we have up there. But this will always run. Now, if you don’t have any errors, let me get rid of this. We don’t have a type error. We don’t have a file error. None of these exceptions will be hit, but this will still always run.

So let’s run this and boom. There you go. Notice it printed the contents of the file. This will always run regardless of whether we have an exception or not, okay, which is the finally block. And then this line was run. So that’s the finale. And this is very important. In situations when you’re trying to access the Internet, you have a request response type of thing, and you don’t want to leave the request running. So in a final block, you want to make sure you always close the request.

Or if you’re opening a file, you want to make sure you close the file right now with this, with Open clause. This automatically closes the file when this particular block of code is finished. But if you open the file in the traditional way, like I’ve shown you before, by just using the Open method and then saving that file object into a variable and then using that variable in your code later on, you’re going to have to close the file. And that’s typically done in the finally block. Okay? So just to remind you again, if I was to let’s get rid of this with Open, we’ll just say that we’re just going to directly open the file and we’re going to save that file in a variable called we’ll just say my file is equal to Open. And then I could just of course, there won’t be any indentation here because this is just we’re not in a block of the with clause. This is just straight up variable assignment using the Open function. And so now we can say my file dot read. I’ll put that in a print line statement.

So at this point, we also want to make sure we close my file so we can do myfile close. Notice that this myfile object is not accessible down here because this is a different block of code than the tri block of code. Even though we’re assigning the variable here, we can’t access that variable down here. All right? So what we need to do is and this is a much older way of doing things, it’s much easier to use the with clause because then you don’t have to do the close down here.

But one way to handle this situation is we can maybe put another tribe block here, okay, and move all this stuff inside of that particular try block like that. And now notice my file is fine. It’s not highlighting that, giving that error. And then finally, we need to also have an accept block outside right here. Try accept. And we can say some error happened, okay? If any code in here gets into trouble, there’s some kind of exception. This is a catch all exception on the outer try block, okay?

So the idea is we are moving this my file object in the outer try. And so since it’s on the outer try, anything that is on the inside. Like this is a block of code that is inside of this try. All of the variables that are outside will be accessible on the inside. And so let’s run this and you’ll see that it’s working as expected. Now, this is of course, you can see it gets kind of hairy, right? It gets confusing. Just to keep things a little simple, I know we don’t have a type error now, so let’s get rid of that. And I know we don’t have a file not found error. Well, I’ll leave that there.

As a matter of fact, I’m going to change this to another type of error, which is I O error, which is an overall input output error. So if the file is not found, if the file is not accessible, if the file does not have a permission or there’s any issue reading or writing to the file, the I O error is going to take care of that. Instead of saying file does not exist, we could just say issue with working with the file, all right? And then we don’t need this accept block, because we have an overall accept block out here. And then we have this finally. Okay? So this probably is a little more easier for you to read and digest what’s going on. Again, notice my file is defined outside. And on the inner block of code, we have this my file being used.

You cannot use this outside of this try block such as this accept. You can use it only inside. And that’s why we had to nest another try catch in here. Another thing people do is they define this Myfile variable on the outside. So first they do My file is equal to none, all right? So they initialize this Myfile variable so that it’s accessible anywhere in the code that follows this particular line, all right? So we don’t run into the issue where the variable is not defined.

So it defines this variable. And then there is this idiom that’s typically followed. Let me just bring the code back to the way it was before we’ll get rid of this, and we’ll move all of this code back. So we’ll define this My file inside of this try block, okay? And this My file variable is being defined on the outside. So, now, notice we don’t have that issue here. An Idiom that’s often checked is to close the file in the finally block people use. If file not equal to none, then that means that they would have to close the file. And of course, that’s My file not file.

So if this variable is not equal to none, then that means we must have been able to successfully open it, right? And so, in that situation, we must close the file. Now, if we open this particular file with instead of R mode, we can open it in W mode. Then in the finally block, you can, of course, do anything with the My file object, such as write to it.

And if you have W plus, then you can write and read with it, do all sorts of things depending on the mode that’s given here. All right? So we’ll leave it at R. But understand that this My file object, if it has the permission to read, write, and you have that mode for RW, you should be able to read and write files in this finally block or anywhere in the Try block. And so, now let’s run this. Notice everything works fine. So a lot of ground recovered in this lecture. We talked about accept finally the block. We talked about multiple types of exceptions that you can have. You can have more fine grained control as to what to do when certain exceptions take place, such as type error, exception, file not found exception, or this IO exception. Now, there’s a lot more others that you can look into. I’ll.

4. OS Module

In Python that does actual practical stuff such as moving files, going into directories, creating directories, that sort of thing. Because Python is used as a scripting language a lot in enterprise software as well as in startups. And so we’re going to learn about those practical matters and how to use Python to do various useful tasks and build utilities for any operating system system really using this very handy programming language. All right, so that’s what this section is about.

I was getting tired of that old project we were working on. So let’s create a new project. I have Pie charm open here. I’m going to click on create new project, and we’ll call this project app. It doesn’t really matter what you call it, but that’s what I gave for now. And so it’s creating that project space for us. And here it is. So I’ll create a new Python file and we’ll call this application PY. We’ll keep the same naming convention that we’ve been using. So a lot of operating system tasks could be done using the OS module, which is really important. So that’s what we’re going to COVID now. So we import the OS module like that by using import OS, and we can invoke various methods using this module.

So if I wanted to, for example, get the current working directory of where I am on the file system, I can use OS getcwd, which stands for current working directory. And let’s print the result of what that screen show us. So if I let’s right click this file and run it and notice it gives us the exact location of where we are. So we are in users. This is my home directory on this Mac operating system in PyCharm projects, and the project name is app, okay? And so this is where I am. This is the current working directory. I can change the directory to someplace else and for that I would use Osdir chdir, which stands for change directory. And now if I wanted to, for example, just go to my desktop, I can just copy the home directory here, make sure you put it in quotes, you paste that there and then do a slash desktop. I notice polychrome is smart enough to figure out where exactly we are. And so it gives us the recommendations in my home folder.

These are some of the things that start with the word D. And so desktop is there. And so now if I print the working directory, I could say OS get. CWD and you’ll see that the working directory is no longer pi charm projects, it’s actually going to be the desktop, okay? Because this line right here changes the directory to the desktop. So let’s run this application again and notice now this is the current working directory. So let me just add comments as we go through this. So this is change directory and this is get current working directory. And if I wanted to list the contents of a directory, I could do OS listdir. Okay? And so let’s also print the result of that. And so this will print all the contents of this directory. So basically everything on my desktop will be printed. So that’s why I click and run this and notice we’ve got various folders. All right, so these are some of the files that I have on my desktop. This is a particular folder. This is a file. This is a file. And you can just kind of scroll down and see what are some of the things that I have. I got those two sample text files that we created earlier on in the course. If you remember with file I O, I’ve got the process pi and so on. This is a bunch of other stuff. So let’s comment here and say that this lists contents of directory.

Now if I wanted to make a directory, if I wanted to make a folder, basically I’d use OS make mkdir and then specify the name of the folder that I want to make. So we’ll call it my folder. And what this line is going to do is going to create the folder called My Folder in the current working directory, whichever directory that is. And at this point we are on our desktop. So let’s right click here and run this application and notice that it’s not going to print anything here. We didn’t ask for it to print anything, but rest assured that a folder called My Folder was created on the desktop.

Let’s double check. If I head over to the desktop folder here, there it is, My folder. It’s been created. Now if I try to run this again, it’s going to give me an error. Let’s run this again. Notice it gives me an error. It says hey, file exists already. That my folder already exists. So if I wanted to make a conditional statement here and if statement to check if a folder exists or not, if it does, then don’t create it. To otherwise do, all I would have to do is just add an if statement going My Folder. If My folder not in OS list der, right? We’ve already seen listed and this list der by the way. It gives us a list of all the folders and files, right? And so this will check whether this particular file name or folder exists within that list of directories or folders. Make sure you indent to bring that inside of the if statement.

And so now I can just run this over and over again and it’s not going to try to recreate the folder every single time. It’s not going to do that because we’ll never get to this line. If I delete that folder, then it will get to this line and it will create that folder again. Now what if I wanted to make multiple directories, a directory within a directory within a directory. One way of doing that is I would have to go and say OS changed, and then go into this particular folder. And then if there was another folder inside of that, I would change Dir and go into that further and then make the directory invoking the Makedir command. But that’s just really an overcomplicated way of doing it. It’s much simpler than to use a different command. And that command is called makers with an S. So it goes like this. OS make. You actually have to spell out the word make in this case. So make dirs and then specify the directory location. So I could do, let’s say if I wanted to create a subfolder inside of my folder, I could do myfolder subfolder data folder or something, whatever.

So what this is going to do is inside of the My folder, it’s going to create another folder called subfolder. And in that subfolder, it’s going to create another folder called data folder. Okay? So now if I right click, let’s run this again. It’s not going to print anything. We’re not printing. But let’s go over to the desktop and check that My folder. Now there you go. We have that subfolder. If we click on that, we have the data folder, right? So it was able to create the entire hierarchy, the entire tree of directories here because of this make Dirs. Now if I run this again, let’s run it again, it’s going to give me an error because the data folder already exists, okay? And we’re trying to create this data folder in this location. It already exists. So we can do a similar check to see if the top level directory exists in there. By doing that, if My folder not in OS list, and then only then would create the subfolders there. All right, now what if you wanted to remove a given directory, you use the OS RM OS. Rmder method, and we specify the given folder that we want to remove. So let’s take this entire directory location. It has to be in quotes, paste that there. And so what this command is going to do is it’s going to remove not all of these directories. It’s only going to remove the last directory, which is right here, not the root. It’s going to remove the base directory. So I’ll put a comment here saying delete specific directory. And let’s say before I run this, I want to create a file inside of that directory just to show you something. So let’s go back to the desktop in my folder, in subfolder, in the data folder, let’s create another file here.

I’m just going to really quickly open that up in my command prompt, go to users MTS mod, basically go to my home and then go to desktop and let’s go to my folder and subfolder. Now that I’m in here, I’m going to create a file in the data folder. Let me go one step further into the data folder. Obviously, I’m using Linux commands here. You don’t have to learn Linux just yet. That deserves a course on its own. But I’m just going to really quickly create a file here. Myfile. TXT. So I just created a file in that particular folder using a bunch of Linux commands. And so now in the data folder, you can see we have the Myfile. TXT. Now I want to show you that it’s not only going to try to delete that folder, it’s going to check to see if it contains any contents. Does this particular folder have whoops. Does this particular folder, data folder have any files or further directories in there? And so we can try to run it and you’ll see we’re going to run into an error. Notice we get an error. It says that the data folder is not empty. Okay? So you have to make sure that the folder is empty before trying to delete it.

Now, if I was to do myfile TXT, I think that’s what I titled that file. Let’s just double check. Yeah, it’s called myfile. TXT. Let’s run this notice. We can’t delete it. We can’t delete the file like this because saying it’s not a directory. So what method would we need to remove this file? Well, there’s a very handy method called remove. Okay, so you do OS remove and you specify the path of the file. So you take this entire thing and you paste that in here. I’m just going to move this, of course, above this remove directory command. And for now, I just want to comment it out just so that we can see that the file is deleted first. So I’m going to right click this run. And now the file should have been removed. If we go to that location in my folder, subfolder data folder is now empty. That file is gone. So this could be used to let me just delete specific file like that, the remove method. Now we can remove the directory and let me just comment this out again before removing the directory. Let’s see if we can run this program again. Is it going to try to remove that file again? Let’s see if it gives an error. Let’s right click and run this. Notice it says no such file, it doesn’t find this file.

So it’s not sure what we’re trying to do. It’s saying, hey, you want to remove a file, but it doesn’t exist. So it does error out here. So to prevent errors like this, you can of course use the if statement checks, something similar to the one we were doing up there. And I’ll show you how to do that later. But for now, let’s just comment this out and let’s uncomment this and let’s right click and run this again. And there we go. We should have been able to remove the directory. Whoops, I still have the file extension here. I got to get rid of that. Okay, so let’s run that again. And there we go. The data folder is now gone. Let’s double check. Let’s go to desktop My folder. And there we go. There’s no more data folder in the subfolder. So I’m just going to put a comment here saying delete specific folder. And I’m showing you all these tools because we’re going to be building something as an assignment for this section and it’s going to be an involved assignment, a very comprehensive assignment where you get to experiment with all of these different methods and perform various tasks in an operating system.

Now there’s a somewhat dangerous method that I want to introduce to you that’s part of the OS module. If I do OS Removed, we spell the entire Remove word and then we type in DERs. What this is going to do is going to eliminate everything in this particular location, all of these folders, okay? Not only is it going to delete the subfolder, but it’s also going to delete the My folder. So let me comment this out because we’ve already removed that directory. Let’s right click and run this. And we don’t have a data folder anymore. So as you can see, every time we try to run it and the folder is there, it complains. It says no such file or directory. So we have to be conscientious of the fact that when we delete something, we have to put if checks around it to make sure so that we don’t run into an error or we have an exception handling mechanism. But right now we have the My folder and we have the subfolder. So if I run this, it’s going to delete both folders. Okay? The removed ers deleted both the folder subfolder as well as My folder. And this could be a bit dangerous. So I typically don’t use removed ers. I like to be specific in removing the particular file or folder that I want to get rid of. And that’s why I use the Rmdr as well as the Remove for removing files.

Okay? So I’m just going to put a comment here saying this is dangerous because it could pretty much wipe out entire directory structures. So I’m going to comment out this line for now. Removes entire directory structures. Now back on our desktop we have a file called Sample text. I don’t know if you recall. Let’s just go back to the desktop. This is a file that we worked on several lessons ago when I was talking about file IO. And so Sample Text, if I wanted to rename it, I could use the OS rename method and specify the name of the file that I want to rename and that’s sample. TXT. And then the next argument to this method is going to be what I want it renamed to. And so we’ll call it rename TXT. So now Sample Tax is going to be renamed to this new name, which is called renamed text. So let’s right click and run this and you’ll see that it’s now renamed. All right, the contents of the file are still there. So that doesn’t change of course. So that’s all for this lesson. I don’t want to introduce too many OS concepts in just one video.

There’s another thing that is hugely important, which is known as walking directories. OS walk. The Walk method in the OS module is used to traverse directories. All right. And that’s really useful. And you’re going to see all types of use cases for utilizing that and we’re going to get into that in the next lecture. Okay. So I’m going to wrap up this lesson. Thanks for watching. I’ll see you in the next one. Scrolling down further. Now we go into the data directory. All right. And again, the first thing here, A, is the path b here is the directory name if there are any directories. And then the C here is the file name if there are any files. In this case we have two files in the My folder. In the Stuff folder we’ve also got two files and we’ve got one directory. And then scrolling further down the data directory is the third iteration and this is the final. We don’t have anything further going into Data except for just this image file, which we have MTL PNG. Now notice B for this third example here, for the third folder, data b is empty. We don’t have any directories inside of the data folder, we just have a file.

Okay. So that’s how this is broken down. So a more appropriate name for ABC would be this is going to be the path, or let’s call it the dur path. And then the B is going to be the actual whether there are any folders in here. So we can say der names. If there are any der names, they would be in this list where we have data and stuff. And for C, this is actually the file names if we have any files. And so we’ll call it file names. And as you can see, each of these are lists, the directory names if there are any directories in the given path, and the file names if there are any files in the given directory path. And so let’s rename these variables up here as well. Durpath, der names and file names. Okay. Now dirpath is not plural. As we explore, we go deeper into the given path. There’s only one path, right? And inside of the path is where we see whether it has any folders or files. Now just to solidify this example, what I’m going to do is I’m going to create some more folders in each of these subfolders. So let’s go back to the desktop. Let’s go into my folder. Now inside of Stuff, I’m going to create another folder and we’ll call this other Stuff. All right. So now inside of the Stuff folder, we’ve got Data as well as Other Stuff. So let’s see how that looks when we utilize this OS Walk functionality. So I’m just going to run this again. And now let’s explore this further. Notice we’re in the My data folder. Now we go into Stuff, and notice now inside of Stuff, we’ve got a list here with two folders, not just one. We have the data folder from before and we have this new folder called Other Stuff.

And again, notice it’s in a list. Now the OS Walk is going to of course take each of these folders, go further deep into them, right? One at a time. So the next thing what it does is it goes into the Other Stuff folder, which is this path right here. And since Other Stuff doesn’t really have anything in it, it just has empty the directory names variable is empty here, as well as the file names. There’s no file names either. And Other Stuff, it’s just an empty folder. And then it moves on to the data folder, which doesn’t have any further subfolders, but it does have a file. Okay, so hopefully you understand how this functionality works. We’re going to be using this quite a bit.