Lab 14: Python Scripting for System Administration

Introduction

In this lab you will perform the following tasks:

  • Create and modify basic python scripts for system administration (shell commands)

You will be introduced to the following commands:

  • python3

Preliminaries

  1. Have completed the reading on BASH and Python scripting. If you haven’t done this you will almost certainly have difficulty completing the lab!

  2. Have completed the BASH scripting lab, this lab will take you through several similar tasks at a much faster rate

  3. Open an SSH remote terminal session to your Linux server’s IP address

    1. Connect to ITCnet from the computer you will be using as your administrative PC. In most cases this means connecting to the ITC Student VPN (unless you are using the Netlab Windows Administrative PC).

    2. Run the PuTTY software on your computer (or the Windows Administrative PC) and enter in the IP address of your Linux server VM in the "Host Name" box and click the "Open" button.

      Remember that if you do not have a Windows computer to connect from you can either figure out how to SSH from your own computer over the VPN to your Linux server or you can use the Windows Administrative PC that is provided for you in Netlab.
  4. Login with your standard user’s username and password

Creating Simple Python Scripts

  1. Just like with BASH scripts we want to specify the interpreter at the beginning of our script with a sha-bang. We will be learning how to write Python3 as it’s the most recent major version.

  2. Create a new text file named hello.py with the following text:

    #!/usr/bin/env python3
    print("Hello World from Python!")
  3. Set the hello.py file to be executable by all users on the system

  4. Try running your hello.py script now. If your system outputs Hello World from Python! you have successfully written your first Python script!

    ben@2480-Z:~$ ./hello.py
    Hello World from Python!
    ben@2480-Z:~$
    You are now printing text to the screen using the Python print command and not the BASH echo command. If you try running the command print("Hello World!") from your command line you will get an error. This is what will make scripting in Python a bit different than BASH scripting. When scripting in BASH every line in your script file could be entered directly on the command line, in Python the commands need to be run through the Python interpreter to be executed.
  5. It is still possible to run system commands from inside of a Python script as well (that’s what ultimately makes them useful for shell scripting purposes after all). To do this we will need to import a special library of functions that Python can use to interact with the underlying system.

    1. Try modifying your hello.py file like this:

      #!/usr/bin/env python3
      import os
      print("Hello World from Python!")
      os.system("echo Hello World from BASH inside of Python!")
    2. In this example we have imported the os library and then run a special function named os.system to execute a command on the system (in this case the echo Hello World from BASH inside of Python! command).

      There are other ways of running shell commands from inside Python3 including subprocess.run and subprocess.Popen which may be better choices in many circumstances though they are slightly more complex to use. We’ll learn a little more about these later in this lab.
    3. Try running this updated script and see how it works.

      ben@2480-Z:~$ ./hello.py
      Hello World from Python!
      Hello World from BASH inside of Python!
      ben@2480-Z:~$
  6. Now try using what you have learned to create and test another script named simple-backup.py which executes the command tar -czf /tmp/jsmith-backup.tar.gz /home/jsmith/ when the script is run (substitue your own username for jsmith).

Using Variables and Conditionals in Python Scripts

  1. Argument parsing in Python is not quite as simple as it is in BASH but with practice it actually allows for much more complex scripts to be developed. Arguments can accessed as entries in an array through the sys library. First, we need to import the sys library and then arguments will be accessed through the variables sys.argv[0] (for the script name) sys.argv[1] (for the first argument) and so on. These variables with square brackets after them are together called an array which is just a single variable that stores multiple values in different "slots" which are identified by numbers starting with zero in this case.

  2. Try adjusting your hello.py script to look like this:

    #!/usr/bin/env python3
    import sys
    print("The script you are running is: " + sys.argv[0])
    print("Hello " + sys.argv[1] + " from Python!")
    1. What do you think will happen if you run the script like this ./hello.py yourfirstname? Try it and find out if you were correct in your hypothesis!

      ben@2480-Z:~$ ./hello.py Ben
      The script you are running is: ./hello.py
      Hello Ben from Python!
      ben@2480-Z:~$
      As with many things in a complex language like Python there are multiple ways to do things. The sys.argv method for accessing arguments is the most rudimentary method and as you advance in your knowledge of Python you can learn about libraries which make building complex scripts much easier and make them more resilient. For example, the argparse library exists specifically to handle parsing command line arguments to scripts in a standard robust way and avoid re-creating the wheel in each script.
  3. Wait, what if someone doesn’t know they are supposed to include their name as an argument though? Try running ./hello.py and see what happens.

    ben@2480-Z:~$ ./hello.py
    The script you are running is: ./hello.py
    Traceback (most recent call last):
      File "/home/ben/./hello.py", line 4, in <module>
        print("Hello " + sys.argv[1] + " from Python!")
                         ~~~~~~~~^^^
    IndexError: list index out of range
    ben@2480-Z:~$
  4. Unlike BASH Python is a little more particular about things and if you try to access a variable that doesn’t exist it will give you an error. This isn’t very user friendly so let’s add a conditional to check and see if an argument was supplied or not.

    1. Try updating your hello.py script again and adding a conditional like this:

      #!/usr/bin/env python3
      import sys
      
      if len(sys.argv) > 1:
          name = sys.argv[1]
      else:
          name = "World"
          print("Try running " + sys.argv[0] + " yourfirstnamehere to get a personal greeting!")
      print("Hello " + name + " from Python!")
    2. See if you can predict what these changes will do and then try it out to see if you are correct

      ben@2480-Z:~$ ./hello.py Ben
      Hello Ben from Python!
      ben@2480-Z:~$ ./hello.py
      Try running ./hello.py yourfirstnamehere to get a personal greeting!
      Hello World from Python!
      ben@2480-Z:~$
    3. Because python stores all of the arguments into a special type of variable which can hold multiple pieces of data, called an array, we can use the len() function to check the length of the sys.argv array. If it’s more than one (because sys.argv[0] always holds the script name so there is always at least one) we know arguments have been added. If not we know they have not been added. You’ll also notice that indenting properly is important in Python scripts this is how Python knows if a line is part of the if/else statement or something that comes after it.

  5. It’s also possible to capture the standard output of a command and put that into a variable for use just as we did with BASH. We can use the output of the date +%Y%m%d command in a variable to make our backup script a little bit nicer (though again, there are better ways to get a date in Python itself once you get more familiar with the language). You’ll also see that in order to capture the output we need to use the subprocess library instead of the os library we were using before.

    1. Create a script better-backup.py with the appropriate execution permissions and these contents (substituting your own username for jsmith):

      #!/usr/bin/env python3
      import subprocess
      USER = "jsmith"
      TODAY = subprocess.check_output("date +%Y%m%d", shell=True)
      TODAY = TODAY.decode('ascii').rstrip()
      BACKUPFILE = "/tmp/"+USER+"-backup-"+TODAY+".tar.gz"
      print("Beginning backup for "+USER+" on "+TODAY)
      subprocess.call(["tar", "-czf", BACKUPFILE, "/home/"+USER])
      print("Backup completed for "+USER+" saved to "+BACKUPFILE)
    2. Try running the above example and see if it works as you expect. This script used the output from the date command to get date information to be used as part of the filename but, as mentioned above, if you were actually writing the script in python you would probably use some of the Python built in date functions instead. The advantages of doing it with the built-in functionality are that you will no longer be depending on the date program being installed on the system and you won’t have to convert the output of subprocess with the .decode('ascii') function and remove the newline from the end of the date output with the .rstrip() function, and you will just generally have cleaner code.

    3. After testing the above script and seeing how it works try modifying it like this to use built in date library functionality:

      #!/usr/bin/env python3
      import subprocess
      import datetime
      USER = "jsmith"
      TODAY = datetime.date.today().strftime("%Y%m%d")
      BACKUPFILE = "/tmp/"+USER+"-backup-"+TODAY+".tar.gz"
      print("Beginning backup for "+USER+" on "+TODAY)
      subprocess.call(["tar", "-czf", BACKUPFILE, "/home/"+USER])
      print("Backup completed for "+USER+" saved to "+BACKUPFILE)
    4. See if you can predict what this script will do and then try it out to see if you are correct. Once you have figured out how this script works see if you can modify it so you can specify a username as an argument instead of as a fixed variable.

    5. After successfully doing that try modifying the script so that it gives instructions about how to use the script if you forget to give a username as an argument.

  6. It is important to note that while this script works, it’s just a simple example and not well written because it is easily broken by giving a username which doesn’t actually exist on the system, or trying to backup files you don’t have permission to access, etc. A better script could be written to check for all of these things as well as other errors. If the script is to be used by more people than the original author (and in the best cases, even then) it is important for usability and security purposes to do these sorts of checks, particularly if the script accepts user input such as an argument.

Using Loops and Reading User Input

  1. Create a file-generator.py script on your system like the one below and see if you can figure out how it works before testing it:

    #!/usr/bin/env python3
    import subprocess
    BASENAME = "sample-file"
    print("Welcome to the sample file generation utility!")
    print("Files will be created using the filename format " + BASENAME + "X.extension")
    NUM = int(input("Enter the number of files you wish to create, then press Enter: "))
    EXTENSION = input("Enter the extension (without the leading period) to put on the files: ")
    for CURNUM in range(0, NUM):
        subprocess.call(["touch", BASENAME + str(CURNUM) + "." + EXTENSION])
    print("Created " + str(NUM) + " file(s).")
  2. There are a couple of differences to note when comparing this with the similar BASH script we created.

    1. First, instead of using a separate program, like the BASH read program, we are obtaining user input with the input() function. This program is then storing the user’s input into a certain variable. By default the input() function saves information as a string which is a type of variable which can hold letters and numbers but can’t be used for numeric operations like adding, subtracting, looping, etc. Because of this we use the int() and str() funtions to switch the data back and forth from the string type to the integer numeric type as needed in various parts of the script.

    2. Second, the for statement is using the built-in range() function instead of the seq command as we did in BASH to generate the number of loops needed.

  3. Try running the script and see if you were correct about what it does and how it works.

    While this script mostly works, it’s just a simple example and not well written because it is easily broken by the user entering something other than a number for the number of files to be created, etc. A better script could be written to check for all of these things as well as other errors. If the script is to be used by more people than the original author (and in the best cases, even then) it is important for usability and security purposes to do these sorts of checks, particularly if the script accepts user input! This is how security bugs are born.
  4. See if you can modify the script so you can specify a starting and an ending number for the files rather than just a number of files to be created.

  5. See if you can add a check to the script which will display an example of a filename that will be created and asking the user if that looks correct before actually creating all the files. If the user enters N then do not create the files. Hint, you will need to do a little further research on using python if statements to check user input.

Functions and Experimenting

  1. Just as in BASH we can create functions in python as well. Try entering this simple sample as hello-func.py:

    #!/usr/bin/env python3
    def helloname():
        name = input("Enter your name, followed by enter: ")
        print("Hello " + name + "!")
    
    
    print("This program will welcome you twice to be extra nice!")
    helloname()
    print("At this point we could be doing lots of other stuff in our script or be in a loop, etc.")
    helloname()
    print("Thanks for playing!")
  2. This should work pretty much the same way that the BASH equivalent script did. See if you can figure out what will happen when you run it, then try it out and see if you were correct. Note that functions need to be declared (listed) before you use them in your script so it would be common to see them at the beginning of the file as in this example.

  3. One difference with functions in Python is how you pass information in and out of functions. Instead of using arguments you include the information in the function call itself and you can return information from the function which can be used immediately or can be stored into a variable. Try modifying hello-func.py like this:

    #!/usr/bin/env python3
    def askname():
        yourname = input("Enter your name, followed by enter: ")
        return yourname
    
    
    def helloname(thename):
        print("Hello " + thename + "!")
    
    
    print("There are lots of ways to make use of the two functions in this script. We can pass a pre-specified string to a function:")
    helloname("world")
    print("We can ask for your name and use it immediately as input into the hello function:")
    helloname(askname())
    print("We can take a string already stored in a variable and pass the variable to a function:")
    somevariable = "Linux Lover"
    helloname(somevariable)
    print("Or we can ask you your name again and store your name in a variable once and then use it a few times:")
    userentry = askname()
    helloname(userentry)
    print("Thanks for playing " + userentry + "!")
  4. See if you can figure out what this script will do and how it works before running it, then try it out and see if you were correct.

  5. Now that you have a grasp on the fundamentals of Python scripting for system administration see if you can write a Python script which can be used to modify the name of all files in the current directory which share the same extension such as those created by file-generator.sh. The goals for your script should be:

    1. The user has a choice to enter the original and the new extension for the files either using command line arguments or interactively by answering questions your script asks.

    2. The script should display each old file name and then the new file name next to it so you can see what is going on

    3. You will want to use conditionals to figure out if the user entered arguments to use as the old and new extensions or should be prompted for them interactively.

    4. You should use functions and loops in order to make the most efficient script possible. Hints: The actual renaming of the files and displaying old and new names would be a good thing to put in a function because you will be calling it over and over again.

  6. Don’t be afraid to try using search engines for help with this. The goal of this lab is not to make you into a professional programmer but to introduce you enough to Python shell scripting so that you can create useful scripts with resources such as search engines available to you.

Wrapping Up

  1. Close the SSH session

    1. Type exit to close the connection while leaving your Linux server VM running.

  2. If you are using the Administrative PC in Netlab instead of your own computer as the administrative computer you should also shut down that system in the usual way each time you are done with the Netlab system and then end your Netlab Reservation. You should do these steps each time you finish using the adminsitrative PC in future labs as well.

You can keep your Linux Server running, you do not need to shut it down.

Document Build Time: 2024-10-30 23:55:42 UTC
Page Version: 2024.08
Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License