Introducing Backutil: A Python‐based Windows backup utility

2021-01-01  Programming,   Technology

Back in the spring, I decided that 2020 would be the year I would finally see a coding project through to completion. A recent work project shone a light on backup and recovery, and I realised that I should probably be a bit more consistent with my own backups from my personal PC. Wanting to avoid paying another annual subscription, I decided to write a script myself. Thus Backutil was born - and the project only grew from there as I added more features along the way.

I’m still not quite at the point when I’m ready to release a v1.0, but I told myself a few months ago that I wanted to put together a minimum viable product by the new year - so here it is! It has a few bugs and is missing a couple of features, but Backutil is now a functioning Python-based utility for backing up files on Windows systems, complete with options for incremental backups and backup rotation.

Contents

1. Introduction
          a. Testing and limitations
2. Configuration
          a. Configuration file
          b. Backup list file
          a. Command line options
3. Download
4. Changelog
5. Future development

Introduction

Backutil is a simple, Python-based utility for backing up files from Windows systems to compressed, password-protected local archives. It has features for performing incremental backups and automatically rotating backup files. This is achieved using robocopy and 7-Zip, which must be installed.

Backutil is a learning/hobby project and some aspects of its code may not follow best practices. While you're welcome to use it, you do so at your own risk. Make sure you take a manual backup of your files before trying it out, and don't go relying on it to back up your production servers.

To back up your files, simply ensure you have configured Backutil (see the sections below) and run backutil.exe from the Command Prompt or PowerShell. The utility will report on its progress until the backup is successfully completed. More detail can also be found in backutil_log.csv.

When the utility is finished, you should find your complete backup files in your designated backup folder. The number and size of these backup files can be configured using the incremental backup and rotation settings, which are set in the configuration file or as command line options.

As Backutil automatically manages your backup files, it can be configured to run automatically at the desired interval using the Windows Task Scheduler. Backutil’s features can be used to generate rolling full or incremental backups as required by your backup objectives and disk size.

Testing and limitations

Aside from all the testing that comes naturally during the development process, I have been using Backutil to back up my personal files for the last month or so using a Windows scheduled task to run the utility on a weekly basis. My configuration performs incremental backups on a five-file rotation and so far has worked without a hitch, to a level where I occasionally even forgot it was running.

One slight limitation, which will be improved with future development, is the speed of the backup process. My current backups include around 83GB of data (about 50GB once compressed), and the initial “big” backup can take a couple of hours to run. For this reason, I recommend using Backutil to back up a focused set of directories rather than your whole hard drive, at least for the moment.

Configuration

Backutil can be configured via three main means: a configuration file, a file containing a list of directories to be backed up, and a series of command line options that override other settings. If a configuration file and backup list file are present, Backutil can be run using the following simple command.

.\backutil.exe

In terms of an installation directory, I put the executable and configuration files in C:\backutil\bin\ and use C:\backutil\ as the staging folder for the temporary files and records. However, you can put these files wherever you like as long as your settings are configured accordingly.

Configuration file

Backutil automatically loads settings from a file named config.ini, including the location of the list of directories to back up, folders for backups and temporary files, and incremental backup and rotation options. The configuration file should be located in the same folder as backutil.exe.

The contents of an example configuration file are shown below.

[LOCAL]
computer_name = matts-pc
backup_list = backup-list.txt
staging_folder = C:\backutil\
archive_pass = supersecretpassword
incremental = True
rotation = True
retained = 5

[SERVER]
server_directory = D:\backups\

The table below sets out what each option in the config.ini configuration file does. Note that all directories supplied via the configuration file must include the trailing backslash.

Section Key Purpose
LOCAL computer_name Sets backup folder/record name
LOCAL backup_list Sets the backup list filename
LOCAL staging_folder Sets folder for temporary file storage
LOCAL archive_pass Sets 7-Zip backup file password
LOCAL incremental Turns incremental backups on/off (True/False)
LOCAL rotation Turns backup rotation on/off (True/False)
LOCAL retained Sets number of backups to retain if rotation is on
SERVER server_directory Sets folder for backup storage

Backup list file

The backup list file is a text file containing a list of directories. When Backutil is run, it will automatically generate a list of files to back up by scanning the contents of these directories and all subdirectories. The format of the backup list file should look something like the example below.

C:/Users/Matt/Desktop
C:/Users/Matt/Downloads
C:/Users/Matt/Music/iTunes/iTunes Media/Music
C:/Users/Matt/Pictures
C:/Users/Matt/Videos

Command line options

Backutil also supports several options if you wish to set certain configuration parameters manually from the Command Prompt or PowerShell. Note that any parameters set via command line options will override the respective parameters in the config.ini configuration file.

Short Long Purpose
-h --help Displays the help file
-n <name> --name <name> Manually sets the backup folder/record name
-l <file> --list <file> Manually sets the backup list file
-i --incremental Manually turns on incremental backups
-r <no> --rotate <no> Manually turns on backup rotation and sets number of backups

The following command shows an example of how the command line options may be used.

.\backutil.exe -n matts-pc -l locations.txt -i -r 5

Running Backutil with the options above will save backup files to a folder called matts-pc (note that this folder name is also how previous backups are tracked). The list of directories to back up files from will be retrieved from locations.txt. Backups will be incremental (only changed files will be backed up each time Backutil runs) and five previous backups will be retained.

Download

Use the link below to download Backutil. You’re free to run it for personal use - just please let me know if you encounter any bugs so I can work on fixing them in future realeases!

Backutil is a learning/hobby project and some aspects of its code may not follow best practices. While you're welcome to use it, you do so at your own risk. Make sure you take a manual backup of your files before trying it out, and don't go relying on it to back up your production servers.

Download Backutil v0.52
36.6MB, ZIP
The downloadable archive contains backutil.exe and an example config.ini file. Interested in the source code? The full Python script is available in the Backutil GitHub repository.

Changelog

Date Version Changes
19/02/2021 v0.52 Small bug fixes and improvements from v0.51:
- 7-Zip file now generated directly in destination folder
- Hash file now only generated after successful backup
- Blank line at end of backup list file no longer required
- Help page consistent with online documentation
- Fixed –help and –incremental arguments

Future development

My determination to build a minimum viable product before the end of 2020 means that I have a backlog of bug fixes and new features to add during 2021. These include:

Have you got more ideas for new Backutil features? Or have you found bugs that I haven’t? Please send me an email to let me know so I can add them to the development backlog.

If you’re interested in the project, check back regularly for new releases. I’ll also announce any updates on my Twitter account, and may add some form of banner to my site’s homepage.

Looking for the comments? My website doesn't have a comments section because it would take a fair amount of effort to maintain and wouldn't usually present much value to readers. However, if you have thoughts to share I'd love to hear from you - feel free to send me a tweet or an email.