Introducing Backutil: A Python‐based Windows backup utility
Back in the spring, I decided that 2020 would be the year I would finally see a coding project through to completion. A recent work project shone a light on backup and recovery, and I realised that I should probably be a bit more consistent with my own backups from my personal PC. Wanting to avoid paying another annual subscription, I decided to write a script myself. Thus Backutil was born - and the project only grew from there as I added more features along the way.
I’m still not quite at the point when I’m ready to release a v1.0, but I told myself a few months ago that I wanted to put together a minimum viable product by the new year - so here it is! It has a few bugs and is missing a couple of features, but Backutil is now a functioning Python-based utility for backing up files on Windows systems, complete with options for incremental backups and backup rotation.
Backutil is a simple, Python-based utility for backing up files from Windows systems to compressed, password-protected local archives. It has features for performing incremental backups and automatically rotating backup files. This is achieved using
robocopy and 7-Zip, which must be installed.
To back up your files, simply ensure you have configured Backutil (see the sections below) and run
backutil.exe from the Command Prompt or PowerShell. The utility will report on its progress until the backup is successfully completed. More detail can also be found in
When the utility is finished, you should find your complete backup files in your designated backup folder. The number and size of these backup files can be configured using the incremental backup and rotation settings, which are set in the configuration file or as command line options.
As Backutil automatically manages your backup files, it can be configured to run automatically at the desired interval using the Windows Task Scheduler. Backutil’s features can be used to generate rolling full or incremental backups as required by your backup objectives and disk size.
Testing and limitations
Aside from all the testing that comes naturally during the development process, I have been using Backutil to back up my personal files since the start of 2021 using a Windows scheduled task to run the utility on a weekly basis. My configuration performs incremental backups on a five-file rotation and so far has worked without a hitch, to a level where I occasionally even forgot it was running.
Backutil can be configured via three main means: a configuration file, a file containing a list of directories to be backed up, and a series of command line options that override other settings. If a configuration file and backup list file are present, Backutil can be run using the following simple command.
In terms of an installation directory, I put the executable and configuration files in
C:\backutil\bin\ and use
C:\backutil\ as the staging folder for the temporary files and records. However, you can put these files wherever you like as long as your settings are configured accordingly.
Backutil automatically loads settings from a file named
config.ini, including the location of the list of directories to back up, folders for backups and temporary files, and incremental backup and rotation options. The configuration file should be located in the same folder as
The contents of an example configuration file are shown below.
The table below sets out what each option in the
config.ini configuration file does. Note that all directories supplied via the configuration file must include the trailing backslash.
|LOCAL||computer_name||Sets backup folder/record name|
|LOCAL||backup_list||Sets the backup list filename|
|LOCAL||staging_folder||Sets folder for temporary file storage|
|LOCAL||archive_pass||Sets 7-Zip backup file password|
|LOCAL||incremental||Turns incremental backups on/off (True/False)|
|LOCAL||rotation||Turns backup rotation on/off (True/False)|
|LOCAL||retained||Sets number of backups to retain if rotation is on|
|LOCAL||max_threads||Sets maximum number of threads for multiprocessing|
|SERVER||server_directory||Sets folder for backup storage|
Backup list file
The backup list file is a text file containing a list of directories. When Backutil is run, it will automatically generate a list of files to back up by scanning the contents of these directories and all subdirectories. The format of the backup list file should look something like the example below.
Command line options
Backutil also supports several options if you wish to set certain configuration parameters manually from the Command Prompt or PowerShell. Note that any parameters set via command line options will override the respective parameters in the
config.ini configuration file.
|-h||--help||Displays the help file|
|-n <name>||--name <name>||Manually sets the backup folder/record name|
|-l <file>||--list <file>||Manually sets the backup list file|
|-i||--incremental||Manually turns on incremental backups|
|-r <no>||--rotate <no>||Manually turns on backup rotation and sets number of backups|
|-t <no>||--threads <no>||Manually sets max threads for multiprocessing|
The following command shows an example of how the command line options may be used.
Running Backutil with the options above will save backup files to a folder called
matts-pc (note that this folder name is also how previous backups are tracked). The list of directories to back up files from will be retrieved from
locations.txt. Backups will be incremental (only changed files will be backed up each time Backutil runs) and five previous backups will be retained. A maximum of six threads will be used for hash generation and file copy operations.
Use the link below to download Backutil. You’re free to run it for personal use - just please let me know if you encounter any bugs so I can work on fixing them in future realeases!
|Download Backutil v0.70
|The downloadable archive contains
|03/05/2021||v0.70||Implemented multiprocessing and other improvements:
- Faster hash generation and file copying with multiprocessing
- Improved terminal output and logging
- General code improvements/tidying
|26/03/2021||v0.61||Implemented SQLite and other speed improvements:
- All data processed using SQLite
- Hashes generated using bigger file chunks
- File size cut by 80 per cent due to Pandas removal
|19/02/2021||v0.52||Small bug fixes and improvements from v0.51:
- 7-Zip file now generated directly in destination folder
- Hash file now only generated after successful backup
- Blank line at end of backup list file no longer required
- Help page consistent with online documentation
- Fixed –help and –incremental arguments
My determination to build a minimum viable product before the end of 2020 means that I have a backlog of bug fixes and new features to add during 2021. These include:
Remote backups - You’ll notice that some parts of Backutil use terminology associated with remote backups (for example, the Server section in the configuration file). This is because Backutil could originally be configured to use WinSCP to send backup files to a remote server. This has been removed for the initial release, but I hope to reinstate it in a future version.
Graphical user interface (GUI) - I’ve played around with Python GUIs a couple of times before, but have never had a script worth implementing one for. Depending on time limitations, I might develop a GUI for Backutil to increase ease of use for less experienced users.
Inconsistencies and bug fixes - The more time you spend with a piece of code, the more flaws you find in it. I’m sure I’ll spot plenty to fix along the way.
Have you got more ideas for new Backutil features? Or have you found bugs that I haven’t? Please send me an email to let me know so I can add them to the development backlog.