Penumbra: shady backups for Windows

I’m proud to announce that, today, I am presenting another one of my slowly progressing, yet promising projects: Penumbra! I created this application for myself because I am basically too much of a nitpicker to find backup software to suit my needs. It started as a script of some sorts, but things soon got out of hand and before I knew it, I was building a GUI (even though a command-line interface would have been sufficient for me).

The GUI is fairly complete, but it will remain offline for a while until I’m satisfied with the back-end code. So for now, Penumbra is a CLI application.

How to be picky about backup software

There are two types of backup software: complicated, enterprise-ready behemoths and dumbed-down, user-friendly eye-candies. Together, they offer everything I need (and more), but I can only use one for each backup. Let the nitpicking begin…

The concept of making backups is really simple. It involves copying files and that’s it. In practice, it can be a real pain to get proper copies of files: A file may be in use or locked for some other reason by the operating system; the file has special security settings that do not survive a basic copy operation; the file is a junction or symbolic link of some sort; etc.. This is usually not a problem if you’re backing up your own documents, music and downloads, but files such as application data are often a problem for basic backup methods. Active programs – most of which run in the background – often keep a lock on their application data, preventing direct access. Of course one could copy a complete disk image when the operating system isn’t running (probably using an application from bootable CD or USB), but this is not ideal for most users… including me.

Enter the shadow

Windows XP and upwards provide a way to avoid locked files and thereby solve one of the biggest issues of making a proper backup. It’s called VSS, or Volume Shadow Copy service, or just Shadow Copy. What it does is create a snapshot of the filesystem while allowing applications (i.e., the ones that implement its interface) to bring their data in a consistent state. This snapshot has no locked files; permissions still count, but that’s what the administrator account is for.

This feature hasn’t gone unnoticed with many developers of backup software, but unfortunately most implementations are part of complicated applications (enterprise software gets all the goodies first…) and/or proprietary.

Shedding a light on Penumbra

This is where Penumbra comes in to fill the gap. Penumbra wants to be powerful backup application with a simple, yet flexible, interface and an open attitude towards extensibility. Before you mistake that sentence for manager mumbo-jumbo, let me elaborate a little. By ‘powerful’ I am not aiming too high; there will be no transaction management, resuming after network errors or anything that would be vital in an enterprise environment. Coorporations have money to buy these features elsewhere, and they probably already do so there is no need to target that group. Penumbra will be ‘powerful’ in that it does one thing and does it well, namely copying the files from a home computer that are or may be important to its users and making sure that they can be restored perfectly as if they were never molten/burnt/scratched off the hard disk.

The interface will be ‘flexible’ in a way that allows users to easily customize the way backups are made. Skilled users shouldn’t feel limited and novice users shouldn’t feel intimidated. That’s why I also intend to keep Penumbra simple. I hope I don’t have to clarify ‘simple.’ Another goal that fits the keyword ‘flexible’ is that backups should be restorable even if Penumbra is somehow not available or preferred. At this moment Penumbra can’t restore backups, so it’s a good thing backups can be restored by hand (and a file archiver).

Open ended

Because I love experimenting with other APIs even more than the concept of open source, this project will have both. First of all, after careful consideration, the code is available under a GPL license. Secondly, I intend to make certain parts of the backup engine extensible and others even replaceable through a plugin system of some sorts. I first have to figure this all out for C#/.NET, but after that it will be one of the first things I do (a complete GUI is also on that todo-list, by the way).

On the cutting edge

I know you can’t wait to make your next backup with Penumbra. Unfortunately, at the moment, it will require a little effort. You can grab the code from Penumbra’s GitHub repository, fire up Visual Studio (the free Express edition might work), start building and hope I didn’t forget to commit a critical file. Alternatively, I could provide a compiled binary, but that would take away the thrill of trying development code… also, it would be irresponsible of me to encourage using it for real backup purposes. Even though you can already make good backups that are probably 100% safe, I shouldn’t recommend it.

So, some final words on using the program: Check out the sample .json file and of course, run penumbra.exe /help before anything else. Next time I will write some more on using Penumbra and its development. Thanks for reading!