CL Script Maildir Backup

Introduction

Inspired by the book “Automate the boring stuff with Python” I constantly automate boring stuff on my Linux machine. But using Common Lisp instead of Python. I think learning a programming language is like learning everything else in life. Apply theoretical knowledge (programming language) to something helpful or useful and you will be more motivated to learn it and it will be harder to forget about it. Programming is all about solving computer problems. If you do not have a computer problem, what is the point of learning a programming language?

What is a computer problem?

A computer problem is the following: I am using a program called mbsync (or isync) to have my emails available on my computer even without internet access.

isync is a command line application which synchronizes mailboxes;

isync solves that problem very well. Here is another problem I do have: I want to periodically backup the ~/Maildir folder (the folder my emails are synced), versioned by date and zipped on my computer to my Nextcloud instance. The reason for this is: What if my Maildir folder got corrupted? Or my email provider experienced a service outage?

Let us break down the problem into smaller steps:

Call isync to synchronize emails.
Create a zip file of ~/Maildir folder.
Move this zip file to a sync folder of Nextcloud.
Remove older backups.
Do this periodically.

The bash script phenomenon

In the world of Linux people often tend to solve their computer problems using bash scripting. And a bash script which solves my computer problem backing up my Maildir folder to Nextcloud could be like the following:

#!/bin/bash
# -*- mode: shell-script; coding: utf-8; -*-

# Set variables
MAILDIR="$HOME/Maildir"
BACKUP_DIR="$HOME/Documents"
DATE=$(date +%Y%m%d)
BACKUP_FILE="maildir_backup_$DATE.zip"

# Run mbsync
mbsync -a

# Create zip of Maildir
zip -r "/tmp/$BACKUP_FILE" "$MAILDIR"

# Move zip file to Documents folder
mv "/tmp/$BACKUP_FILE" "$BACKUP_DIR"

# Optional: Remove old backups (keeping last 5)
cd "$BACKUP_DIR" && ls -t maildir_backup_*.zip | tail -n +6 | xargs -r rm

A nice and easy bash script. But often computer problems tend to grow and scripts has to grow too. And when source code is growing it is getting difficult to maintain. This script is using global variables which can be overwritten easily by other scripts or name conflicts can happen at the time of execution. DATE seems like a very general name for a variable. Maybe it is already defined in my .bashrc file or otherwise loaded into my bash environment?

Some important things

Before we beginn to rewrite the solution to my computer problem using Common Lisp I need to introduce you to some important things in Common Lisp:

UIOP Library
Pathname Objects

UIOP Library

the Utilities for Implementation- and OS- Portability.

UIOP is the portability layer of ASDF. It provides utilities that abstract over discrepancies between implementations, between operating systems, and between what the standard provides and what programmers actually need, to write portable Common Lisp programs.

UIOP simplifies to write implementation independent Common Lisp code for the programmer. So the first line of code needed in our Common Lisp file is:

(require :uiop) ; Load UIOP Library

We will make use of uiop:run-program to call mbsync and zip programs.

Pathname objects

There are no such thing as files in Common Lisp. Instead Common Lisp implementations are using a framework in which files are named by a Lisp data object of type pathname.¹

Here are some important pathname functions to remember:

user-homedir-pathname: Returns the user home directory as pathname.
merge-pathnames: Merges two pathnames into one.

(merge-pathnames #P"Maildir/" (user-homedir-pathname))

#P"/home/marcus/Maildir/"

Using pathname objects in Common Lisp instead of strings for working with files offers several benefits, aligning well with functional programming principles. Here are the key advantages:

Abstraction and platform independence: Pathname objects provide a higher-level abstraction for file paths, making your code more portable across different operating systems. The system automatically handles differences in path separators and conventions.
Component-based manipulation: Pathnames allow you to work with individual components (device, directory, name, type) separately, which is more functional and less error-prone than string manipulation.
Type safety: Using pathnames provides better type safety, as the system can check for valid pathname objects rather than arbitrary strings.
Built-in operations: Common Lisp provides many built-in functions for working with pathnames, making file operations more convenient and less error-prone.

Rewrite the bash script

Get today as date

The first smaller problem we want to address is getting a versioned pathname as maildir_backup_20240911.zip. To achieve this we need to get the date at the time the script is executed with a function called get-date. Common Lisp implementations provides us with two important functions related to date and time:

get-universal-time: Counts seconds since January 1, 1900, at midnight UTC.
decode-universal-time: Parses get-universal-time and returns multiple values related to date and time.

(multiple-value-bind (second minute hour date month year)
    (decode-universal-time (get-universal-time) 0)
  (format nil "~4,'0d~2,'0d~2,'0d" year month date))

20240923

By using the function get-date we are able to construct a versioned file name.

(defun get-date ()
  (multiple-value-bind (second minute hour date month year)
      (decode-universal-time (get-universal-time))
    (format nil "~4,'0d~2,'0d~2,'0d" year month date)))

Footnotes:

http://cltl2.lisp.se/cltl/clm/node204.html#SECTION002711000000000000000