Latest news

Amazon EC2 plugin

Our team is currently working on finalizing the development phase of the new EC2 backup plugin. This is the first cloud plugin for Bacularis and an important step towards broader cloud integrations within the project.

As we have already discovered during development, backing up EC2 virtual machines is quite challenging due to the number of implementation details involved and the overall complexity of the process. Fortunately, Amazon's documentation is very well prepared, which significantly helped us during the implementation work.

Why EC2 backup?

The goal of the Bacularis EC2 plugin is to provide consistent block-level backup and restore capabilities for EC2 virtual machines within AWS infrastructure.

Block-level backup often means transferring large amounts of data. In this case, however, we used incremental EBS snapshots together with the ability to retrieve only changed blocks between snapshots for Bacula incremental backups. This significantly reduces the amount of data required both for backup creation and for later restore operations.

Challenges

Surprisingly, the biggest challenge during the development of the Bacularis EC2 plugin was not the backup and restore process itself, but rather designing and implementing a robust transport layer responsible for preparing data for cloud transfer and retrieving it back from AWS.

The transport layer supports features such as:

• error detection,
• queuing,
• retries,
• exponential backoff,
• and other reliability-related mechanisms.

We designed this layer in a reusable way so it can also be used in future Bacularis features that communicate with remote network services and transfer data over the network. Additionally, some of its parameters will be configurable directly from the advanced EC2 plugin settings.

The second major technical challenge was designing the plugin to stream data directly between AWS EC2 and Bacula volumes without any intermediate local storage writes. The same applies in the opposite direction - restoring backup data by streaming it directly from backup volumes to AWS without temporary local storage usage.

Initially, this turned out to be even more difficult than the first challenge because web application operations are somewhat limited in this area, especially regarding Bacula ↔ Bacularis plugin communication. Fortunately, this challenge has also been successfully solved.

Below is a screenshot of one of the plugin configuration windows. The screenshot presents the backup job creation window together with plugin configuration and Bacula fileset settings.

In the coming days, we plan to finalize the development phase and begin testing. If you already have any questions or feedback, feel free to join the Bacularis User Group. We will be happy to hear from you.