How to back up a huge file with Bacula?
https://stackoverflow.com/questions/244337
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianQuestion
It currently is 700MB but it's conceivable that it'll grow beyond the 1GB. Normally I just copy this file to another location (for the curious, it's the database of a Zope instance, a ZODB file).
This file changes little from day to day, but I understand Bacula can't do inside-the-file subdivision for incremental backups. Anyway, it doesn't matter. What I want to do is a full backup daily and keep two of them and a full backup weekly and also keep two of them. So at any given time I can get yesterday, the day before yesterday, a week ago and two weeks ago. Would you think that's a good idea?
I suppose I should make two schedules, daily and weekly. But which numbers should I have on the volumes and the pools to achieve this? Two volumes of 1.5GB? Any hints or guidance is welcome, I'm not a sysadmin and my experience with Bacula is very limited.
Solution
Online backup of a large database file is risky business, as the file might change while you are reading it, rendering the backup inconsistent and possibly useless. I believe you should not be making backups of the ZODB file itself, but rather of diffs created daily by the repozo tool. This way, you also outsource the job of handling the inside-the-file subdivisions that you say Bacula is incapable of dealing with.
OTHER TIPS
In my experience with bacula and backup to disk, it is best to keep one volume per backup job. That way there is no dead space in the files as jobs expire. Bacula can reuse the whole volume and it cuts down on disk utilization. Use the "Set Maximum Volume Jobs = 1" directive in the pool resource.
I would set up two pools, a daily and weekly. Set the volume retention to two days in the daily and two weeks in the weekly. Schedule the daily on say, mon-sat, and the weekly on sunday.
Depending on your infrastructure, I would recommend taking a snapshot of the volume you are backing up to "freeze" it and make the backup from there.
For some of our backups we are using LVM snapshots (http://tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html), to avoid locking any of our databases (we have terabytes of data to backup and a lock would have a huge impact on the service)
Then, as you said that the database is not moving too much, I would go and have a 6 days retention period, 6 volumes for Dailies and 2 volumes for weeklies. Your Dailies should hit the Incremental backup pool, and the Weeklies should be the fulls.
For example, make the Weeklies (Fulls) run on Monday and then an incremental every day (Tue-Sun). This will allow you to come back any day of the week if you realize your data is corrupted, without taking too much space or time during the backup.
EDIT: And... I should check the post dates before answering. Haha. 3 Years late.
For the open source bacula (bacula.org) the best idea is to use the "Set Maximum Volume Jobs = 1" directive indeed.
If you want the "inside-the-file subdivision for incremental backups", please consider the Delta Plugin from Bacula Systems - https://www.baculasystems.com/products/bacula-enterprise-plugins/delta.
