Borg backup
Torej, privzeta aplikacija za backup v Ubuntuju je DejaDup, vendar ta program deluje nezanesljivo. Ko sem prijavil nek kritičen bug, mi je razvijalec začel razlagati, da je problem, ker se on in Ubuntu ne strinjata glede označevanja novih verzij (vir). Zadeva še vedno ni popravljena...
Za določene KRITIČNE hrošče traja skoraj leto, da jih odpravijo:
https://launchpad.net/deja-dup/+bug/1600299
Določenih funkcionalnosti preprosto nočejo implementirat, argumenti za to pa so precej trhli:
https://bugs.launchpad.net/deja-dup/+bug/324631
Iz bug trackerja se da razbrati, da je kar nekaj ljudi, ki so izgubili svoje varnostne kopije oz. jih niso uspeli restavrirati... Skratka, čas, da se razgledamo po alternativah.
Tako sem našel Borg backup, ki ga sedaj preskušam.
Namestitev na Ubuntu 16.04.2 LTS:
To je del na odjemalcu. Če varnostno arhiviranje izvajamo na zunanji strežnik, mora enaka različica programa teči tudi tam. Ker uporabljam RaspberryPi, sem uporabil kar ustrezen Borg ARM build. Na voljo so tudi za NAS naprave (ARM 5 procesorji).
Na RPi sem zadevo namestil takole:
Ko zdaj rečem:
OK. Zdaj je treba narediti arhiv. Le-ta se nahaja na zunanjem disku, priklopljenem na RPi. Rečem:
Vnesem geslo:
Nato ustvarim arhiv z imenom "matej_prenosnik":
Vnesem geslo:
Zdaj mi dela backup (glede na to, da to dela preko interneta, zna trajat...).
Ker sem uporabil parameter --progress, mi izpisuje kaj dela:
1.95 GB O 1.78 GB C 136.76 MB D 6765 N home/matej/.cache/mozilla/firefox/we3453.default/cache2/entries/47AB91F5A8E3EC0064CDF942E08689321B4B19BE
In priročnika:
Vsaj tako razumem zadevo.
Kontaktiral sem developerje in dobil par uporabnih odgovorov.
Kaj se zgodi, če se backup prekine? Npr. računalnik izgubi povezavo, gre v spanje, itd...
Odgovor:
Borg is explicitly desinged to take care of that! It creates a 'checkpoit' of it's index every 5 minutes (changable via --checkpoint-interval). If you connection drops (or long hibernate) or PC crashes while backing up, all chunks up to the last checkpoint will be in the index.
You snapshot you tried to created in the repository will only be saved as '<snapsotname>.checkpoint' and if you run the command again, borg will skip all already known chunks, backup the rest and if successfull, remove the old .checkpoint snapshot for you (if the name was the same).
So, you can happily interrupt your backup (if it ran for at least 5-10 minutes) and run it again until you managed to save everything. Perfect for slow, initial backups over the internet.
Ali računalnik lahko uporabljam med backupiranjem?
Odgovor:
Na kratko, da.
Borg usues regular filesystem syscalls to walk though your files. If it has not worked through directory X yet and you change something, borg will backup this change once it is in directory X. The other way around, borg will not notice any changes done to directories it already processed.
When working on a file WHILE borg is chunking it… it might cause locking issues (borg will warn and skip it) or in the worst-case (if not
locked), it will store an inconsistent state. It is your responsibility to avoid that.
Generally speaking, for a regular desktop-user this is a very rare case.
Most of the time programms lock their files or save them in a burst. And even if, the next backup the consistent version will be saved.
So for regular desktop usage you will not have any issue in 99% of the cases if you work while backup up - tho simply fix that by doing your backup while you make a pause or at night.
Problem so pa datoteke, ki se med varnostnim arhiviranjem spreminjajo. Recimo virtualni diski (VDI).
For stuff like (active) virtual machine images (borg loves those, deduplication is *great* on these) of course the chance is a little
higher as they are constantly written to. Same with database files and such. For critical data, you should make sure to have a consistant sate.
Dumps (DB-Dumps, VM-Snapshots) or on filesystem-level (btfs or zfs snapshot) or on block level (lvm snapshot) for borg to work on.
(That being said, I have an active VM on my homeserver which I backup 'the wrong way' without snapshots or pausing it. Did one test and had one actual restore - both times it worked without issues. Tho the data is not exactly super-critical, that's why I do it. Sitll not
recommended. :P)
Za določene KRITIČNE hrošče traja skoraj leto, da jih odpravijo:
https://launchpad.net/deja-dup/+bug/1600299
Določenih funkcionalnosti preprosto nočejo implementirat, argumenti za to pa so precej trhli:
https://bugs.launchpad.net/deja-dup/+bug/324631
Iz bug trackerja se da razbrati, da je kar nekaj ljudi, ki so izgubili svoje varnostne kopije oz. jih niso uspeli restavrirati... Skratka, čas, da se razgledamo po alternativah.
Tako sem našel Borg backup, ki ga sedaj preskušam.
Namestitev na Ubuntu 16.04.2 LTS:
sudo apt-get install borgbackup borgbackup-docUkaz:
borg -Vpravi, da imam različico 1.0.7 (trenutna stabilna različica je sicer 1.10):
borg 1.0.7
To je del na odjemalcu. Če varnostno arhiviranje izvajamo na zunanji strežnik, mora enaka različica programa teči tudi tam. Ker uporabljam RaspberryPi, sem uporabil kar ustrezen Borg ARM build. Na voljo so tudi za NAS naprave (ARM 5 procesorji).
Na RPi sem zadevo namestil takole:
wget https://borg.bauerj.eu/borg-1.0.7-armv6
sudo cp borg-1.0.7-armv6 /usr/local/bin/borg
sudo chown root:root /usr/local/bin/borg
sudo chmod 755 /usr/local/bin/borg
Ko zdaj rečem:
borg -Vmi na RPi reče:
borg 1.0.7
OK. Zdaj je treba narediti arhiv. Le-ta se nahaja na zunanjem disku, priklopljenem na RPi. Rečem:
borg init ssh://pi@backup.rpi/media/backup/matej(backup.rpi mi v hosts datoteki kaže na moj IP naslov, seveda)
Vnesem geslo:
Enter new passphrase:Geslo je obvezno, razen če ga izrecno ne želiš. Namesto gesla je mogoče uporabiti datoteko s ključem (keyfile).
Enter same passphrase again:
Do you want your passphrase to be displayed for verification? [yN]:
Nato ustvarim arhiv z imenom "matej_prenosnik":
borg create --compression zlib,4 ssh://pi@backup.rpi/media/backup/matej::matej_prenosnik ~ --exclude '*.vdi' --exclude ~/Prejemi --stats --progress
Vnesem geslo:
Enter passphrase for key ssh://pi@backup.rpi/media/backup/matej:
Zdaj mi dela backup (glede na to, da to dela preko interneta, zna trajat...).
Ker sem uporabil parameter --progress, mi izpisuje kaj dela:
1.95 GB O 1.78 GB C 136.76 MB D 6765 N home/matej/.cache/mozilla/firefox/we3453.default/cache2/entries/47AB91F5A8E3EC0064CDF942E08689321B4B19BE
In priročnika:
-p, --progress show progress display while creating the archive,Torej, 1.95 GB je original size, 1.78 GB je compressed size, 136.76 MB je deduplicated size, arhiviral je 6765 datotek, sledi trenutna datoteka, ki jo arhivira.
showing Original, Compressed and Deduplicated sizes,
followed by the Number of files seen and the path
being processed, default: False
Vsaj tako razumem zadevo.
Kontaktiral sem developerje in dobil par uporabnih odgovorov.
Kaj se zgodi, če se backup prekine? Npr. računalnik izgubi povezavo, gre v spanje, itd...
Odgovor:
Borg is explicitly desinged to take care of that! It creates a 'checkpoit' of it's index every 5 minutes (changable via --checkpoint-interval). If you connection drops (or long hibernate) or PC crashes while backing up, all chunks up to the last checkpoint will be in the index.
You snapshot you tried to created in the repository will only be saved as '<snapsotname>.checkpoint' and if you run the command again, borg will skip all already known chunks, backup the rest and if successfull, remove the old .checkpoint snapshot for you (if the name was the same).
So, you can happily interrupt your backup (if it ran for at least 5-10 minutes) and run it again until you managed to save everything. Perfect for slow, initial backups over the internet.
Ali računalnik lahko uporabljam med backupiranjem?
Odgovor:
Na kratko, da.
Borg usues regular filesystem syscalls to walk though your files. If it has not worked through directory X yet and you change something, borg will backup this change once it is in directory X. The other way around, borg will not notice any changes done to directories it already processed.
When working on a file WHILE borg is chunking it… it might cause locking issues (borg will warn and skip it) or in the worst-case (if not
locked), it will store an inconsistent state. It is your responsibility to avoid that.
Generally speaking, for a regular desktop-user this is a very rare case.
Most of the time programms lock their files or save them in a burst. And even if, the next backup the consistent version will be saved.
So for regular desktop usage you will not have any issue in 99% of the cases if you work while backup up - tho simply fix that by doing your backup while you make a pause or at night.
Problem so pa datoteke, ki se med varnostnim arhiviranjem spreminjajo. Recimo virtualni diski (VDI).
For stuff like (active) virtual machine images (borg loves those, deduplication is *great* on these) of course the chance is a little
higher as they are constantly written to. Same with database files and such. For critical data, you should make sure to have a consistant sate.
Dumps (DB-Dumps, VM-Snapshots) or on filesystem-level (btfs or zfs snapshot) or on block level (lvm snapshot) for borg to work on.
(That being said, I have an active VM on my homeserver which I backup 'the wrong way' without snapshots or pausing it. Did one test and had one actual restore - both times it worked without issues. Tho the data is not exactly super-critical, that's why I do it. Sitll not
recommended. :P)
Komentarji
Obstaja še nek "GUI" - https://github.com/borgbackup/borgweb, vendar zelo osnoven in nedodelan.
Ko so arhivi narejeni, je posamezen arhiv mogoče priklopiti kot FUSE datotečni sistem.
Še par zanimivih ukazov:
Bom nadalje preskušal, ko končam prvi arhiv. Predvsem bi si rad postavil sistem, da mi samodejno arhivira npr. vsak dan in vzdržuje zadnjih npr. 10 kopij.
Sicer pa - ima kdo izkušnje s to aplikacijo?
Izpisuje:
Na koncu zahteva geslo.
Poleg tega so mi encrypted backupi zelo pomembni.
Rešitev:
1. Na oddaljeni lokaciji:
2. Lokalno:
Zdaj preverja, in če bo OK nadaljujemo. In ja, to je security feature.
od ostalih je zanimiv še Duplicati, potem CLI program rdiff-backup in pa obstaja še en ki vleče podatke iz klientov na strežnik: backupPC.
v glavnem Areca ima več variant za backup in pošiljaš na strežnik preko SSH. lahko so kriptirane datoteke, v eni veliki ali posamično...
v službi sem si jo tudi namestil in sicer tako, da mi dela backup na USB ključek. za povrnitev je tudi enostavno. lahko jo delaš preko GUI vmesnika ali pa kar "ročno" preko upravljalnika datotek. npr poiščeš datoteko, ki bi jo rad povrnil in si jo kopiraš na disk.
rdiff je sicer v redu, ampak ne zna kriptirat, pa tudi razvoj je malo zastal.
Za BackupP sem že slišal in ga pohvalijo, ne vem pa kako je težak za vzdrževanje?
Duplicati pa izgleda precej moderno. Ima kdo izkušnje z njim?
Server side:
https://borg.bauerj.eu/borg-1.0.10-armv6
sudo cp borg-1.0.10-armv6 /usr/local/bin/borg
sudo chown root:root /usr/local/bin/borg
sudo chmod 755 /usr/local/bin/borg
borg -V
borg 1.0.10
Client side:
sudo add-apt-repository ppa:costamagnagianfranco/borgbackup
sudo apt-get update
sudo apt-get install borgbackup borgbackup-doc
borg -V
borg 1.0.10
borg upgrade --tam ssh://pi@backup.rpi/media/backup/matej
Mount backupa na določen datum:
sudo mkdir /media/matej/borgbackup
sudo chown matej.matej /media/matej/borgbackup
borg mount ssh://pi@backup.rpi/media/backup/matej::matej_17-07-2017 /media/matej/borgbackup/
borg umount /media/matej/borgbackup/
Po mojih izkušnjah dela zelo zanesljivo (tudi pri izpadih), je pa vseeno program malo čudno zasnovan, ker verzije med seboj niso čisto kompatibilne. Tako da če imate namen sinhronizirati na dveh ali več računalnikih, ki nimajo iste verzije unisona v repozitoriju, se boste precej namatrali s kompilanjem, drugače pa dela b.p.
Skratka
sudo apt-get install unison
Na BorgBackup odjemalcu: