Conversation
Notices
-
good morning guten Morgen !tzag !fediverse! going to try today to set up rsync (or somthing else?) on my laptop Dennis so I can make backups even when not at home...
-
@mk Good morning! I'm currently using obnam to a network server to achieve that. Since I don't have control over that server (although I do trust the non-profit owners), I'm using encryption.
-
@sazius great - looking that up, I'm also finding Attic, Duplicity (uses encrypted tar) and bup (uses gitpackfile format); have you heard of any of those or tried them? Encryption (for now) is only relevant for transport (SSH or so) across Internet, not for storage, since it goes to my #NAS
-
@mk I used duplicity for a long time, and it's good too. I like obnam, though in that it stores only changed chunks of files. So if your huge binary file had one bit changed in it, it doesn't need to backup the whole file again, only the affected chunk.
-
@sazius It seems Attic also has a 'deduplication' algo that sends only changed chunks of files; this is important since I will have to use it over slow Internet connections. Attic supports encryption, as an option. But another factor is I cannot just install 'anything' on the #NAS - it's embedded Linux, and comes with (installable) rsync; there's a user-suported package for rsnapshot (but the archive is called "unstable"); Python and Perl are available, too, so rsync + scripting in one of those should work.
-
@mk if I'm not mistaken obnam doesn't require anything installed on the server. Just ssh.
-
@sazius good - Attic doesn't either, as long as you mount the remote repository via sshfs. As I understand it now Obnam or Attic would be faster than rsnapshot, because they 'deduplicate' on a block/chunk basis while rsnapshot works at file level - that's important for the slow connection. (I have free mobile internet via a dongle which I finally got working this week, but it's slow - security updates for Thunderbird and Firefox were painful, but worked.)
-
@mk hmm, reading: "If the file on the target is different than the one on the source then only the parts of it that are different are transferred." [1] - that suggests rsync (and thus rsnapshot) uses blocks/chunks for deduplication, too. [1] http://www.sanitarium.net/golug/rsync_backups_2010.html @sazius
-
@mk I think you may be right. If you don't need encrypted storage rsync might be the simplest solution. However keep in mind that rsync is about syncronising, not backing up. You can't retrieve the old version of a file for example (unless you keep track of that some other way).
-
@sazius yeah - but I had been thinking about using rsnapshot (which is about backing up) since it's basically a set of scripts wrapped around rsync that takes care of the creation of the 'generational' directories (if I understand correctly). I like the idea of being able to go back to generations of a file (within a set limit, of course). attic also looks attractive because it seems to be very simple to set up.
-
@sazius major problem: I still need to tweak the server itself because I don't like to have to login to the server with the root (or admin) account. And I haven't found where exactly that restriction is defined. :( I'd be happier if I could block direct root access and rename the "admin" account. Ging to mull it over a bit and then (very carefully!) do some experiments *sigh* #NAS
-
@mk ah, I see. Obnam's biggest minus is that it's backups are a bit opaque, you need to use obnam's own tools to look at your files or restore them. But there's really no way around that if you want to store only changed chunks. I suppose with rsnapshot you would get real directories? In that case it will probably store multiple copies of big changed files?
-
@mk A typical scenario is you have 100 MB's of music files, and you change the way you spell the artist's name, and it needs to store the entire files again as "new" files.
-
@sazius real directories is what I understand rsnapshot does. Maybe attic, too. Locally (on the #NAS) I'd really like to have 'real' files, or at least they should be accessible as such. But from my reading I also understand that rsync is much faster than obnam, and obnam actually isn't very good at dedup either. No idea how that compares to attic, though. https://attic-backup.org/ http://blog.karssen.org/2013/01/04/comparing-rsnapshot-and-obnam-for-scheduled-large-backups/
-
@sazius more close to what I need: I have a photo file, and adapt the embedded meta data - in some cases that does not change the internal location of image data, but in other cases (as in changing rather than adding meta data) it would - and the file would have to be transferred completely, unless somehow 'chunks' (of the actual image data) can still be recognized even if they have "shifted". My most urgent need is for office files though.
-
@mk For transfer rsync might be better, but for storage it cannot be as good as obnam, since it uses real files.
-
@sazius if "good" means "fast" I suspect rebuilding a 'real' file from chunks with a new chunk on the server would be much faster than transferring the whole file over a slow connection. Or do you mean another "good"? :)
-
@mk Well, if you have real directory structure and a big file changes slightly you're going to have it twice, i.e. it takes 2x the space. So "good" in terms of taking less space :-)
-
@sazius with a little less than 8Tb (in rAID 10) I'm not worried about space for now ;-)
-
@mk if you change the genre setup of your 1TB of music you might be ;-)
-
@sazius hehe, but my photos take up far more space than my music (most of which isn't ripped yet) ;-)
-
@mk well, you can imagine a similar scenario where you change the metadata of many photos :-)
-
@sazius yes. But I store originals and edited files separately anyway
-
@mk @sazius meh. editing the sshd_config on the #NAS works (the changes stick, even after modifying the port via the web UI) - but the 'AllowUsers' line I added is *not* accepted. So I cannot login as another user, and thus *must* login as root even though I want to forbid that. :( So, my only 'extra' security for SSH is the non-standard port. #nothappy
-
@mk Can't you have a real Linux server as a "proxy", i.e. you'd connect to that and it has the NAS mounted e.g. over NFS or Samba.
-
@sazius oh, that sounds like a good idea! I guess I could (a Raspberry pi?) - but not quickly. Guess I'll make backups on a partable drive for now... but I'll definitely come back to that idea.
-
@sazius and I just thought of another possible option: OwnCloud - I have it installed on the #NAS but not set it up yet (so: also not quickly - but it might at least work for 'remote' backup