[Web4lib] Off-site backups and scripted SCP/SFTP

[Hankinson, Andrew]

>From the "many ways to skin a cat" department....

You could also tar the file locally and then scp/sftp the tar to your
remote server.  Makes managing backups a little easier, as you have
single-file archives (you could even append a date on the filename) and
you don't have to overwrite older versions.

However, I'm still interested to hear why, if using key-based
authentication, a password is even necessary.  I've set up passwordless
(key-authenticated) logins and remote SCP backup scripts before, and
haven't had a problem.

-----Original Message-----
From: web4lib-bounces at webjunction.org
[mailto:web4lib-bounces at webjunction.org] On Behalf Of Joerg Messer
Sent: July 25, 2006 1:32 PM
Cc: Web4Lib
Subject: Re: [Web4lib] Off-site backups and scripted SCP/SFTP


I recommend going with TAR over SSH instead of SCP.  TAR will actually
give you an exact copy where CP/SCP doesn't play nice with symbolic
links.  Just pipe TAR through an SSH pipe to another TAR on the remote
machine.

tar cf - /data |  ssh user at backup.system.org "cd /backup; tar xvpf -"

Works like a charm.

Another popular approach is to use RSYNC.

Francis Kayiwa wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> On Jul 25, 2006, at 7:36 AM, Keith D. Engwall wrote:
> 
>> I'm curious as to whether anyone is using commercial off-site backups

>> and if so, what service they are using.
>>
>> We've got about 5GB (mostly from our LIS), and would prefer to use 
>> SFTP or SCP with keys (because of scripts).
>>
>> Also,
>> I've found some information online about setting up SSH Keys for 
>> scripts, but it seems like either
>>
>> a) you have to not password protect the key
>>
>> or
>>
>> b) you have to manually enter the password after a reboot of the 
>> computer (using ssh-agent or similar solution)
>>
>> I'm trying to find a third, in between method... where the password 
>> is stored somewhere obscure.  Obviously this is still a security 
>> risk, but it's at least less of one than using cleartext ftp with the

>> username and password in the script.
>>
>> Any ideas?
> 
> Why not use SSH instead of SFTP/SCP?
> 
> SSH will pass STDIN through the encrypted session to STDOUT on the 
> remote end. This takes care of it no? Also, forced commands permit 
> immediate command execution method through normal shell notation
> 
> To specify a forced command to a system using OpenSSH, one prepends 
> 'command="some command" to the beginning of the line containing the 
> public key. Other SSH servers may use different syntax. Consult your 
> documentation on the correct format, or start using OpenSSH :-)
> 
> I would probably use dd for something like this. Mostly to protect the

> user from overwriting the public keys.
> 
> command="/usr/bin/dd of=/path/to/file"
> 
> This assuming you are using "put" (to borrow from FTP parlance)
> 
> otherwise
> 
> command="dd if=/file/to/send" for "get."
> 
> All the above would be the modification at the "server" end.
> 
> On the client end
> 
> "cat file | ssh remotehost"
> 
> or on Windows using putty
> 
> "type file | plink remotehost"
> 
> Depending on if the outside datacenter accepts SSH connections you may

> need a "Man in the Middle" host but we would be getting way ahead of 
> ourselves. :-)
> 
> regards,
> ./fxk
> 
> ===============
> Francis Kayiwa
> Library Systems Team
> 4-180, MC 234
> T: +1.312.996.2716
> W: http://www.uic.edu/~kayiwa
> Key: http://tigger.uic.edu/~kayiwa/kayiwa.gpg
> 
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (Darwin)
> 
> iD8DBQFExjfkN+YGKSXdLhkRAjLZAJ0ePy1g2fXzMONXPyH9yG94z8iUiwCeKutj
> UYUxhnRlo7jCXHrWDZhAvpg=
> =NoLv
> -----END PGP SIGNATURE-----
> _______________________________________________
> Web4lib mailing list
> Web4lib at webjunction.org
> http://lists.webjunction.org/web4lib/

--
Joerg Messer
Programmer/Analyst
University of British Columbia Library
2206 East Mall, Vancouver, BC, Canada, V6T 1Z3 t. 1.604.822.5091 f.
1.604.822.3201 e. joerg.messer at ubc.ca
_______________________________________________
Web4lib mailing list
Web4lib at webjunction.org
http://lists.webjunction.org/web4lib/