rsync backup - Security

This is a discussion on rsync backup - Security ; I am trying to figure out the best way to set up rsync over ssh. Background: I provide backups to commercial clients, using rsync over vpn. Unfortunately, one customer sits behind a router that only allows me to set up ...

+ Reply to Thread
Results 1 to 19 of 19

Thread: rsync backup

  1. rsync backup

    I am trying to figure out the best way to set up rsync over ssh.

    Background:

    I provide backups to commercial clients, using rsync over vpn.
    Unfortunately, one customer sits behind a router that only allows me to
    set up a TCP (not UDP or anything else) tunnel, which collapses under
    load...

    So I'm trying to figure out how to get rsync over ssh with near-root
    permissions.

    I am backing up user home directories which typically have 700
    permissions, so group membership for rsync isn't going to work.

    I also have to back up data directories, which are typically 775 or some
    such permissions. That's much less of a problem.

    I don't like the idea of having password-less root access.

    So, in a nutshell:

    I want to connect from my server to their server with rsync, in such a
    way that I can read their user and data dirs, but at the same time limit
    root ssh access only to me.

    Any suggestions?

    Thanks,

    --Yan

  2. Re: rsync backup

    CptDondo coughed up some electrons that declared:

    > I am trying to figure out the best way to set up rsync over ssh.
    >
    > Background:
    >
    > I provide backups to commercial clients, using rsync over vpn.
    > Unfortunately, one customer sits behind a router that only allows me to
    > set up a TCP (not UDP or anything else) tunnel, which collapses under
    > load...
    >
    > So I'm trying to figure out how to get rsync over ssh with near-root
    > permissions.
    >
    > I am backing up user home directories which typically have 700
    > permissions, so group membership for rsync isn't going to work.
    >
    > I also have to back up data directories, which are typically 775 or some
    > such permissions. That's much less of a problem.
    >
    > I don't like the idea of having password-less root access.
    > So, in a nutshell:
    >
    > I want to connect from my server to their server with rsync, in such a
    > way that I can read their user and data dirs, but at the same time limit
    > root ssh access only to me.
    >


    Yes, it's easier than that.

    What I did in similar circumstances was:

    1) Local (on customer server) rsync daemon, listening only on localhost with
    read only access plus rsync root password. That stops anyone with loopback
    access (eg shell on same box) from reading the entire of /

    2) DSA key to allow password-less (well, the DSA key *is* the password, but
    non interactive) to a totally non privileged account. If you can figure it
    out, one with a non-usefull shell would be good too - perhaps a script that
    does a sleep(1000000) or something, then exits?...

    3) Now ssh-port-forward the rsyncd listening port on the remote box to your
    box using the otherwise useless account. Now run a local rsync client
    against the local port with the remote rsync password. Total remote root
    access to filesystem (read only), but encrypted an tunnelled.

    I expect there are variations possible on that.

    HTH

    Tim

  3. Re: rsync backup

    On 2007-07-09, Tim Southerwood wrote:
    > CptDondo coughed up some electrons that declared:
    >
    >> I am trying to figure out the best way to set up rsync over ssh.
    >>
    >> Background:
    >>
    >> I provide backups to commercial clients, using rsync over vpn.
    >> Unfortunately, one customer sits behind a router that only allows me to
    >> set up a TCP (not UDP or anything else) tunnel, which collapses under
    >> load...
    >>
    >> So I'm trying to figure out how to get rsync over ssh with near-root
    >> permissions.
    >>
    >> I am backing up user home directories which typically have 700
    >> permissions, so group membership for rsync isn't going to work.
    >>
    >> I also have to back up data directories, which are typically 775 or some
    >> such permissions. That's much less of a problem.
    >>
    >> I don't like the idea of having password-less root access.
    >> So, in a nutshell:
    >>
    >> I want to connect from my server to their server with rsync, in such a
    >> way that I can read their user and data dirs, but at the same time limit
    >> root ssh access only to me.
    >>

    >
    > Yes, it's easier than that.
    >
    > What I did in similar circumstances was:
    >
    > 1) Local (on customer server) rsync daemon, listening only on localhost with
    > read only access plus rsync root password. That stops anyone with loopback
    > access (eg shell on same box) from reading the entire of /
    >
    > 2) DSA key to allow password-less (well, the DSA key *is* the password, but
    > non interactive) to a totally non privileged account. If you can figure it
    > out, one with a non-usefull shell would be good too - perhaps a script that
    > does a sleep(1000000) or something, then exits?...
    >
    > 3) Now ssh-port-forward the rsyncd listening port on the remote box to your
    > box using the otherwise useless account. Now run a local rsync client
    > against the local port with the remote rsync password. Total remote root
    > access to filesystem (read only), but encrypted an tunnelled.


    This still leaves him tunneling TCP over TCP, though, which he says
    is something he'd like to avoid. Having Rsync use SSH as a sort of
    transport layer instead of tunneled TCP (using the --rsh parameter)
    is a better approach from the networking point of view.

    CptDondo, I'm not sure exactly what you mean regarding password-less
    root access. Are you referring to the issue of having your backup
    program log in as root on the client's server, by SSH, in an
    automated fashion? If that's your concern, you can use public key
    SSH authentication to log into an OpenSSH server without user
    interaction:

    http://www.ibm.com/developerworks/library/l-keyc.html

    Is this what you're getting at? (apologies if I've misunderstood the
    question...)

    Mark

    --
    Mark Shroyer
    http://markshroyer.com/

  4. Re: rsync backup

    Mark Shroyer coughed up some electrons that declared:

    > This still leaves him tunneling TCP over TCP, though, which he says
    > is something he'd like to avoid. Having Rsync use SSH as a sort of
    > transport layer instead of tunneled TCP (using the --rsh parameter)
    > is a better approach from the networking point of view.


    Hi Mark,

    Although it's giving me brain cancer to think about it, I'm not convinced
    that rsync over an ssh tunnel is in the same category as TCP over a PPP VPN
    over SSH.

    On balance though, my solution is, I believe, one of the best from the
    security POV (it avoids all of the ssh-as-root nastiness), which would lead
    me, if it were my problem, to live with the networking issues and try to
    tune them down by adjusting the TCP stack at my end, or if necessary, both
    ends, or get the customer to fix their broken (IMHO) router.

    Cheers

    Tim

  5. Re: rsync backup

    Tim Southerwood wrote:
    > Mark Shroyer coughed up some electrons that declared:
    >
    >> This still leaves him tunneling TCP over TCP, though, which he says
    >> is something he'd like to avoid. Having Rsync use SSH as a sort of
    >> transport layer instead of tunneled TCP (using the --rsh parameter)
    >> is a better approach from the networking point of view.

    >
    > Hi Mark,
    >
    > Although it's giving me brain cancer to think about it, I'm not convinced
    > that rsync over an ssh tunnel is in the same category as TCP over a PPP VPN
    > over SSH.
    >
    > On balance though, my solution is, I believe, one of the best from the
    > security POV (it avoids all of the ssh-as-root nastiness), which would lead
    > me, if it were my problem, to live with the networking issues and try to
    > tune them down by adjusting the TCP stack at my end, or if necessary, both
    > ends, or get the customer to fix their broken (IMHO) router.


    Amen. I *think* I solved the problem by changing to a UDP based VPN
    from their end. (They can connect out using UDP, but the router won't
    forward incoming UDP connections - yes, it's broken - but it came free
    from their ISP.)

    At least preliminary testing shows its working. :-)

    --Yan

  6. Re: rsync backup

    CptDondo wrote:

    > Tim Southerwood wrote:
    >> Mark Shroyer coughed up some electrons that declared:
    >>
    >>> This still leaves him tunneling TCP over TCP, though, which he says
    >>> is something he'd like to avoid. Having Rsync use SSH as a sort of
    >>> transport layer instead of tunneled TCP (using the --rsh parameter)
    >>> is a better approach from the networking point of view.

    >>
    >> Hi Mark,
    >>
    >> Although it's giving me brain cancer to think about it, I'm not convinced
    >> that rsync over an ssh tunnel is in the same category as TCP over a PPP
    >> VPN over SSH.
    >>
    >> On balance though, my solution is, I believe, one of the best from the
    >> security POV (it avoids all of the ssh-as-root nastiness), which would
    >> lead me, if it were my problem, to live with the networking issues and
    >> try to tune them down by adjusting the TCP stack at my end, or if
    >> necessary, both ends, or get the customer to fix their broken (IMHO)
    >> router.

    >
    > Amen. I *think* I solved the problem by changing to a UDP based VPN
    > from their end. (They can connect out using UDP, but the router won't
    > forward incoming UDP connections - yes, it's broken - but it came free
    > from their ISP.)
    >
    > At least preliminary testing shows its working. :-)
    >
    > --Yan


    Hi Yan,

    Just out of interest, do you really need to run a VPN between you and the
    client at all? Wouldn't it be easier and just as secure to run ssh to a
    nominated box and tunnel through there as needed.

    Sure, I don't know what else you need to do - but I thought it was worth
    mentioning...

    Cheers

    Tim

  7. Re: rsync backup

    On 10 Jul, 10:11, Tim Southerwood wrote:

    > Sure, I don't know what else you need to do - but I thought it was worth
    > mentioning...
    >
    > Cheers
    >
    > Tim- Hide quoted text -


    And definitely read Richard Silvarman's book on SSH and the part about
    password free key handling. I also highly recommend using rsnapshot to
    do this kind of rsync job.


  8. Re: rsync backup

    Tim Southerwood wrote:
    > On balance though, my solution is, I believe, one of the best from the
    > security POV (it avoids all of the ssh-as-root nastiness) [...]


    What ssh-as-root problems do you see?

    Ideally, you'll be using certificate based authentication rather than
    passwords, and you'll have "PermitRootLogin without-password" in your
    sshd_config.

    Chris

  9. Re: rsync backup

    Chris Davies wrote:

    > Tim Southerwood wrote:
    >> On balance though, my solution is, I believe, one of the best from the
    >> security POV (it avoids all of the ssh-as-root nastiness) [...]

    >
    > What ssh-as-root problems do you see?
    >
    > Ideally, you'll be using certificate based authentication rather than
    > passwords, and you'll have "PermitRootLogin without-password" in your
    > sshd_config.
    >
    > Chris


    Hi Chris,

    A: The don't-do-anything-as-root-you-don't-have-to principle, especially
    when it makes a root shell potentially available over the Internet.

    It doesn't matter how good the DSA key auth mechanism is, it's one more
    unnecessary hole in the config waiting for an SSH bug (and there have been
    a few, not many, but a few) to let some git in. The less he/she/it can do
    if it gets in, the better. Notwithstanding theft of the key of course as
    another way in.

    Most sites turn off PermitRootLogin totally. At the very least, I might make
    another uid=0 account with a restricted shell for this, but there is no way
    on the face of the planet I would ever allow direct generic root access
    from an untrusted domain (Internet) into any box I was responsible for.

    Especially when there is another way...

    Cheers

    Tim

  10. Re: rsync backup

    Tim Southerwood wrote:
    > Chris Davies wrote:
    >
    >> Tim Southerwood wrote:
    >>> On balance though, my solution is, I believe, one of the best from the
    >>> security POV (it avoids all of the ssh-as-root nastiness) [...]

    >> What ssh-as-root problems do you see?
    >>
    >> Ideally, you'll be using certificate based authentication rather than
    >> passwords, and you'll have "PermitRootLogin without-password" in your
    >> sshd_config.
    >>
    >> Chris

    >
    > Hi Chris,
    >
    > A: The don't-do-anything-as-root-you-don't-have-to principle, especially
    > when it makes a root shell potentially available over the Internet.
    >
    > It doesn't matter how good the DSA key auth mechanism is, it's one more
    > unnecessary hole in the config waiting for an SSH bug (and there have been
    > a few, not many, but a few) to let some git in. The less he/she/it can do
    > if it gets in, the better. Notwithstanding theft of the key of course as
    > another way in.
    >
    > Most sites turn off PermitRootLogin totally. At the very least, I might make
    > another uid=0 account with a restricted shell for this, but there is no way
    > on the face of the planet I would ever allow direct generic root access
    > from an untrusted domain (Internet) into any box I was responsible for.
    >
    > Especially when there is another way...
    >



    There is the "permitRootLogin forced-commands-only" option, which this
    whole discussion led me to discover.

    That seems to be the way to go.

    --Yan

  11. Re: rsync backup

    CptDondo coughed up some electrons that declared:


    >
    > There is the "permitRootLogin forced-commands-only" option, which this
    > whole discussion led me to discover.
    >
    > That seems to be the way to go.
    >
    > --Yan


    Hey, that is neat - new feature I didn't notice before. Ta.

    Tim

  12. Re: rsync backup

    Tim Southerwood wrote:
    > CptDondo coughed up some electrons that declared:
    >
    >
    >> There is the "permitRootLogin forced-commands-only" option, which this
    >> whole discussion led me to discover.
    >>
    >> That seems to be the way to go.
    >>
    >> --Yan

    >
    > Hey, that is neat - new feature I didn't notice before. Ta.


    That's what I love about FOSS - someone must have asked the same
    question I did, and then wrote the code.....

    Heck, I'm still waiting on critical fixes to some proprietary software I
    work with - and it's been a year since I reported the bug.... Heck *I*
    can't fix it and I wish they would....

  13. Re: rsync backup

    On 12 Jul, 20:52, CptDondo wrote:
    > Tim Southerwood wrote:
    > > CptDondo coughed up some electrons that declared:

    >
    > >> There is the "permitRootLogin forced-commands-only" option, which this
    > >> whole discussion led me to discover.

    >


    STOP!

    You only need root permissions on the writing side (assuming its a one-
    way replication). If you initiate the connection from this end too
    then you don't need root ssh access (as long as the read end is
    readable by the ssh user).

    FWIW I suspect you'll find that if your tunnel falls over under load,
    then you'll find ssh does the same (I've successfully run ppp over SSL
    on smallish VPNs (3-100 sites)).

    Seems like you're working awfully hard to solve the problems created
    by the 'free' router - routers aren't that expensive!

    C.


  14. Re: rsync backup

    Chris Davies wrote:
    > What ssh-as-root problems do you see?


    Tim Southerwood wrote:
    > A: The don't-do-anything-as-root-you-don't-have-to principle, especially
    > when it makes a root shell potentially available over the Internet.


    Fair enough.


    > Most sites turn off PermitRootLogin totally.


    Yep. Including me.


    > At the very least, I might make another uid=0 account with a restricted
    > shell for this [...]


    I assume this is so that "root" cannot work but a root equivalent
    account could. I'm not sure how this differs from "PermitRootLogin
    forced-commands-only"

    Cheers,
    Chris

  15. Re: rsync backup

    Chris Davies wrote:

    > Chris Davies wrote:
    > I assume this is so that "root" cannot work but a root equivalent
    > account could. I'm not sure how this differs from "PermitRootLogin
    > forced-commands-only"


    I guess it doesn't - I didn't know about "forced-commands-only" when I wrote
    that, and "forced-commands-only" is much better than my suggestion (total
    hack).

    Cheers

    Tim


  16. Re: rsync backup

    CptDondo writes:

    >I am trying to figure out the best way to set up rsync over ssh.


    >Background:


    >I provide backups to commercial clients, using rsync over vpn.
    >Unfortunately, one customer sits behind a router that only allows me to
    >set up a TCP (not UDP or anything else) tunnel, which collapses under
    >load...


    Then you are sunk.

    >So I'm trying to figure out how to get rsync over ssh with near-root
    >permissions.


    ssh uses tcp.


    >I am backing up user home directories which typically have 700
    >permissions, so group membership for rsync isn't going to work.


    >I also have to back up data directories, which are typically 775 or some
    >such permissions. That's much less of a problem.


    >I don't like the idea of having password-less root access.


    Why? this is the kind of think ssh is for.



    >So, in a nutshell:


    >I want to connect from my server to their server with rsync, in such a
    >way that I can read their user and data dirs, but at the same time limit
    >root ssh access only to me.


    Set up a public/private key for the ssh for your system and put the public
    key on that systems .ssh/authorized_hosts file.


    But if the connection collapses under load, rsync will put a load on it.


    >Any suggestions?


    >Thanks,


    >--Yan


  17. Re: rsync backup

    CptDondo writes:

    >Tim Southerwood wrote:
    >> Mark Shroyer coughed up some electrons that declared:
    >>
    >>> This still leaves him tunneling TCP over TCP, though, which he says
    >>> is something he'd like to avoid. Having Rsync use SSH as a sort of
    >>> transport layer instead of tunneled TCP (using the --rsh parameter)
    >>> is a better approach from the networking point of view.

    >>
    >> Hi Mark,
    >>
    >> Although it's giving me brain cancer to think about it, I'm not convinced
    >> that rsync over an ssh tunnel is in the same category as TCP over a PPP VPN
    >> over SSH.
    >>
    >> On balance though, my solution is, I believe, one of the best from the
    >> security POV (it avoids all of the ssh-as-root nastiness), which would lead
    >> me, if it were my problem, to live with the networking issues and try to
    >> tune them down by adjusting the TCP stack at my end, or if necessary, both
    >> ends, or get the customer to fix their broken (IMHO) router.


    >Amen. I *think* I solved the problem by changing to a UDP based VPN
    >from their end. (They can connect out using UDP, but the router won't
    >forward incoming UDP connections - yes, it's broken - but it came free
    >from their ISP.)


    Buy them a new router and put it on y our backup bill.

    By this time your time you have spent asking here has been worth far more
    than a new router.




    >At least preliminary testing shows its working. :-)


    >--Yan


  18. Re: rsync backup

    Tim Southerwood writes:

    >Chris Davies wrote:


    >> Tim Southerwood wrote:
    >>> On balance though, my solution is, I believe, one of the best from the
    >>> security POV (it avoids all of the ssh-as-root nastiness) [...]

    >>
    >> What ssh-as-root problems do you see?
    >>
    >> Ideally, you'll be using certificate based authentication rather than
    >> passwords, and you'll have "PermitRootLogin without-password" in your
    >> sshd_config.
    >>
    >> Chris


    >Hi Chris,


    >A: The don't-do-anything-as-root-you-don't-have-to principle, especially
    >when it makes a root shell potentially available over the Internet.


    To backup a system you have to be root.

    And how in the world does it make a root shell potentially available over
    the internet?


    >It doesn't matter how good the DSA key auth mechanism is, it's one more
    >unnecessary hole in the config waiting for an SSH bug (and there have been
    >a few, not many, but a few) to let some git in. The less he/she/it can do
    >if it gets in, the better. Notwithstanding theft of the key of course as
    >another way in.


    >Most sites turn off PermitRootLogin totally. At the very least, I might make
    >another uid=0 account with a restricted shell for this, but there is no way
    >on the face of the planet I would ever allow direct generic root access
    >from an untrusted domain (Internet) into any box I was responsible for.


    >Especially when there is another way...


    >Cheers


    `:wq>Tim

  19. Re: rsync backup

    Unruh wrote:

    > Tim Southerwood writes:
    >
    >
    > To backup a system you have to be root.


    On the host containing the local filesystem. That does *not* mean you have
    to permit root logins from untrusted networks.

    > And how in the world does it make a root shell potentially available over
    > the internet?


    The word is "potentially". Configuring sshd to allow root access from
    untrusted networks is generally considered undesirable.

    The principle of security is to enable as little as possible. If root ssh
    access is really necessary, then DSA key authentication combined with a
    pass phrase is pretty good, if you REALLY have to. A restricted shell or
    the equivalent measures previously mentioned help.

    But the whole point of what I wrote is that *none* of that is necessary. I
    describe a method of running rsync where rsyncd holds the root access on
    the backup-candidate host, but ssh access is limited to a non priviledged
    account purely for tunnelling purposes.

    You *should not* allow any type of root ssh access to hosts from untrusted
    networks unless it's really necessary, because you leave a potential hole.

    That hole will bite you in the arse when a) you pass-phraseless DSA key gets
    nicked; b) sshd is discovered to have a vulnerability and someone takes
    advantage [this is rare but it *has* happened].

    Please read what I wrote again

    Tim

+ Reply to Thread