Linux multihomed routing (2 ISP, 1 internal network) problem - Networking

This is a discussion on Linux multihomed routing (2 ISP, 1 internal network) problem - Networking ; Hi all !! I'm having a little trouble with a multihomed setup I am running here. I'll try to give as much information as possible. I am using the latest Debian release with kernel 2.6.18-4-686. I am doing the routing ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Linux multihomed routing (2 ISP, 1 internal network) problem

  1. Linux multihomed routing (2 ISP, 1 internal network) problem

    Hi all !!

    I'm having a little trouble with a multihomed setup I am running here.
    I'll try to give as much information as possible.

    I am using the latest Debian release with kernel 2.6.18-4-686.
    I am doing the routing using iproute2. the rules are setup with
    iptables.
    The system has 3 NICs: eth0 has the cable ISP (called "TELENET"), eth1
    has the ADSL ISP (called "SKYNET") (with external modem) and eth2 is
    my internal network (called "INTERN"). The ADSL line has a fixed IP,
    and is used to connect to one of our servers from remote locations
    (the modem is setup to forward everything that hits it to my IP on
    eth1)
    The cable ISP is the preferred ISP (ie: all undefined outgoing traffic
    goes thru the cable provider).
    All clients on the lan have full access, no restrictions, to do
    whatever they want online.
    I am using ip route rules and iptables with --set-mark to tell certain
    traffic which routing table to use.

    When I finished configuring Saturday evening, everything was working
    like a charm.
    When I tested again today, suddenly it didn't work as good anymore ...
    Something has changed overnight but I have no clue what it is.
    After some testing, I figured out 2 things: my dns traffic suddenly
    wants to go thru the DSL line and my clients can not directly connect
    to the internet anymore. After setting a clients dns servers to the
    DSL ISP's dns servers, they could resolve again, but they can still
    not surf anymore. The Linux machine itself can still perfectly do
    everything.

    I hope somebody can help me out here, because I am at a loss; I've
    once been able to set this up about 6 years ago, but that knowledge
    has faded a bit (and isn't apparantly very useful anymore anyways). I
    am also not a linux noob ;-)

    Below you can find all technical stuff.

    Thanks to all of you in advance for your help !!

    Cheers,
    Andy


    ------------ MAIN ROUTING TABLE -----------
    # ip route show table main
    192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.254
    192.168.254.0/24 dev eth1 proto kernel scope link src 192.168.254.2
    81.82.0.0/19 dev eth0 proto kernel scope link src 81.82.x.x
    default via 81.82.0.1 dev eth0

    ------------ EXTRA ROUTING TABLE-----------
    # ip route show table 4
    192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.254
    192.168.254.0/24 dev eth1 proto kernel scope link src 192.168.254.2
    81.82.0.0/19 dev eth0 proto kernel scope link src 81.82.x.x
    default via 192.168.254.1 dev eth1

    ----------- ROUTING RULES -----------
    # ip rule show
    0: from all lookup 255
    32764: from 192.168.254.1 lookup 4
    32765: from all fwmark 0x4 lookup 4
    32766: from all lookup main
    32767: from all lookup default

    ---------- FIREWALL (rules) SCRIPT (partial) ----------

    IPTABLES=/sbin/iptables
    TELENET="eth0"
    SKYNET="eth1"
    INTERN="eth2"
    INTNET="192.168.0.0/24"
    $IPTABLES -F
    $IPTABLES -F -t nat
    $IPTABLES -F -t mangle
    $IPTABLES -A INPUT -i lo -s 127.0.0.1/8 -d 0.0.0.0/0 -j ACCEPT
    $IPTABLES -A OUTPUT -o lo -s 127.0.0.1/8 -d 0.0.0.0/0 -j ACCEPT
    $IPTABLES -A INPUT -i $TELENET -s 0.0.0.0/0 -d $TELENETIP -m state --
    state ESTABLISHED,RELATED -j ACCEPT
    $IPTABLES -A INPUT -i $SKYNET -s 0.0.0.0/0 -d $SKYNETIP -m state --
    state ESTABLISHED,RELATED -j ACCEPT
    $IPTABLES -A OUTPUT -o $TELENET -s $TELENETIP -d 0.0.0.0/0 -j ACCEPT
    $IPTABLES -A OUTPUT -o $SKYNET -s $SKYNETIP -d 0.0.0.0/0 -j ACCEPT
    $IPTABLES -A INPUT -i $INTERN -s $INTNET -d 0.0.0.0/0 -j ACCEPT
    $IPTABLES -A OUTPUT -o $INTERN -s $INTERNIP -d $INTNET -j ACCEPT
    $IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
    443 -j MARK --set-mark 0x4
    $IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
    444 -j MARK --set-mark 0x4
    $IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
    1723 -j MARK --set-mark 0x4
    $IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
    4125 -j MARK --set-mark 0x4
    $IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 443 -
    m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
    $SERVER1IP:443
    $IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 444 -
    m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
    $SERVER1IP:444
    $IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 1723
    -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
    $SERVER1IP:1723
    $IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 4125
    -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
    $SERVER1IP:4125
    $IPTABLES -t nat -A POSTROUTING -o $TELENET -j SNAT --to-source
    $TELENETIP
    $IPTABLES -t nat -A POSTROUTING -o $SKYNET -j SNAT --to-source
    $SKYNETIP
    $IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
    1024:65535 --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
    1024:65535 --dport 444 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
    1024:65535 --dport 1723 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
    1024:65535 --dport 4125 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPTABLES -A FORWARD -d $INTNET -j ACCEPT
    $IPTABLES -A FORWARD -s $INTNET -j ACCEPT
    $IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 443 -
    m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
    $IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 444 -
    m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
    $IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 1723 -
    m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
    $IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 4125 -
    m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
    $IPTABLES -P FORWARD DROP
    $IPTABLES -P INPUT DROP
    $IPTABLES -P OUTPUT DROP


  2. Re: Linux multihomed routing (2 ISP, 1 internal network) problem

    andy_occ@hotmail.com wrote:
    > Hi all !!


    > I'm having a little trouble with a multihomed setup I am running here.
    > I'll try to give as much information as possible.


    > I am using the latest Debian release with kernel 2.6.18-4-686.
    > I am doing the routing using iproute2. the rules are setup with
    > iptables.
    > The system has 3 NICs: eth0 has the cable ISP (called "TELENET"), eth1
    > has the ADSL ISP (called "SKYNET") (with external modem) and eth2 is
    > my internal network (called "INTERN"). The ADSL line has a fixed IP,
    > and is used to connect to one of our servers from remote locations
    > (the modem is setup to forward everything that hits it to my IP on
    > eth1)
    > The cable ISP is the preferred ISP (ie: all undefined outgoing traffic
    > goes thru the cable provider).
    > All clients on the lan have full access, no restrictions, to do
    > whatever they want online.
    > I am using ip route rules and iptables with --set-mark to tell certain
    > traffic which routing table to use.


    > When I finished configuring Saturday evening, everything was working
    > like a charm. When I tested again today, suddenly it didn't work
    > as good anymore ...


    > Something has changed overnight but I have no clue what it is.
    > After some testing, I figured out 2 things: my dns traffic suddenly
    > wants to go thru the DSL line and my clients can not directly connect
    > to the internet anymore. After setting a clients dns servers to the
    > DSL ISP's dns servers, they could resolve again, but they can still
    > not surf anymore. The Linux machine itself can still perfectly do
    > everything.


    All posted information considered, one guess would be that IP forwarding
    is not working. Does cat /proc/sys/net/ipv4/ip_forward yield 1 or 0?
    Another guess would be the LAN interface isn't working, but that would
    also prevent LAN<->Linux_box traffic.

    Although I'm no expert, the two iptables network forwarding rules for
    LAN<->TELENET looked reasonable. However, the unfettered inbound Internet
    traffic would worry me.



    --
    Clifford Kite
    /* Speak softly and carry a +6 two-handed sword. */

  3. Re: Linux multihomed routing (2 ISP, 1 internal network) problem

    Thank you for your response Clifford !

    Forwarding is definately enabled, as the $SERVERIP server is still
    able to do dns queries thru the DSL line (although, and that's the
    problem, I see no reason why it suddenly wanted to go thru the DSL
    connection, none of my rules are set up to do that ... like i said,
    my standard random tcp/ip traffic should be routed thru the $TELENET
    connection

    also, all interfaces are definately working : when i was testing the
    setup before, i downloaded things thru the DSL and Cable straight from
    the lan...

    what is this unferttered inbound Internet traffic
    you are referring to ?

    another thing i suddenly remember : when the setup stopped working,
    and i logged on, the screen had all these "martian source" things on
    them, which I am sure are part of the problem.
    the thing is, i fixed those on saturday, and on monday it was broken
    again

    I'll keep on looking, and if anybody else has an idea, please do tell
    me !

    Andy

    (ps: i replied to my own post instead of to Cliffords post because
    Google Groups isn't showing his message yet, and my news client
    program won't let me post messages


  4. Re: Linux multihomed routing (2 ISP, 1 internal network) problem

    andy_occ@hotmail.com wrote:

    > what is this unferttered inbound Internet traffic
    > you are referring to ?


    My bad. I was thinking (or not...) that the rules

    $IPTABLES -A FORWARD -d $INTNET -j ACCEPT
    $IPTABLES -A FORWARD -s $INTNET -j ACCEPT

    would allow virtually unrestricted access to $INTNET from the Internet,
    which is not true with SNATed RFC 1918 networks. The only non-local
    network access possible should be directly from an ISP to $INTNET or to
    the DNATed server and server-ports from the Internet via the $SKYNETIP
    address.

    --
    Clifford Kite
    /* I hear and I forget. I see and I remember. I do and I understand.
    --Confucius, 551-479 BC */

+ Reply to Thread