Problem seeing correct originating IP in Ubuntu linux - Networking

This is a discussion on Problem seeing correct originating IP in Ubuntu linux - Networking ; Hi all, I have my own linux Ubuntu 6.10 server running a variety of services. Since I last upgraded my kernel to version 2.6.17-12-generic, I have the strangest problem. A little background on my very simple network configuration: | [10.0.0.1 ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Problem seeing correct originating IP in Ubuntu linux

  1. Problem seeing correct originating IP in Ubuntu linux

    Hi all,

    I have my own linux Ubuntu 6.10 server running a variety of services.
    Since I last upgraded my kernel to version 2.6.17-12-generic, I have the
    strangest problem. A little background on my very simple network
    configuration:


    |
    [10.0.0.1 - WRT54G running DD-WRT with DHCP and dnsmasq] -
    |
    [10.0.0.3 - server]
    |
    [10.0.0.5 - xbox]
    |
    [10.0.0.25 - laptop]

    Now, I often log into my server with SSH from the laptop, and normally I
    would see the login coming from 10.0.0.25 like this (I think this is
    what it looked like, at least I am sure I could see the source IP in
    some form, but I cannot reproduce it anymore):

    user@server:~$ netstat -tn
    Active Internet connections (w/o servers)
    Proto Recv-Q Send-Q Local Address Foreign Address
    State
    ESTABLISHED
    tcp6 0 0 ::ffff:10.0.0.3:22 ::ffff:10.0.0.25:2276
    user@server:~$

    ...but after this kernel update (I think this is what changed it), I have:

    user@server:~$ netstat -tn
    Active Internet connections (w/o servers)
    Proto Recv-Q Send-Q Local Address Foreign Address
    State
    ESTABLISHED
    tcp6 0 0 ::ffff:10.0.0.3:22 ::ffff:10.0.0.1:2276
    user@server:~$

    So it looks like my SSH connection is coming from the router, while it
    is really coming from my laptop. This strange behavior is also present
    in all log-files showing information about originating IP (syslog for
    instance).

    One funny thing is that if the xbox connects via samba to the server,
    everything looks ok:

    user@server:~$ netstat -tn
    Active Internet connections (w/o servers)
    Proto Recv-Q Send-Q Local Address Foreign Address
    State
    tcp 0 0 10.0.0.3:445 10.0.0.5:1040
    ESTABLISHED

    So to me this looks like a problem displaying ipv6 ip's? Or is it me not
    having understood ipv6 at all?

    Ifconfig of eth0 on the server gives:

    eth0 Link encap:Ethernet HWaddr 00:40:63:E6:5B:8B
    inet addr:10.0.0.3 Bcast:10.0.0.255 Mask:255.255.255.0

    ...so netmask is correct, and it is the same on the windows laptop. So
    traffic really shouldn't be routed. And if I ping the laptop:

    root@server:~# ping 10.0.0.25
    PING 10.0.0.25 (10.0.0.25) 56(84) bytes of data.
    64 bytes from 10.0.0.25: icmp_seq=1 ttl=128 time=2.79 ms

    Last but not least - if I run a "arp -a" everything seems correct:

    root@server:# arp -a
    wrt54g.mydomain.dk (10.0.0.1) at 00:18:39:C0:1A:30 [ether] on eth0
    laptop.mydomain.dk (10.0.0.25) at 00:0E:35E:8F:49 [ether] on eth0
    xbox.mydomain.dk (10.0.0.5) at 00:12:5A:20:C5:C5 [ether] on eth0
    root@server:#

    I am not sure whether this is something to be worried about (I guess it
    isn't), but I would like the correct information in my logfiles and in
    my netstat. Any clues?

    Best regards,
    Henrik

  2. Re: Problem seeing correct originating IP in Ubuntu linux

    Hello,

    Henrik Uhrenfeldt a écrit :
    [...]
    > ..but after this kernel update (I think this is what changed it), I have:
    >
    > user@server:~$ netstat -tn
    > Active Internet connections (w/o servers)
    > Proto Recv-Q Send-Q Local Address Foreign Address State
    > ESTABLISHED
    > tcp6 0 0 ::ffff:10.0.0.3:22 ::ffff:10.0.0.1:2276
    > user@server:~$
    >
    > So it looks like my SSH connection is coming from the router, while it
    > is really coming from my laptop. This strange behavior is also present
    > in all log-files showing information about originating IP (syslog for
    > instance).


    How do you connect to the server ? By name, by address ?
    It looks like your laptop is connecting to the server through the router
    and some source NAT is involved. Could it be that the ssh client uses
    the router's external address and the router has a port forwarding NAT
    rule redirecting port 22 to the server address ? For this to work, the
    router has to replace the original client source address with its own.

    [...]
    > So to me this looks like a problem displaying ipv6 ip's?


    No. The tcp6 and ::ffff: IPv6 prefix are caused by sshd receiving the
    IPv4 connections on an IPv6 socket, so IPv4 addresses are seen as
    IPv4-mapped IPv6 addresses. I didn't like that so I set up my box so
    that sshd listens on separate IPv4 and IPv6 sockets.

  3. Re: Problem seeing correct originating IP in Ubuntu linux

    Hi,

    I inlined some answers below..

    Pascal Hambourg wrote:
    > Hello,
    >
    > Henrik Uhrenfeldt a écrit :
    > [...]
    >> ..but after this kernel update (I think this is what changed it), I have:
    >>
    >> user@server:~$ netstat -tn
    >> Active Internet connections (w/o servers)
    >> Proto Recv-Q Send-Q Local Address Foreign Address State
    >> ESTABLISHED
    >> tcp6 0 0 ::ffff:10.0.0.3:22 ::ffff:10.0.0.1:2276
    >> user@server:~$
    >>
    >> So it looks like my SSH connection is coming from the router, while it
    >> is really coming from my laptop. This strange behavior is also present
    >> in all log-files showing information about originating IP (syslog for
    >> instance).

    >
    > How do you connect to the server ? By name, by address ?
    > It looks like your laptop is connecting to the server through the router
    > and some source NAT is involved. Could it be that the ssh client uses
    > the router's external address and the router has a port forwarding NAT
    > rule redirecting port 22 to the server address ? For this to work, the
    > router has to replace the original client source address with its own.
    >


    I connect directly to IP 10.0.0.3 from the laptop, and since netmask is
    255.255.255.0 on the laptop, I really think it should do a direct
    connection. But yes - it does look like it connects through the router;
    this is what puzzles me.

    > [...]
    >> So to me this looks like a problem displaying ipv6 ip's?

    >
    > No. The tcp6 and ::ffff: IPv6 prefix are caused by sshd receiving the
    > IPv4 connections on an IPv6 socket, so IPv4 addresses are seen as
    > IPv4-mapped IPv6 addresses. I didn't like that so I set up my box so
    > that sshd listens on separate IPv4 and IPv6 sockets.


    Ah, so this is why some connections look like IPv6 and others don't. I
    was beginning to think I was running an IPv6 network :-)
    Where did you change this - in sshd config?

    - Henrik

  4. Re: Problem seeing correct originating IP in Ubuntu linux

    Henrik Uhrenfeldt a écrit :
    >
    > I connect directly to IP 10.0.0.3 from the laptop, and since netmask is
    > 255.255.255.0 on the laptop, I really think it should do a direct
    > connection. But yes - it does look like it connects through the router;
    > this is what puzzles me.


    tcpdump or (t)ethereal/wireshark/tshark on the client and server may
    help tracking the actual packets path.

    >> No. The tcp6 and ::ffff: IPv6 prefix are caused by sshd receiving the
    >> IPv4 connections on an IPv6 socket, so IPv4 addresses are seen as
    >> IPv4-mapped IPv6 addresses. I didn't like that so I set up my box so
    >> that sshd listens on separate IPv4 and IPv6 sockets.

    >
    > Ah, so this is why some connections look like IPv6 and others don't. I
    > was beginning to think I was running an IPv6 network :-)
    > Where did you change this - in sshd config?


    If you have an IPv4-only network, add or uncomment the following
    directive in /etc/ssh/sshd_config, so sshd opens an IPv4 socket :

    ListenAddress 0.0.0.0

    And delete or comment any ListenAddress directive with address "::" (any
    IPv6 address) so sshd does not open an IPv6 socket. Then restart sshd.

    If you have and IPv4+IPv6 network, you must set the kernel sysctl
    /proc/sys/net/ipv6/bindv6only to 1 to prevent the use of IPv4
    communications with IPv6 sockets. Add or uncomment the following two
    directives in sshd_config, so sshd opens IPv4 and IPv6 sockets :

    ListenAddress ::
    ListenAddress 0.0.0.0

    Then restard sshd. Note that setting bindv6only to 1 may have issues
    with other dual stack services, because IPv6 sockets won't accept IPv4
    connections any more.

  5. Re: Problem seeing correct originating IP in Ubuntu linux

    Hi again,

    The plot thickens.

    I just sniffed my server eth0 with wireshark. I see broadcast netbios
    name traffic from my laptop MAC address using IP 10.0.0.25, and then I
    see a direct connection to my server from the same MAC, but with ip
    10.0.0.1 as source... Does my laptop spoof it's IP? And if so - why?

    Come to think of it, my router actually bridges the wireless part (on
    which the laptop is) together with the physical LAN ethernet. But I
    would think that MAC's would still be traceable in the packet header. I
    could understand if it had been NAT translated, but it shouldn't when
    bridged, right?

    - Henrik

    Henrik Uhrenfeldt wrote:
    > Hi,
    >
    > I inlined some answers below..
    >
    > Pascal Hambourg wrote:
    >> Hello,
    >>
    >> Henrik Uhrenfeldt a écrit :
    >> [...]
    >>> ..but after this kernel update (I think this is what changed it), I
    >>> have:
    >>>
    >>> user@server:~$ netstat -tn
    >>> Active Internet connections (w/o servers)
    >>> Proto Recv-Q Send-Q Local Address Foreign Address State
    >>> ESTABLISHED
    >>> tcp6 0 0 ::ffff:10.0.0.3:22 ::ffff:10.0.0.1:2276
    >>> user@server:~$
    >>>
    >>> So it looks like my SSH connection is coming from the router, while
    >>> it is really coming from my laptop. This strange behavior is also
    >>> present in all log-files showing information about originating IP
    >>> (syslog for instance).

    >>
    >> How do you connect to the server ? By name, by address ?
    >> It looks like your laptop is connecting to the server through the
    >> router and some source NAT is involved. Could it be that the ssh
    >> client uses the router's external address and the router has a port
    >> forwarding NAT rule redirecting port 22 to the server address ? For
    >> this to work, the router has to replace the original client source
    >> address with its own.
    >>

    >
    > I connect directly to IP 10.0.0.3 from the laptop, and since netmask is
    > 255.255.255.0 on the laptop, I really think it should do a direct
    > connection. But yes - it does look like it connects through the router;
    > this is what puzzles me.
    >
    >> [...]
    >>> So to me this looks like a problem displaying ipv6 ip's?

    >>
    >> No. The tcp6 and ::ffff: IPv6 prefix are caused by sshd receiving the
    >> IPv4 connections on an IPv6 socket, so IPv4 addresses are seen as
    >> IPv4-mapped IPv6 addresses. I didn't like that so I set up my box so
    >> that sshd listens on separate IPv4 and IPv6 sockets.

    >
    > Ah, so this is why some connections look like IPv6 and others don't. I
    > was beginning to think I was running an IPv6 network :-)
    > Where did you change this - in sshd config?
    >
    > - Henrik


  6. Re: Problem seeing correct originating IP in Ubuntu linux

    Henrik Uhrenfeldt a écrit :
    >
    > The plot thickens.
    >
    > I just sniffed my server eth0 with wireshark. I see broadcast netbios
    > name traffic from my laptop MAC address using IP 10.0.0.25, and then I
    > see a direct connection to my server from the same MAC, but with ip
    > 10.0.0.1 as source... Does my laptop spoof it's IP? And if so - why?


    Rather unlikely. If so, it wouldn't receive the replies and the SSH
    session could not be established.

    > Come to think of it, my router actually bridges the wireless part (on
    > which the laptop is) together with the physical LAN ethernet. But I
    > would think that MAC's would still be traceable in the packet header. I
    > could understand if it had been NAT translated, but it shouldn't when
    > bridged, right?


    Bridged IPv4 packets traverse iptables chains - including NAT - in 2.6
    kernels compiled with bridge-nf support (CONFIG_BRIDGE_NETFILTER=y) if
    /proc/sys/net/bridge/bridge-nf-call-iptables=1 (this is the default).
    However the 'physdev' iptables match has an option
    '--physdev-is-bridged' which matches only bridged packets, not routed
    ones. If /proc/sys/net/bridge/bridge-nf-call-iptables is present on your
    router and set to 1, try to set it to 0.
    However 2.4 kernels do not support bridge-nf.

  7. Re: Problem seeing correct originating IP in Ubuntu linux

    Pascal Hambourg a écrit :
    >
    > However 2.4 kernels do not support bridge-nf.


    Unless patched with the ebtables-brnf patch from
    .

  8. Re: Problem seeing correct originating IP in Ubuntu linux

    Pascal Hambourg wrote:
    > Henrik Uhrenfeldt a écrit :
    >>
    >> I connect directly to IP 10.0.0.3 from the laptop, and since netmask
    >> is 255.255.255.0 on the laptop, I really think it should do a direct
    >> connection. But yes - it does look like it connects through the
    >> router; this is what puzzles me.

    >
    > tcpdump or (t)ethereal/wireshark/tshark on the client and server may
    > help tracking the actual packets path.
    >
    >>> No. The tcp6 and ::ffff: IPv6 prefix are caused by sshd receiving the
    >>> IPv4 connections on an IPv6 socket, so IPv4 addresses are seen as
    >>> IPv4-mapped IPv6 addresses. I didn't like that so I set up my box so
    >>> that sshd listens on separate IPv4 and IPv6 sockets.

    >>
    >> Ah, so this is why some connections look like IPv6 and others don't. I
    >> was beginning to think I was running an IPv6 network :-)
    >> Where did you change this - in sshd config?

    >
    > If you have an IPv4-only network, add or uncomment the following
    > directive in /etc/ssh/sshd_config, so sshd opens an IPv4 socket :
    >
    > ListenAddress 0.0.0.0
    >
    > And delete or comment any ListenAddress directive with address "::" (any
    > IPv6 address) so sshd does not open an IPv6 socket. Then restart sshd.
    >
    > If you have and IPv4+IPv6 network, you must set the kernel sysctl
    > /proc/sys/net/ipv6/bindv6only to 1 to prevent the use of IPv4
    > communications with IPv6 sockets. Add or uncomment the following two
    > directives in sshd_config, so sshd opens IPv4 and IPv6 sockets :
    >
    > ListenAddress ::
    > ListenAddress 0.0.0.0
    >
    > Then restard sshd. Note that setting bindv6only to 1 may have issues
    > with other dual stack services, because IPv6 sockets won't accept IPv4
    > connections any more.


    Pascal,

    Thanks for your help. I started suspected that something fishy was up
    with the ethernet bridge in the router, and I ended up installing a
    newer firmware on it, and with a fresh router firmware and default
    settings, things started to work again.

    I'll try your SSH tip - thanks for the info.

    - Henrik

  9. Re: Problem seeing correct originating IP in Ubuntu linux

    Pascal Hambourg wrote:
    > Henrik Uhrenfeldt a écrit :
    >>
    >> I connect directly to IP 10.0.0.3 from the laptop, and since netmask
    >> is 255.255.255.0 on the laptop, I really think it should do a direct
    >> connection. But yes - it does look like it connects through the
    >> router; this is what puzzles me.

    >
    > tcpdump or (t)ethereal/wireshark/tshark on the client and server may
    > help tracking the actual packets path.
    >
    >>> No. The tcp6 and ::ffff: IPv6 prefix are caused by sshd receiving the
    >>> IPv4 connections on an IPv6 socket, so IPv4 addresses are seen as
    >>> IPv4-mapped IPv6 addresses. I didn't like that so I set up my box so
    >>> that sshd listens on separate IPv4 and IPv6 sockets.

    >>
    >> Ah, so this is why some connections look like IPv6 and others don't. I
    >> was beginning to think I was running an IPv6 network :-)
    >> Where did you change this - in sshd config?

    >
    > If you have an IPv4-only network, add or uncomment the following
    > directive in /etc/ssh/sshd_config, so sshd opens an IPv4 socket :
    >
    > ListenAddress 0.0.0.0
    >
    > And delete or comment any ListenAddress directive with address "::" (any
    > IPv6 address) so sshd does not open an IPv6 socket. Then restart sshd.
    >
    > If you have and IPv4+IPv6 network, you must set the kernel sysctl
    > /proc/sys/net/ipv6/bindv6only to 1 to prevent the use of IPv4
    > communications with IPv6 sockets. Add or uncomment the following two
    > directives in sshd_config, so sshd opens IPv4 and IPv6 sockets :
    >
    > ListenAddress ::
    > ListenAddress 0.0.0.0
    >
    > Then restard sshd. Note that setting bindv6only to 1 may have issues
    > with other dual stack services, because IPv6 sockets won't accept IPv4
    > connections any more.


    Pascal,

    Thanks for your help. I started suspected that something fishy was up
    with the ethernet bridge in the router, and I ended up installing a
    newer firmware on it, and with a fresh router firmware and default
    settings, things started to work again.

    I'll try your SSH tip - thanks for the info.

    - Henrik

+ Reply to Thread