>>> On Nov 29, 2007 11:21 PM, Vitezslav Novy wrote:
>>>> Hello,
>>>> my configuration is
>>>> kernel GENERIC
>>>> em0: flags=8843 metric 0 mtu 1500
>>>> options=18b
>>>> ether 00:19:d1:0f:1c:18
>>>> inet netmask 0xffffff00 broadcast
>>>> media: Ethernet autoselect (100baseTX )
>>>> status: active
>>>> and standard "open" ipfw firewall and
>>>> natd -u -s -m -d -dynamic -n em0
>>>> I experience very slow TCP upload from this host - cca 50kbps.
>>>> I have some debug prints in kernel (mostly in ip_output and ipfw log)
>>>> and I see:
>>>> 1/ outgoing packet appears in ip_output with ip_len 2924 and
>>>> m->pkthdr.csum_flags=1
>>>> 2/ is diverted by firewall
>>>> 3/ Packet appears immediately again in ip_output with ip_len 2924 and
>>>> m->pkthdr.csum_flags=1
>>>> 4/ Packet is accepted by firewall and dropped by ip_output with error 40
>>>> 5/ After cca 0.4s (tcp retransmit timeout?) new packet appears in
>>>> ip_output with ip_len 1488 and m->pkthdr.csum_flags=1
>>>> 6/ is successfully diverted and accepted by ipfw and sent to wire.
>>>> 7/ after tcp ack is received new packet appears in ip_output with ip_len
>>>> 2924 and everything repeats
>>>> Packets are not changed by natd, beacause have src address of em0.
>>>> nat
>>>> Upload has normal speed (512kbps) if
>>>> I unset TSO on interface OR set net.inet.tcp.tso=0 OR (strange thing)
>>>> delete ipfw divert rule
>>>> If necessary I will collect and send more info.
>>> TSO is silly at 100Mb, turn it off

After more debugging everything looks clear.
Problem is TSO+divert related.

TCP layer sends large packet with CSUM_TSO set, packet is diverted and
ip_output returns 0 to TCP layer.

When packet is reinjected into ip_output CSUM_TSO flag is lost.
Packet is dropped by ip_output with error EMSGSIZE, but this error is
propagated to natd, which cannot do anything with it.

After retransmit timeout, TCP layer send packet again and because it is
retransmit, TSO is not used and packet is successfully sent
Because TCP layer has no feedback about problem with TSO, next packet is
sent with TSO flag again.

I'm not sure if it is possible to protect CSUM_TSO flag during divert

I tested simple patch which makes ipfw to refuse divert packet with
CSUM_TSO flag a returns EMSGSIZE immediately. It works well for me.
Maybe it can be direction where solution can be found, but
it can break use of divert for purposes other than natd.


freebsd-stable@freebsd.org mailing list
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"