At CEC2006, Brendan and Bryan showed how we can use a neat feature of DTrace in conjunction with the DTrace tcp provider - we can represent byte sizes sent and received using a quantized aggregation, which visually represents the number of values that fall within ranges n to 2n, 2n to 4n, etc. This is particularly useful for TCP, since for peak throughput we would expect a bimodal distribution with one spike at 0 (for control segments with no payload) and another in the interval containing the maximum segment size (MSS) for the TCP connection. Viewing send and receive quantized aggregations (aggregated by remote host IP address) in conjunction with the mean MSS for connections to the remote host, we do indeed see this pattern on the receive side for 199.222.69.17 in the case below, but obviously the results will vary depending on the TCP connection load characteristics. In this case, the load consisted of a HTTP request/response, the former of which will often fall below the MSS in size (we see 1 TCP send in the 512-1024 range), while the receive segments are nearly all (32) in the MSS range (1024-2047). For localhost (127.0.0.1, :::1), we see a number of control segments of 0 size only.

# dtrace -s tcp_payloadsize.d ^C MSS (bytes) 199.222.69.17 1372 MSS (bytes) 127.0.0.1 4116 MSS (bytes) ::1 4126 Sent segment size (bytes) 127.0.0.1 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 18 1 | 0 Sent segment size (bytes) ::1 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 18 1 | 0 Sent segment size (bytes) 199.222.69.17 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 21 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 0 64 | 0 128 | 0 256 | 0 512 |@@ 1 1024 | 0 Received segment size (bytes) 127.0.0.1 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 18 1 | 0 Received segment size (bytes) ::1 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 18 1 | 0 Received segment size (bytes) 199.222.69.17 value ------------- Distribution ------------- count -1 | 0 0 |@@ 2 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 0 64 | 0 128 | 0 256 |@ 1 512 | 0 1024 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 32 2048 | 0 Here is the script that gathered these results. The MSS is averaged as it can change over time - more on that later.

#!/usr/sbin/dtrace -s #pragma D option quiet tcp:::send { @transmit["Sent segment size (bytes)", args[2]->ip_daddr] = quantize(args[2]->ip_plength - args[4]->tcp_offset); @mss["MSS (bytes)", args[2]->ip_daddr] = avg(args[3]->tcps_mss); } tcp:::receive { @receive["Received segment size (bytes)", args[2]->ip_saddr] = quantize(args[2]->ip_plength - args[4]->tcp_offset); @mss["MSS (bytes)", args[2]->ip_saddr] = avg(args[3]->tcps_mss); } END { printa("%-20s %-24s %@-20d\n", @mss); }

Read More about [DTrace tcp provider and TCP segment sizes...