rlim_fd_max and 99% IO Wait - Solaris 8 Sun server - Unix

This is a discussion on rlim_fd_max and 99% IO Wait - Solaris 8 Sun server - Unix ; Hello, Given the following output from vmstat, mpstat and iostat. What would you say the bottleneck is on our Sun E6500 server ?? I noticed that in "top" the "% iowait" is consistently higher 99% while the cpu % idle ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: rlim_fd_max and 99% IO Wait - Solaris 8 Sun server

  1. rlim_fd_max and 99% IO Wait - Solaris 8 Sun server

    Hello,

    Given the following output from vmstat, mpstat and iostat. What would
    you say the bottleneck is on our Sun E6500 server ?? I noticed that in
    "top" the "% iowait" is consistently higher 99% while the cpu % idle
    remains zero. The only change made to the system was the value of
    rlim_fd_max from 8096 to 65536 in the /etc/system. The "ulimit -Hn"
    now shows 65535. The change was one of the Oracle 10G installation
    requirement, together with other Sun patches. Memory doesn't seem to
    be an issue. Judging from everything I've seen, may be the I/O wait is
    the bottleneck. Any idea what's happening at our server and how we may
    trouble-shoot this.


    The server is Sun E6500 / 20 x 400MHz / 26G memory. It's running the
    following database and application:

    - Solaris 8 (kernel 108528-29)
    - Veritas Oracle DB Edition (VxDBED, VxVM, VxFS - 3.5 MP3) with Vxdmp
    - Oracle 8.1.7.4 (2 instances)
    - Oralce 9 release 1 (1 instance)
    - Oracle 10G release 1 (or OEM)
    - Oracle Application Server 1.0.2.2

    server is connected to the following storage :

    - 2 Emulex HBAs connection to SAN fabrics to luns at Hitachi 9585
    storage array
    - 2 Emulex HBAs connection to SAN fabrics to luns at EMC DMX storage
    array


    Thanks,

    Humphrey



    # vmstat 5 5
    procs memory page disk faults
    cpu
    r b w swap free re mf pi po fr de sr s6 s9 s1 s1 in sy cs
    us sy id
    0 11001 0 19304088 16932208 26 166 8 0 0 0 0 0 3 0 3 2258 6235 1625
    5 1 94
    0 607115 0 16708216 14533000 42 247 0 0 0 0 0 0 1 0 1 2458 17527 2577
    10 2 88
    0 607278 0 16708312 14532232 22 185 0 0 0 0 0 0 0 0 1 2452 4797 2504
    10 1 89
    0 607492 0 16708152 14531136 36 136 0 0 0 0 0 0 0 0 0 2461 4922 2380
    10 2 89
    0 607705 0 16712784 14534904 7 0 0 0 0 0 0 0 0 0 0 2430 2597 2238
    5 1 94



    # mpstat 3 3
    CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys
    wt idl
    0 7 0 2326 365 274 56 0 1 13 0 243 3 1
    88 8
    1 7 0 412 98 96 82 0 2 12 0 319 7 1
    83 9
    4 8 0 371 94 93 73 0 2 10 0 278 3 2
    87 8
    5 8 0 445 96 94 85 0 2 12 0 325 7 1
    84 8
    8 8 0 391 97 95 81 0 2 9 0 268 3 2
    87 8
    9 7 0 415 96 94 92 0 2 12 0 306 7 1
    83 9
    12 8 0 382 94 92 83 0 2 9 0 327 3 1
    87 8
    13 8 0 440 98 96 93 1 2 12 0 346 7 1
    83 9
    14 9 0 648 177 175 89 0 2 10 0 277 3 1
    87 9
    15 7 0 438 137 135 86 0 2 12 0 319 6 1
    85 8
    16 8 0 393 96 94 89 0 2 9 0 318 4 1
    87 8
    17 7 0 467 104 102 89 0 2 11 0 308 6 1
    84 8
    18 8 0 397 118 116 78 0 2 9 0 347 3 1
    87 8
    19 7 0 428 96 94 83 0 2 13 0 329 7 1
    84 8
    20 9 0 426 97 95 75 0 2 9 0 313 3 1
    87 8
    21 9 0 437 97 95 79 0 2 12 0 352 7 1
    83 8
    22 9 0 416 97 95 75 0 2 10 0 284 3 1
    87 8
    23 8 0 436 96 94 76 0 2 12 0 319 7 1
    84 8
    24 8 0 378 94 93 72 0 2 10 0 315 3 1
    87 8
    25 8 0 434 96 95 81 0 2 12 0 337 6 1
    84 9
    CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys
    wt idl
    0 0 0 2467 385 289 76 0 5 14 0 101 0 1
    99 0
    1 0 0 131 100 99 57 0 6 5 0 73 0 0
    100 0
    4 2 0 593 101 100 116 0 2 6 0 128 0 0
    100 0
    5 0 0 31 101 99 88 0 6 4 0 62 0 0
    99 0
    8 0 0 75 103 100 116 2 4 5 0 188 3 0
    97 0
    9 0 0 239 101 99 107 1 6 7 0 68 0 0
    100 0
    12 0 0 105 100 98 133 0 3 4 0 55 0 11
    89 0
    13 0 0 224 101 99 106 0 5 5 0 110 1 0
    99 0
    14 0 0 890 186 180 49 5 3 10 0 240 40 0
    60 0
    15 0 0 62 180 176 109 1 8 9 0 195 4 1
    95 0
    16 0 0 169 101 99 72 0 5 4 0 86 0 0
    100 0
    17 0 0 88 101 99 84 1 3 8 0 133 3 0
    96 0
    18 0 0 107 113 111 70 0 4 4 0 80 0 0
    100 0
    19 0 0 299 101 99 105 0 7 5 0 74 0 1
    99 0
    20 0 0 37 100 98 132 0 5 7 0 124 1 1
    99 0
    21 0 0 749 99 98 127 0 5 6 0 77 0 1
    99 0
    22 0 0 328 99 98 110 0 4 5 0 110 0 0
    100 0
    23 0 0 107 101 100 70 0 5 4 0 118 0 0
    100 0
    24 0 0 0 101 100 160 0 4 18 0 267 2 0
    98 0
    25 0 0 551 104 99 57 4 3 6 0 257 53 1
    46 0
    CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys
    wt idl
    0 0 0 2312 388 291 79 0 2 22 0 169 0 1
    99 0
    1 0 0 175 100 99 96 0 4 9 0 74 0 0
    100 0
    4 0 0 351 100 99 132 0 4 5 0 173 0 0
    100 0
    5 0 0 536 105 100 56 7 4 9 0 297 61 1
    38 0
    8 0 0 137 100 99 94 0 5 1 0 53 0 1
    99 0
    9 0 0 449 101 99 172 1 8 10 0 167 0 1
    99 0
    12 0 0 160 95 93 157 0 7 10 0 77 1 0
    99 0
    13 0 0 267 101 99 123 1 6 7 0 66 0 1
    99 0
    14 0 0 925 195 191 156 3 5 10 0 168 25 0
    75 0
    15 0 0 63 187 182 105 3 4 10 0 207 10 0
    90 0
    16 0 0 218 100 98 129 1 8 14 0 151 0 0
    99 0
    17 0 0 139 102 100 124 2 5 12 0 122 3 0
    97 0
    18 0 0 131 139 137 122 1 5 7 0 79 0 0
    100 0
    19 0 0 320 101 98 150 1 7 11 0 136 0 0
    100 0
    20 0 0 59 103 100 91 1 6 7 0 84 2 0
    98 0
    21 0 0 618 103 100 178 1 7 9 0 191 1 1
    98 0
    22 0 0 555 91 90 78 0 5 3 0 123 0 0
    100 0
    23 0 0 228 101 99 96 0 5 6 0 98 0 0
    100 0
    24 0 0 44 101 100 95 0 4 7 0 90 0 0
    100 0
    25 0 0 138 100 99 125 0 4 8 0 92 0 0
    100 0



    # iostat -xn 5
    extended device statistics
    r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
    1.5 1.7 56.5 53.8 0.0 0.0 0.0 11.2 0 2 c0t10d0
    0.1 0.0 1.8 1.7 0.0 0.0 0.0 11.7 0 0 c0t11d0
    1.5 1.7 56.3 54.0 0.0 0.0 0.0 11.1 0 2 c1t2d0
    1.3 6.4 11.0 71.2 0.0 0.0 0.0 0.7 0 1 c2t24d1
    1.8 6.7 14.7 74.1 0.0 0.0 0.0 0.7 0 1 c2t24d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d4
    0.0 3.5 0.4 31.2 0.0 0.0 0.0 0.7 0 0 c2t24d5
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d6
    0.4 0.8 7.3 10.4 0.0 0.0 0.0 3.2 0 0 c2t24d7
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d8
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d11
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d12
    0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.1 0 0 c2t24d13
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d14
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d15
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d16
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d17
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d18
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0 0 c2t24d19
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d1
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d4
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d5
    0.3 1.0 5.7 12.6 0.0 0.0 0.0 3.4 0 0 c3t25d6
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d7
    0.0 0.0 0.1 0.1 0.0 0.0 0.0 1.5 0 0 c3t25d8
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d11
    0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.7 0 0 c3t25d12
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d13
    0.0 0.0 0.2 0.4 0.0 0.0 0.0 2.6 0 0 c3t25d14
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d15
    0.0 0.0 0.1 0.1 0.0 0.0 0.0 1.2 0 0 c3t25d16
    1.8 4.2 21.4 41.7 0.0 0.0 0.5 2.2 0 1 c3t25d17
    0.0 0.0 0.2 0.4 0.0 0.0 0.0 3.0 0 0 c3t25d18
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d19
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d20
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d22
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.2 0 0 c2t24d23
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d24
    0.0 0.3 1.4 2.2 0.0 0.0 0.0 1.1 0 0 c2t24d25
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d26
    0.0 0.0 0.3 0.1 0.0 0.0 0.0 2.6 0 0 c2t24d27
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.3 0 0 c3t25d20
    0.0 0.0 0.2 0.4 0.0 0.0 0.0 2.4 0 0 c3t25d22
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d23
    2.9 16.6 66.3 134.6 0.0 0.0 0.0 1.7 0 3 c3t25d24
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d25
    0.0 0.3 0.4 5.3 0.0 0.0 0.0 2.2 0 0 c3t25d26
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d27
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 st5
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
    antibes:vold(pid706)
    extended device statistics
    r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t10d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t11d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t2d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d1
    5.6 14.4 44.7 164.5 0.0 0.0 0.0 0.6 0 1 c2t24d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d4
    0.0 2.8 0.0 22.4 0.0 0.0 0.0 0.6 0 0 c2t24d5
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d6
    0.0 1.0 0.0 11.2 0.0 0.0 0.0 0.8 0 0 c2t24d7
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d8
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d11
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d12
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d13
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d14
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d15
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d16
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d17
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d18
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d19
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d1
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d4
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d5
    0.0 1.4 0.0 17.6 0.0 0.0 0.0 1.1 0 0 c3t25d6
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d7
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d8
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d11
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d12
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d13
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d14
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d15
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d16
    0.0 7.2 0.0 61.4 0.0 0.0 0.0 1.1 0 0 c3t25d17
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d18
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d19
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d20
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d22
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d23
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d24
    0.0 0.4 0.0 3.2 0.0 0.0 0.0 0.6 0 0 c2t24d25
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d26
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d27
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d20
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d22
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d23
    8.2 17.4 230.0 142.2 0.0 0.1 0.0 2.5 0 3 c3t25d24
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d25
    0.0 0.4 0.0 6.4 0.0 0.0 0.0 1.4 0 0 c3t25d26
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d27
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 st5
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
    antibes:vold(pid706)
    extended device statistics
    r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t10d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t11d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t2d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d1
    5.6 13.4 44.8 147.2 0.0 0.0 0.0 0.6 0 1 c2t24d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d4
    0.0 6.6 0.0 52.8 0.0 0.0 0.0 0.6 0 0 c2t24d5
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d6
    0.0 0.6 0.0 8.0 0.0 0.0 0.0 0.8 0 0 c2t24d7
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d8
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d11
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d12
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d13
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d14
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d15
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d16
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d17
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d18
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d19
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d1
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d4
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d5
    0.0 1.0 0.0 12.8 0.0 0.0 0.0 1.2 0 0 c3t25d6
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d7
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d8
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d11
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d12
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d13
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d14
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d15
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d16
    0.0 8.2 0.0 95.1 0.0 0.0 0.0 1.0 0 1 c3t25d17
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d18
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d19
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d20
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d22
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d23
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d24
    0.0 0.2 0.0 1.6 0.0 0.0 0.0 0.6 0 0 c2t24d25
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d26
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d27
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d20
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d22
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d23
    7.2 20.6 192.0 166.4 0.0 0.1 0.0 2.7 0 4 c3t25d24
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d25
    0.0 0.4 0.0 4.8 0.0 0.0 0.0 1.3 0 0 c3t25d26
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d27
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 st5
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
    antibes:vold(pid706)


  2. Re: rlim_fd_max and 99% IO Wait - Solaris 8 Sun server

    Well, I'd ask the basic questions, first, are your paths active/active or
    active/passive. Second, are you using mpxio, or powerpath. Third, how many
    hops between you and the frames... etc.

    Now on to the data itself, your page faults seem high to me, as do the
    number of blocked processes; it looks almost as though your thrashing,
    nothing else indicates a reason for that. Was this snapshot at program
    instantiation or something?? Nothing particularly screams IO wait from your
    storage, but there's not a lot to gauge by here. If you have some sar data,
    a longer running vmstat, and a more concise iostat, that would help; I
    think the mpstat is sufficient to guess that nothing is cpu bound.

    System is nominally idle according to this output, reads/writes look sparse
    and fairly timely.

    If thrashing is somehow occurring, what are your minfree/maxfree settings?

    Wish I could offer more help, but I don't really have enough to look at to
    make a comprehensive analysis.

    If you haven't resolved your issue yet, feel free to post more data, and
    I'll be glad to help.

    Thanks.

    underh20 wrote:

    > Hello,
    >
    > Given the following output from vmstat, mpstat and iostat. What would
    > you say the bottleneck is on our Sun E6500 server ?? I noticed that in
    > "top" the "% iowait" is consistently higher 99% while the cpu % idle
    > remains zero. The only change made to the system was the value of
    > rlim_fd_max from 8096 to 65536 in the /etc/system. The "ulimit -Hn"
    > now shows 65535. The change was one of the Oracle 10G installation
    > requirement, together with other Sun patches. Memory doesn't seem to
    > be an issue. Judging from everything I've seen, may be the I/O wait is
    > the bottleneck. Any idea what's happening at our server and how we may
    > trouble-shoot this.
    >
    >
    > The server is Sun E6500 / 20 x 400MHz / 26G memory. It's running the
    > following database and application:
    >
    > - Solaris 8 (kernel 108528-29)
    > - Veritas Oracle DB Edition (VxDBED, VxVM, VxFS - 3.5 MP3) with Vxdmp
    > - Oracle 8.1.7.4 (2 instances)
    > - Oralce 9 release 1 (1 instance)
    > - Oracle 10G release 1 (or OEM)
    > - Oracle Application Server 1.0.2.2
    >
    > server is connected to the following storage :
    >
    > - 2 Emulex HBAs connection to SAN fabrics to luns at Hitachi 9585
    > storage array
    > - 2 Emulex HBAs connection to SAN fabrics to luns at EMC DMX storage
    > array
    >
    >
    > Thanks,
    >
    > Humphrey
    >
    >
    >
    > # vmstat 5 5
    > procs memory page disk faults
    > cpu
    > r b w swap free re mf pi po fr de sr s6 s9 s1 s1 in sy cs
    > us sy id
    > 0 11001 0 19304088 16932208 26 166 8 0 0 0 0 0 3 0 3 2258 6235 1625
    > 5 1 94
    > 0 607115 0 16708216 14533000 42 247 0 0 0 0 0 0 1 0 1 2458 17527 2577
    > 10 2 88
    > 0 607278 0 16708312 14532232 22 185 0 0 0 0 0 0 0 0 1 2452 4797 2504
    > 10 1 89
    > 0 607492 0 16708152 14531136 36 136 0 0 0 0 0 0 0 0 0 2461 4922 2380
    > 10 2 89
    > 0 607705 0 16712784 14534904 7 0 0 0 0 0 0 0 0 0 0 2430 2597 2238
    > 5 1 94
    >
    >
    >
    > # mpstat 3 3
    > CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys
    > wt idl
    > 0 7 0 2326 365 274 56 0 1 13 0 243 3 1
    > 88 8
    > 1 7 0 412 98 96 82 0 2 12 0 319 7 1
    > 83 9
    > 4 8 0 371 94 93 73 0 2 10 0 278 3 2
    > 87 8
    > 5 8 0 445 96 94 85 0 2 12 0 325 7 1
    > 84 8
    > 8 8 0 391 97 95 81 0 2 9 0 268 3 2
    > 87 8
    > 9 7 0 415 96 94 92 0 2 12 0 306 7 1
    > 83 9
    > 12 8 0 382 94 92 83 0 2 9 0 327 3 1
    > 87 8
    > 13 8 0 440 98 96 93 1 2 12 0 346 7 1
    > 83 9
    > 14 9 0 648 177 175 89 0 2 10 0 277 3 1
    > 87 9
    > 15 7 0 438 137 135 86 0 2 12 0 319 6 1
    > 85 8
    > 16 8 0 393 96 94 89 0 2 9 0 318 4 1
    > 87 8
    > 17 7 0 467 104 102 89 0 2 11 0 308 6 1
    > 84 8
    > 18 8 0 397 118 116 78 0 2 9 0 347 3 1
    > 87 8
    > 19 7 0 428 96 94 83 0 2 13 0 329 7 1
    > 84 8
    > 20 9 0 426 97 95 75 0 2 9 0 313 3 1
    > 87 8
    > 21 9 0 437 97 95 79 0 2 12 0 352 7 1
    > 83 8
    > 22 9 0 416 97 95 75 0 2 10 0 284 3 1
    > 87 8
    > 23 8 0 436 96 94 76 0 2 12 0 319 7 1
    > 84 8
    > 24 8 0 378 94 93 72 0 2 10 0 315 3 1
    > 87 8
    > 25 8 0 434 96 95 81 0 2 12 0 337 6 1
    > 84 9
    > CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys
    > wt idl
    > 0 0 0 2467 385 289 76 0 5 14 0 101 0 1
    > 99 0
    > 1 0 0 131 100 99 57 0 6 5 0 73 0 0
    > 100 0
    > 4 2 0 593 101 100 116 0 2 6 0 128 0 0
    > 100 0
    > 5 0 0 31 101 99 88 0 6 4 0 62 0 0
    > 99 0
    > 8 0 0 75 103 100 116 2 4 5 0 188 3 0
    > 97 0
    > 9 0 0 239 101 99 107 1 6 7 0 68 0 0
    > 100 0
    > 12 0 0 105 100 98 133 0 3 4 0 55 0 11
    > 89 0
    > 13 0 0 224 101 99 106 0 5 5 0 110 1 0
    > 99 0
    > 14 0 0 890 186 180 49 5 3 10 0 240 40 0
    > 60 0
    > 15 0 0 62 180 176 109 1 8 9 0 195 4 1
    > 95 0
    > 16 0 0 169 101 99 72 0 5 4 0 86 0 0
    > 100 0
    > 17 0 0 88 101 99 84 1 3 8 0 133 3 0
    > 96 0
    > 18 0 0 107 113 111 70 0 4 4 0 80 0 0
    > 100 0
    > 19 0 0 299 101 99 105 0 7 5 0 74 0 1
    > 99 0
    > 20 0 0 37 100 98 132 0 5 7 0 124 1 1
    > 99 0
    > 21 0 0 749 99 98 127 0 5 6 0 77 0 1
    > 99 0
    > 22 0 0 328 99 98 110 0 4 5 0 110 0 0
    > 100 0
    > 23 0 0 107 101 100 70 0 5 4 0 118 0 0
    > 100 0
    > 24 0 0 0 101 100 160 0 4 18 0 267 2 0
    > 98 0
    > 25 0 0 551 104 99 57 4 3 6 0 257 53 1
    > 46 0
    > CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys
    > wt idl
    > 0 0 0 2312 388 291 79 0 2 22 0 169 0 1
    > 99 0
    > 1 0 0 175 100 99 96 0 4 9 0 74 0 0
    > 100 0
    > 4 0 0 351 100 99 132 0 4 5 0 173 0 0
    > 100 0
    > 5 0 0 536 105 100 56 7 4 9 0 297 61 1
    > 38 0
    > 8 0 0 137 100 99 94 0 5 1 0 53 0 1
    > 99 0
    > 9 0 0 449 101 99 172 1 8 10 0 167 0 1
    > 99 0
    > 12 0 0 160 95 93 157 0 7 10 0 77 1 0
    > 99 0
    > 13 0 0 267 101 99 123 1 6 7 0 66 0 1
    > 99 0
    > 14 0 0 925 195 191 156 3 5 10 0 168 25 0
    > 75 0
    > 15 0 0 63 187 182 105 3 4 10 0 207 10 0
    > 90 0
    > 16 0 0 218 100 98 129 1 8 14 0 151 0 0
    > 99 0
    > 17 0 0 139 102 100 124 2 5 12 0 122 3 0
    > 97 0
    > 18 0 0 131 139 137 122 1 5 7 0 79 0 0
    > 100 0
    > 19 0 0 320 101 98 150 1 7 11 0 136 0 0
    > 100 0
    > 20 0 0 59 103 100 91 1 6 7 0 84 2 0
    > 98 0
    > 21 0 0 618 103 100 178 1 7 9 0 191 1 1
    > 98 0
    > 22 0 0 555 91 90 78 0 5 3 0 123 0 0
    > 100 0
    > 23 0 0 228 101 99 96 0 5 6 0 98 0 0
    > 100 0
    > 24 0 0 44 101 100 95 0 4 7 0 90 0 0
    > 100 0
    > 25 0 0 138 100 99 125 0 4 8 0 92 0 0
    > 100 0
    >
    >
    >
    > # iostat -xn 5
    > extended device statistics
    > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
    > 1.5 1.7 56.5 53.8 0.0 0.0 0.0 11.2 0 2 c0t10d0
    > 0.1 0.0 1.8 1.7 0.0 0.0 0.0 11.7 0 0 c0t11d0
    > 1.5 1.7 56.3 54.0 0.0 0.0 0.0 11.1 0 2 c1t2d0
    > 1.3 6.4 11.0 71.2 0.0 0.0 0.0 0.7 0 1 c2t24d1
    > 1.8 6.7 14.7 74.1 0.0 0.0 0.0 0.7 0 1 c2t24d2
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d3
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d4
    > 0.0 3.5 0.4 31.2 0.0 0.0 0.0 0.7 0 0 c2t24d5
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d6
    > 0.4 0.8 7.3 10.4 0.0 0.0 0.0 3.2 0 0 c2t24d7
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d8
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d11
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d12
    > 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.1 0 0 c2t24d13
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d14
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0 0 c2t24d15
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d16
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d17
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d18
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0 0 c2t24d19
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d1
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d2
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d3
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d4
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d5
    > 0.3 1.0 5.7 12.6 0.0 0.0 0.0 3.4 0 0 c3t25d6
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d7
    > 0.0 0.0 0.1 0.1 0.0 0.0 0.0 1.5 0 0 c3t25d8
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d11
    > 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.7 0 0 c3t25d12
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d13
    > 0.0 0.0 0.2 0.4 0.0 0.0 0.0 2.6 0 0 c3t25d14
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d15
    > 0.0 0.0 0.1 0.1 0.0 0.0 0.0 1.2 0 0 c3t25d16
    > 1.8 4.2 21.4 41.7 0.0 0.0 0.5 2.2 0 1 c3t25d17
    > 0.0 0.0 0.2 0.4 0.0 0.0 0.0 3.0 0 0 c3t25d18
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d19
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d20
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d22
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.2 0 0 c2t24d23
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d24
    > 0.0 0.3 1.4 2.2 0.0 0.0 0.0 1.1 0 0 c2t24d25
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d26
    > 0.0 0.0 0.3 0.1 0.0 0.0 0.0 2.6 0 0 c2t24d27
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.3 0 0 c3t25d20
    > 0.0 0.0 0.2 0.4 0.0 0.0 0.0 2.4 0 0 c3t25d22
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d23
    > 2.9 16.6 66.3 134.6 0.0 0.0 0.0 1.7 0 3 c3t25d24
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d25
    > 0.0 0.3 0.4 5.3 0.0 0.0 0.0 2.2 0 0 c3t25d26
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d27
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 st5
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
    > antibes:vold(pid706)
    > extended device statistics
    > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t10d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t11d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t2d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d1
    > 5.6 14.4 44.7 164.5 0.0 0.0 0.0 0.6 0 1 c2t24d2
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d3
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d4
    > 0.0 2.8 0.0 22.4 0.0 0.0 0.0 0.6 0 0 c2t24d5
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d6
    > 0.0 1.0 0.0 11.2 0.0 0.0 0.0 0.8 0 0 c2t24d7
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d8
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d11
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d12
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d13
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d14
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d15
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d16
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d17
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d18
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d19
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d1
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d2
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d3
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d4
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d5
    > 0.0 1.4 0.0 17.6 0.0 0.0 0.0 1.1 0 0 c3t25d6
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d7
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d8
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d11
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d12
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d13
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d14
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d15
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d16
    > 0.0 7.2 0.0 61.4 0.0 0.0 0.0 1.1 0 0 c3t25d17
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d18
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d19
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d20
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d22
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d23
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d24
    > 0.0 0.4 0.0 3.2 0.0 0.0 0.0 0.6 0 0 c2t24d25
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d26
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d27
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d20
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d22
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d23
    > 8.2 17.4 230.0 142.2 0.0 0.1 0.0 2.5 0 3 c3t25d24
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d25
    > 0.0 0.4 0.0 6.4 0.0 0.0 0.0 1.4 0 0 c3t25d26
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d27
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 st5
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
    > antibes:vold(pid706)
    > extended device statistics
    > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t10d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t11d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t2d0
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d1
    > 5.6 13.4 44.8 147.2 0.0 0.0 0.0 0.6 0 1 c2t24d2
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d3
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d4
    > 0.0 6.6 0.0 52.8 0.0 0.0 0.0 0.6 0 0 c2t24d5
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d6
    > 0.0 0.6 0.0 8.0 0.0 0.0 0.0 0.8 0 0 c2t24d7
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d8
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d11
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d12
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d13
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d14
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d15
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d16
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d17
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d18
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d19
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d1
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d2
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d3
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d4
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d5
    > 0.0 1.0 0.0 12.8 0.0 0.0 0.0 1.2 0 0 c3t25d6
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d7
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d8
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d11
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d12
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d13
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d14
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d15
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d16
    > 0.0 8.2 0.0 95.1 0.0 0.0 0.0 1.0 0 1 c3t25d17
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d18
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d19
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d20
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d22
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d23
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d24
    > 0.0 0.2 0.0 1.6 0.0 0.0 0.0 0.6 0 0 c2t24d25
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d26
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t24d27
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d20
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d22
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d23
    > 7.2 20.6 192.0 166.4 0.0 0.1 0.0 2.7 0 4 c3t25d24
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d25
    > 0.0 0.4 0.0 4.8 0.0 0.0 0.0 1.3 0 0 c3t25d26
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c3t25d27
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 st5
    > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
    > antibes:vold(pid706)



  3. Re: rlim_fd_max and 99% IO Wait - Solaris 8 Sun server

    I saw this same thing happen on an apache server that was being beaten
    up very badly. The access log grew to just over 2GB and stopped
    growing. Turned out to be a file-size limitation (I think with
    Solaris, since apache was not compiled with 64-bit support, although
    I'm not 100% that's the reason).

    Once the file size limit was hit, cpu was really low, but iowait was
    consistently high.

    Something to check out, anyway.

    Patrick


+ Reply to Thread