How to get usage statistics - SNMP

This is a discussion on How to get usage statistics - SNMP ; Hi. I have some hundred routers that I must gather statistical information like usage, traffic, etc. Basically those that MRTG collects to create graphs. And I need to get this information every 5 minutes for all the links. Nowadays I ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: How to get usage statistics

  1. How to get usage statistics

    Hi.

    I have some hundred routers that I must gather statistical information
    like usage, traffic, etc. Basically those that MRTG collects to create
    graphs. And I need to get this information every 5 minutes for all the
    links.

    Nowadays I use MRTG to get them, but I don't think it's the
    best way to do it. What I want is to get that information and store in
    a database for later analysis, and MRTG takes a lot of cpu to create
    graphics that nobody sees.

    I checked out net-snmp and played a little with it, but i don't know
    how to use it exactly to get only this information.

    Can anyone help me telling if this is the way, or if there is any
    better way to collect this data?

    Thanks.

  2. Re: How to get usage statistics

    nvivo.misc@mandic.com.br (Natan Vivo) writes:

    > Nowadays I use MRTG to get them, but I don't think it's the best way
    > to do it. What I want is to get that information and store in a
    > database for later analysis, and MRTG takes a lot of cpu to create
    > graphics that nobody sees.


    Have you looked at RRD Tool (written by the same guy as MRTG)? Its a
    re-implementation of MRTG's logging and graphing features minus the
    built-in SNMP support. There are several systems that have been built
    on top of it that might be of interest. One such example is Cacti
    (http://www.cacti.net). There are other 3rd party systems you can
    find on the RRD Tool webpage:

    http://people.ee.ethz.ch/~oetiker/we...rld/index.html

    Most of these systems are GUI front-ends to some SNMP tools that poll
    your network, store the results in RRD files, generate graphs on the
    fly, and some generate reports. If you are looking to do more
    extensive analysis, you may have to roll your own system or pay for a
    commercial offering such as Concord's NetHealth.

    > I checked out net-snmp and played a little with it, but i don't know
    > how to use it exactly to get only this information.
    >
    > Can anyone help me telling if this is the way, or if there is any
    > better way to collect this data?


    You could roll your own system. You'd basically write some sort of
    SNMP poller (either using net-snmp or some other SNMP library) that
    could poll multiple devices in parallel (so you can poll all the
    devices every 5 minutes as you indicated), collect the stats and store
    them in some database. You could then hook up some graphing utility
    (ploticus or something like it) to generate the graphs from the
    database. Finally, you'd have to roll up your data periodically (like
    MRTG and RRD do for you automatically). With the data in the
    database, you could then do all the analysis you want.

    Rolling your own is definitely more work, but you get much more
    flexibility with regards to reporting. Using the RRD Tool based
    systems, you get the rolling of data for free as well as the graphing,
    but you lose the ability with flexible reporting because it requires
    reading through hundreds of RRD files on the file system. I'm not
    sure of any other solutions in between and would be interested if
    anyone else could provide additional insight.

    Pete


  3. Re: How to get usage statistics

    Natan Vivo wrote:

    > I have some hundred routers that I must gather statistical information
    > like usage, traffic, etc. Basically those that MRTG collects to create
    > graphs. And I need to get this information every 5 minutes for all the
    > links.
    >
    > Nowadays I use MRTG to get them, but I don't think it's the
    > best way to do it. What I want is to get that information and store in
    > a database for later analysis, and MRTG takes a lot of cpu to create
    > graphics that nobody sees.


    Never MRTG versions use RRDs and only generate graphics on the fly.
    I strongly suggest to consider RRDs for storing the time series as
    they are compact and allow to drive different tools. You can use
    an RRD API (several bindings to different languages do exit) to
    extract data and do whatever you need to do with the data. To use
    RRDs, you have however to accept the automatic data consolidation
    that RRDs to. If your requirement is that you have to keep all 5
    minute readings forever in that granularity, you have to roll your
    own system (and accept the increasing storage space to keep the
    statistics).

    /js

    --
    Juergen Schoenwaelder International University Bremen
    P.O. Box 750 561, 28725 Bremen, Germany

  4. Re: How to get usage statistics

    Pete Kazmier wrote in message news:<87zmvub83a.fsf@coco.kazmier.com>...

    > You could roll your own system. You'd basically write some sort of
    > SNMP poller (either using net-snmp or some other SNMP library) that
    > could poll multiple devices in parallel (so you can poll all the
    > devices every 5 minutes as you indicated), collect the stats and store
    > them in some database. You could then hook up some graphing utility
    > (ploticus or something like it) to generate the graphs from the
    > database. Finally, you'd have to roll up your data periodically (like
    > MRTG and RRD do for you automatically). With the data in the
    > database, you could then do all the analysis you want.


    Yes, that is what I was thinking about. I got rrdtool last night and
    read about it, it seems nice to store and generate graphs on the fly.

    So I will create a service that check all the devices in parallel, use
    net-snmp to get the status information, store in rrds and generate
    graphs on the fly or based on a schedule depending on the graph.

    Since the creation and managing of those files will be automatic by
    the system and we have a dedicated server, there will be no problem to
    store hundreds of databases.

    Thanks for the help!

  5. Re: How to get usage statistics

    Juergen Schoenwaelder wrote in message news:<4265eaa8$0$21460$9b622d9e@news.freenet.de>...
    > Never MRTG versions use RRDs and only generate graphics on the fly.
    > I strongly suggest to consider RRDs for storing the time series as
    > they are compact and allow to drive different tools. You can use
    > an RRD API (several bindings to different languages do exit) to
    > extract data and do whatever you need to do with the data. To use
    > RRDs, you have however to accept the automatic data consolidation
    > that RRDs to. If your requirement is that you have to keep all 5
    > minute readings forever in that granularity, you have to roll your
    > own system (and accept the increasing storage space to keep the
    > statistics).


    Hi Juergan!

    Yes, I decided to use RRDtool to store. Since I space is no problem, I
    can create huge logs for each device, and that is what I want!

    Thanks you both for the answer!

  6. Re: How to get usage statistics

    Juergen Schoenwaelder writes:

    > To use RRDs, you have however to accept the automatic data
    > consolidation that RRDs to. If your requirement is that you have to
    > keep all 5 minute readings forever in that granularity, you have to
    > roll your own system (and accept the increasing storage space to
    > keep the statistics).


    As far as I understand it, this is not true. Even if Natan wanted to
    keep all 5-minute readings for 5 years without having to accept the
    automatic data consolidation, he could just create an RRA that does
    not do any consolidation as follows:

    rrdtool create host1.rrd --step 300 \
    DS:cpu:GAUGE:600:0:100 \
    RRA:AVERAGE:0.5:1:525600
    ^^^

    The key number above is that '1' which says that one primary data
    point is used for a consolidated data point. Granted, this would take
    a large amount of disk space (about 4.2M per DS), but he could still
    take advantage of using RRD Tool instead of rolling his own.

  7. Re: How to get usage statistics

    Pete Kazmier wrote:

    > As far as I understand it, this is not true. Even if Natan wanted to
    > keep all 5-minute readings for 5 years without having to accept the
    > automatic data consolidation, he could just create an RRA that does
    > not do any consolidation as follows:
    >
    > rrdtool create host1.rrd --step 300 \
    > DS:cpu:GAUGE:600:0:100 \
    > RRA:AVERAGE:0.5:1:525600
    > ^^^
    >
    > The key number above is that '1' which says that one primary data
    > point is used for a consolidated data point. Granted, this would take
    > a large amount of disk space (about 4.2M per DS), but he could still
    > take advantage of using RRD Tool instead of rolling his own.


    Thanks for correcting me. And what are 4.2M in 5 years from now...

    /js

    --
    Juergen Schoenwaelder International University Bremen
    P.O. Box 750 561, 28725 Bremen, Germany

+ Reply to Thread