Script - Slackware

This is a discussion on Script - Slackware ; Hello Well I am trying to get a script to sort out a huge xfer log file. Basically this is the field from the Xfer log Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/ AccountFolder/download.tst b _ o r USER ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: Script

  1. Script

    Hello

    Well I am trying to get a script to sort out a huge xfer log
    file.

    Basically this is the field from the Xfer log
    Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    AccountFolder/download.tst b _ o r USER ftp 0 * c

    I need a script that will take three files xferlog1,
    xferlog2,xferlog3

    append them to one file and Sort out the Data

    Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    AccountFolder/download.tst b _ o r USER ftp 0 * c
    Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    AccountFolder/download.tst b _ i r USER ftp 0 * c

    Each user would have two lines an incomming xfer and an outgoing xfer.

    so it would look like

    Mon Jan 28 23:56:54 2008 0 *.*.*.* 5 /home3/FTP-protected/IBES/Account/
    download.tst b _ o r USEr ftp 0 * c
    Tue Jan 29 00:00:19 2008 0 *.*.*.* 12477 /home3/FTP-protected/TFN/v2/
    Account/download.tst b _ i r USEr ftp 0 * c

    Thank you .

    I am learning and I have no clue as to how to do any of this.


  2. Re: Script

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    On 2008-02-14, Andy wrote:
    > I need a script that will take three files xferlog1,
    > xferlog2,xferlog3
    >
    > append them to one file and Sort out the Data
    >
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    > AccountFolder/download.tst b _ o r USER ftp 0 * c
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    > AccountFolder/download.tst b _ i r USER ftp 0 * c
    >
    > Each user would have two lines an incomming xfer and an outgoing xfer.
    >
    > so it would look like
    >
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* 5 /home3/FTP-protected/IBES/Account/
    > download.tst b _ o r USEr ftp 0 * c
    > Tue Jan 29 00:00:19 2008 0 *.*.*.* 12477 /home3/FTP-protected/TFN/v2/
    > Account/download.tst b _ i r USEr ftp 0 * c


    I'm not entirely sure what you're attempting to do here. How exactly
    do you want this sorted out? You might be able to get away with just
    using sort(1).

    # cat file1 file2 file3 | sort

    If you need something more robust, there are lots of perl scripts or
    small applications floating around for working with wuftp style
    xferlogs. Try a search for xferlog onf freshmeat.

    - --
    It is better to hear the rebuke of the wise,
    Than for a man to hear the song of fools.
    Ecclesiastes 7:5
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (GNU/Linux)

    iD8DBQFHtK1erZS6hX/gvjoRAt/eAJ9R61oXS47edZzt7x174hJzqgtdRgCfSZHD
    aAPUQMKAAuprpVO2VhHZK2A=
    =uvpq
    -----END PGP SIGNATURE-----

  3. Re: Script

    +Alan Hicks+ wrote:
    > # cat file1 file2 file3 | sort


    Yeah, but does this work in KDE?



    -Beej


  4. Re: Script

    On 2008-02-14, Beej Jorgensen wrote:
    > +Alan Hicks+ wrote:
    >> # cat file1 file2 file3 | sort

    >
    > Yeah, but does this work in KDE?
    >
    >
    >
    > -Beej
    >


    Doesn't matter whether it does or not. You don't need
    KDE to do that. Or anything else.

    Except wasting most of your system resources.
    It does that very well indeed.




    Tom

    --
    calhobbit (at) | Artificial Intelligence:
    gmail [DOT] com | When the real thing just won't do.


  5. Re: Script

    On 2008-02-14, Beej Jorgensen wrote:
    > +Alan Hicks+ wrote:
    >> # cat file1 file2 file3 | sort

    >
    > Yeah, but does this work in KDE?


    No, but he might as well be using KDE for such an egregious waste of
    resources! In ODE, which is the best desktop environment ever invented
    by a trolling netkook, the above is a UUOC. It's teraquads more
    efficient to do

    sort file1 file2 file3

    On my ODE system, sort without a pipe was a whole 0.05 seconds faster!

    I'd like to personally thank My Hero, trolling netkook and inventor of
    ODE ''Tom Newton'', for introducing us all to the ODE Desktop
    Environment, without which we'd all still be doing UUOC.

    --keith

    --
    kkeller-usenet@wombat.san-francisco.ca.us
    (try just my userid to email me)
    AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
    see X- headers for PGP signature information


  6. Re: Script

    On 2008-02-15, Keith Keller wrote:
    > On 2008-02-14, Beej Jorgensen wrote:
    >> +Alan Hicks+ wrote:
    >>> # cat file1 file2 file3 | sort

    >>
    >> Yeah, but does this work in KDE?

    >
    > No, but he might as well be using KDE for such an egregious waste of
    > resources! In ODE, which is the best desktop environment ever invented
    > by a trolling netkook, the above is a UUOC. It's teraquads more
    > efficient to do
    >
    > sort file1 file2 file3
    >
    > On my ODE system, sort without a pipe was a whole 0.05 seconds faster!
    >
    > I'd like to personally thank My Hero, trolling netkook and inventor of
    > ODE ''Tom Newton'', for introducing us all to the ODE Desktop
    > Environment, without which we'd all still be doing UUOC.


    Yes. I've heard it all before. Like everyone else on the Usenet
    worth talking to, I don't take a word you post seriously, regardless
    of which of the thousand names you post under you are using at the
    moment.

    The fact that you never seem to comprehend the simple fact that
    none of your childish posts has any significant effect on me, and
    their very nature, tells everyone all they need to know about
    you: You are not mentally competent.

    I'm here. I post what I want and I almost always ignore your posts
    and any replies to them (regardless of which name you are hiding
    behind at the moment). As do most people. Obviosuly. That's why
    most of the responses to your posts are from you, using other
    names.

    So what are you going to do about it?

    Answer: Live with it. You can't do anything at all except run your
    mouth for the entertainment of your fellow trolls, who are,
    literally, the dregs of the Internet. They have no power at all.
    No one takes them seriously except themselves.

    I don't take you seriously. I don't care what a loser like you
    thinks. Nor does anyone else who matters.

    In fact, you are so powerless that I don't even have to read
    your posts to find out what you are saying. It just doesn't
    matter.

    Here's the ticket: If you don't like me being on the Usenet
    or what I post, you have one option: Kiss my butt.

    If you'd ever like to get together (will your mother let even
    let you out of the house?), email me and we'll set it up.

    Please note that if you were any kind of a threat at all, I'd
    put out the $5K it would take to have you tracked down and
    sue you in civil court.

    The thing is, in order to make libel and defamation of
    character charges stick, I'd have to prove that at least one
    person with credibility takes you seriously.

    And I doubt that I could do that.

    Blabber on, loser. Everyone needs a hobby.

    Tom

    --
    calhobbit (at) | Artificial Intelligence:
    gmail [DOT] com | When the real thing just won't do.


  7. Re: Script

    Tom Newton wrote:
    >
    > The fact that you never seem to comprehend the simple fact that
    > none of your childish posts has any significant effect on me, and
    >


    It obviously does given the length of the response you posted.

    - Kurt

  8. Re: Script

    On Fri, 15 Feb 2008, ~kurt wrote:

    >
    > Tom Newton wrote:
    >>
    >> The fact that you never seem to comprehend the simple fact that
    >> none of your childish posts has any significant effect on me, and
    >>

    >
    > It obviously does given the length of the response you posted.


    and the tremendously long list of named he 'claims' he has killfiled
    muwhahaha

    >
    > - Kurt
    >


    --
    Cheers
    Res

    mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';

  9. Re: Script

    On 2008-02-15, Tom Newton wrote:
    >
    > Please note that if you were any kind of a threat at all, I'd
    > put out the $5K it would take to have you tracked down and
    > sue you in civil court.


    If you think it takes $5000 to track me down, I'd hold off on your civil
    suit, as it's bound to be as much a failure as your usenet postings.

    --keith



    --
    kkeller-usenet@wombat.san-francisco.ca.us
    (try just my userid to email me)
    AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
    see X- headers for PGP signature information


  10. Re: Script

    On Thu, 14 Feb 2008 17:31:53 -0800, Tom Newton wrote:

    > Please note that if you were any kind of a threat at all, I'd
    > put out the $5K it would take to have you tracked down and
    > sue you in civil court.


    Hey dickmunch. Since you already publicly stated that you had a "thick
    manila envelope" of information about me, why don't you go ahead and post
    my name, address, and phone number right here in this public forum. You
    have my permission.

    Go ahead, Alan. Let's see what you got. Or, all you all mouth?


    --
    "Ubuntu" -- an African word, meaning "Slackware is too hard for me".


  11. Re: Script

    On 2008-02-15, Tom Newton wrote:
    >
    > The thing is, in order to make libel and defamation of
    > character charges stick, I'd have to prove that at least one
    > person with credibility takes you seriously.
    >


    Hell, you can't even prove that one person with credibility takes
    ***YOU*** seriously.

    ken

  12. Re: Script

    On Feb 14, 8:16 pm, Andy wrote:
    > Hello
    >
    > Well I am trying to get a script to sort out a huge xfer log
    > file.
    >
    > Basically this is the field from the Xfer log
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    > AccountFolder/download.tst b _ o r USER ftp 0 * c
    >
    > I need a script that will take three files xferlog1,
    > xferlog2,xferlog3
    >
    > append them to one file and Sort out the Data
    >
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    > AccountFolder/download.tst b _ o r USER ftp 0 * c
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* /home3/FTP-protected/Server/
    > AccountFolder/download.tst b _ i r USER ftp 0 * c
    >
    > Each user would have two lines an incomming xfer and an outgoing xfer.
    >
    > so it would look like
    >
    > Mon Jan 28 23:56:54 2008 0 *.*.*.* 5 /home3/FTP-protected/IBES/Account/
    > download.tst b _ o r USEr ftp 0 * c
    > Tue Jan 29 00:00:19 2008 0 *.*.*.* 12477 /home3/FTP-protected/TFN/v2/
    > Account/download.tst b _ i r USEr ftp 0 * c
    >
    > Thank you .
    >
    > I am learning and I have no clue as to how to do any of this.


    It does matter whether you are going to write script or looking for
    implemented solutions. If first, here is some clue. awk has a power
    and often underestimated feature - associated arrays, where literal
    strings can be used to index elements of array. At first sight looks
    like useless, but only at first. You have already sorted list and want
    to resort it, to have combined input and output records for each
    transfer. First we use username and file name from each record to
    create index, i.e. $(NF-4) $(NF-9) (for awk space means catenation of
    2 strings, $(NF-4) means 4 record from end). Then what we have to do
    is just, have we found an matching record for pair, then print them
    both, else store the record.

    cat xfer1 xfer2 xfer3 |awk '{s=$(NF-4) $(NF-9); if( x[s] ) {print
    x[s] "\n", $0; delete x[s]} else x[s]=$0}'
    This is only a sketch, but I think quite good, because we do not
    perform a complete sort
    on whole huge files, but only resort them.
    Note the delete statement, once we have printed record we don't need
    to store it anymore
    and waste memory (RAM), this is important for huge files. While
    debugging, replace
    cat xfer1 xfer2 xfer3 to something like, head -n 50 xfer1.
    Hope this helps.

    ---
    Bogdan

+ Reply to Thread