adding up field columns in a file - Unix

This is a discussion on adding up field columns in a file - Unix ; Hello, I have a file with some columns of numeric data e.g. 5 5 5 6 5 7 I need to total them, currently im using the following method for i in cat | awk print column2 (for example) do ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: adding up field columns in a file

  1. adding up field columns in a file

    Hello,
    I have a file with some columns of numeric data e.g.

    5 5
    5 6
    5 7

    I need to total them, currently im using the following method


    for i in cat | awk print column2 (for example)
    do
    total=$i
    total=xpr $total + $i`
    done

    However this method is really slow on large files, could someone
    recommend a quicker way?

    Thanks


  2. Re: adding up field columns in a file

    On 2006-01-23, cconnell_1@lycos.com wrote:
    > Hello,
    > I have a file with some columns of numeric data e.g.
    >
    > 5 5
    > 5 6
    > 5 7
    >
    > I need to total them, currently im using the following method
    >
    >
    > for i in cat | awk print column2 (for example)
    > do
    > total=$i
    > total=xpr $total + $i`
    > done
    >
    > However this method is really slow on large files, could someone
    > recommend a quicker way?


    To add the second column:

    awk -v col=2 '{ total += $col } END { print total }' FILENAME

    To add the first column, change col=2 to col=1

    --
    Chris F.A. Johnson, author |
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence

  3. Re: adding up field columns in a file


    Chris F.A. Johnson wrote:
    > On 2006-01-23, cconnell_1@lycos.com wrote:
    > > Hello,
    > > I have a file with some columns of numeric data e.g.
    > >
    > > 5 5
    > > 5 6
    > > 5 7
    > >
    > > I need to total them, currently im using the following method
    > >
    > >
    > > for i in cat | awk print column2 (for example)
    > > do
    > > total=$i
    > > total=xpr $total + $i`
    > > done
    > >
    > > However this method is really slow on large files, could someone
    > > recommend a quicker way?

    >
    > To add the second column:
    >
    > awk -v col=2 '{ total += $col } END { print total }' FILENAME
    >
    > To add the first column, change col=2 to col=1
    >
    > --
    > Chris F.A. Johnson, author |
    > Shell Scripting Recipes: | My code in this post, if any,
    > A Problem-Solution Approach | is released under the
    > 2005, Apress | GNU General Public Licence



    Thanks, I think there is a typo somewhere as I get byte@osn01:/tmp-->
    awk -v col=2 '{ total += $col } END { print total }' test
    awk: syntax error near line 1
    awk: bailing out near line 1

    Any ideas?


  4. Re: adding up field columns in a file

    On 2006-01-23, cconnell_1@lycos.com wrote:
    >
    > Chris F.A. Johnson wrote:
    >> On 2006-01-23, cconnell_1@lycos.com wrote:
    >> > Hello,
    >> > I have a file with some columns of numeric data e.g.
    >> >
    >> > 5 5
    >> > 5 6
    >> > 5 7
    >> >
    >> > I need to total them, currently im using the following method
    >> >
    >> >
    >> > for i in cat | awk print column2 (for example)
    >> > do
    >> > total=$i
    >> > total=xpr $total + $i`
    >> > done
    >> >
    >> > However this method is really slow on large files, could someone
    >> > recommend a quicker way?

    >>
    >> To add the second column:
    >>
    >> awk -v col=2 '{ total += $col } END { print total }' FILENAME
    >>
    >> To add the first column, change col=2 to col=1

    >
    > Thanks, I think there is a typo somewhere as I get byte@osn01:/tmp-->
    > awk -v col=2 '{ total += $col } END { print total }' test
    > awk: syntax error near line 1
    > awk: bailing out near line 1
    >
    > Any ideas?


    Are you using Solaris? If so, use nawk or xpg4/awk (I forget the
    exact location).


    --
    Chris F.A. Johnson, author |
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence

  5. Re: adding up field columns in a file

    Yes its solaris.
    nawk works fantasic.Thx


  6. Re: adding up field columns in a file

    >>>>> "cconnell" == cconnell 1 writes:

    cconnell> for i in cat | awk print column2 (for example)

    Why the unnecessary cat?

    Why not just:

    cat filename | cat | cat | cat | tr | cat | awk ...

    --
    Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095

    Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
    See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

  7. Re: adding up field columns in a file

    On 2006-01-23, Randal L. Schwartz wrote:
    >>>>>> "cconnell" == cconnell 1 writes:

    >
    >cconnell> for i in cat | awk print column2 (for example)
    >
    > Why the unnecessary cat?
    >
    > Why not just:
    >
    > cat filename | cat | cat | cat | tr | cat | awk ...


    usage: tr [-csu] string1 string2
    tr [-cu] -d string1
    tr [-cu] -s string1
    tr [-cu] -ds string1 string2

    --
    Chris F.A. Johnson, author |
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence

  8. Re: adding up field columns in a file

    While on the subject, if one wants to print selective columns in a file
    using awk, is it possible to do it without cat to make it faster?
    i.e I always pipe the output from a file into awk from cat, can you use
    awk on its own?
    Thanks


  9. Re: adding up field columns in a file

    cconnell_1@lycos.com wrote:
    > While on the subject, if one wants to print selective columns in a file
    > using awk, is it possible to do it without cat to make it faster?
    > i.e I always pipe the output from a file into awk from cat, can you use
    > awk on its own?


    Yes. In general, you can almost always eliminate redundant "cat"s
    by changing this form:

    cat foo | bar

    into this form:

    bar < foo

    With awk, there is an even shorter way, because awk can take an input
    filename as a command line argument directly (just like cat does), so
    you can change this

    cat /etc/passwd | awk -F: '{print $1}'

    into this:

    awk -F: '{print $1}' /etc/passwd

    You could of course use redirection so that the shell opens the file
    and makes it awk's standard input:

    awk -F: '{print $1}' < /etc/passwd

    but that more typing.

    - Logan

  10. Re: adding up field columns in a file

    On 2006-01-24, cconnell_1@lycos.com wrote:
    > While on the subject, if one wants to print selective columns in a file
    > using awk, is it possible to do it without cat to make it faster?
    > i.e I always pipe the output from a file into awk from cat, can you use
    > awk on its own?


    Did you look at the solutions you were given? Don't they answer
    your question?

    --
    Chris F.A. Johnson, author |
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence

  11. Re: adding up field columns in a file

    cconnell_1@lycos.com wrote:
    >
    > While on the subject, if one wants to print selective columns in a file
    > using awk, is it possible to do it without cat to make it faster?
    > i.e I always pipe the output from a file into awk from cat, can you use
    > awk on its own?


    A large percentage of the time using cat to prime a pipeline
    will end up being labelled "gratitous use of cat". You should
    nearly always be able to avoid it with I/O redirection other
    than a pipe, putting filenames on the command line and
    so on.

    And still, after decades of writing scripts I often prime a
    pipeline with a cat. I especially like cat ... | while ... Why?
    Stylistic reasons. I don't like while ... one < file


  12. Re: adding up field columns in a file

    Thats great to know. Thanks for the help, I will experiment with it.
    And Chris yes your answer did answer my question as there was the
    filename at the end of the awk statement- doh!


  13. Re: adding up field columns in a file

    cconnell_1@lycos.com wrote:
    > Hello,
    > I have a file with some columns of numeric data e.g.
    >
    > 5 5
    > 5 6
    > 5 7
    >
    > I need to total them, currently im using the following method


    you can sum up the 2nd column by:
    awk '{t+=$2}END{print t}' FILENAME

    sum up two columns:
    awk '{t1+=$1;t2+=$2}END{print "total:",t1,t2}' FILENAME

    sum up all columns:
    perl -alne '$t[$_]+=$F[$_]for(0..$#F);END{print "total: @t\n"}'
    FILENAME

    Xicheng


+ Reply to Thread