Windows freeware unique sort technique for large text files (hosts) - Unix

This is a discussion on Windows freeware unique sort technique for large text files (hosts) - Unix ; On Sun, 17 Aug 2008 18:46:33 +0200, "B. R. 'BeAr' Ederson" wrote: (...) > >> The only manual change needed was to move this line back to the top: >> 127.0.0.1 localhost # this needs to be the first line ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 24 of 24

Thread: Windows freeware unique sort technique for large text files (hosts)

  1. Re: Windows freeware unique sort technique for large text files(hosts)

    On Sun, 17 Aug 2008 18:46:33 +0200, "B. R. 'BeAr' Ederson"
    wrote:

    (...)
    >
    >> The only manual change needed was to move this line back to the top:
    >> 127.0.0.1 localhost # this needs to be the first line for some reason
    >>
    >> I'm digging for the sort command that only sorts from the second line
    >> down but haven't found it yet.

    >
    > The following command line should contain all commands in a one liner:
    >
    > sed "/127\.0\.0\.1/d" hosts | tr '[A-Z]' '[a-z]' | sort -u | sed
    > "1i127.0.0.1 localhost" > hosts
    >


    Bad idea. My guess of how the OP is using the hosts file is to set
    the
    IP address of known malicious sites as 127.0.0.1. You'd at least
    want
    to append your sed's search expression with '[:space:]*localhost'
    before
    deleting *ALL* lines that contain 127.0.0.1.

    sort has a -f option, so the tr is not required.



    (...)
    > Although the above should work fine, it usually is better to create a
    > hosts.new file first and rename it afterwards. But that's up to you.
    >


    That's actually excellent advice, so much that, that should have been
    reflected in your command line. Had the OP used your above command
    line,
    he'd have lost all entries that corresponded to malicious web-sites
    in
    his hosts file.


    > There are other ways to do the above. I settled with deleting lines
    > containing the localhost (127.0.0.1) entries instead of just preserving
    > the first line, because the merging of several hosts files may result in
    > more than one localhost line...
    >


    I guess that explains your deleting lines containing 127.0.0.1, but
    there
    are a number of web sites out there (one even given in an else
    thread)
    that provide HOSTS files that redirect all requests to known
    malicious
    sites to 127.0.0.1.


    > HTH.
    > BeAr



  2. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sun, 17 Aug 2008 12:07:23 -0700 (PDT), Anand Hariharan wrote:

    >> The following command line should contain all commands in a one liner:
    >>
    >> sed "/127\.0\.0\.1/d" hosts | tr '[A-Z]' '[a-z]' | sort -u | sed "1i127.0.0.1 localhost" > hosts


    > Bad idea. My guess of how the OP is using the hosts file is to set the
    > IP address of known malicious sites as 127.0.0.1. You'd at least want
    > to append your sed's search expression with '[:space:]*localhost' before
    > deleting *ALL* lines that contain 127.0.0.1.


    You are absolutely right. :-( Actually, it should have been:

    sed "/^127\.0\.0\.1/d"...

    I thought about adding a filter for possible leading whitespace, since
    some hosts files are formatted this way. But the already long command
    line got a bit too unreadable. While deleting the whitespace class
    operator I must have killed the leading caret on accident... :-(

    > sort has a -f option, so the tr is not required.


    I used tr, because it already had been suggested in this thread and
    will produce nicer looking output. The mixed case result of the sort
    process using the -f option is probably harder to look through, if
    need arises.

    > Had the OP used your above command line, he'd have lost all entries that
    > corresponded to malicious web-sites in his hosts file.


    At least she would have a more manageable hosts file size. ;-)

    BeAr
    --
    ================================================== =========================
    = What do you mean with: "Perfection is always an illusion"? =
    ================================================== =============--(Oops!)===

  3. Re: Windows freeware unique sort technique for large text files (hosts)

    Donita Luddington writes:

    > Do you know if sort can be told to sort all but the first line?


    Usually people extract the line, and put it back.
    Here is a simple way to do this.

    head -1 hosts >hosts.head
    sed '1d' hosts | sort >hosts.rest
    mv hosts hosts.backup
    cat hosts.head hosts.rest >hosts
    rm hosts.head hosts.rest


  4. Re: Windows freeware unique sort technique for large text files(hosts)

    On Aug 17, 10:13*am, Donita Luddington wrote:

    > What I did was add native Win32 port of the UnixUtils athttp://unxutils.sourceforge.netto my WinXP laptop.
    >


    If you enjoyed doing this, I STRONGLY recommend instaling Cygwin on
    your PC. Since I discovered Cygwin a few years back, it's one of the
    first packages that I install on any Windows machine that I need to
    work with.
    Almost makes me forget that I'm on Windows!

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2