Windows freeware unique sort technique for large text files (hosts) - Unix

This is a discussion on Windows freeware unique sort technique for large text files (hosts) - Unix ; Is there a way, using windows freeware, to sort unique a huge hosts file? I've concatonated all the freeware windows hosts files I can find into a single huge fifty-thousand line C:\Windows\System\Drivers\Etc\hosts file but the resulting hosts file is so ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 24

Thread: Windows freeware unique sort technique for large text files (hosts)

  1. Windows freeware unique sort technique for large text files (hosts)

    Is there a way, using windows freeware, to sort unique a huge hosts file?

    I've concatonated all the freeware windows hosts files I can find into a
    single huge fifty-thousand line C:\Windows\System\Drivers\Etc\hosts file
    but the resulting hosts file is so huge, replete with duplicates, that it's
    slowing down windows browsing.

    I would like to pare the hosts file to remove duplicates. How?

    I tried sorting with windows vim 7.1 freeware but I can't get the unique
    sort option to work inside of vim. What am I doing wrong?

    Here is a vim 7.1 command that works inside the huge hosts file:
    :%!sort (this sorts the huge windows hosts file just fine)

    This vim 7.1 sort unique command should work but it does not:
    :%!sort -u (this is supposed to sort uniquely)

    The syntax is:
    : (begin a windows vim 7.1 command)
    !sort -u (run the following command "sort -u" inside of vim freeware)

    When I run ":!sort -u" inside of vim, it pares the hosts file down to
    a single (empty) line.

    Is there another free way to sort uniquely a large windows text file?

  2. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 11:00:26 -0700, Donita Luddington wrote:

    > This vim 7.1 sort unique command should work but it does not:
    > :%!sort -u (this is supposed to sort uniquely)


    This syntax means: execute external sort command with the unique
    option. It will most likely trigger Windows own sort command,
    which has no -u option. Therefore, it fails.

    Get a sort.exe which supports -u as part of the UnxUtils package:

    http://sourceforge.net/projects/unxutils

    An updated version is part of UnxUpdates.zip, which at the moment
    is not available via SourceForge. So you may get it here:

    http://www.weihenstephan.de/~syring/...UtilsDist.html

    When you're at it, you can execute it directly, giving your file
    as input source. (No need to bother Vim with this task.) If you
    start using UnxUtils on a more regular basis (they are worthwhile),
    you may consider to copy all *.exe files into a directory and add
    this to your search path. I explained the how-to only yesterday.
    So you may have a look at:

    Message-ID:

    or look the posting up through Google Groups:

    http://groups.google.de/group/alt.co...621001ce95ea22

    HTH.
    BeAr
    --
    ================================================== =========================
    = What do you mean with: "Perfection is always an illusion"? =
    ================================================== =============--(Oops!)===

  3. Re: Windows freeware unique sort technique for large text files (hosts)

    [FollowUp-To: alt.comp.freeware]

    Donita Luddington wrote:

    > Newsgroups: alt.comp.freeware,comp.unix.questions,comp.editors
    >



    When you do crosspost you should set followup-to


    > From: Donita Luddington
    > NNTP-Posting-Host: 69.110.47.74
    > Message-ID:
    >
    > Is there a way, using windows freeware, to sort unique a huge
    > hosts file?
    >
    > I've concatonated all the freeware windows hosts files I can find
    > into a single huge fifty-thousand line
    > C:\Windows\System\Drivers\Etc\hosts file but the resulting hosts
    > file is so huge, replete with duplicates, that it's slowing down
    > windows browsing.
    >
    > I would like to pare the hosts file to remove duplicates. How?
    >



    Start with pushing concatenated file through rpsort

    rpsort /q /d /n < infile > outfile

    Then view outfile for "semi-duplicate" entries to
    determine if further processing necessary.

    e.g. spacing, comment, or... something else differences

    --
    OpenPGP: id=18795161E22D3905; preference=signencrypt;
    url=http://guysalias.fateback.com/pgpkeys.txt

  4. Re: Windows freeware unique sort technique for large text files (hosts)

    "Donita Luddington" wrote in message
    news:Oi1lk.6166$np7.5749@flpi149.ffdc.sbc.com...
    > Is there a way, using windows freeware, to sort unique a huge hosts file?
    >
    > I've concatonated all the freeware windows hosts files I can find into a
    > single huge fifty-thousand line C:\Windows\System\Drivers\Etc\hosts file
    > but the resulting hosts file is so huge, replete with duplicates, that
    > it's
    > slowing down windows browsing.
    >
    > I would like to pare the hosts file to remove duplicates. How?
    >
    > I tried sorting with windows vim 7.1 freeware but I can't get the unique
    > sort option to work inside of vim. What am I doing wrong?
    >
    > Here is a vim 7.1 command that works inside the huge hosts file:
    > :%!sort (this sorts the huge windows hosts file just fine)
    >
    > This vim 7.1 sort unique command should work but it does not:
    > :%!sort -u (this is supposed to sort uniquely)
    >
    > The syntax is:
    > : (begin a windows vim 7.1 command)
    > !sort -u (run the following command "sort -u" inside of vim freeware)
    >
    > When I run ":!sort -u" inside of vim, it pares the hosts file down to
    > a single (empty) line.
    >
    > Is there another free way to sort uniquely a large windows text file?



    I'd use a Linux VM.

    Then it's a simple

    tr '[A-Z]' '[a-z]' < file.txt | sort | uniq > file2.txt





  5. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 20:01:20 +0100, John Stubbings wrote:
    > tr '[A-Z]' '[a-z]' < file.txt | sort | uniq > file2.txt


    Hey John,
    I guess your command does this

    tr (translate character by character)
    '[A-Z]' (from any character in the set of capital A to capital Z)
    '[a-z]' (to the corresponding character in the set of small a to small z)
    < file.txt (from the input file "file.txt")
    | (pipe that output to the next command)
    sort (sort alphabetically those results)
    | (pipe that output to the next command)
    uniq (uniquify the results, line by line,
    > file2.txt (and save as "file2.txt)


    If I had Linux installed, I'd run that in a flash but I don't have any
    Linux and, I guess, it would take many hours to install so I'm looking for
    a native windows freeware solution instead.

    There must be a way to uniquify a file from within vi freeware on windows.

  6. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 20:29:35 +0200, B. R. 'BeAr' Ederson wrote:

    >> This vim 7.1 sort unique command should work but it does not:
    >> :%!sort -u (this is supposed to sort uniquely)

    >
    > This syntax means: execute external sort command with the unique
    > option. It will most likely trigger Windows own sort command,
    > which has no -u option. Therefore, it fails.


    Hey BeAr,
    I did not realize this was calling a windows "sort.exe" command.
    I thought it was calling a "vim" sort command.

    I just looked to see if the windows sort command has a "unique" option.

    c:\> sort /? > c:\temp\sort.txt
    c:\> type c:\temp\sort.txt
    SORT [/R] [/+n] [/M kilobytes] [/L locale] [/REC recordbytes]
    [[drive1:][path1]filename1] [/T [drive2:][path2]]
    [/O [drive3:][path3]filename3]
    /+n Specifies the character number, n, to
    begin each comparison. /+3 indicates that
    each comparison should begin at the 3rd
    character in each line. Lines with fewer
    than n characters collate before other lines.
    By default comparisons start at the first
    character in each line.
    /L[OCALE] locale Overrides the system default locale with
    the specified one. The ""C"" locale yields
    the fastest collating sequence and is
    currently the only alternative. The sort
    is always case insensitive.
    /M[EMORY] kilobytes Specifies amount of main memory to use for
    the sort, in kilobytes. The memory size is
    always constrained to be a minimum of 160
    kilobytes. If the memory size is specified
    the exact amount will be used for the sort,
    regardless of how much main memory is
    available.

    The best performance is usually achieved by
    not specifying a memory size. By default the
    sort will be done with one pass (no temporary
    file) if it fits in the default maximum
    memory size, otherwise the sort will be done
    in two passes (with the partially sorted data
    being stored in a temporary file) such that
    the amounts of memory used for both the sort
    and merge passes are equal. The default
    maximum memory size is 90% of available main
    memory if both the input and output are
    files, and 45% of main memory otherwise.
    /REC[ORD_MAXIMUM] characters Specifies the maximum number of characters
    in a record (default 4096, maximum 65535).
    /R[EVERSE] Reverses the sort order; that is,
    sorts Z to A, then 9 to 0.
    [drive1:][path1]filename1 Specifies the file to be sorted. If not
    specified, the standard input is sorted.
    Specifying the input file is faster than
    redirecting the same file as standard input.
    /T[EMPORARY]
    [drive2:][path2] Specifies the path of the directory to hold
    the sort's working storage, in case the data
    does not fit in main memory. The default is
    to use the system temporary directory.
    /O[UTPUT]
    [drive3:][path3]filename3 Specifies the file where the sorted input is
    to be stored. If not specified, the data is
    written to the standard output. Specifying
    the output file is faster than redirecting
    standard output to the same file.


    You're right. There is no "-unique" option to the windows sort command.
    There must be a unique option inside of a windows text editor somewhere!

  7. Re: Windows freeware unique sort technique for large text files (hosts)

    Donita Luddington:

    > Is there another free way to sort uniquely a large windows text file?


    Notepad++ with the TextFX plug-in. I am not sure about the maximun
    file size it can handle though.
    http://notepad-plus.sourceforge.net/uk/site.htm

    Yrrah

  8. Re: Windows freeware unique sort technique for large text files (hosts)

    Donita Luddington wrote:

    > The syntax is:
    > : (begin a windows vim 7.1 command)
    >!sort -u (run the following command "sort -u" inside of vim freeware)
    >



    If you want to stick with vi try...

    :%!sort |uniq

    (yes, there is space between external "sort" and the pipe "|")

    But you may still have "semi-duplicate" lines.

    e.g.

    0.0.0.0 example.com
    0.0.0.0 example.com
    0.0.0.0 example.com #bad domain

    --
    OpenPGP: id=18795161E22D3905; preference=signencrypt;
    url=http://guysalias.fateback.com/pgpkeys.txt

  9. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 12:35:29 -0700, Donita Luddington wrote:

    > There must be a way to uniquify a file from within vi freeware on windows.


    I found these pointers for removing duplicate lines in vi
    http://rayninfo.co.uk/vimtips.html
    :%s/^\(.*\)\n\1$/\1/ : delete duplicate lines

    http://www.vim.org/tips/tip.php?tip_id=305
    :%s/^\(.*\)\n\1/\1$/ delete duplicate lines

    But, executed in vim 7.1 on Windows, this syntax returns an error.
    I'll try some more after I figure out what it's trying to do.

    I think it's trying to do this:
    :%s/ ==> search the whole file for
    ^ ==> the beginning of a line
    \(.*\) ==> any character and any number of characters after that
    \n ==> a newline character
    \1 ==> what you found the first time (ie, the whole line)
    / ==> replace it with
    \1 ==> what you found that first time (again, the original line)
    $ ==> I'm not sure why this "end of line" character is needed
    / ==> I'm not sure why this "end of command" character is needed

    Before I debug why removal of duplicate lines in Lemmy and Vim freeware
    isn't working, can you tell me if I have the command to do so right?


  10. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 12:35:29 -0700, Donita Luddington wrote:

    > If I had Linux installed, I'd run that in a flash but I don't have any
    > Linux and, I guess, it would take many hours to install so I'm looking for
    > a native windows freeware solution instead.
    >
    > There must be a way to uniquify a file from within vi freeware on windows.


    As I wrote: Get an appropriate sort command. You can even replace
    the Windows one, as long as you don't need exactly this implementation
    for some purpose. (Better set up a dedicated UnxUtils directory with
    entry in the search path, though.)

    Btw.: All the tools John suggested are part of the UnxUtils package.
    So it shouldn't take "hours" to follow his good advice... ;-)

    Notepad++ - as Yrrah suggested - should do fine in general. But it
    gets awfully slow when sorting huge files. That's the case with
    most internal editor solutions.

    The UnxUtils variant (either sort -u alone or (better) piped
    through tr, beforehand, as John suggested) should take barely
    a second.

    When the UnxUtils sort will be found in search path *before* the
    Windows one (or *instead*, if you just replace it), your Vim
    command should do fine. (If it has to be an "editor solution".)

    BeAr
    --
    ================================================== =========================
    = What do you mean with: "Perfection is always an illusion"? =
    ================================================== =============--(Oops!)===

  11. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 13:25:30 -0700, Donita Luddington wrote:

    >> There must be a way to uniquify a file from within vi freeware on windows.

    >
    > I found these pointers for removing duplicate lines in vi
    > http://rayninfo.co.uk/vimtips.html
    >:%s/^\(.*\)\n\1$/\1/ : delete duplicate lines
    >
    > http://www.vim.org/tips/tip.php?tip_id=305
    >:%s/^\(.*\)\n\1/\1$/ delete duplicate lines
    >
    > But, executed in vim 7.1 on Windows, this syntax returns an error.


    Try this:

    :%s/^\([^\n]*\)\n\1$/\1/

    Please note, that you have to sort the file *beforehand*! The above
    will only remove *consecutive* duplicate lines. And if you don't
    have any consecutive duplicate lines, you *will* get an "error".
    (Pattern not found.)

    BeAr
    --
    ================================================== =========================
    = What do you mean with: "Perfection is always an illusion"? =
    ================================================== =============--(Oops!)===

  12. Re: Windows freeware unique sort technique for large text files (hosts)

    Donita Luddington has brought this to us :
    > Is there a way, using windows freeware, to sort unique a huge hosts file?


    Here is some info, that may interest.

    Hosts File
    http://home.comcast.net/~SupportCD/XPMyths.html
    Myth - "Special AntiSpyware Hosts Files are necessary to prevent
    Spyware infections."
    Reality - "Using Special AntiSpyware Hosts Files are a waste of time
    and leads to a false sense of security. Any Malware/Spyware can easily
    modify the Hosts File at will, even if it is set to Read-only. It is
    impossible to "lock-down" a Hosts File unless you are running as a
    limited user which makes using it in this case irrelevant anyway.
    Various Malware/Spyware uses the Hosts File to redirect your Web
    Browser to other sites. They can also redirect Windows to use a Hosts
    File that has nothing to do with the one you keep updating. The Hosts
    file is an archaic part of networking setups that was originally meant
    to be used on a LAN and was the legacy way to look up Domain Names on
    the ARPANET. It tells a PC the fixed numeric address of the internal
    server(s) so the PC doesn't have to go looking for them through all
    possible addresses. It can save time when "discovering" a LAN. I don't
    consider 1970's ARPANET technology useful against modern
    Malware/Spyware. When cleaning Malware/Spyware from a PC, it is much
    easier to check a clean Hosts File then one filled with thousands of
    lines of addresses. Considering how easily a Hosts File can be
    exploited, redirected and potentially block good sites, it is strongly
    recommended NOT to waste time using Special Hosts Files. Especially
    when proper Malware/Spyware protection can be achieved by simply using
    these steps, all without ever using a Hosts File."



  13. Re: Windows freeware unique sort technique for large text files(hosts)

    On Aug 2, 2:00*pm, Donita Luddington wrote:
    > Is there a way, using windows freeware, to sort unique a huge hosts file?
    >
    > I've concatonated all the freeware windows hosts files I can find into a
    > single huge fifty-thousand line C:\Windows\System\Drivers\Etc\hosts file
    > but the resulting hosts file is so huge, replete with duplicates, that it's
    > slowing down windows browsing.
    >
    > I would like to pare the hosts file to remove duplicates. How?
    >
    > I tried sorting with windows vim 7.1 freeware but I can't get the unique
    > sort option to work inside of vim. What am I doing wrong?
    >
    > Here is a vim 7.1 command that works inside the huge hosts file:
    > * :%!sort *(this sorts the huge windows hosts file just fine)
    >
    > This vim 7.1 sort unique command should work but it does not:
    > * :%!sort -u (this is supposed to sort uniquely)
    >
    > The syntax is:
    > : * *(begin a windows vim 7.1 command)
    > !sort -u *(run the following command "sort -u" inside of vim freeware)
    >
    > When I run ":!sort -u" inside of vim, it pares the hosts file down to
    > a single (empty) line.
    >
    > Is there another free way to sort uniquely a large windows text file?


    For the best info on HOSTS files and managing them I have found this
    site : http://www.mvps.org/winhelp2002/hosts.htm to be very useful.
    Not only do they publish a very capable HOSTS file, they have free and
    non-free software listed that will allow you to manage your HOSTS
    file. As well there are several other tips and tricks that I find
    useful.

    The Carnie

  14. Re: Windows freeware unique sort technique for large text files (hosts)

    Donita Luddington wrote:

    > Is there a way, using windows freeware, to sort unique a huge hosts file?
    >
    > I've concatonated all the freeware windows hosts files I can find into a
    > single huge fifty-thousand line C:\Windows\System\Drivers\Etc\hosts file
    > but the resulting hosts file is so huge, replete with duplicates, that
    > it's
    > slowing down windows browsing.
    >
    > I would like to pare the hosts file to remove duplicates. How?
    >
    > I tried sorting with windows vim 7.1 freeware but I can't get the unique
    > sort option to work inside of vim. What am I doing wrong?
    >
    > Here is a vim 7.1 command that works inside the huge hosts file:
    > :%!sort (this sorts the huge windows hosts file just fine)
    >
    > This vim 7.1 sort unique command should work but it does not:
    > :%!sort -u (this is supposed to sort uniquely)
    >
    > The syntax is:
    > : (begin a windows vim 7.1 command)
    > !sort -u (run the following command "sort -u" inside of vim freeware)
    >
    > When I run ":!sort -u" inside of vim, it pares the hosts file down to
    > a single (empty) line.
    >
    > Is there another free way to sort uniquely a large windows text file?


    Unduplicate, downloadable from http://adriancarter.homestead.com/
    might be able to do it, depending on how wide are the lines in your
    file. You mention a 50,000 line file; to test Unduplicate just now
    I created ~65,500 random lines in Excel, with a high degree of
    duplication, then copied to the clipboard. Unduplicate reduced it
    to about 9900 unique values in less than 10 seconds. I then created
    the same data but in ~130,000 lines of a text file, and it didn't take
    much longer.
    I'm away from my development setup at present, using an old slow
    early XP system with 250Mb memory. The reason I can't give
    exact timings is that Unduplicate gives no signal after it has done
    its thing with the clipboard. A weakness I intend to remedy as soon
    as I get back home. But it will probably work for you - you just
    have to select all in an editor, copy, click on the Unduplicate tray
    icon, wait a while, then paste.

    --
    beerwolf



  15. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 11:00:26 -0700,
    Donita Luddington wrote:
    > Is there a way, using windows freeware, to sort unique a huge hosts file?
    >
    > I've concatonated all the freeware windows hosts files I can find into a
    > single huge fifty-thousand line C:\Windows\System\Drivers\Etc\hosts file
    > but the resulting hosts file is so huge, replete with duplicates, that it's
    > slowing down windows browsing.


    I suspect that even with removing all the duplicates you'll still end up
    with a file that's a tad big for the usual hosts lookup implementation.
    Likely, each lookup will end up reading the entire file line-by-line
    until the first hit or end-of-file, whichever comes first.

    I think you may need to look at a better solution; firefox with adblock
    for example. I assume but have not verified whether adblock's lookup is
    faster, mind. I do know that abusing the hosts file for keeping huge
    blacklists is more likely to hurt than to help, and not just in slowness.


    > I would like to pare the hosts file to remove duplicates. How?


    The easy way for someone with unix experience is to run it through sort,
    then uniq. Various editors (emacs, vi(m), probably more) can do it too.
    Various ways for obtaining a unix toolset have already been mentioned.

    There is a freeware windows implementation available of the programming
    (scripting, really) language ``awk''[awk]. The installation consists of
    fetching a single executable and putting it somewhere convenient, then
    run it with the appropriate arguments (program to execute or file where
    the program to execute resides, input files, perhaps output redirection).

    Implementing sort in it would be a bit involved, but an in-place ``uniq''
    that doesn't need sorting turns out to be easy. In a dos-box, run:

    awk '!_[$0]++' inputfile > outputfile

    On unix shells you may need to escape the !, but I don't think you need
    to on a windows command line, though I'm not sure just how it handles
    quoting. This is a bit of a hack in that it is nigh-on unreadable for a
    beginner, so let me reassure you that it is entirely possible to write
    very readable awk programs. It has been deployed with success as a
    language for non-programmers, in fact.


    [awk] http://plan9.bell-labs.com/cm/cs/awkbook/ which links to
    http://plan9.bell-labs.com/cm/cs/who/bwk/awk95.exe

    --
    j p d (at) d s b (dot) t u d e l f t (dot) n l .
    This message was originally posted on Usenet in plain text.
    Any other representation, additions, or changes do not have my
    consent and may be a violation of international copyright law.

  16. Re: Windows freeware unique sort technique for large text files (hosts)

    Hi Guys,

    By way of update, I followed Bear's and others' original advice and was
    able to sort the now fifty-thousand line hosts file in about a second or
    two on Windows.

    What I did was add native Win32 port of the UnixUtils at
    http://unxutils.sourceforge.net to my WinXP laptop.

    This created c:\bin and c:\usr and, more specifically
    C:\usr\local\wbin\sort.exe

    Thanks to you, this more powerful sort, containing the "unique" and "ouput"
    -u and -o options is part of my Windows command-line repertoire.

    It wasn't at first obvious (to me), but, Wikipedia helped with syntax:
    http://en.wikipedia.org/wiki/Sort_(Unix)

    For others, here's the command to pare down the hosts file after you've
    combined all those hosts files you can find on the Internet using sort:

    Start->Run->cmd
    type c:\windows\system32\drivers\etc\hosts | c:\usr\local\wbin\sort.exe -u
    -o c:\windows\system32\drivers\etc\hosts

    The only manual change needed was to move this line back to the top:
    127.0.0.1 localhost # this needs to be the first line for some reason

    Do you know if sort can be told to sort all but the first line?

    It would be nice if the sort command could sort from line 2 to the end so
    that the extra step of moving the localhost line wasn't needed.

  17. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sat, 2 Aug 2008 22:27:18 +0200, B. R. 'BeAr' Ederson wrote:

    > (Better set up a dedicated UnxUtils directory with
    > entry in the search path, though.)


    Thanks Bear!

    Your unxutils advice worked beautifully.

    Start->Run->cmd
    type c:\windows\system32\drivers\etc\hosts | c:\usr\local\wbin\sort.exe -u
    -o c:\windows\system32\drivers\etc\hosts

    The only manual change needed was to move this line back to the top:
    127.0.0.1 localhost # this needs to be the first line for some reason

    I'm digging for the sort command that only sorts from the second line down
    but haven't found it yet.

  18. Re: Windows freeware unique sort technique for large text files(hosts)

    On Sun, 17 Aug 2008 08:16:14 -0700, Donita Luddington
    wrote:

    (...)
    >
    > Start->Run->cmd
    > type c:\windows\system32\drivers\etc\hosts | c:\usr\local\wbin\sort.exe
    > -u -o c:\windows\system32\drivers\etc\hosts
    >


    That qualifies for a UUOC (well, 'type' in this case).


    > The only manual change needed was to move this line back to the top:
    > 127.0.0.1 localhost # this needs to be the first line for some reason
    >
    > I'm digging for the sort command that only sorts from the second line
    > down but haven't found it yet.


    Am guessing there must be some variant/clone of 'sed' included in
    UnxUtils. If not, since you are so keen on calling sort from within vim,
    you can simply do -

    :2,$! C:\usr\local\wbin\sort -u

    - from within a vim session that is editing your hosts file.

    - Anand


  19. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sun, 17 Aug 2008 08:16:14 -0700, Donita Luddington wrote:

    > On Sat, 2 Aug 2008 22:27:18 +0200, B. R. 'BeAr' Ederson wrote:
    >
    >> (Better set up a dedicated UnxUtils directory with
    >> entry in the search path, though.)

    >
    > Thanks Bear!


    You're welcome. :-) Besides, it is BeAr, not Bear. ;-)

    > The only manual change needed was to move this line back to the top:
    > 127.0.0.1 localhost # this needs to be the first line for some reason
    >
    > I'm digging for the sort command that only sorts from the second line down
    > but haven't found it yet.


    The following command line should contain all commands in a one liner:

    sed "/127\.0\.0\.1/d" hosts | tr '[A-Z]' '[a-z]' | sort -u | sed "1i127.0.0.1 localhost" > hosts

    If UnxUtils are not part of the PATH search string, all utilities
    need to be called with fully qualified name. The "hosts" entries
    have to be substituted with the full name including directory
    components, if the command is not executed from the directory
    containing that hosts file. (Which would be easier...)

    Although the above should work fine, it usually is better to create
    a hosts.new file first and rename it afterwards. But that's up to you.

    There are other ways to do the above. I settled with deleting lines
    containing the localhost (127.0.0.1) entries instead of just preserving
    the first line, because the merging of several hosts files may result
    in more than one localhost line...

    HTH.
    BeAr
    --
    ================================================== =========================
    = What do you mean with: "Perfection is always an illusion"? =
    ================================================== =============--(Oops!)===

  20. Re: Windows freeware unique sort technique for large text files (hosts)

    On Sun, 17 Aug 2008 18:19:56 +0200 (CEST), Anand Hariharan wrote:

    >> The only manual change needed was to move this line back to the top:
    >> 127.0.0.1 localhost # this needs to be the first line for some reason
    >>
    >> I'm digging for the sort command that only sorts from the second line
    >> down but haven't found it yet.

    >
    > Am guessing there must be some variant/clone of 'sed' included in
    > UnxUtils.


    There is. ;-)

    BeAr
    --
    ================================================== =========================
    = What do you mean with: "Perfection is always an illusion"? =
    ================================================== =============--(Oops!)===

+ Reply to Thread
Page 1 of 2 1 2 LastLast