How to merge two text files - Linux

This is a discussion on How to merge two text files - Linux ; Hi Everyone, I wrote a script to merge files while read line; do fgrep "$line" $FILE1 >tmp result=$? if [ $result -eq 1 ]; then echo $line >> $1 fi done it works fine but some times it repeats few ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: How to merge two text files

  1. How to merge two text files

    Hi Everyone,

    I wrote a script to merge files

    while read line; do
    fgrep "$line" $FILE1 >tmp
    result=$?

    if [ $result -eq 1 ]; then
    echo $line >> $1
    fi
    done < "$FILE2"

    it works fine but some times it repeats few lines.
    Is there some other utility to merge two files non-interactively.
    What I need is common lines and different lines from two files to a
    third file.

    Thanks,
    Ratnesh.


  2. Re: How to merge two text files

    Ratnesh Maurya coughed up some electrons that declared:

    > Hi Everyone,
    >
    > I wrote a script to merge files
    >
    > while read line; do
    > fgrep "$line" $FILE1 >tmp
    > result=$?
    >
    > if [ $result -eq 1 ]; then
    > echo $line >> $1
    > fi
    > done < "$FILE2"
    >
    > it works fine but some times it repeats few lines.
    > Is there some other utility to merge two files non-interactively.
    > What I need is common lines and different lines from two files to a
    > third file.
    >
    > Thanks,
    > Ratnesh.


    Hi

    If you don't mind having sorted lines, you could just use a GNU (and
    otherwise) command called

    comm

    If you really want it in perl, or with original line order preserved, just
    say so - I'll have another go.

    Cheers

    Tim

    PS

    > man comm

    COMM(1) User Commands
    COMM(1)



    NAME
    comm - compare two sorted files line by line

    SYNOPSIS
    comm [OPTION]... FILE1 FILE2

    DESCRIPTION
    Compare sorted files FILE1 and FILE2 line by line.

    With no options, produce three-column output. Column one
    contains
    lines unique to FILE1, column two contains lines unique to FILE2,
    and
    column three contains lines common to both files.

    -1 suppress lines unique to FILE1

    -2 suppress lines unique to FILE2

    -3 suppress lines that appear in both files

  3. Re: How to merge two text files

    On 18 Jul., 07:31, Ratnesh Maurya wrote:
    > What I need is common lines and different lines from two files to a
    > third file.

    Just to understand your request:
    - You have two files file1 and file2.
    - There can be missing lines or additional lines (compared
    to file1) in file2.
    - The result file (file3) should contain all lines from file1 and
    file2.
    - If a line is in file1 and file2 the result file3 should contain the
    line once.

    If file3 can be sorted I suggest:

    cat file1 file2 | sort | uniq > file3

    Otherwise it is not so simple. When a little script
    compares the files line by line it has a problem.
    If two lines differ it can be an insertion or deletion.
    To find out which, some read ahead is necessary.
    (the diff utility works with read ahead to find this out)
    So maybe you use the output of diff as base.
    Something like: Do the insertions and ignore the
    removes.

    Greetings Thomas Mertes

    Seed7 Homepage: http://seed7.sourceforge.net
    Seed7 - The extensible programming language: User defined statements
    and operators, abstract data types, templates without special
    syntax, OO with interfaces and multiple dispatch.


  4. Re: How to merge two text files

    On 18 Jul., 07:31, Ratnesh Maurya wrote:
    > Hi Everyone,
    >
    > I wrote a script to merge files
    >
    > while read line; do
    > fgrep "$line" $FILE1 >tmp
    > result=$?
    >
    > if [ $result -eq 1 ]; then
    > echo $line >> $1
    > fi
    > done < "$FILE2"

    While I would never write something in perl, I wrote a
    Seed7 program which can merge two files, when there
    are small inserts and deletes:

    $ include "seed7_05.s7i";
    include "getf.s7i";

    const integer: LIMIT is 10;

    const func integer: pos (in array string: mainArray,
    in string: searched, in integer: fromIndex) is func
    result
    var integer: result is 0;
    local
    var integer: index is 0;
    begin
    for index range fromIndex to fromIndex + LIMIT do
    if result = 0 and
    index <= length(mainArray) and
    searched = mainArray[index] then
    result := index;
    end if;
    end for;
    end func;

    const proc: main is func
    local
    var array string: file1 is 0 times "";
    var array string: file2 is 0 times "";
    var array string: file3 is 0 times "";
    var integer: idx1 is 1;
    var integer: idx2 is 1;
    var string: stri1 is "";
    var string: stri2 is "";
    var integer: found1 is 0;
    var integer: found2 is 0;
    begin
    if length(argv(PROGRAM)) <> 3 then
    writeln("usage: hi merge inFile1 inFile2 outFile3");
    else
    file1 := readf(argv(PROGRAM)[1]);
    file2 := readf(argv(PROGRAM)[2]);
    while idx1 <= length(file1) and idx2 <= length(file2) do
    stri1 := file1[idx1];
    stri2 := file2[idx2];
    if stri1 = stri2 then
    file3 &:= [] stri1;
    elsif idx1 < length(file1) and idx2 < length(file2) and
    file1[succ(idx1)] = file2[succ(idx2)] then
    file3 &:= [] stri1;
    file3 &:= [] stri2;
    else
    found1 := pos(file1, stri2, succ(idx1));
    found2 := pos(file2, stri1, succ(idx2));
    if found1 <> 0 then
    if found2 <> 0 then
    if found1 < found2 then
    file3 &:= file1[idx1 .. found1];
    idx1 := found1;
    else
    file3 &:= file2[idx2 .. found2];
    idx2 := found2;
    end if;
    else
    file3 &:= file1[idx1 .. found1];
    idx1 := found1;
    end if;
    else
    if found2 <> 0 then
    file3 &:= file2[idx2 .. found2];
    idx2 := found2;
    else
    file3 &:= [] stri1;
    file3 &:= [] stri2;
    end if;
    end if;
    end if;
    incr(idx1);
    incr(idx2);
    end while;
    writef(argv(PROGRAM)[3], file3);
    end if;
    end func;

    Greetings Thomas Mertes

    Seed7 Homepage: http://seed7.sourceforge.net
    Seed7 - The extensible programming language: User defined statements
    and operators, abstract data types, templates without special
    syntax, OO with interfaces and multiple dispatch.


+ Reply to Thread