Transaction processing revisited - Protocols

This is a discussion on Transaction processing revisited - Protocols ; It is an increasingly common question: how to upload a file in such a way that another process won't try to process it before the upload is complete? This sort of operation -- in which a variety of clients upload ...

+ Reply to Thread
Results 1 to 2 of 2

Thread: Transaction processing revisited

  1. Transaction processing revisited

    It is an increasingly common question: how to upload a file in such
    a way that another process won't try to process it before the upload
    is complete? This sort of operation -- in which a variety of clients
    upload transactions (files) to a central service for processing --
    is commonly used in EDI applications, insurance claims, etc. You want
    to be sure that each transaction is processed exactly once; not zero,
    or two or more times, and that processing takes place only when the
    upload is complete and successful.

    Discussions of transaction processing have been available on the
    Kermit website for some time:

    http://www.columbia.edu/kermit/case10.html
    http://www.columbia.edu/kermit/ftpscripts.html

    The second one concerns FTP but the principles are the same -- the
    transaction processing example is about halfway down the document, after
    the introductory tutorial material.

    In these documents we talk about uploading files to a temporary working
    directory and the moving each file to its final destination only after
    the upload is complete and successful.

    In some situations, however, it is possible that multiple clients might
    try to upload different files having the same name to the same working
    directory at the same time, and although Kermit's file collision features
    can be used to prevent one file from destroying the other, it complicates
    the moving or renaming process.

    Another approach is for the server to ensure that each incoming file has
    a unique name. This way, no collisions will ever take place and no
    renaming within the temporary working directory will be necessary.
    This can be done with the following command:

    set receive rename-to xxx

    where xxx is a template, as explained here:

    http://www.columbia.edu/kermit/ckermit70.html#x4.1

    Let's say the incoming filename is ABCD1234.FIL, and you want the new name
    to incorporate the original name and still have the ".FIL" extension. Here
    are some variables you can use:

    \v(filename) = the name the file was sent was (e.g. ABCD1234.FIL).
    \v(ntime) = current local time, seconds since midnight (e.g. 45369).
    \v(ndate) = current date, numeric (e.g. 20040716).
    \v(pid) = Kermit's numeric process ID.

    The date, time, and PID combined would produce a guaranteed unique filename
    because at any given moment, every process (including every Kermit receiver)
    has a unique process ID.

    How to construct the new filename? Let's assume that the filename contains
    only one dot (.). First use string functions to separate the basename from
    the extension:

    \fstripx(\v(filename)) = ABCD1234
    \flop(\v(filename)) = FIL

    Then concatenate with the date, time, and pid; you can use \flpad() to
    left-pad variable-length fields (like \v(ntime) or \v(pid)) if you want to
    make them fixed-length. So here is your renaming function:

    \fstripx(\v(filename))_\v(ndate)_\flpad(\v(ntime), 5,0)_\v(pid).-
    \flop(\v(filename))

    (This is a single line, which I've broken for benefit of netnews. Rejoin
    by removing the hyphen and the linebreak and indentation.)

    I've separated the fields with underscores but of course you don't have to
    do that, or you can use a different separator. The command would be:

    set receive rename-to -
    \fstripx(\v(filename))_\v(ndate)_\flpad(\v(ntime), 5,0)_\v(pid).-
    \flop(\v(filename))

    which you would execute before putting the Kermit into receive or server
    mode. With this command in effect, a file called ABCD1234.FIL might be
    received as ABCD1234_20040721_04739_632.FIL. Your SET FILE COLLISION
    setting should be irrelevant because you won't have any collisions.

    If you also want to move the file to a different directory at the same
    time you rename it, you can include that in the string, e.g.:

    set receive rename-to ../Ready/\fstripx(\v(filename))_\v(ndate)-
    _\flpad(\v(ntime),5,0)_\v(pid).\flop(\v(filename))

    - Frank

  2. Re: Transaction processing revisited


    "Frank da Cruz" wrote in message
    news:slrncfth02.t2m.fdc@sesame.cc.columbia.edu...
    > It is an increasingly common question: how to upload a file in such
    > a way that another process won't try to process it before the upload
    > is complete? This sort of operation -- in which a variety of clients
    > upload transactions (files) to a central service for processing --
    > is commonly used in EDI applications, insurance claims, etc. You want
    > to be sure that each transaction is processed exactly once; not zero,
    > or two or more times, and that processing takes place only when the
    > upload is complete and successful.


    > - Frank


    It must me my morning to grouse.
    I've noticed several postings on this subject on this group
    and others in the last few weeks.

    Seem to me that folks are looking for something like:

    connect
    do transaction
    quit


    How come we pay good programmers to write easy programs
    like accounts receivable and payroll and airline reservations.
    We provide them with training, detail specifications, and constant
    supervision.

    The same organization will then assign a total newbee to construct
    a complex data communications system which exchanges critical
    business data among incompatable computers with no more than
    a CD (K-95, Procomm, PCAnywhere etc) and some general outline
    of the system goals.


    Regards...Dan.



+ Reply to Thread