Synchronizing Multiple Computers - Unix
This is a discussion on Synchronizing Multiple Computers - Unix ; These days parallel programming seems to be growing in popularity,
especially with the advent of dual and quad-core processors. I've
spent a lot of time experimenting with various methods of parallel
programming, such as multi-threaded and multi-process programming, and
as ...
-
Synchronizing Multiple Computers
These days parallel programming seems to be growing in popularity,
especially with the advent of dual and quad-core processors. I've
spent a lot of time experimenting with various methods of parallel
programming, such as multi-threaded and multi-process programming, and
as such have also experimented with various methods of synchronization
such as mutexes, semaphores, and file locks.
But there are some projects which require massively parallel execution
in order to do a job efficiently, (such as building an inverted index
for example) and this usually means employing multiple computers. Of
course, that also means coming up with ways to synchronize jobs
between computers. As far as I know, there's no real standardized way
to do this, and so I suppose most people implement it through some
kind of customized network communication.
One possible way that occurred to me to implement multiple computer
synchronization was to create a sort of "mutex server." A mutex
server would just basically be a server which sits and waits for lock
requests from clients. Each client atomically connects to the mutex
server, and then requests a lock. Once a client gets a lock, it can
then get access to some shared resource, such as a file on a shared
NFS directory or something.
That seems to me to be the simplest way to synchronize multiple
computers, but I haven't really explored the problem in depth. I'd
appreciate any comments, insights, anecdotes, or links to information
resources that discuss this issue.
Thanks.
-
Re: Synchronizing Multiple Computers
In article <1180898577.252415.206950@q66g2000hsg.googlegroups. com>,
chsalvia@gmail.com wrote:
> One possible way that occurred to me to implement multiple computer
> synchronization was to create a sort of "mutex server." A mutex
> server would just basically be a server which sits and waits for lock
> requests from clients. Each client atomically connects to the mutex
> server, and then requests a lock. Once a client gets a lock, it can
> then get access to some shared resource, such as a file on a shared
> NFS directory or something.
>
> That seems to me to be the simplest way to synchronize multiple
> computers, but I haven't really explored the problem in depth. I'd
> appreciate any comments, insights, anecdotes, or links to information
> resources that discuss this issue.
Synchronization is one of the most difficult problems in distributed
computing, and has probably been the topic of hundreds of journal
articles over the past 30-40 years.
Your solution is simple, but not very robust (what if the mutex server
goes down) or high performance (all mutex operations are waiting on the
one server).
I suggest you take a look at the research on the subject to find more
advanced solutions.
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
-
Re: Synchronizing Multiple Computers
On Sun, 03 Jun 2007 15:55:43 -0400, Barry Margolin wrote:
> In article <1180898577.252415.206950@q66g2000hsg.googlegroups. com>,
> chsalvia@gmail.com wrote:
>> That seems to me to be the simplest way to synchronize multiple
>> computers, but I haven't really explored the problem in depth. I'd
>> appreciate any comments, insights, anecdotes, or links to information
>> resources that discuss this issue.
>
Synchronisation is an artifact of time.
> Synchronization is one of the most difficult problems in distributed
> computing, and has probably been the topic of hundreds of journal
> articles over the past 30-40 years.
Amen!
> Your solution is simple, but not very robust (what if the mutex server
> goes down) or high performance (all mutex operations are waiting on the
> one server).
>
> I suggest you take a look at the research on the subject to find more
> advanced solutions.
See, for instance, Lesley Lamport's writings:
http://research.microsoft.com/users/...pubs/pubs.html
(that will keep you busy for the next few years!)
HTH,
AvK
