My employer completed migration of apps to Linux - Linux
This is a discussion on My employer completed migration of apps to Linux - Linux ; Jean-David Beyer wrote:
> dennis@home wrote:
>>
>> "Ignoramus31261" wrote in
>> message news:vs2dnaf7U-03zY_UnZ2dnUVZ_vninZ2d@giganews.com...
>>> On 2008-11-06, Maxwell Lol wrote:
>>>> Erik Funkenbusch writes:
>>>>
>>>>> On Wed, 05 Nov 2008 14:03:18 -0600, Ignoramus31261 wrote:
>>>>>
>>>>>>> Which is entirely ...
-
Re: My employer completed migration of apps to Linux
Jean-David Beyer wrote:
> dennis@home wrote:
>>
>> "Ignoramus31261" wrote in
>> message news:vs2dnaf7U-03zY_UnZ2dnUVZ_vninZ2d@giganews.com...
>>> On 2008-11-06, Maxwell Lol wrote:
>>>> Erik Funkenbusch writes:
>>>>
>>>>> On Wed, 05 Nov 2008 14:03:18 -0600, Ignoramus31261 wrote:
>>>>>
>>>>>>> Which is entirely different from async I/O.
>>>>>> Not "entirely". They are both nonblocking and do not use CPU.
>>>>> No. Async I/O is not "non-blocking". Non-blocking means that you
>>>>> call the I/O function and either it returns with data, or it
>>>>> returns with a return code indicating there is no data yet.
>>>> And when you call select with a timeout of zero, it is also
>>>> non-blocking.
>>> It is also easy to implement "async I/O" with select(). Just have a
>>> thread run select() and call those callbacks. (or many threads)
>>
>> As long as you don't care about the order the data is written! Are you
>> sure you have ever programmed anything? I get the idea you are making
>> this stuff up as you have committed several errors so far that would make
>> most applications fail.
>>
> I wrote an OS once for a special purpose.
>
> The IO could do the regular IO similar to what the normal UNIX read(),
> write(). seek() would do. But for the really heavy-duty IO, where we
> were reading a television camera in real time, we could not do that
> because we could not stop when the machine was busy or we would lose
> data. So I put in additional constructs, such as (I no longer remember
> the exact syntax):
>
> startread(filedescriptor, &buffer, size, recordID);
> startwrite(filedescriptor, &buffer, size, recordID);
> result = iostatus(filedescriptor, &buffer);
>
> All three of these returned essentially immediately.
> The reason we needed these is that we double-buffered the IO and did not
> have the time between when one IO finished and the next one started to
> set up the device again. We put two controllers on each device and cold
> set tbe next one up when the first one finished. The startio functions
> did all the device setup work, put the commends in a FIFO queue, and as
> one finished with a hardware interrupt, the OS could start the next one
> quickly enough. It was easy enough to setup one controller during the
> time the IO transfer was occurring on the other controller. I called
> this model asynchronous IO because the IO took place in parallel with
> execution of the application; i.e., the application could continue to
> run even though it had issued one or more IO requests.
>
> In the UNIX model, this was impossible because once a read or a write
> was called, the application stalled until the IO was completed (what I
> would call serial IO) and only when the IO was complete could the nest
> read or write could be issued. The return of one of these IO functions
> implicitly informed the application that the operation was complete (or
> failed, as the case might be).
>
>
I suspect you had set up blocking writes then.
Normally you wouldn't block on writes unless you ad run out of buffer space.
as far as reads go, that is a question of writing a decent device driver
that can buffer enough data so that even if read relatively
infrequently, no data is lost. A big ring buffer is good.
Then do all the transfers under interrupt.
Linux the kernel is a slender shim between the user and the decvice
driver: if you want super performance, write a better device driver,
don't change OS!
Of purse MS has always got round crap device drivers by allowing you to
hit the hardware from pretty much anywhere. Whether that is a plus or a
minus depends on your religion.
-
Re: My employer completed migration of apps to Linux
On 2008-11-06, Johan Lindquist wrote:
> So anyway, it was like, 09:50 CET Nov 06 2008, you know? Oh, and, yeah,
> Christopher Hunter was all like, "Dude,
>> Johan Lindquist wrote:
>>> So anyway, it was like, 09:14 CET Nov 06 2008, you know? Oh, and, yeah,
>>> Christopher Hunter was all like, "Dude,
>
>>>> a) Games are best played on a dedicated games console.
>>>
>>> Do you play any computer games yourself?
>>
>> Not any more - I have better things to do with my time.
>
> This is the other common trait of non-gamers. Beside the "games are
> better played on a console" mantra, they all seem to share this
> condescending attitude that computer games are pointless and a waste
I would say that actual game programmers give consoles far more
credit than "PC gaming snobs" do. They are also in a much better
position to give a sound informed evaluation of what is being discussed
here.
[deletia]
> I mainly play Starcraft (eagerly waiting for Starcraft II right now),
> and I can simply not see how it would even be playable on a regular
> console, given that you have neither mouse nor keyboard to control
> gameplay.
[deltia]
--
Apple: because TRANS.TBL is an mp3 file. It really is! |||
/ | \
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
http://www.usenet.com
-
Re: My employer completed migration of apps to Linux
The Natural Philosopher wrote:
> Jean-David Beyer wrote:
>> I wrote an OS once for a special purpose.
>>
>> The IO could do the regular IO similar to what the normal UNIX read(),
>> write(). seek() would do. But for the really heavy-duty IO, where we
>> were reading a television camera in real time, we could not do that
>> because we could not stop when the machine was busy or we would lose
>> data. So I put in additional constructs, such as (I no longer remember
>> the exact syntax):
>>
>> startread(filedescriptor, &buffer, size, recordID);
>> startwrite(filedescriptor, &buffer, size, recordID);
>> result = iostatus(filedescriptor, &buffer);
>>
>> All three of these returned essentially immediately.
>> The reason we needed these is that we double-buffered the IO and did not
>> have the time between when one IO finished and the next one started to
>> set up the device again. We put two controllers on each device and cold
>> set tbe next one up when the first one finished. The startio functions
>> did all the device setup work, put the commends in a FIFO queue, and as
>> one finished with a hardware interrupt, the OS could start the next one
>> quickly enough. It was easy enough to setup one controller during the
>> time the IO transfer was occurring on the other controller. I called
>> this model asynchronous IO because the IO took place in parallel with
>> execution of the application; i.e., the application could continue to
>> run even though it had issued one or more IO requests.
>>
>> In the UNIX model, this was impossible because once a read or a write
>> was called, the application stalled until the IO was completed (what I
>> would call serial IO) and only when the IO was complete could the nest
>> read or write could be issued. The return of one of these IO functions
>> implicitly informed the application that the operation was complete (or
>> failed, as the case might be).
>>
>>
> I suspect you had set up blocking writes then.
Not really; not with what I called asynchronous IO; the normal UNIX-like IO
had blocking reads and writes.
> Normally you wouldn't block on writes unless you ad run out of buffer space.
The app sometimes had no buffer space. The IO in this system was at many
levels. There was a device, such as a digitized TV camera that would write
into a buffer that held far less than a frame of data (this was in the early
1970s when a whole frame of memory at those speeds was out of the question.
So there was a controller between the camera and the buffer memory.
Then there was a controller between the buffer memory and a large magnetic
drum memory. Then there was a controller between the magnetic drum memory
and another buffer memory that was 4096 bytes of memory addressable also
from the computer.
>
> as far as reads go, that is a question of writing a decent device driver
> that can buffer enough data so that even if read relatively
> infrequently, no data is lost. A big ring buffer is good.
>
We would need a device driver that could buffer up about a frame of data
(256 x 256 x 24 bits of data at a pop that would have to be read in a very
few milliseconds of time. Tough at the time because you could not buy that
much memory, especially at the speeds we needed to run.
>
> Then do all the transfers under interrupt.
>
Yes. But the trouble was that one did not have time to post the IO complete
and setup the device again before the deadline -- typically the horizontal
retrace time of the camera. This had to be done during the time of one
horizontal sweep.
> Linux the kernel is a slender shim between the user and the decvice
> driver: if you want super performance, write a better device driver,
> don't change OS!
Of course, in those days there was no Linux. UNIX was just becoming
available, but it could support files only up to 65536 bytes and we would
have to write 32 such files per second. We could not do that if we waited
for one write to complete before attempting to do the next.
>
> Of purse MS has always got round crap device drivers by allowing you to
> hit the hardware from pretty much anywhere. Whether that is a plus or a
> minus depends on your religion.
>
According to my religion, that is a serious minus.
--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 13:10:01 up 1 day, 2:40, 4 users, load average: 4.12, 4.10, 4.09
-
Re: My employer completed migration of apps to Linux
-
Re: My employer completed migration of apps to Linux
On 2008-11-06, Chris Ahlstrom wrote:
> After takin' a swig o' grog, dennis@home belched out
> this bit o' wisdom:
>
>> "Ignoramus31261" wrote in message
>>
>>> It is also easy to implement "async I/O" with select(). Just have a
>>> thread run select() and call those callbacks. (or many threads)
>>
>> As long as you don't care about the order the data is written!
>> Are you sure you have ever programmed anything?
>> I get the idea you are making this stuff up as you have committed several
>> errors so far that would make most applications fail.
>
> Ah, a person who can debug a design and some code without ever having seen
> it!
>
> Say! You might be /just/ the guy to explain the ins and outs of MD5 sums!
>
You write to sockets, from queues, in the order that the sockets are
available, that's a very standard practice.
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/
-
Re: My employer completed migration of apps to Linux
In article ,
Terry Porter wrote:
> > The truth is that Linux is a great server, but struggles as a dektop due
> > to lack of apps.
>
> That's ***NOT*** my truth in anyway, shape or form.
>
> Let's take FwBuilder as just one example,(and only one example is needed
> to prove your statement is not accurate) it's a very cool GUI FLOSS OOP
When someone says there is a lack of apps for a platform, they do not
mean there are *no* apps for that platform.
--
--Tim Smith
-
Re: My employer completed migration of apps to Linux
In article <6nfpg5Fl9c1aU3@mid.individual.net>,
Christopher Hunter wrote:
> > Do you play any computer games yourself?
>
> Not any more - I have better things to do with my time.
>
> > It seems to me only non-gamers believe this to be true. In my personal
> > experience, there are several games which can never be as good on a
> > console as they are on a proper computer.
>
> Games that require clever graphics are /certainly/ better done on a dedicated
> games console. However, the XBox is (as with everything from that company)
Consoles usually have an advantage at launch, but then PC graphics cards
catch up and surpass the consoles.
--
--Tim Smith
-
Re: My employer completed migration of apps to Linux
On Wed, 05 Nov 2008 20:04:26 -0600, Ignoramus31261 wrote:
> On 2008-11-06, Maxwell Lol wrote:
>> Erik Funkenbusch writes:
>>
>>> On Wed, 05 Nov 2008 14:03:18 -0600, Ignoramus31261 wrote:
>>>
>>>>> Which is entirely different from async I/O.
>>>>
>>>> Not "entirely". They are both nonblocking and do not use CPU.
>>>
>>> No. Async I/O is not "non-blocking". Non-blocking means that you call the
>>> I/O function and either it returns with data, or it returns with a return
>>> code indicating there is no data yet.
>>
>> And when you call select with a timeout of zero, it is also non-blocking.
>
> It is also easy to implement "async I/O" with select(). Just have a
> thread run select() and call those callbacks. (or many threads)
Thus showing, once more, that you have no idea how I/O really works and why
there are such things as Async I/O in the first place.
Do you really think the kernel devs would have gone to all the trouble to
develop Async I/O if you could easily be implemented with select()? If you
do, then you're not very bright.
One of the advantages of Async I/O is that it runs in kernel context, which
means I/O operations will be responded to at a higher priority than user
mode code. For requirements that need to process significant amounts of
data, waiting for the user process to beome ready is a serious bottleneck,
especially if there is a lot of I/O (either disk or network) going on.
Merely creating threads, even if you make them a high priority, is not the
same kind of performance as async I/O.
-
Re: My employer completed migration of apps to Linux
On Thu, 06 Nov 2008 12:10:06 -0800, Tim Smith wrote:
> In article <6nfpg5Fl9c1aU3@mid.individual.net>,
> Christopher Hunter wrote:
>>> Do you play any computer games yourself?
>>
>> Not any more - I have better things to do with my time.
>>
>>> It seems to me only non-gamers believe this to be true. In my personal
>>> experience, there are several games which can never be as good on a
>>> console as they are on a proper computer.
>>
>> Games that require clever graphics are /certainly/ better done on a dedicated
>> games console. However, the XBox is (as with everything from that company)
>
> Consoles usually have an advantage at launch, but then PC graphics cards
> catch up and surpass the consoles.
Not to mention that a PC typically has more CPU power, more memory, and
numerous input devices (keybaord, mouse, gamepad, and is typically used at
a desk rather than on a couch in front of a TV.
Many kinds of games are difficult to play on a gamepad, or must be severely
modified to play well on a gamepad. Civilization is a prime example, a
game I have played in various versions for nearly 20 years.
-
Re: My employer completed migration of apps to Linux
On 2008-11-06, Tim Smith wrote:
> In article <6nfpg5Fl9c1aU3@mid.individual.net>,
> Christopher Hunter wrote:
>> > Do you play any computer games yourself?
>>
>> Not any more - I have better things to do with my time.
>>
>> > It seems to me only non-gamers believe this to be true. In my personal
>> > experience, there are several games which can never be as good on a
>> > console as they are on a proper computer.
>>
>> Games that require clever graphics are /certainly/ better done on a dedicated
>> games console. However, the XBox is (as with everything from that company)
>
> Consoles usually have an advantage at launch, but then PC graphics cards
> catch up and surpass the consoles.
That sort of narrowly focused argument was much more meaningful
in 1985 than it is these days.
--
Some people have this nutty idea that in 1997 |||
reading to a hard disk and writing to a hard disk / | \
both at the same time was something worth patenting.
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
http://www.usenet.com
-
Re: My employer completed migration of apps to Linux
In article ,
Erik Funkenbusch wrote:
> >
> > It is also easy to implement "async I/O" with select(). Just have a
> > thread run select() and call those callbacks. (or many threads)
>
> Thus showing, once more, that you have no idea how I/O really works and why
> there are such things as Async I/O in the first place.
>
> Do you really think the kernel devs would have gone to all the trouble to
> develop Async I/O if you could easily be implemented with select()? If you
> do, then you're not very bright.
>
> One of the advantages of Async I/O is that it runs in kernel context, which
> means I/O operations will be responded to at a higher priority than user
> mode code. For requirements that need to process significant amounts of
> data, waiting for the user process to beome ready is a serious bottleneck,
> especially if there is a lot of I/O (either disk or network) going on.
>
> Merely creating threads, even if you make them a high priority, is not the
> same kind of performance as async I/O.
One of the best references on this whole general topic is this:
Most programs that need to do a large amount of asynchronous I/O
nowadays are network programs. E.g., a server serving a large number of
clients. The current network architecture on most operating systems
sucks at this for a very simple reason. Let's consider a web server.
It is communicating over port 80 via TCP to, say, 10000 clients.
The data from all 10000 clients is coming in on one stream (e.g., the
data stream coming on on the ethernet port). The network stack in the
kernel splits this up, by connection, so the web server software has to
get it from 10000 different sockets.
And what does the web server most likely do with those 10000 streams it
sees the data as? Reads them to build up complete requests, and then
probably tosses those on a single work queue, that worker threads pull
from!
So, you've got the data coming in to the system as a stream, being split
into 10000 streams by the kernel, forcing the web server software to
deal with 10000 streams, just so the web server software can put the
data back into a single stream for its worker threads to process.
It could be made much cleaner, and probably much more efficient, if the
web server software could tell the kernel it doesn't want the streams
separated. Turn every incoming packet for port 80 into a message
consisting of a header with source information and the packet data, and
deliver all these messages on a single socket, and let the web server
deal with further stream splitting, if it needs it.
--
--Tim Smith
-
Re: My employer completed migration of apps to Linux
-
Re: My employer completed migration of apps to Linux
Erik Funkenbusch wrote:
> On Wed, 05 Nov 2008 20:04:26 -0600, Ignoramus31261 wrote:
>
>> On 2008-11-06, Maxwell Lol wrote:
>>> Erik Funkenbusch writes:
>>>
>>>> On Wed, 05 Nov 2008 14:03:18 -0600, Ignoramus31261 wrote:
>>>>
>>>>>> Which is entirely different from async I/O.
>>>>> Not "entirely". They are both nonblocking and do not use CPU.
>>>> No. Async I/O is not "non-blocking". Non-blocking means that you call the
>>>> I/O function and either it returns with data, or it returns with a return
>>>> code indicating there is no data yet.
>>> And when you call select with a timeout of zero, it is also non-blocking.
>> It is also easy to implement "async I/O" with select(). Just have a
>> thread run select() and call those callbacks. (or many threads)
>
> Thus showing, once more, that you have no idea how I/O really works and why
> there are such things as Async I/O in the first place.
>
> Do you really think the kernel devs would have gone to all the trouble to
> develop Async I/O if you could easily be implemented with select()? If you
> do, then you're not very bright.
>
> One of the advantages of Async I/O is that it runs in kernel context, which
> means I/O operations will be responded to at a higher priority than user
> mode code. For requirements that need to process significant amounts of
> data, waiting for the user process to beome ready is a serious bottleneck,
> especially if there is a lot of I/O (either disk or network) going on.
>
> Merely creating threads, even if you make them a high priority, is not the
> same kind of performance as async I/O.
You are dennis@home and I claim my £5 you don't understand what it means
either.
-
Re: My employer completed migration of apps to Linux
-
Re: My employer completed migration of apps to Linux
After takin' a swig o' grog, Tim Smith belched out
this bit o' wisdom:
> In article ,
> Terry Porter wrote:
>> > The truth is that Linux is a great server, but struggles as a dektop due
>> > to lack of apps.
>>
>> That's ***NOT*** my truth in anyway, shape or form.
>>
>> Let's take FwBuilder as just one example,(and only one example is needed
>> to prove your statement is not accurate) it's a very cool GUI FLOSS OOP
>
> When someone says there is a lack of apps for a platform, they do not
> mean there are *no* apps for that platform.
True. They mean to present the fiction that Linux doesn't have enough apps
to be useful.
The trouble with counting apps is the redundancy of functionality. Windows
has a lot more apps, but a greater overlap in functionality.
But Linux does have a lot more window managers! And file systems!
--
Lend money to a bad debtor and he will hate you.
-
Re: My employer completed migration of apps to Linux
On Thu, 06 Nov 2008 13:59:28 -0800, Tim Smith wrote:
> One of the best references on this whole general topic is this:
>
>
Interesting quote from that article:
"AIO is normally used with edge-triggered completion notification, i.e. a
signal is queued when the operation is complete. (It can also be used with
level triggered completion notification by calling aio_suspend(), though I
suspect few people do this.)"
"glibc 2.1 and later provide a generic implementation written for standards
compliance rather than performance."
So even Linux doesn't have performance in mind when it wraps AIO with
standard functions.
-
Re: My employer completed migration of apps to Linux
On Thu, 6 Nov 2008 18:24:22 -0500, Chris Ahlstrom wrote:
>> When someone says there is a lack of apps for a platform, they do not
>> mean there are *no* apps for that platform.
>
> True. They mean to present the fiction that Linux doesn't have enough apps
> to be useful.
No, they mean to present the opinion that it doesn't have enough of the
kind of apps they want to make it worth the hassle of conversion and
relearning a new OS. In particular, they don't want to have to use Two
OS's to get what they want.
> The trouble with counting apps is the redundancy of functionality. Windows
> has a lot more apps, but a greater overlap in functionality.
Overlap has nothing to do with it, and nobody but a moron counts raw app
numbers, they count how many apps fullfil given needs, and Linux lacks
fullfilling solutions in many of those categories.
> But Linux does have a lot more window managers! And file systems!
Indeed.
-
Re: My employer completed migration of apps to Linux
"dennis@home" writes:
> I don't need to see it.
> If it does what he states it will make the data order somewhat mixed.
> If he wants it debugged then he will have to post the code and agree
> the fees.
If he's doing select(), then he can handle several input and output
files simultaneously.
-
Re: My employer completed migration of apps to Linux
-
Re: My employer completed migration of apps to Linux
Jean-David Beyer writes:
> In the UNIX model, this was impossible because once a read or a write
> was called, the application stalled until the IO was completed (what I
> would call serial IO) and only when the IO was complete could the nest
> read or write could be issued. The return of one of these IO functions
> implicitly informed the application that the operation was complete
> (or failed, as the case might be).
One way to solve that problem would be to use threads.