On 9/6/07, LluÃ*s Batlle wrote:
> Hello,
>
> I understand from [1] that named pipes are suposed to use the
> filesystem as an "unlimited" buffer for the pipe. This would be an
> advantage against a pipe deadlock I'm experiencing using unnamed pipes
> (a CGI communicating wit Apache), but I cannot reproduce the
> "unlimited buffer" in Linux.

On linux you want fifo(7), which says "When processes are exchanging
data via the FIFO, the kernel passes all data internally without
writing it to the file system.". Should answer the rest of your
questions.
-sqweek

2007/9/6, sqweek :
> On linux you want fifo(7), which says "When processes are exchanging
> data via the FIFO, the kernel passes all data internally without
> writing it to the file system.". Should answer the rest of your
> questions.
Oh, thank you. I expected that mkfifo(1) would point to that page if
it existed, and in my installation it doesn't point.

On 9/6/07, LluÃ*s Batlle wrote:
> 2007/9/6, sqweek :
> > On linux you want fifo(7),
> Oh, thank you. I expected that mkfifo(1) would point to that page if
> it existed, and in my installation it doesn't point.

You expected a GNU man page to be useful? Are you kidding me??
-sqweek

I was more used to blame GNU's code than man pages

To Enrico, in fact I want to use the disk-buffered pipe program in a shell.
Something like:
generate_data | diskpipe | data_eater
I don't think a 9p server would do anything here.

Thank you,
Lluís.

2007/9/6, sqweek :
> On 9/6/07, Lluís Batlle wrote:
> > 2007/9/6, sqweek :
> > > On linux you want fifo(7),
> > Oh, thank you. I expected that mkfifo(1) would point to that page if
> > it existed, and in my installation it doesn't point.
>
> You expected a GNU man page to be useful? Are you kidding me??
> -sqweek
>

On 9/6/07, Lluís Batlle wrote:
> I was more used to blame GNU's code than man pages
>
> To Enrico, in fact I want to use the disk-buffered pipe program in a shell.
> Something like:
> generate_data | diskpipe | data_eater
> I don't think a 9p server would do anything here.
>

generate_data > input
cat input > output
data_eater < output

with output and input being files from a file server.

--
- curiosity sKilled the cat

I was thinking in a general solution for having pipes with very big buffers....
but your much simpler proposal fixes perfectly my problem

I'm having too much problems on over-engineering these last days...

Thank you,
Lluís.

2007/9/6, Gorka Guardiola :
> On 9/6/07, Lluís Batlle wrote:
> > I was more used to blame GNU's code than man pages
> >
> > To Enrico, in fact I want to use the disk-buffered pipe program in a shell.
> > Something like:
> > generate_data | diskpipe | data_eater
> > I don't think a 9p server would do anything here.
> >
>
> generate_data > input
> cat input > output
> data_eater < output
>
> with output and input being files from a file server.
>
> --
> - curiosity sKilled the cat
>

On 9/6/07, Gorka Guardiola wrote:

> generate_data > input
> cat input > output
> data_eater < output

This in the general case is a bad idea, from many angles :-)

ron

I wrote it without coffee... would be more like:

generate_data > input &
data_eater < output

:-).

And input and output can come from a stupid fileserver like pipefs...
or pipebackedbyaharddiskfs :-).
--
- curiosity sKilled the cat

I better explain my problem.
I have a CGI for Apache. I wrote it in C, and it simply parses text
from a html textarea, giving another html in output. I made it as a
state machine which outputs and outputs according to the input
characters.

Apache communicates with the CGI process with two pipes linked to the
process' stdin and stdout.
But I think that apache code is like this:
while()
{
read(socket,buffer)
write(pipe_process1, buffer)
}
while()
{
read(pipe_process2, buffer)
write(socket,buffer)
}

Therefore, if I had a CGI similar to 'cat' (and I have), it would
deadlock due to the pipe_process2 not being read by apache if it gets
filled, the process blocks, and then pipe_process1 gets filled.

As I want the CGI to accept input from almost any length, I need a
"pipe buffer" big enough.

Then, based on Gorka suggestion (which I didn't understand as a
9pserver , I finally agreed on the easiest solution wrapping the CGI
with a sh script:
#!/bin/sh
head -n 10000000 > inputfile # limiting the disk ussage to 10MB, raw cutting.
mycgi < inputfile
#EOF

That's all. I will not write any "superpipe" program, or any 9pserver,
because at the end I need the disk. The sh wrapper works well enough.

Thank you for your help in this non-plan9-related problem! Simply the
unicon documentation pointed to something strange in named pipes which
I couldn't check anywhere.

Regards,
Lluís

2007/9/6, Gorka Guardiola :
> I wrote it without coffee... would be more like:
>
> generate_data > input &
> data_eater < output
>
> :-).
>
> And input and output can come from a stupid fileserver like pipefs...
> or pipebackedbyaharddiskfs :-).
> --
> - curiosity sKilled the cat
>

> I better explain my problem.
> I have a CGI for Apache. I wrote it in C, and it simply parses text
> from a html textarea, giving another html in output. I made it as a
> state machine which outputs and outputs according to the input
> characters.
>
> Apache communicates with the CGI process with two pipes linked to the
> process' stdin and stdout.
> But I think that apache code is like this:
> while()
> {
> read(socket,buffer)
> write(pipe_process1, buffer)
> }
> while()
> {
> read(pipe_process2, buffer)
> write(socket,buffer)
> }
>
> Therefore, if I had a CGI similar to 'cat' (and I have), it would
> deadlock due to the pipe_process2 not being read by apache if it gets
> filled, the process blocks, and then pipe_process1 gets filled.

no, you wouldn't.

i believe the problem is the cgi and apache disagree on the termination
condition. make sure you close the fd 0 when you are done reading.

- erik

2007/9/6, erik quanstrom :
> > Therefore, if I had a CGI similar to 'cat' (and I have), it would
> > deadlock due to the pipe_process2 not being read by apache if it gets
> > filled, the process blocks, and then pipe_process1 gets filled.
>
> no, you wouldn't.
>
> i believe the problem is the cgi and apache disagree on the termination
> condition. make sure you close the fd 0 when you are done reading.
I used gdb, and my process doesn't reach at all the end of reading. It
gets blocked writting to stdout. The system input TCP queue to apache
is still filled with 70KB of data in the test I did. And the browser
didn't receive any byte. And I know my program processed an amount of
data (it makes the hard disk pain a bit, so I can 'hear' when it does
something).

> Thank you for your help in this non-plan9-related problem! Simply the
> unicon documentation pointed to something strange in named pipes which
> I couldn't check anywhere.

i couldn't work out where they got that impression or information.
i'm sure i've never seen that in any of the unix-y implementations i've read.