requests and sub-requests - modperl

This is a discussion on requests and sub-requests - modperl ; Hi. I am writing a new PerlAuthHandler module. It is working fine in most cases, but.. In an attempt at being clever, I put the following code in the handler : unless ($r->is_initial_req) { if (defined $r->prev) { # we ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: requests and sub-requests

  1. requests and sub-requests

    Hi.

    I am writing a new PerlAuthHandler module.
    It is working fine in most cases, but..

    In an attempt at being clever, I put the following code in the handler :

    unless ($r->is_initial_req) {
    if (defined $r->prev) {
    # we are in a subrequest. Just copy user from main request.
    $r->user( $r->prev->user );
    }
    # Also disable authorization phase
    $r->set_handlers(PerlAuthzHandler => undef);
    return OK;
    }

    The idea being that if we are in a sub-request, there is no point in
    authenticating/authorizing it again, since the main request should
    already do that, right ? Optimisation..

    Now the above works very nicely, except in the case where, before this
    handler gets called, there is an intervention by mod_rewrite.
    It seems as if mod_rewrite makes the above fail, even when the rewrite
    condition does not apply and the URL is considered as a "pass-through".

    I suspect that it is because mod_rewrite, no matter what, invoques the
    original (or modified) URL as a sub-request of the original request.
    This would cause the above to fail, because in such a case, the above
    conditional code would be invoked, but there is no $r->prev->user to be
    copied.

    So,
    1) is my suspicion above correct ?
    2) is there a way to modify the above code to still allow some
    optimisation in that case ?

    Thanks.


  2. Re: requests and sub-requests

    On Sun 12 Oct 2008, André Warnier wrote:
    > In an attempt at being clever, I put the following code in the
    > handler :
    >
    > * * *unless ($r->is_initial_req) {
    > * * * * *if (defined $r->prev) {
    > * * * * * * *# we are in a subrequest. *Just copy user from main
    > request. $r->user( $r->prev->user );
    > * * * * *}
    > * * * * *# Also disable authorization phase
    > * * * * *$r->set_handlers(PerlAuthzHandler => undef);
    > * * * * *return OK;
    > * * *}


    You have to distinguish between subrequests and internal redirects. The
    former result from $r->lookup_uri, $r->lookup_file or similar (there
    are a few more such functions in the C API) and internal redirects that
    result from $r->internal_redirect (internal_fast_redirect() is not as
    the name suggests an internal redirect but simply overrides the current
    request). Subrequests are used for example by mod_rewrite, mod_include,
    mod_negotiation to look for some characteristics of a document and
    perhaps pull it in (run() it). Internal redirects are used in mod_cgi
    when the CGI output indicates a status 200 (HTTP_OK) but also contains
    a Location header. But the main usage of internal redirects is the
    ErrorDocument.

    Now, is_initial_req() checks if the current $r is the result of a
    subrequest or the result of a internal redirect and returns false if
    so. prev() returns the parent request if the current $r is the result
    of an internal redirect and main() returns the main request if the
    current $r is a subrequest. So, your code checks only for internal
    redirects (ErrorDocument).

    Now, have a look at httpd-2.x.y/server/request.c around line 170. You'll
    see this piece of code:

    /* Skip authn/authz if the parent or prior request passed the
    * authn/authz,
    * and that configuration didn't change (this requires
    * optimized _walk()
    * functions in map_to_storage that use the same merge results given
    * identical input.) If the config changes, we must re-auth.
    */
    if (r->main && (r->main->per_dir_config == r->per_dir_config)) {
    r->user = r->main->user;
    r->ap_auth_type = r->main->ap_auth_type;
    }
    else if (r->prev && (r->prev->per_dir_config == r->per_dir_config))
    {
    r->user = r->prev->user;
    r->ap_auth_type = r->prev->ap_auth_type;
    }
    else {
    switch (ap_satisfies(r)) {
    case SATISFY_ALL:
    case SATISFY_NOSPEC:
    if ((access_status = ap_run_access_checker(r)) != 0) {
    return decl_die(access_status, "check access", r);
    ...

    You see, you are not the first who had had the idea of reusing an
    established identity. If your subreq or internal redirect hits the same
    Location or Directory container the AAA phases are completely skipped.
    Maybe this is enough optimization if you shift a few directives around
    in your httpd.conf.

    If not, the code above shows you how to do it. But you must ask yourself
    if it really is valid to reuse the identity. I believe, you can safely
    inherit the identity from $r->main or $r->prev but you must not skip
    the other 2 A's. If you can't it would mean you have one realm of
    identities for the main request and another for the subreq. That, I'd
    say, is a configuration error.

    > The idea being that if we are in a sub-request, there is no point in
    > authenticating/authorizing it again, since the main request should
    > already do that, right ? *Optimisation..
    >
    > Now the above works very nicely, except in the case where, before
    > this handler gets called, there is an intervention by mod_rewrite. It
    > seems as if mod_rewrite makes the above fail, even when the rewrite
    > condition does not apply and the URL is considered as a
    > "pass-through".
    >
    > I suspect that it is because mod_rewrite, no matter what, invoques
    > the original (or modified) URL as a sub-request of the original
    > request. This would cause the above to fail, because in such a case,
    > the above conditional code would be invoked, but there is no
    > $r->prev->user to be copied.


    mod_rewrite doesn't make subrequests if not asked to. I know only of 2
    ways to have mod_rewrite perform a subreq: %{LA-U:variable}
    and %{LA-F:variable} in a RewriteCond.

    Torsten

    --
    Need professional mod_perl support?
    Just hire me: torsten.foertsch@gmx.net


  3. Re: requests and sub-requests

    Torsten,

    Many thanks for the excellent information, I will ponder that.

    More below, but one more question here :
    Where does $r->internal_redirect "live" (in which package) ?
    I am having trouble finding it.

    Torsten Foertsch wrote:
    > On Sun 12 Oct 2008, André Warnier wrote:
    >> In an attempt at being clever, I put the following code in the
    >> handler :
    >>
    >> unless ($r->is_initial_req) {
    >> if (defined $r->prev) {
    >> # we are in a subrequest. Just copy user from main
    >> request. $r->user( $r->prev->user );
    >> }
    >> # Also disable authorization phase
    >> $r->set_handlers(PerlAuthzHandler => undef);
    >> return OK;
    >> }

    >
    > You have to distinguish between subrequests and internal redirects. The
    > former result from $r->lookup_uri, $r->lookup_file or similar (there
    > are a few more such functions in the C API) and internal redirects that
    > result from $r->internal_redirect (internal_fast_redirect() is not as
    > the name suggests an internal redirect but simply overrides the current
    > request). Subrequests are used for example by mod_rewrite, mod_include,
    > mod_negotiation to look for some characteristics of a document and
    > perhaps pull it in (run() it). Internal redirects are used in mod_cgi
    > when the CGI output indicates a status 200 (HTTP_OK) but also contains
    > a Location header. But the main usage of internal redirects is the
    > ErrorDocument.
    >
    > Now, is_initial_req() checks if the current $r is the result of a
    > subrequest or the result of a internal redirect and returns false if
    > so. prev() returns the parent request if the current $r is the result
    > of an internal redirect and main() returns the main request if the
    > current $r is a subrequest. So, your code checks only for internal
    > redirects (ErrorDocument).
    >
    > Now, have a look at httpd-2.x.y/server/request.c around line 170. You'll
    > see this piece of code:
    >
    > /* Skip authn/authz if the parent or prior request passed the
    > * authn/authz,
    > * and that configuration didn't change (this requires
    > * optimized _walk()
    > * functions in map_to_storage that use the same merge results given
    > * identical input.) If the config changes, we must re-auth.
    > */
    > if (r->main && (r->main->per_dir_config == r->per_dir_config)) {
    > r->user = r->main->user;
    > r->ap_auth_type = r->main->ap_auth_type;
    > }
    > else if (r->prev && (r->prev->per_dir_config == r->per_dir_config))
    > {
    > r->user = r->prev->user;
    > r->ap_auth_type = r->prev->ap_auth_type;
    > }
    > else {
    > switch (ap_satisfies(r)) {
    > case SATISFY_ALL:
    > case SATISFY_NOSPEC:
    > if ((access_status = ap_run_access_checker(r)) != 0) {
    > return decl_die(access_status, "check access", r);
    > ...
    >

    Ok, I get it.

    I have a little question related to the above, but not very urgent : why
    the check on the configuration change ? what can change between a
    request and a sub-request (or internal redirect) ?

    > You see, you are not the first who had had the idea of reusing an
    > established identity.

    I did not think I would be.

    If your subreq or internal redirect hits the same
    > Location or Directory container the AAA phases are completely skipped.
    > Maybe this is enough optimization if you shift a few directives around
    > in your httpd.conf.

    I don't think so, because this is a really specific authentication
    method, for a special case.
    And I don't think that Apache will skip the mod_perl AAA phases, will it ?

    >
    > If not, the code above shows you how to do it. But you must ask yourself
    > if it really is valid to reuse the identity. I believe, you can safely
    > inherit the identity from $r->main or $r->prev but you must not skip
    > the other 2 A's. If you can't it would mean you have one realm of
    > identities for the main request and another for the subreq. That, I'd
    > say, is a configuration error.
    >

    As a first stage of the AAA, for some Locations, there is a filtering on
    the remote IP of the caller. Some IP's get an "automatic" user-id,
    which can vary according to the IP. In some cases, this is authoritative
    (no access unless you have the right IP), in some cases not (you get a
    second chance). Some Locations don't have the IP filter, they always
    get the second chance below. This IP filter is implemented as a
    PerlAccessHandler. This is the main reason for trying to optimise,
    because it is expensive : the IP of the caller must be compared to
    several ranges of IP, not necessarily matching regular subnets.

    The second step is a PerlAuthenHandler, which can re-direct to a login
    page.
    Then there is a PerlAuthenzHandler to check if this user is allowed to
    access that resource.
    It also combines with SSO, with some URL rewriting, and with trying to
    control access to a Tomcat application behind the Apache.

    The back-end for the authentication is a special DB system, whose access
    for that is rather heavy, but required.
    On the positive side, this is for a limited range of well-known
    applications, for a limited public and for a reasonable number of
    expected transactions/s.
    So I am trying to wring out the optimisations I can, without going too far.
    I started this module wanting to keep it "clean and lean and mean", but
    as I discover more and more twists, it is getting to look like the
    classical spaghetti bowl..

    I am also, but on a separate thread, looking at tying this AAA stuff to
    the $r->connection (with notes()).

    I'm also having fun doing this, it's interesting.

    >> The idea being that if we are in a sub-request, there is no point in
    >> authenticating/authorizing it again, since the main request should
    >> already do that, right ? Optimisation..
    >>
    >> Now the above works very nicely, except in the case where, before
    >> this handler gets called, there is an intervention by mod_rewrite. It
    >> seems as if mod_rewrite makes the above fail, even when the rewrite
    >> condition does not apply and the URL is considered as a
    >> "pass-through".
    >>
    >> I suspect that it is because mod_rewrite, no matter what, invoques
    >> the original (or modified) URL as a sub-request of the original
    >> request. This would cause the above to fail, because in such a case,
    >> the above conditional code would be invoked, but there is no
    >> $r->prev->user to be copied.

    >
    > mod_rewrite doesn't make subrequests if not asked to. I know only of 2
    > ways to have mod_rewrite perform a subreq: %{LA-U:variable}
    > and %{LA-F:variable} in a RewriteCond.
    >


    This :
    http://perl.apache.org/docs/2.0/api/...s_initial_req_
    may be missing ".. or an internal redirect" in a couple of places.


  4. Re: requests and sub-requests

    André Warnier wrote:
    > Torsten,
    >
    > Many thanks for the excellent information, I will ponder that.
    >
    > More below, but one more question here :
    > Where does $r->internal_redirect "live" (in which package) ?
    > I am having trouble finding it.


    http://perl.apache.org/docs/2.0/api/...rnal_redirect_

    > As a first stage of the AAA, for some Locations, there is a filtering on
    > the remote IP of the caller. Some IP's get an "automatic" user-id,
    > which can vary according to the IP. In some cases, this is authoritative
    > (no access unless you have the right IP), in some cases not (you get a
    > second chance). Some Locations don't have the IP filter, they always
    > get the second chance below. This IP filter is implemented as a
    > PerlAccessHandler. This is the main reason for trying to optimise,
    > because it is expensive : the IP of the caller must be compared to
    > several ranges of IP, not necessarily matching regular subnets.


    It seems odd to me to set $r->user in an AccessHandler. It's probably
    not a problem, but it seems (at least to me) that that would make more
    sense as a part of the Authen code. You can then control all your
    'second chance' stuff with normal state checking within your Authen
    Handler instead of doing funky stuff with set_handlers (which seems to
    be what you're doing)

    Adam


  5. Re: requests and sub-requests

    Adam Prime wrote:

    >
    > http://perl.apache.org/docs/2.0/api/...rnal_redirect_

    Thanks.
    Although considering Torsten previous answer and explanation, that's
    kind of an odd place, no ? ;-)

    [...]

    >
    > It seems odd to me to set $r->user in an AccessHandler. It's probably
    > not a problem, but it seems (at least to me) that that would make more
    > sense as a part of the Authen code. You can then control all your
    > 'second chance' stuff with normal state checking within your Authen
    > Handler instead of doing funky stuff with set_handlers (which seems to
    > be what you're doing)
    >


    Well, yeah, I guess it can be debated, on the philosophical level.

    Access Control is supposed to be allowing or denying access, based on
    criteria other than the user identity.
    This handler does 2 things :
    It checks the remote IP of the caller, and can deny access if it is not
    in its list. That's clearly part of Access.
    Now in addition, if the IP is in it's list, it can have a "standard"
    user-id associated with it. So, as long as we're there, we might as
    well pick it up and set it in $r->user.

    The Authentication runs right after it, and that's another handler.
    This one is supposed to check that we know who the user is.
    It does that, by checking first $r->user.
    If it finds it there, it means that something put it there before, and
    it this case that can only be the Access handler. If the Access module
    did not put it there, then this handler would have to re-read the IP
    list, and look again for the caller IP, which does not seem to make sense.
    Now if there is no $r->user, it means the user needs to login.
    For the login, we want to send back a nice-looking form. And after the
    user fills it in, we would like - if the user-id is ok - to proceed
    back to the original page that the user tried to access. For that we
    need to fill-in some parameters in the login form, so that when the user
    posts it, we know where to go (back) to.

    That's where the set_handlers kicks in.

    Instead of sending an external redirect to the browser toward a static
    login form (which I could then not fill in), or to a server location
    where they would get the login form (which would nean another
    section in the configuration, and another browser round-trip), I thought
    it was more elegant and mod_perl-ish and not so funky, to use
    set_handlers() to install a ResponseHandler for the current call, and
    then return OK from the current Authen handler.

    The next phase is the response (I'm skipping the AuthenzHandler for
    clarity).
    Now we have two cases :
    - either the access/authentication/authorization was ok, and we have the
    default Apache handler sending the normal response
    - or the AAA was not ok, and we have the custom previously-installed
    perl ResponseHandler, who will send the login form, while filling it in
    on the way based on information stored in $r->pnotes previously.
    Without a browser round-trip.

    I personally find this rather elegant, and not too bizarre.
    It also works very nicely, in most cases.

    The problem is that, if the location desired by the user originally
    already has a non-default handler, then things don't happen exactly as I
    wanted : despite installing modperl as the handler and my
    ResponseHandler under that, the original Apache-level handler is still
    being called.

    But that's another thread on this same list.


  6. Re: requests and sub-requests

    On Sun 12 Oct 2008, André Warnier wrote:
    > I have a little question related to the above, but not very urgent :
    > why the check on the configuration change ? what can change between a
    > request and a sub-request (or internal redirect) ?


    Suppose this:


    Require group foo



    Require group bar


    If during the processing of /mainreq a subreq is issued to /subreq the
    other "Require group" must be respected. Of course there also can be
    another AuthUserFile or so, so that the identity could not be
    established and the subreq results in a 401. I'd consider that as a
    configuration error.

    The configuration change mentioned above means the different Location
    containers. They are represented as $r->per_dir_config.

    > I don't think so, because this is a really specific authentication
    > method, for a special case.
    > And I don't think that Apache will skip the mod_perl AAA phases, will
    > it ?


    Yes, mod_perl handlers are called inside these ap_run_... functions.
    ap_run_access_checker() is the first of the 3 A's. A PerlAccessHandler
    is called from this function.

    Torsten

    --
    Need professional mod_perl support?
    Just hire me: torsten.foertsch@gmx.net


  7. Re: requests and sub-requests

    Torsten Foertsch wrote:
    [...]
    >> And I don't think that Apache will skip the mod_perl AAA phases, will
    >> it ?

    >
    > Yes, mod_perl handlers are called inside these ap_run_... functions.
    > ap_run_access_checker() is the first of the 3 A's. A PerlAccessHandler
    > is called from this function.
    >

    Thanks again for another valuable bit of information.
    That removes the incentive for many of my tentative optimisations.
    I seems that I can just let Apache deal with this then.
    I have to think some more about it, but at the moment I don't see a
    downside of that.
    I do not use lookup_uri in this module, at least not explicitly.

    Would you by any chance like to have a look too at the "sethandlers
    question" thread a bit higher up ?


+ Reply to Thread