IPSec Linux - Longhorn one way. - Networking

This is a discussion on IPSec Linux - Longhorn one way. - Networking ; I've this (let's say strange) problem in a communication between one Linux server with kernel 2.6.18-6 (Debian) and one Windows Server 2008 Enterprise; the policy requires AH with x.509 certificates. If the Linux machine tries beginning the communication (on e ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: IPSec Linux - Longhorn one way.

  1. IPSec Linux - Longhorn one way.

    I've this (let's say strange) problem in a communication between one Linux
    server with kernel 2.6.18-6 (Debian) and one Windows Server 2008
    Enterprise; the policy requires AH with x.509 certificates. If the Linux
    machine tries beginning the communication (on e new connection) the Quick
    Mode SA negotiation fails (but the phase 1, main mode negotiation
    succedes). When the communication begins from the Windows server, all go
    just fine and from that moment, obviously, the communication works in both
    ways.
    In the syslog I can read:
    racoon: ERROR: mismatched ID was returned.
    racoon: ERROR: failed to pre-process packet.
    racoon: ERROR: phase2 negotiation failed.
    In the Windows event viewer i have one failure for any one Main mode
    nagotiation (failed or succeded):
    Event ID: 4976
    Task Category: IPsec Main Mode
    Level: Information
    Keywords: Audit Failure
    Description:
    During Main Mode negotiation, IPsec received an invalid negotiation packet.
    If this problem persists, it could indicate a network issue or an attempt
    to modify or replay this negotiation.
    Any idea?
    Thanks.

    --
    Lorenzo Vaina,
    MCSA Windows Server 2003,
    MCTS SQL Server 2005.
    messaggi privati: http://www.vaina.it/posta.html

  2. Re: IPSec Linux - Longhorn one way.

    Am Tue, 08 Jul 2008 20:38:02 +0200 schrieb Lorenzo Vaina:

    > I've this (let's say strange) problem in a communication between one Linux
    > server with kernel 2.6.18-6 (Debian) and one Windows Server 2008
    > Enterprise; the policy requires AH with x.509 certificates. If the Linux
    > machine tries beginning the communication (on e new connection) the Quick
    > Mode SA negotiation fails (but the phase 1, main mode negotiation
    > succedes). When the communication begins from the Windows server, all go
    > just fine and from that moment, obviously, the communication works in both
    > ways.
    > In the syslog I can read:
    > racoon: ERROR: mismatched ID was returned.
    > racoon: ERROR: failed to pre-process packet.
    > racoon: ERROR: phase2 negotiation failed.
    > In the Windows event viewer i have one failure for any one Main mode
    > nagotiation (failed or succeded):
    > Event ID: 4976
    > Task Category: IPsec Main Mode
    > Level: Information
    > Keywords: Audit Failure
    > Description:
    > During Main Mode negotiation, IPsec received an invalid negotiation packet.
    > If this problem persists, it could indicate a network issue or an attempt
    > to modify or replay this negotiation.
    > Any idea?
    > Thanks.
    >


    check your phase2 proposals on debian.

  3. Re: IPSec Linux - Longhorn one way.

    Burkhard Ott scripsit:

    > check your phase2 proposals on debian.


    Thank you for your reply. I re-checked the sainfo stanza and it seems fine.
    Oh, it's so simple:

    sainfo anonymous
    {
    pfs_group modp2048;
    lifetime time 15 min;
    encryption_algorithm 3des,null_enc;
    authentication_algorithm hmac_sha1;
    compression_algorithm deflate;
    }

    I added null_enc as a fallback trying but obviously all I got was to load an
    unsuitable module in my poor kernel memory.
    I think that if something was wrong here, the opposite way communication
    failed too.

    Please, do you know with ID is referred to, in the stanza:
    racoon: ERROR: mismatched ID was returned.
    ?

    Thanks.

    --
    Lorenzo Vaina,
    MCSA Windows Server 2003,
    MCTS SQL Server 2005.
    messaggi privati: http://www.vaina.it/posta.html

  4. Re: IPSec Linux - Longhorn one way.

    Am Wed, 09 Jul 2008 13:52:01 +0200 schrieb Lorenzo Vaina:

    > Burkhard Ott scripsit:
    >
    >> check your phase2 proposals on debian.

    >
    > Thank you for your reply. I re-checked the sainfo stanza and it seems fine.
    > Oh, it's so simple:
    >
    > sainfo anonymous
    > {
    > pfs_group modp2048;
    > lifetime time 15 min;
    > encryption_algorithm 3des,null_enc;
    > authentication_algorithm hmac_sha1;
    > compression_algorithm deflate;
    > }
    >
    > I added null_enc as a fallback trying but obviously all I got was to load an
    > unsuitable module in my poor kernel memory.
    > I think that if something was wrong here, the opposite way communication
    > failed too.
    >
    > Please, do you know with ID is referred to, in the stanza:
    > racoon: ERROR: mismatched ID was returned.
    > ?
    >
    > Thanks.
    >


    Usually if no other ID is configured the IP is taken, bu tyou could also
    set an ID with my_identifier.
    Try also you IPSec without IPComp (compression_algorithm deflate
    probably Windows doesn't like that.

    cheers

  5. Re: IPSec Linux - Longhorn one way.

    Burkhard Ott scripsit:

    > Usually if no other ID is configured the IP is taken, bu tyou could also
    > set an ID with my_identifier.


    Please do you know witch ID is sent by the Windows side?

    > Try also you IPSec without IPComp (compression_algorithm deflate
    > probably Windows doesn't like that.


    Setting a compression algorithm is mandatory in racoon. If it will use or
    not use IPComp is setted at kernel level, if I'm not wrong.

    > cheers


    Thank you for your interesting.

    --
    Lorenzo Vaina,
    MCSA Windows Server 2003,
    MCTS SQL Server 2005.
    messaggi privati: http://www.vaina.it/posta.html

  6. Re: IPSec Linux - Longhorn one way.

    Am Wed, 09 Jul 2008 21:32:59 +0200 schrieb Lorenzo Vaina:

    > Burkhard Ott scripsit:
    >
    >> Usually if no other ID is configured the IP is taken, bu tyou could also
    >> set an ID with my_identifier.

    >
    > Please do you know witch ID is sent by the Windows side?


    I assume the IP Adress, if not a ID configured.


    >> Try also you IPSec without IPComp (compression_algorithm deflate
    >> probably Windows doesn't like that.

    >
    > Setting a compression algorithm is mandatory in racoon. If it will use or
    > not use IPComp is setted at kernel level, if I'm not wrong.


    You are wrong. Thats might be the reason why windows can't read the packet
    correctly.

    cheers

  7. Re: IPSec Linux - Longhorn one way.

    Burkhard Ott scripsit:

    >> Setting a compression algorithm is mandatory in racoon. If it will use or
    >> not use IPComp is setted at kernel level, if I'm not wrong.

    >
    > You are wrong. Thats might be the reason why windows can't read the packet
    > correctly.


    racoon: ERROR: /etc/racoon/racoon.conf:50: "}" no compression algorithm at
    anonymous
    racoon: ERROR: fatal parse failure (1 errors)

    I found a Microsoft Knowledge Base article (950826) stating "You cannot
    establish an IPsec connection between a Linux operating system and a
    Windows Vista operating system when you initiate the connection from the
    Linux operating system", but the cause description is a swithing of AH and
    ESP order. However I use only AH and I have the same issue using ESP only.

    >
    > cheers

    Thank you.

    --
    Lorenzo Vaina,
    MCSA Windows Server 2003,
    MCTS SQL Server 2005.
    messaggi privati: http://www.vaina.it/posta.html

  8. Re: IPSec Linux - Longhorn one way.

    Am Thu, 10 Jul 2008 10:23:43 +0200 schrieb Lorenzo Vaina:

    > racoon: ERROR: /etc/racoon/racoon.conf:50: "}" no compression algorithm at
    > anonymous
    > racoon: ERROR: fatal parse failure (1 errors)


    AFAIK The "compression_algorithm" option in the racoon.conf only specifies
    the algorithm. Compression will not be used unless you enable it in the
    SPD entry.

    > I found a Microsoft Knowledge Base article (950826) stating "You cannot
    > establish an IPsec connection between a Linux operating system and a
    > Windows Vista operating system when you initiate the connection from the
    > Linux operating system", but the cause description is a swithing of AH
    > and ESP order. However I use only AH and I have the same issue using ESP
    > only.


    I've read the article, funny issue. Try to disable AH, esp is you payload.
    My idea is when MS switch teh format to esp+ah instead of ah+esp, what
    happens if there is no AH attached?
    Maybe it helps.

    cheers

  9. Re: IPSec Linux - Longhorn one way.

    Burkhard Ott scripsit:
    >
    > I've read the article, funny issue. Try to disable AH, esp is you payload.
    > My idea is when MS switch teh format to esp+ah instead of ah+esp, what
    > happens if there is no AH attached?
    > Maybe it helps.
    >
    > cheers


    I confirm that my problem is only with the Windows Server 2008 operating
    system. If the Windows side is an older Microsoft system, all works as
    expected.
    I tried the configuration described in the Microsoft KB article and I found
    that the communication is impossible in both way, not only if Linux is the
    initiator. However I need only AH and I tried using only ESP without luck:
    in every cases only Windows can initiate the connection.
    Now I think this is a bug in the Microsoft operating system and I'm going to
    write this result on the Microsoft Winserver group.
    Thank you for your help. Please continue writing here if you (or somebody
    else) have some other ideas.

    --
    Lorenzo Vaina,
    MCSA Windows Server 2003,
    MCTS SQL Server 2005.
    messaggi privati: http://www.vaina.it/posta.html

+ Reply to Thread