very annoying problem - Aix

This is a discussion on very annoying problem - Aix ; I am running apache on aix 5.3 with very few problems. However, we occasionally see a perl job (perl es f0 a a post) running as the web server user. This job is sending out tens of thousands of emails ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: very annoying problem

  1. very annoying problem

    I am running apache on aix 5.3 with very few problems. However, we
    occasionally see a perl job (perl es f0 a a post) running as the web
    server user. This job is sending out tens of thousands of emails to
    Brazil!

    I cannot find any script called 'es' anywhere on the system. The job is
    spawned from the web-server process. It seems to startup automatically,
    but, not from any cron jobs.

    Has anyone seen anything like this?

    Any help greatly appreciated.

  2. Re: very annoying problem

    Mike Klein wrote:
    > I am running apache on aix 5.3 with very few problems. However, we
    > occasionally see a perl job (perl es f0 a a post) running as the web
    > server user. This job is sending out tens of thousands of emails to
    > Brazil!
    >
    > I cannot find any script called 'es' anywhere on the system. The job is
    > spawned from the web-server process. It seems to startup automatically,
    > but, not from any cron jobs.
    >
    > Has anyone seen anything like this?


    You are kidding, right? You see your webserver doing things you
    don't understand, you obviously don't intend it to do those things
    in the first place and they also could be considered as very heavy
    spamming. Are there no alarm bells are ringing at all? And *NO*
    IBM does *NOT* ship its AIX OS with the default of sending high
    volumes of mail to brazil!
    Immediately:
    - Restrict incoming and outgoing traffic, on a network as well
    as on the system.
    - Redirect SMTP traffic over a controlled mail relay. Make sure
    you're not being used by spammers, otherwise you might end up
    on basically every blacklist there is.
    - Check the webservers access logs and correlate with the last
    run of the Perl command. It might be triggered from the outside.
    - Check with the admin of the installed websoft/CMS/etc. Stuff
    like that has a tendency to be exploitable if not written well.
    - If you can't find a culprit/script, boot the server from a
    clean, known-good, read-only media (CD/DVD) into maintainance
    mode and search again.

    As soon as possible:
    - Get your server and network secured and audited. Pay someone
    if you can't do it yourself!
    - If you're hosting sites for customers, establish policies for
    what is allowed. Implement checks to monitor those policies!

    After all you're legally liable if you're running the box ...

    Regards,

    Frank

  3. Re: very annoying problem

    In article ,
    Frank Fegert wrote:

    > Mike Klein wrote:
    > > I am running apache on aix 5.3 with very few problems. However, we
    > > occasionally see a perl job (perl es f0 a a post) running as the web
    > > server user. This job is sending out tens of thousands of emails to
    > > Brazil!
    > >
    > > I cannot find any script called 'es' anywhere on the system. The job is
    > > spawned from the web-server process. It seems to startup automatically,
    > > but, not from any cron jobs.
    > >
    > > Has anyone seen anything like this?

    >
    > You are kidding, right? You see your webserver doing things you
    > don't understand, you obviously don't intend it to do those things
    > in the first place and they also could be considered as very heavy
    > spamming. Are there no alarm bells are ringing at all? And *NO*
    > IBM does *NOT* ship its AIX OS with the default of sending high
    > volumes of mail to brazil!
    > Immediately:
    > - Restrict incoming and outgoing traffic, on a network as well
    > as on the system.
    > - Redirect SMTP traffic over a controlled mail relay. Make sure
    > you're not being used by spammers, otherwise you might end up
    > on basically every blacklist there is.
    > - Check the webservers access logs and correlate with the last
    > run of the Perl command. It might be triggered from the outside.
    > - Check with the admin of the installed websoft/CMS/etc. Stuff
    > like that has a tendency to be exploitable if not written well.
    > - If you can't find a culprit/script, boot the server from a
    > clean, known-good, read-only media (CD/DVD) into maintainance
    > mode and search again.
    >
    > As soon as possible:
    > - Get your server and network secured and audited. Pay someone
    > if you can't do it yourself!
    > - If you're hosting sites for customers, establish policies for
    > what is allowed. Implement checks to monitor those policies!
    >
    > After all you're legally liable if you're running the box ...
    >
    > Regards,
    >
    > Frank


    What I am trying to determine is how a script that doesn't exist on my
    system can be run by the web server user who doesn't have a shell
    account. There are also no files containing the email address domains on
    the system! The mail logs are full of messages from the web server user.
    Apache access and error logs similarly show nothing useful. Email
    relaying is not possible! We do not host external web sites and shutting
    down the system or email is not an option.

  4. Re: very annoying problem

    Mike Klein wrote:
    > What I am trying to determine is how a script that doesn't exist on my
    > system can be run by the web server user who doesn't have a shell
    > account. There are also no files containing the email address domains on
    > the system! The mail logs are full of messages from the web server user.
    > Apache access and error logs similarly show nothing useful. Email
    > relaying is not possible! We do not host external web sites and shutting
    > down the system or email is not an option.


    As said, it might be just a badly written part of what you are
    serving with the apache. E.g. a guest book or message board which
    is basically a mailer and gets its parameters from the URL. If
    this is widely accessible via the internet, anyone can turn your
    system into a spam relay. Check whatever piece of software you
    have running on your webserver for known security issues and up-
    dates.
    Another example of poor configuration is e.g. PHP with url_fopen
    enabled. This basically enables anyone to include remote code
    into scripts running on your server. I don't know off hand if
    this is also possible for e.g. Perl CGIs. But if so, the "bad"
    code and lists of email addresses need not to be located on the
    server itself.
    If you can't find any correlations from the log files, you may
    have to wade through the code of your webapps :-(

    Regards,

    Frank

  5. Re: very annoying problem

    On 2006-12-12, Mike Klein wrote:

    > I am running apache on aix 5.3 with very few problems. However, we
    > occasionally see a perl job (perl es f0 a a post) running as the web
    > server user. This job is sending out tens of thousands of emails to
    > Brazil!

    [...]
    > Has anyone seen anything like this?


    Most probable explanation:

    You've been hacked, and the machine is now being abused to send spam. It
    could be that a rootkit was installed.

    Disconnect the machine from *all* network connections immediately.
    Format and reinstall it from scratch. Make sure that you're not using
    insecure software/scripts.

    --
    Jurjen Oskam

    Savage's Law of Expediency:
    You want it bad, you'll get it bad.

  6. Re: very annoying problem

    On 2006-12-12, Mike Klein wrote:

    > relaying is not possible! We do not host external web sites and shutting
    > down the system or email is not an option.


    I'm quite sure your ISP doesn't agree. If you let this situation continue,
    you'll likely find the machine cut off from the Internet, and your company
    appearing on many blacklists around the Net.

    --
    Jurjen Oskam

    Savage's Law of Expediency:
    You want it bad, you'll get it bad.

  7. Re: very annoying problem

    Mike Klein wrote:

    > What I am trying to determine is how a script that doesn't exist on my
    > system can be run by the web server user who doesn't have a shell
    > account.


    May be the script exist but you cannot see it. If your system has been
    hacked, may be ls, find ... commands have been modified too!

    Frank is right when saying "reboot the server from a
    clean, known-good, read-only media (CD/DVD) into maintainance
    mode and search again."

    And may be there is no other solution than a full re-install.

    Patrick

+ Reply to Thread