Blog (my web musings)

I was recently contacted by a friend who was in a bit of a quandary owing to hundreds of hourly delivery failure messages that were suddenly landing in their inbox. They weren't sending the emails themselves (via an email client) but the wave of activity had caught the attention of the web host who was threatening to suspend the account if such nefarious activity wasn't stopped immediately. Concerned by the threat, and alarmed that they were being blamed for something that they had no knowledge of, I was asked if I could help.

Change passwords, minimise risk

I had a chat with my friend about the email accounts associated with the domain; There were a handful of official business ones, a few accounts used for personal correspondence between family members, and a couple of old test accounts that had long been forgotten about. I advised him to delete all the accounts that were infrequently used or that were expendable. I also suggested that he changed passwords for everything that remained, including FTP settings and the main user password for the hosting account. I didn't really expect this to make a difference to the immediate email flood though because I suspected that the emails were being sent programatically via a script on the server, but at least it showed his web host that he was taking steps to rectify the problem. As predicted, the emails kept on coming.

Track traffic

In addition to the main website, the hosting account was home to a few additional sub-domains that individually had a total overall traffic score logged for the last 7 days. These were again made up of old test sites (CMS software trials) and personal hobby sites used by his children. I could see from the figures that one was particularly busy, with hundreds of MBs of daily activity, compared with similar sub-domains that only reported a few bytes of throughput. This looked very wrong and indicated where the problem was most likely originating from.

Look for dodgy code

A cursory look at the contents of the affected sub-domain showed that it was a playground for test scripts and home to a lot of old freebie software. It was a hobby site that was currently being used for college coursework and could therefore not be deleted outright until the work had been assessed, so I decided to download the entire website via FTP and scan the contents for likely hacker code, using a free tool called AstroGrep.

With AstroGrep I searched on the following text / code snippets. Each are known to be functions that are commonly used by hackers;

• \x68\x74\x74\x70 - hex encoded version of "http" that could be used to reference files on another domain
• curl_exec - could be used to reference files on another domain
• eval(stripslashes - used to decode obfuscated code
• base64_decode - used to decode obfuscated code, but be aware that this is a legitimate function, for example, it is used in core Joomla! files, so check each occurance for unnatural / ill-formatted / dodgy code
• str_rot13 - used to decode obfuscated code

I returned to my friend with a list of files that did indeed contain suspicious-looking code, and we systematically worked through them, with the help of his college-student son, either deleting files entirely, or replacing them with clean versions from backups. I thought this might be the end of all the emails, but the next day, my phone rang again... More spam delivery failure notifications.

Access logs

So if a spammer wasn't making use of hacked files and nefarious code, I deduced that they must be using legitimate files that were somehow less-than-secure - maybe a contact form, or something similar.

I checked the access logs for the sub-domain and scanned for files / pages that had been very frequently hit over the last week. I was looking for something that had been accessed hundreds of times per day, in very quick succession, in the hopes that I could identify and block the attacking IP(s) via .htaccess. I found it in the form of a guestbook script. Frustratingly, the access log only contained obfuscated IP addresses (new GDPR compliance by the web host?), so I decided to take matters into my own hands; Assuming that the repeatedly hit guestbook page would continue to be a hot target for at least the next few days / weeks, I would use it against the attackers, turning it into a logging script to record their offending IP(s) myself.

Identify spammer, deny from site

The guestbook script was removed but I inserted the following PHP script inside the page to instead record accessing IPs;

<?php 

$log_file = '_access_log.txt'; // file to store logged IPs

function getIP() { 
    $ip; 
    if (getenv("HTTP_CLIENT_IP")) { $ip = getenv("HTTP_CLIENT_IP"); }
    else if (getenv("HTTP_X_FORWARDED_FOR")) { $ip = getenv("HTTP_X_FORWARDED_FOR"); }
    else if (getenv("REMOTE_ADDR")) { $ip = getenv("REMOTE_ADDR"); }
    else { $ip = "UNKNOWN"; }
    return $ip;
    }
$accessip = getIP(); // get IP to use in '_access_log.txt'

echo "<p>Sorry, IP <b>$accessip</b> does not have permission to access this area.</p>"; // message to display in browser

// log IPs to file
$lines = file($log_file);
$fpl = fopen($log_file,'w+');
$text = date('jS M Y, h:ia').' - [ACCESS ATTEMPT] - IP: '.$accessip; // write this text per line
fwrite($fpl, $text."\r\n");
foreach ($lines as $line) { fwrite($fpl, $line); }
fclose($fpl);

// delete old lines in log file
$numlines = count(file($log_file));
if ($numlines > 1000) { // keep 1000 lines in log file
    $oldlines = file($log_file); 
    $lastline = sizeof($oldlines) -1; 
    unset($oldlines[$lastline]); 
    $fp = fopen($log_file, 'w'); 
    fwrite($fp, implode('', $oldlines)); 
    fclose($fp);
    }

?>

I also created a blank file called "_access_log.txt", with 755 permissions, to log all the access attempts to the guestbook page.

As it currently stands, the script is only logging IPs that hit the guestbook page. It isn't doing anything with them once the IPs are identified. That's where the human eye comes in to play. For several days, I extracted the contents of the log file and noted down any IPs that appeared multiple times (i.e. hundreds). I then blocked them from return visits using the deny from directive in an .htaccess file at the root of the domain.

And the flood of spam emails? You'll be pleased to learn that they stopped almost immediately. I check the file once a month now to wheedle out any persistent stragglers, but the bulk of the attackers have now been blocked. Problem solved!