Thomas' notes on Oracle DB

As I wrote in my post “New beginning” I need to track changes made to files on my webserver. When “Shiraz” hacked my site, it wasn’t too hard to analyse what they did to display their own content on my blog. A simple search for “Shiraz” in all files on the webserver reported a file that belonged to the installed and active wordpress theme. Instead of the regular template this file contained plain html which displayed their content.

Bringing back my own content was fairly easy, because the admin panel was still accessible. I simply deleted the active theme and reinstalled it. But what else had they changed? Did they manage to put a backdoor in other files? Did they have shell access? Unfortunately I don’t know. So what to do in this case? The server automatically backups the ~/www directory regularly so I could restore a backup. But of which date? I didn’t know how long I had to go back to get rid of the Shiraz changes, because I didn’t know when Shiraz changed the files. Even if I looked for the template files in the backups, and found the latest backup that does not contain the changes I know of, it could be possible that there was an earlier change to some other files. Shiraz might have made these changes days  before they changed the template. Who knows?

As I only have a shared and managed server with limited shell access, I chose a simple way to get rid of all backdoors: I changed my login password, backed up all user files and removed them completely afterwards. Finally I reinstalled wordpress and I am glad I got it back online ;-)

To get rid of all this time-consuming work the next time someone starts to automatically abuse a security hole, I decided it would be great to know if and when someone changed my files ;-). There are some scripts out there that already exist for this purpose. In general most of them hash file contents and meta data and compare them from run to run. This is a good method but for me, having just a shared server with minimal hardware resources, it has one main disadvantage: It’s resource intensive and time consuming. I wanted to have a somewhat faster and less resource intensive solution.

To achieve this I wrote a little shell script myself. It uses “find” to find files below a (user defined) path that have been modified x minutes ago (where x is defined via parameter to the script). It then mails the list to a user specified address. To avoid false alerts, it supports a blacklistfile where you can put files that change periodically (e.g. log files). As this scripts needs to be run periodically with overlapping scans (so that you don’t miss any changes), it has also a built in mechanism to avoid multiple alerts for the same file change.

#!/bin/bash

######################################################################
#
# dfc.sh - A script to detect file changes
#
#
#
# PREREQUISITES:
#    - mailx must be properly configured to send mails
#    - Working cron daemon and access to crontab
#
# DESCRIPTION:
#    This script is some kind of a very simple IDS. It
#    searches for files in the directory determined
#    by <path_to_search> that have been
#    modified within the last x minutes (determined
#    by <search_period>). It then mails the file list
#    to the mail address specified by <mail_address>.
#    Optionally, you can specify a blacklist containing files
#    for which you don't want to be informed in case of
#    changes (see OPTIONAL FILES for details).
#
#    This script is supposed to be run by cron. The
#    interval in which you should run the script,
#    depends on <search_period>. If the interval
#    is equal or greater than <search_period>, you
#    risk to miss filechanges. It's recommended
#    to run this script every <search_period>/2
#    minutes. The script will take care of
#    files, for which an alert has already been
#    sent.
#
#    This script can safely be run multiple times
#    on the same host.
#
# AUTHOR:
#    Thomas Friedlein - www.thomas-friedlein.de
#
# CREATION:
#    23.06.2011
#
# USAGE:
#    dfc.sh <path_to_search> <mail_address>
#           <search_period> [<blacklist_file>]
#
# FILES:
#    - [<blacklist_file>]:
#        You can specify files to exclude in the output here (e.g.
#        list logfiles here to avoid misleading alerts). List one
#        file per line in this file. Those lines are passed to
#        grep. Excluding /tmp/etc also excludes /tmp/etc/abc and
#        /tmp/etc/asdas. If you just want to exclude /tmp/etc
#        then put a dollar sign at the end, e.g. /tmp/etc$
#
#    - $TEMPFILE (/tmp/dfc_<md5sum of path_to_search>):
#        Stores all files that have been detected at the last
#        run. It is used to avoid multiple alerts for the
#        same change. The filename consists of a prefix
#        and the md5sum of the <path_to_search>, to make this
#        script safe to run multiple times on the same host.
#
#    - $LOCKFILE (/tmp/dfc_<md5sum of path_to_search>):
#        Used to serialize execution of this script
#
# CHANGELOG:
#    - 23.06.2011 Creation
#    - 24.06.2011 Don't send out alert for a changed file more
#                 than once.
#
#####################################################################

# Avoid using uninitialized variables and
# stop script on errors
set -u
set -e

# Check whether user provided less than 3 or more than 4 arguments
if [[ $# -lt 3 || $# -gt 4 ]]; then
 echo "USAGE: dfc.sh <path_to_search> <mail_address>"
 echo "       <search_period> [<blacklist_file>]"
 exit 1
fi

# Check whether the first argument is an existing directory
if [[ ! -d $1 ]]; then
 echo "$1 does not exist or is not a directory"
 exit 1
fi

# If a fourth argument is supplied, check whether it's a file.
if [[ $# -eq 4 && ! -f $4 ]]; then
 echo "$4 does not exist or is not a file"
 exit 1
fi

# Variables
HASH=$(echo $1 | md5sum | awk '{print $1}')
TEMPFILE="/tmp/dfc_$HASH"
LOCKFILE="/tmp/dfc_lock_$HASH"

# Check whether another instance of this script is already
# running by checking the lockfile. If it does not exist
# then create it
if [[ -f $LOCKFILE ]]; then
 echo "Another instance is already running. If "
 echo "not, then manually delete $LOCKFILE"
 exit 1
else
 touch $LOCKFILE
 # If we need to exit unexpectedly,
 # make sure the file is deleted properly
 trap "rm -f $LOCKFILE; exit" INT TERM EXIT
fi

# Execute command and store output in variable
# for further processing.
CHANGED_FILES=$(find $1 -mmin -$3 -ls)

# Stop if no files changed at all. Also delete the tempfile
# if it exists in this case
if [[ $(echo "$CHANGED_FILES" | grep -v "^$" | wc -l) -eq 0 ]]; then
 echo "No files changed"
 rm -f $LOCKFILE
 if [[ -f $TEMPFILE ]]; then
 rm -f $TEMPFILE
 fi
 exit 0
fi

# If a fourth argument exists, grep
# changed files with blacklist.
if [[ $# -eq 4 ]]; then
 CHANGED_FILES=$(echo "$CHANGED_FILES" | grep -vf $4)
fi

# Again stop if no files changed after filtering. Also
# delete the tempfile if it exists in this case
if [[ $(echo "$CHANGED_FILES" | grep -v "^$" | wc -l) -eq 0 ]]; then
 echo "No files changed"
 rm -f $LOCKFILE
 if [[ -f $TEMPFILE ]]; then
 rm -f $TEMPFILE
 fi
 exit 0
fi

# If the tempfile exists, filter out those files that have
# already been found on last run. If it doesn't exist
# then don't filter anything for display
if [[ -f $TEMPFILE && $(cat $TEMPFILE | grep -v "^$" | wc -l) -gt 0 ]]
then
 # Grep returns 1 if there are no rows returned.
 # Because of this, set +e is need so that the script
 # is not aborted. Afterwards it is set back.
 set +e
 FILES_TO_DISPLAY=$(echo "$CHANGED_FILES" | grep -vf $TEMPFILE)
 set -e
else
 FILES_TO_DISPLAY=$CHANGED_FILES
fi

# Write the datepart and complete path of all found files in tempfile
# to filter them out the next time the script is run. "$" is applied
# to the end to make sure the filenames are matched exactly
echo "$CHANGED_FILES" | awk '{print $8 " " $9 " " $ 10 " " $11 "$"}' \
> $TEMPFILE

# Stop if there are no files to display. This time, do not delete
# tempfile on exit, because although there are no files to display
# we found some files that need to stay in tempfile for next run
if [[ $(echo "$FILES_TO_DISPLAY" | grep -v "^$" | wc -l) -eq 0 ]]; then
 echo "No files changed"
 rm -f $LOCKFILE
 exit 0
fi

echo "$FILES_TO_DISPLAY"
echo "$FILES_TO_DISPLAY" | mailx -s "Files below $1 modified in the \
last $3 minutes" $2

# Delete lockfile and exit
rm -f $LOCKFILE
exit 0

Although this script might not be perfect it’s a simple and fast way for this task. Shrinked to the core task it’s a one-liner, the rest of the lines are just checks or provide some additional, not in every case needed features.