Linux

Bash script for updating Drupal Core

Updating Drupal core always makes me a bit nervous, as it involves ripping out the old version and inserting a new. Not having access to Drush, I decided to make a shell script that would automate the steps, eliminating the usual risk of human error. The following is the script. It’s heavily commented, so I’ll keep extra remarks to a minimum.

The script takes one argument, the full URL of the new Drupal version (e. g., http://ftp. Drupal.org/files/projects/Drupal-7.19.tar.gz). Line 20 in the script is the only one that needs editing to match your installation.

Be sure to read the warnings in the first few lines. You need to put your site into maintenance mode prior to running the script, run update.php after, then test and take out of maintenance mode. Also, remember to back up your site!

  1. #!/bin/bash
  2. # WARNING WARNING WARNING
  3. # Before updating Drupal, BACK UP YOUR SITE. http://Drupal.org/node/129202 explains how. Also, set site into maintenance mode.
  4. # After running script, run update.php, then take out of maintenance mode (if everything is working OK).
  5. # END OF WARNING
  6.  
  7. # Check for proper number of command line args.
  8. EXPECTED_ARGS=1
  9. E_BADARGS=65
  10. if [ $# -ne $EXPECTED_ARGS ]
  11. then
  12.         echo “Usage: `basename $0` {arg}”
  13.         exit $E_BADARGS
  14. fi
  15.  
  16. #for the following steps, stop immediately if something doesn't work
  17. set -e
  18.  
  19. #this is a cd to your Drupal site's root directory, change as needed
  20. cd ~
  21.  
  22. #get the update
  23. wget $1
  24.  
  25. #remove path from the wget argument to get name of file
  26. NEW_VERSION=${1##*/}
  27. tar -zxvf $NEW_VERSION
  28.  
  29. #remove .tar.gz from NEW_VERSION to get untarred directory name
  30. USE_DIRECTORY=${NEW_VERSION%.tar.gz}
  31.  
  32. #run through the rest even if there are errors
  33. set +e
  34.  
  35. #just to be sure, check to see if we can see a file in the untarred directory—otherwise, don't delete the old site!
  36. #if we can see the file, then delete old files (as used in Drupal 7), copy new distribution into site, compare old htaccess and robots to new, remove temporary files
  37. if [ -f $USE_DIRECTORY/index.php” ]
  38. then
  39.         rm -f robots. txt.old htaccess.old
  40.         cp robots.txt robots. txt.old
  41.         cp .htaccess htaccess.old
  42.         rm -R -f includes misc modules profiles scripts themes
  43.         rm -f .gitignore .htaccess authorize.php CHANGELOG.txt COPYRIGHT.txt cron.php index.php INSTALL.txt install.php INSTALL.mysql.txt INSTALL.pgsql.txt INSTALL.sqlite.txt LICENSE.txt MAINTAINERS.txt README.txt robots.txt update.php UPGRADE.txt web.config xmlrpc.php
  44.         cp -R $USE_DIRECTORY/ .
  45.         diff .htaccess htaccess.old
  46.         diff robots.txt robots. txt.old
  47.         rm -R $USE_DIRECTORY
  48.         rm $NEW_VERSION
  49.         rm INSTALL*txt install.php
  50. else
  51.         echo $USE_DIRECTORY not found”
  52. fi

Tags: 

Enabling Error Recovery Control in RAID Drives

Consumer hard drives typically make many attempts to read a bad sector, causing hangs in RAID arrays. A quick way to solve the issue without having to buy more expensive “RAID-compatible” drives is to enable error recovery control, setting a time limit on the retries. Some drives support this; others do not. What follows is a simple one-liner that, when added to the system crontab, sets error recovery control to 7 seconds in all the drives in the array that support ERC.

@reboot /bin/grep -o 'sd[a-z][a-z]\?' /proc/mdstat | /usr/bin/xargs -I '{}' /usr/sbin/smartctl -l scterc,70,70 '/dev/{}'

This line means that, at reboot, grep will identify all the sd? devices in the array and feed them one-by-one to smartctl, where the parameter scterc will be set to 70,70.

Tags: 

Logging router events on an Ubuntu server, and checking with Logwatch

Getting all the pieces working was complicated. There are four different things to set up:

  1. The router needs to forward the log to the syslog server — this is router-dependent so not described here
  2. The Linux server needs to accept the logs
  3. Logwatch needs to know what to do with the logs
  4. Logrotate needs to know what to do

Accepting the logs

Edit /etc/rsyslog.conf to provide UDP and TCP reception, by uncommenting the modules in the first section. In my case, the uncommented modules look like this:

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514


Add /etc/rsyslog.d/01router.conf to handle the incoming data before any of the other rules in rsyslog.d are executed, with the following content:

#rules for netgear router to place in separate file
# do this in FRONT of the local/regular rules
if $fromhost-ip == '192.168.0.1' then /var/log/router.log
& ~
#& ~ means stop processing rules



Telling logwatch what to do with the files


Add /etc/logwatch/conf/logfiles/netgear.conf containing

LogFile = router.log
Archive = router. log.?
Archive = rouger. log.?.gz
# Keep only the lines in the proper date range…
*ApplyStdDate


Add /etc/logwatch/conf/services/netgear.conf containing

Title = “netgear”

# Which logfile group…
LogFile = netgear


And, finally, add the code to process the log. This is what I put together:

In the file /etc/logwatch/scripts/services/netgear

#!/bin/sh
#
#variable initialization
#
SEDSCRIPT='/netgear.sed'
DIR=”$( cd “$( dirname “$0” )” && pwd )”
LASTACTION=
 : ${LOGWATCH_DETAIL_LEVEL:=5}

#
#is there a script for sed?
#
[ -s ${DIR}${SEDSCRIPT} ] && LASTACTION=”-f ${DIR}${SEDSCRIPT}”

#
# Should we remove some of the detail?
# These following lines wrap in the blog, but there are four lines

# of code. Each lines should start with [ and end with /d”

[ “$LOGWATCH_DETAIL_LEVEL” -lt 10 ] && LASTACTION=”$LASTACTION -e /Receive.NTP.Reply/d;/Send. out.NTP.request/d”
[ “$LOGWATCH_DETAIL_LEVEL” -lt 8 ] && LASTACTION=”$LASTACTION -e /DHCP.server. received.REQUEST.from…………………/d”
[ “$LOGWATCH_DETAIL_LEVEL” -lt 5 ] && LASTACTION=”$LASTACTION -e /\[UPnP. set.event/d”
[ “$LOGWATCH_DETAIL_LEVEL” -lt 3 ] && LASTACTION=”$LASTACTION -e /DHCP.server/d”


#
#is there something to do after uniq?
#
if [ “$LASTACTION” ]
then
 
cut -c17- | sort | uniq -c | sed $LASTACTION
else
 
cut -c17- | sort | uniq -c
fi


in the file /etc/logwatch/scripts/services/netgear.sed I put code to give friendly names to known entities on the local network and other entities in the log (e. g., time servers)


s/12:34:6B:49:56:21/& T410s wifi /
s/12:43:06:4E:32:23/& Fred's iPad /
s/192.168.0.3$/& Computer in office /
s/192.168.0.8$/& Printer HP 7555 /
s/69.25.96.13$/& nist1.symmetricom.com /

 

Logrotate


In /etc/logrotate.d, add the following (I have it in the file “custom”)
 

/var/log/router.log {
  rotate 5
  weekly
  compress
  missingok
  notifempty
  create 640 syslog adm
}

Tags: 

One-Line Script to Refresh a Web Site Cache by Spidering all Sitemap Entries

Well, the title says it all. I wanted to refresh my web site’s page cache, as some pages are very slow loading otherwise. The site can (with the Drupal Boost module) serve pages without having to reconstruct them if they are in the cache.

After some digging around and consolidating information from a number of sources, I came up with the following that reads the xml sitemap for the site then checks each page without downloading it. Remember to replace www. example.com/sitemap.xml with the correct URL for your sitemap.

/usr/bin/wget -q http://www.example.com/sitemap.xml -O - | /usr/bin/awk -F'</?loc>' 'NF>1{print $2}' | /usr/bin/wget --spider -q -i -

Note that, while the above may be broken into several lines on your browser, it should be a single line for the shell command line unless you use line continuation characters.

I added it to my crontab to run immediately after the Drupal cron job in crontab by adding it on the same line as the Drupal cron job, with a semicolon (;) in between the cron job and the spider commands.

Tags: 

Taming SAMBA

Optimizing Samba for Windows clients

It took me a while to figure this out. Hopefully this can help others

The following changes to smb.conf:

# in the [global] section
# note that wide links are disabled if UNIX extensions 
# are not set to No
wide links = Yes
UNIX extensions = No
# if the clients are on a LAN
socket options = IPTOS_LOWDELAY TCP_NODELAY
# in the [homes] section
# if [homes] is on certain filesystems, 
# the following will improve performance.
# use on  XFS, ext4, btrfs, ocfs2, JFS2 
# do not use on  ext3, ZFS, HFS+
strict allocate = Yes
# if [homes] is on a RAID array, use the following 
# and set to your RAID chunk size
# run the following as root to find the chunk size 
#(change md0 to your raid device as needed)
# mdadm —detail /dev/md0
write cache size = 65536

Stopping those darn warnings in the logs if you don't have printers attached to the Samba server

The following changes to smb.conf:

# in the [global] section
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes

Stopping the complaints about IDMAP_VERSION if your server is a stand-alone server

Run the following commands as root:

service winbind stop
update-rc.d -f winbind remove

Tags: 

Subscribe to Linux