Uploading files to two separate locations

Question

I'm uploading a clients website early next week and they've just informed me that they use load balancing software to distribute over two servers.

Normally this wouldn't be an issue because content is stored in the database and as long as both files areas can access it then it should be fine.

However, the issue comes in when uploading files. I've been informed that files should be uploaded to both servers.

I wouldn't even know where to begin to make this modification. Can anybody shed some light on it for me?

Is there a global method that's invoked when files are uploaded?

Note: This needs to work with the file field on several content types and even when files are added through modules/cck

Answer 1

I very much suggest a solution which actually shares the files/ directory. My primary recommendation here is some dedicated hardware. To avoid marketing for anyone in particular, I just recommen you read, on Wikipedia, SAN and FAN.

If hardware is out of the question, then we have good experience with NFS for direct sharing. Samba on the other hand has proven to be to slow.

There is also a "native" Drupal solution called Storage API which can do a lot of neat things with your files, but I don't think your particular needs can be solved out of the box, but would require some extra coding.

I would only use some type of syncing as a last resort, but here's a really basic bash-script you could cron run. The major part of this script is a safeguard so it will refuse to start if it's already running.

#!/bin/bash
LOCK_FILE=~/.lockfile

if [ -f "${LOCK_FILE}" ]; then
  PID=$(head -n1 "${LOCK_FILE}")

  TEST_RUNNING=$(ps -p ${PID} | grep ${PID})

  if [ -z "${TEST_RUNNING}" ]; then
    echo $$ > "${LOCK_FILE}"
  else
    exit 0
  fi
else
  echo $$ > "${LOCK_FILE}"
fi

rsync -az local_files/ username@host:/remote/path/to/files
rm -f "${LOCK_FILE}"

exit 0

Answer 2

I would imagine that if the 2 storage servers are accessible from the web server through a share or mount, then you can make some system calls in your custom module to copy the file to the 2 locations.

Or do something really clunky with the sysadmins and do a schedule task or cron job to mirror a directory.