Hugo FTP deployment

deployment-with-sitecopy
Table of Contents

If you have a small Hugo static site and your hosting provider just offers FTP access and not SSH/rsync, synchronizing the site manually can be difficult. With access to your web host via FTP, you can use sitecopy1 to incrementally deploy your entire Hugo site, similar to rsync solution.

sitecopy is a tool for copying locally stored web sites to remote web servers. A single command will upload files to the server which have changed locally, and delete files from the server which have been removed locally, to keep the remote site synchronized with the local site. The aim is to remove the hassle of uploading and deleting individual files using an FTP client. sitecopy will also optionally try to spot files you move locally, and move them remotely.

Assumptions

  • Access to your web host with FTP
  • A functional static website built with Hugo

Installing sitecopy

sitecopy is available in all major distributions, so just use your favourite package manager to install it:

sudo apt-get -y install sitecopy

sitecopy can also be installed on Windows if you are using cygwin2 via apt-cyg3: apt-cyg install sitecopy

Optionally, you should take a quick look at sitecopy’s man page to familiarize yourself with its options:

man sitecopy

Configuring sitecopy

In your home folder create the directory .sitecopy (with permissions 700) – sitecopy uses this directory to store synced files details. After that, we will need to create the sitecopy configuration file .sitecopyrc:

1
2
3
4
cd ~
mkdir -m 700 .sitecopy
touch .sitecopyrc
chmod 600 .sitecopyrc

Open .sitecopyrc file in your editor and fill in the configuration for the hugosite.com site:

vim .sitecopyrc
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
site hugosite.com
  # ftp server name or IP
  server ftp.hugosite.com
  username hugo
  password hugopwd
  # Hugo public folder
  local /home/hugo/sites/hugosite.com/public/
  # ftp remote web root
  remote /public_html/
  exclude *.bak
  exclude *~
  exclude /public_htm/.smileys
  exclude /public_htm/cgi-bin
  # Use checksumming and look for moved and renamed files
  state checksum
  checkmoved renames

The state checksum configuration is important here. Since Hugo does not support incremental builds, all the files in the public folder will be regenerated when you rebuild your Hugo site. By using checksum method for the sitecopy state (and not the default timesize) we make sure that only the files that are truly modified will be synchronized.

Most of the configuration options are self-explanatory. The site directive must be followed by a name for the web site – you can freely choose one, e.g. hugosite.com or ftp.mysite.com – this name will be used later on with the sitecopy commands. The local directive contains the local path of the web site – in this case, Hugo’s public folder, remote contains the path of the web site on the remote server – it can be absolute or relative.

The exclude lines are optional, they allow you to exclude files and folders (last 2 exclude lines) from being synchronised with sitecopy.

Initialising the site

For this example we will use the most common scenario, where the local copy exists – your Hugo site – and you have an empty remote site.

There are other scenarios that might fit your situation for witch you will need to consult the sitecopy man page and modify the script accordingly.

First initialise the site (replace hugosite.com with the name of the site you use in the .sitecopyrc file):

1
2
3
hugo@ubuntu:~$ sitecopy --init hugosite.com
sitecopy: Initializing site 'hugosite.com' (on ftp.hugosite.com in /public_html/)
sitecopy: All the files and directories are marked as NOT updated remotely.

Then, upload the local copy to the remote site using sitecopy --update hugosite.com:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
hugo@ubuntu:~$ sitecopy --update hugosite.com
sitecopy: Updating site 'hugosite.com' (on ftp.hugosite.com in /public_html/)
Creating assets/css/fonts/: done.
Creating archive/: done.
Uploading assets/css/fonts/merriweather-700italic.woff2: [.] done.
Uploading assets/css/fonts/merriweather-700italic.woff: [.] done.
Uploading index.html: [.] done.
Uploading robots.txt: [.] done.
Uploading .htaccess: [.] done.
Uploading favicon.ico: [.] done.
sitecopy: Update completed successfully.

sitecopy will MD5 checksum every file from your public Hugo folder and store the information in the ~/.sitecopy/[sitename] file. Here is a excerpt from the ~/.sitecopy/hugosite.com file generated by the command above:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
<?xml version="1.0" encoding="ISO-8859-1"?>
<sitestate version='1.0'>
    <options>
        <saved-by package='sitecopy' version='0.16.6'/>
        <checksum-algorithm><checksum-MD5/></checksum-algorithm>
        <state-method><state-checksum/></state-method>
        <escaped-filenames/>
    </options>
    <items>
        <item><type><type-file/></type><filename>archive/index.html</filename>
        <protection>775</protection><size>44876</size><checksum>46218346c37c658130ad5d77aae75dc0</checksum><ascii><false/></ascii></item>
        <item><type><type-file/></type><filename>assets/css/fonts/merriweather-700italic.woff2</filename>
        <protection>775</protection><size>28436</size><checksum>89179a1c446815565bc6e67ce3e9e049</checksum><ascii><false/></ascii></item>
        <item><type><type-file/></type><filename>assets/css/fonts/merriweather-700italic.woff</filename></item>
        <item><type><type-directory/></type><filename>assets/css/fonts</filename>
        <protection>775</protection></item>
    </items>
</sitestate>

Next time when you will build your Hugo site and run sitecopy --update , sitecopy will check the MD5 checksum of the files and update them accordingly.

Using Hugo and sitecopy

Hugo FTP sync via sitecopy usage is really easy. Modify your Hugo site as needed and when done build it again, e.g.: hugo --minify --cleanDestinationDir --gc.

Next, an optional step is to run sitecopy hugosite.com to find out which files have changed locally (replace hugosite.com with the name of the site you use in the .sitecopyrc file):

1
2
3
4
5
hugo@ubuntu:~$ sitecopy hugosite.com
sitecopy: Showing changes to site 'hugosite.com' (on ftp.hugosite.com in /public_html/)
* These items have been added since the last update:
index.html
sitecopy: The remote site needs updating (1 item to update).

To synchronize your local Hugo site – upload new and changed files to the remote server and delete files on the remote server that have been deleted locally – you simply run sitecopy --update hugosite.com:

1
2
3
4
hugo@ubuntu:~$ sitecopy --update hugosite.com
sitecopy: Updating site 'hugosite.com' (on ftp.hugosite.com in /public_html/)
Uploading index.html: [] done.
sitecopy: Update completed successfully.

Shell Script

For your convenience, you can find most of above in the shell script below. For this script to work, you need to initialise the site first, as described above.

1
2
3
4
cd ~/hugo/sites/hugosite.com
touch ftp-sync.sh
chmod +x ftp-sync.sh
vim ftp-sync.sh
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
#!/usr/bin/env bash

# Script parameter (if no parameter is passed, use 'check' as default).
COMMAND=${1:-check}

# Site name (from ~/.sitecopyrc).
SITE_NAME="hugosite.com"

# sitecopy 'changes.awk' file path.
# (comes with sitecopy installation and it is used to generate
# a human friendly HTML file with the changes)
AWKF="/usr/share/doc/sitecopy/examples/changes.awk"

# Changes output file.
HTMLF="changes.html"

# Check the script parameter (must be either 'check' or 'sync').
if [[ -z "$COMMAND" ]] || [[ "$COMMAND" != "sync" && "$COMMAND" != "check" ]]; then
  echo
  echo "Usage: $0 [check | sync]"
  exit 1
fi

# Check if there are any changes.
if sitecopy --list ${SITE_NAME} > /dev/null; then
  echo
  echo "No changes to the site ${SITE_NAME}"
  exit 0
fi

# Check or Sync the site.
if [ "$COMMAND" == "check" ]; then
  # Check the changes, put them in a HTML file and display them 
  # via lynx browser.
  echo
  echo "Checking site sync state (see ${HTMLF} file)..."
  sitecopy --flatlist "${SITE_NAME}" | gawk -f "${AWKF}" > "${HTMLF}"
  if [ -e "${HTMLF}" ]; then
      # Optionally, open 'changes.html' in lynx browser.
      lynx "${HTMLF}" -assume_charset=utf8
  fi
else
  echo
  echo "Syncing site ${SITE_NAME}..."
  echo
  sitecopy --update --quiet --show-progress "${SITE_NAME}"
  # Beep for long operations.
  echo -ne '\007'
fi

The script accepts two parameters:

  1. check (default, optional) – to check and display the current changes. Running the script with this option (or without any option) will check the changes in the local site (via the ~/.sitecopy/[sitename] file), put them in a HTML file (see the –flatlist option in man page) and display them via lynx4 browser.

        ./ftp-sync.sh check
        # or
        ./ftp-sync.sh
    chnages.html
    changes.html in lynx
  2. sync – to perform the actual synchronisation if there are any changes available:

        ./ftp-sync.sh sync

Feel free to adapt it to suit your needs and happy Hugo!