How to persist folders and files with dokku and docker-options

If you want to persist folders or files for an application deployed with dokku, here is how I got it to work. To take you through the steps, we will be creating a little node app that creates a file on each startup/deploy and displays all the files created when you hit the / route.

TL;DR: If you don't enjoy the handholding, you can skip straight to the part where we mount our volumes: Initialize the app.

First, let's create a simple package.json to get us started:

package.json

{
    "scripts": {
        "start": "node main"
    },
    "engines": {
        "iojs": "2.x"
    }
}

Install express

Then, let's install our only dependency, express:

$ npm i express -S

Also, let's make that our folder (to which we will write our files) exists:

$ mkdir -p storage

Now for our little application:

main.js

'use strict'
const fs = require('fs')
const express = require('express')
const path = require('path')
const port = process.env.PORT || 3000
const dirName = process.env.STORAGE_DIR || path.join(__dirname, 'storage')
const app = express()
fs.writeFileSync(path.join(dirName, '' + Date.now()), '')
console.log('wrote to ' + dirName)
app.get('/', function(req, res) {
    fs.readdir(dirName, function(err, files) {
        if (err) {
            res.status(500).end('Unkown error')
            console.error(err.stack || err)
            process.exit(1)
        }
        files.forEach(function(file) {
            res.write(file + '\n')
        })
        res.end()
    })
})
app.listen(port)

Notice that I am using an environment variable STORAGE_DIR, because we will set this in our production environment.

If you run node main and visit localhost:3000 you should see one or more newline-separated timestamps. Ok, let's get this ready for deployment with dokku.

.env

BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs
STORAGE_DIR=/storage

Our .env file sets up some important environment variables. First, we'll be using heroku's node.js buildpack that lets us use io.js. Also, we're setting our STORAGE_DIR environment variable as announced previously.

.gitignore

node_modules
storage
.DS_Store

Put the usual suspects in your .gitignore (don't forget the storage folder). Now initialize git:

Initialize git

$ git init
$ git add -a
$ git commit -m "initial commit"
$ git add remote dokku dokku@dokku:persistence

Initialize the app

$ ssh dokku apps:create persistence
$ ssh dokku docker-options:add persistence "run -v /home/apps/persistence/storage:/app/storage"
$ ssh dokku docker-options:add persistence "deploy -v /home/apps/persistence/storage:/app/storage"

Note that it is not a mistake that we are specifying /app/storage to be persisted although we specified STORAGE_DIR=/storage (without /app) in our .env file.

After that is done, we are ready to deploy:

Deploy the app

$ git push dokku master

When that is done, you can run

$ ssh dokku logs persistence

to see something like this:

Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
Recommending WEB_CONCURRENCY=1
> @ start /app
> node main
wrote to /app/storage

Note again that the console says we are writing to /app/storage and not /storage, even though STORAGE_DIR=/storage.

Visiting the app in your browser will give you a single timestamp.

Making sure everything works

First, you will want to see if your options are set correctly:

$ ssh dokku docker-options persistence

You should see something like this:

Deploy options:
    -v /home/apps/persistence/storage:/app/storage
Run options:
    -v /home/apps/persistence/storage:/app/storage

Next, you can ssh into your machine, look at the persisted directory

$ ls /home/apps/persistence/storage/

and you should see a single file with a timestamp as a name.

Ok, everything looks good. Now let's rebuild our app and see if we get a second timestamp added while keeping the first one we created:

$ ssh dokku ps:rebuild persistence

When the build is done, you should be able to revisit your site and see that a second timestamp was added. It works!

Destroying persisted apps

Please be aware that when you destroy your app by running

$ ssh dokku apps:destroy persistence

the persisted folders will stay on the remote system and you will need to delete them manually if you want them gone.