A quick primer on dkron

So I’ve been experimenting with dkron as a way to run distributed crons. Now that I’ve got it all working, I think I can make sense of it – but getting to this part was admittedly a pain, in part because the documentation is a bit obtuse. Hopefully the following helps you if you’re stuck.

Note: I’m using Consul with this, so if you use etcd or something else, your mileage may vary.

When you install dkron from a deb package, it creates a few things for you: (1) an Upstart service; (2) a folder at /usr/share/dkron that has the frontend code; (3) a sample config at /etc/dkron/dkron.json.example. So what you want to do is copy the dkron.json.example to dkron.json and configure it. Then just run service dkron start, and it’s running.

Now, this configuration. Here’s mine (the formatting is for Ansible templating; use as you see fit):

{
  "backend": "consul",
  "backend_machine": "{{ consul_path }}",
  "keyspace": "bkg-process",
  "http_addr": ":8080",
  "server": true,
  "encrypt": "{{ dkron_encrypt }}",
  "tags": {
    "role": "my-role"
  },
  "join": "my-cluster.service.dc1.consul"
}

A couple of things to note here. First, the consul_path is something like 1.2.3.4:8500. All instances that you plan to link together should have the same encrypt, and they should all have a role in common. The last part, the join – that’s the most important part. Here, I’m saying, “use Consul to find other my-clusters and join one of them”. (I’m using dnsmasq along with Consul to achieve that part.) If you’re not using Consul for service discovery this way, then feel free to put a static IP there.

Oh, another thing: if you’re using Consul, note that as of this writing, using Consul’s ACL is not supported – so you’ll have to set permissions on the Anonymous Token. Set everything in the keyspace to be able to write:

key "bkg-process/" {
policy = "write"
}

Once you have the config set up, it should join up with any other servers it finds. You can check this via curl:

curl http://127.0.0.1:8080/v1/members

If you see more than just your instance in there, great – they’re connected. To find out which service is the leader, check:

curl http://127.0.0.1:8080/v1/leader

and you should get back the hostname and address of one of the instances.

Okay, next: how do you actually add a cron job? You hit the API:

curl -n -X POST 127.0.0.1:8080/v1/jobs \
  -H "Content-Type: application/json" \
 \
  -d '{
  "name": "cron_job",
  "schedule": "0 * * * * *",
  "command": "/run-it.sh",
  "owner": "Me",
  "owner_email": "foo@example.com",
  "run_as_user": "ubuntu",
  "disabled": false,
  "tags": {
    "role": "my-role:1"
  }
}'

Once your instances are clustered, I believe you can run this on any of them and it should work correctly. First, notice that I’m curling against the local dkron instance. Next, note the schedule line. Since it’s six items, I believe the first is seconds (don’t quote me on that, though), so be careful what you put there. I’ve also noticed that when making a change, I sometimes had to restart the dkron service. I’m not really sure why that is, though; maybe it gets stuck sometimes. Anyway, the SINGLE MOST IMPORTANT PIECE HERE is the role. Note that it says “my-role:1”. This means that it will only run on ONE instance with role my-role. If you leave off the :1, it will run on ALL the instances with role my-role. This caused me quite a few hours of head-scratching.

I imagine I’ll have more to talk about on this subject, but I wanted to get something written (before it all goes out of my head).