Configure Beanstalkd and Utilities on macOS

Posted on Tuesday September 24th, 2019

Beanstalkd is a "simple, fast work queue". Originally created for web applications, it's so simple that it can be used on any project that requires async job processing. In my case, it's a must for any web project of medium complexity. Thanks to its simplicity and popularity, the beanstalkd service has been compiled and ported to most OSs, including macOS, and there are client libraries for almost every language.

I've written this guide to install and configure the core beanstalkd service on macOS, as well as a helper monitor utility to simplify my web development workflows. (This guide has only been tested on macOS High Sierra and macOS Catalina.)

Install Beanstalkd

Installation requires Brew, the package manager for macOS. With brew, there's already a published tap/formula for beanstalkd, so you only need to type:

brew install beanstalkd

As a note, macOS is not very friendly to some services. If you think your beanstalkd installation is misbehaving, you can nudge it with:

brew services restart beanstalkd

Beanstalkd monitor and utilities

There are some tools recommended by the official beanstalkd project to monitor and debug the service. For me, the simplest and most straight-forward monitor is the PHP-based beanstalk_console by ptrofimov. Also, since this is a PHP project, we can install it locally in macOS and run it with the built-in PHP standalone server.

To locally install the project, you will need Composer, a PHP dependency manager (you can quickly check if it's installed by running composer --version on the Terminal). If you don't already have Composer, installation is as simple as typing:

brew install composer

Once Composer is installed, this one-liner will install and configure the beanstalk_console monitor (replace path/to/install to your desired install location):

composer create-project ptrofimov/beanstalk_console -s dev path/to/install

We'll also create a helper run.sh script inside the path/to/install directory. This script prepares your dev environment by restarting Beanstalkd, launching a local PHP server with the monitor, and finally opening a new tab on Chrome pointing to it (127.0.0.1:8005).

#!/bin/bash

# CD into the script's actual directory
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
cd ${DIR}

# Restart the core Beanstalkd service
brew services restart beanstalkd

# Launch a new Chrome window pointing to the console
open -a "Google Chrome" "http://127.0.0.1:8005"

# Start the local PHP server with the beanstalk_console
php -S 127.0.0.1:8005 -t public

Don't forget to make it executable:

chmod +x run.sh

After running the script, you Terminal should display:

$> ./run.sh

PHP 7.1.23 Development Server started at Tue Sep 24 18:12:18 2019
Listening on http://127.0.0.1:8005
Document root is /path/to/beanstalk_console/public
Press Ctrl-C to quit.

That's it!



Enabling NTP Time Synchronization in Ubuntu 18.04

Posted on Monday November 19th, 2018

Before Ubuntu 16.04, most network time sync was handled with the Network Time Protocol (NTP) and its daemon (ntpd). Using NTP, the local machine connects to a pool of NTP servers that provide it with constant and accurate time updates. However, since Ubuntu 16.04 the default time synchronization deamon being used is timesyncd, and ships enabled by default on all new Ubuntu installations.

You can easily check it by running the timedatectl command:

$ timedatectl
                      Local time: Mon 2018-11-19 17:17:21 UTC
                  Universal time: Mon 2018-11-19 17:17:21 UTC
                        RTC time: Mon 2018-11-19 17:17:22
                       Time zone: Etc/UTC (UTC, +0000)
       System clock synchronized: yes
systemd-timesyncd.service active: yes
                 RTC in local TZ: no

Take a note at the line systemd-timesyncd.service active: yes, which means that the time synchronization deamon being used is timesyncd, which is the default on Ubuntu installations ship with since 16.04:

Since Ubuntu 16.04 timedatectl / timesyncd (which are part of systemd) replace most of ntpdate / ntp.

timesyncd is available by default and replaces not only ntpdate, but also the client portion of chrony (or formerly ntpd). So on top of the one-shot action that ntpdate provided on boot and network activation, now timesyncd by default regularly checks and keeps your local time in sync. It also stores time updates locally, so that after reboots monotonically advances if applicable.

timesyncd should be fine for most purposes, but for some high-precision applications NTP is still the way to go.

Enabling ntpd

Before installing and enabling ntpd (the client or deamon process), we have to disable the default timesyncd

$ sudo timedatectl set-ntp no
$ timedatectl
                      Local time: Mon 2018-11-19 17:35:21 UTC
                  Universal time: Mon 2018-11-19 17:35:21 UTC
                        RTC time: Mon 2018-11-19 17:35:22
                       Time zone: Etc/UTC (UTC, +0000)
       System clock synchronized: yes
systemd-timesyncd.service active: no
                 RTC in local TZ: no

We check that the systemd-timesyncd-service is disabled, and then we proceed to install ntp via apt-get.

$ sudo apt-get install -y ntp

The NTP daemon should have been started by default after the restart. To verify that it was correctly installed and working we can use the query tool for NTP ntpq, using the -p flag to print information about it's peers (NTP servers).

$ sudo ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 0.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 1.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 2.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 3.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
-ntp6.flashdance 192.36.143.152   2 u   38   64    1  159.692   10.730   2.336
+ntp7.flashdance 194.58.202.148   2 u   39   64    1  164.543    9.626   2.791
*time100.stupi.s .PPS.            1 u   39   64    1  155.059    6.076   2.629
+ntp8.flashdance 192.36.143.151   2 u   41   64    1  164.777    6.715   1.999
-ntp2.flashdance 194.58.202.148   2 u   38   64    1  153.563    5.630   3.300
-ntp5.flashdance 192.36.143.151   2 u   41   64    1  149.059    6.564   2.724
 chilipepper.can 17.253.34.253    2 u   48   64    1  131.168    0.113   0.000
 golem.canonical 145.238.203.14   2 u   51   64    1  129.830    0.024   0.000
 pugot.canonical 193.79.237.14    2 u   51   64    1  129.574    0.071   0.000
 alphyn.canonica 17.253.52.125    2 u   50   64    1   67.961   -0.367   0.000

Run timedatectl one last time to check that the clock is synced.

$ timedatectl
                      Local time: Mon 2018-11-19 18:13:22 UTC
                  Universal time: Mon 2018-11-19 18:13:22 UTC
                        RTC time: Mon 2018-11-19 18:13:22
                       Time zone: Etc/UTC (UTC, +0000)
       System clock synchronized: yes
systemd-timesyncd.service active: no
                 RTC in local TZ: no

Even though the timesyncd service is off, the System clock synchronized flag should be yes because we are using ntpd on the background.

Manually Forcing a Sync

If the system's clock is desynchronized by more than ~3 seconds, then ntpd might not be able to automatically sync the clock. In this case, we have to manually force the first sync. To do this, we have to stop the ntp service to release the UDP port 123, then we run the forced sync, and after that's done we turn the ntp back on.

$ sudo service ntp stop
$ sudo ntpd -gq
$ sudo service ntp start

We run the command ntpd -gq, the -gq flags tell the NTP daemon to adjust the time irrespective of the skew (g) and exit (q) immediately.

References