Blog Posts by Andrew B. Collier / @datawookie

The Google Mobility Data (or Community Mobility Reports) refers to the datasets provided by Google which track how people move and congregate in various locations during specific time periods. The data is based on anonymised location information from users who have opted into Location History on their Google accounts.

Fathom Data is working on a project to reproduce the figures from the CORE textbook The Economy using R and {ggplot2}. There’s a strict style guide which specifies the figure aesthetics including colours and font. We’re a team of seven people working on as many different setups. The principle challenges have been package versions and fonts.

I’ve been following an excellent tutorialfor deploying a Docker image on an EC2 instance via GitLab CI/CD. It covers every step in the process in great detail. If you follow the steps then you’ll definitely end up with a working pipeline.

However, I still wasn’t quite sure how to handle the environment variables and credentials that I wanted to bake into the image, and which varied between my local development environment and the final deployed image.

📢 An updated version of this post reflecting recent changes in GitLab Runner can be found here.

I’ve got a project which takes a long time to build. And I rebuild it regularly. I’ve been using the shared runners on GitLab. However, the total time constraint has become a limitation. I’m going to install GitLab Runner as a Docker service on an underutilised EC2 instance.

Two new features in the {emayili} (0.4.6) package for easily sending emails from R.

Package Setup

If you have not already installed the package, then grab it from CRAN or GitHub.

# From CRAN.
install.packages("emayili")
# From GitHub.
remotes::install_github("datawookie/emayili")

Load the package.

library(emayili)

Check that you have the current version.

packageVersion("emayili")

[1] ‘0.4.6’

Let’s quickly set up an SMTP server. We’ll use SMTP Bucket, which is incredibly convenient for testing.

SMTP_SERVER   = "mail.smtpbucket.com"
SMTP_PORT     = 8025

smtp <- server(host = SMTP_SERVER, port = SMTP_PORT)

UTF-8 Characters in Attachment Filenames

It’s now possible to attach files with names that include non-ASCII characters. Suppose I wanted to send this image (source) of Wenceslao Moreno.

I’ve been hosting a MySQL database on a DigitalOcean server for a few of years. The project has been on hold for a while. Entropy kicked in and the server became unreachable. Fortunately I was still able to access the server via a recovery console to export the database using mysqldump and download the resulting SQL dump file.

Now I want to resurrect the database locally but I also want to migrate it to PostgreSQL.

Inspired by the informative post from Jumping Rivers about selecting the correct image file type, I decided to optimise PNG file size as part of this blog’s CI pipeline.

Suppose that you want to use {emayili} to send birthday messages (this post motivated by issue #61).

I ran into a bit of a snag when updating a {blogdown} site. Suddenly, inexplicably, the images were no longer present.

I have a rather obscure situation where I want to launch Selenium… but with JavaScript disabled.

According to the Automobile Association (AA) the fuel price is the sum of four main components:

• the basic fuel price
• the general fuel levy
• the Road Accident Fund (RAF) levy and
• wholesale and retail margins, distribution and transport costs.

This article suggests that almost 70% of the fuel price in South Africa is due to taxes and levies.

How much of South Africa's petrol price goes to taxes https://t.co/m96j0TaKlS

— BusinessTech (@BusinessTechSA) January 29, 2021

I used data from {saffer} to examine this assertion.

I started using rain (a South African ISP) back in March 2019. The coverage was good (the only place I couldn’t get a signal was at Lanseria Airport), while the bandwidth was consistently high. I loved the fact that it was affordable, reliable and portable.

Much has changed.

I have a project where I need to have a persistent Selenium session. There’s a script which will leave a browser window open when it exits. When the script runs again it should connect to the same window.

The transcripts for the South African President’s speeches are available here. I’ve just added these data to the {saffer} package.

I’ve been adding topological maps of South Africa to the saffer-data-map repository. The maps are originally in MrSID format (.sid files), which is a proprietary file format developed by LizardTech (a company which has been consumed by Extensis).

The {saffer} package is a nascent collection of things relating to South Africa. I’ve just added fuel price data.

I have a SQLAlchemy count() query which is being called fairly frequently in my API. The query itself is not terribly inefficient, but it’s being called with sufficient frequency that it has a performance impact.

A couple of days ago I posted an analysis of the price of nitrile gloves at Dischem.

I had a suspicion that there was more data beyond the history that I got from Strava (see previous post). And indeed my hunch was confirmed by downloading my history from Garmin Connect.

My colleague, Matt, noted that nitrile gloves are getting really expensive.