Blog Posts by Andrew B. Collier / @datawookie

A sitemap serves as a navigational blueprint for search engines, ensuring they can efficiently crawl and index all essential pages of a website. By providing a structured list of URLs, a sitemap streamlines the discoverability of content, especially in complex or extensive sites. This not only optimizes search engine ranking and visibility but also ensures that any updates or new content additions are promptly recognized and indexed, thereby enhancing the site’s overall accessibility and user experience.

AsciiDoc is a powerful format for authoring content. Like Markdown it’s simple to use, but offers more features and flexibility. In this post I’ll look at how to adapt a Gatsby site to use AsciiDoc.

By default the code behind a Gatsby site is written in JavaScript. However, as a site scales it can become more difficult to maintain all of that JavaScript. One way to improve the scalability of the site is to migrate from JavaScript to TyepScript.

Gatsby is a modern, fast framework for building optimized, high-performance websites. It’s a static site generator that compiles a site into static files at build time. Under the hood it uses React (user interface library) and GraphQL (data query language).

Compared with tools like WordPress or Joomla, Gatsby feels a lot more technical and less user-friendly. The learning curve is steeper and it takes longer to get things set up. However, the reward is more flexibility and granular control over all aspects of the site.

This post runs through the steps for setting up a minimal Gatsby site.

Many sports trading strategies hinge on odds changing over time. For instance, a strategy might involve laying a market at lower odds, anticipating the opportunity to back it at higher odds later on. Conversely, one might back a market at higher odds, hoping to lay it at lower odds in the future. Some strategies work with short term odds fluctuations, while others depend on longer term odds variations.

In this post I’ll take a look at some examples of odds dynamics and unpack why the odds change.

In a previous post I wrote about an Undetected ChromeDriver Docker image. A container derived from that image exposed a view of the Chrome session via VNC on port 5900. This worked really well. However, it meant having yet another app (the VNC client) running on my already cluttered desktop. I have extended the Docker image to use noVNC which means that I can now view the Chrome session via a web browser. This is very convenient since I always have a browser running.

One day your Gatsby site is building fine and the next it’s breaking with a JavaScript heap out of memory error. What’s gone wrong and how can you fix it?

At present there are two viable formats for the Transparency in Coverage data: JSON and XML. In this post we’ll dig into what the JSON files look like.

I have a challenge: extracting data from an enormous JSON file. The structure of the file is not ideal: it’s a mapping at the top level, which means that for most standard approaches the entire document needs to be loaded before it can be processed. It would have been so much easier if the top level structure was an array. But, alas. It’s almost as if the purveyors of the data have made it intentionally inaccessible.

The Transparency in Coverage Act (bill currently before congress) is a set of regulations that aim to increase transparency in health insurance coverage in the USA. The primary goal of the act is to provide consumers with clear, accessible, and actionable information about the cover that they receive from their health insurance. What services are included? How much will the insurer pay for a specific service? And how does this change from one provider to another? Or from one geographic region to another? Answers to these kinds of questions were previously hard, if not impossible, for a consumer to access.

In principle the information covered by the regulations should include costs, benefits, and other essential details. It should ensure that consumers can make informed healthcare decisions and understand the financial implications of their choices.

Code that moves data to and from S3 can slow down testing. A lot. This post demonstrates how you can speed things up by mocking S3.

Alembic can autogenerate migrations. This is probably its most valuable feature. However, I had a situation where --autogenerate kept on creating migrations for the databasechangelog and databasechangeloglock tables. These are Liquibase tables and should never feature in the Alembic migrations.

The solution was to tell Alembic to ignore these tables by updating the env.py module.

I need a list of medical conditions for a project. There are many potential sources for such a list. I selected the list published by NHS inform.

Marshmallow can readily handle nested schemas. But sometimes it’s preferable to flatten that schema for loading and/or dumping the data. The fields.Pluck() class makes this possible.

In a previous post I documented the process of setting up a GitLab Runner using the gitlab/gitlab-runner Docker image. As of GitLab Runner v16.0.0 the registration process has changed somewhat. This is an update to reflect that change.

A common workflow consists of using GitHub Actions to build a Docker image and then push it to Docker Hub.

Many of my projects now involve building a Docker image. The image is generally pushed to a registry as part of a CI workflow. This is how I push an image to Docker Hub from GitLab CI.

The data in the table below gives (manufacturer) specifications for a selection of kayaks and canoes. The data were originally compiled from two sources:

• a document shared by the Worcester Canoe Club and
• the racing boat data published by Purple Paddler.

The data has been revised and expanded to include other manufacturers and more recent models. It has also been cleaned to some extent, but there is still work to be done. Please let me know if you spot any errors or omissions.

I prefer to have my primary key columns first in a table. I recognise that column order is irrelevant to the performance of the table, but I prefer this for personal aesthetic reasons. However, from SQLAlchemy 2.0.0 there’s a change in the way that column order works with inherited base classes.