Blog Posts by Andrew B. Collier / @datawookie

A quick review of the year.

• I published 55 posts (including this one).
• I spent a lot of time working with GatsbyJS for one of my clients. At first I was quite out of my depth, but I slowly figured out more or less how it works. I documented some of my learning in a series of posts.
• My most popular post is still about Shared Memory & Docker. The runner up looks at how to Install GitLab Runner with Docker.
• I spent some time compiling data on kayak specifications in the hope of producing a definitive table. It’s a work in progress but it’s already getting quite a lot of interest.

Now onto a few interesting articles from this week, mostly announcements of new versions.

• Firefox 121.0
• OpenSSH 9.6
• Julia 1.10
• Keras 3.0.2 and
• Love Songs.

When I containerised Selenium crawlers in the past I normally used a remote driver connection from the crawler to Selenium, running a separate Docker image with Selenium and accessing it via port 4444. This has proven to be a robust design. However, it does mean two containers rather than just one, leading to a higher maintenance burden and elevated resource requirements.

What about simply embedding Chrome and ChromeDriver directly into the crawler image? It requires a bit more work, but it’s worth it. The critical point is ensuring compatible versions of Chrome and ChromeDriver.

With a local or remote SSH tunnel the ports on both the local and remote machines must be specified at the time of creating the tunnel. But what if you need something more flexible? That’s where Dynamic Port Forwarding comes into play.

Interesting articles from the week that was:

• Python 3.12 on AWS Lambda
• Python 3.12.1
• Best Novels Visualised and
• Space Typography.

Local and remote SSH tunnels serve the same fundamental purpose: they make it possible to securely send data across an unsecured network. The implementation details are subtly different though. A local SSH tunnel acts like a secure bridge from a local machine to a remote server. It’s ideal for accessing services on the remote server which aren’t publicly exposed. Conversely, a remote SSH tunnel reverses this direction, forwarding traffic from the remote server back to a local machine (or another machine).

The critical distinction between the two is the direction of the connection between the remote and local machines.

In the previous post we looked at how to set up a collection of static redirects via the vercel.json configuration file. Now we’re going to explore a more flexible and dynamic alternative using Edge Middleware.

In the previous post we set up a Minecraft server on Ubuntu. Now we’re going to install the Minecraft client and connect to that server.

I’m not a gamer, but I have an offspring who is deeply obsessed with Minecraft. I set up a Minecraft server for her so that she can play with her friends online in a safe environment.

A few things that caught my attention this week:

• Gemini
• Docker’s Generative AI & Machine Learning Stack
• Setting up a Minecraft Server on EC2
• Selenium 4.16 and
• Typical Airliner Seating Chart.

SSH tunnels are a powerful and secure method for transmitting data over potentially unsecured networks. They allow users to establish an encrypted connection between their local machine and a remote server, providing a secure and private pathway for data. An SSH tunnel will allow a service running on a remote machine to appear as if it is running on a local machine. This is also known as port forwarding.

A redirect is a rule which sends users to a different URL than the one they requested. They are most commonly used to ensure that browsers still get to the correct page after it has been moved to a new URL.

If you have a relatively small number of redirects and don’t need to do anything too fancy then static (or “configuration”) redirects are a good option. Static redirects are configured on Vercel by adding entries to the vercel.json configuration file. There’s just one major snag: you can only create 1024 redirects using this mechanism.

Sometimes when you run git merge you will be confronted with a huge load of merge conflicts. However, if you are lucky there might be a clear rule which you can apply to each of those conflicts, either

• accept current change (change on current branch or ours) or
• accept incoming change (incoming change from other branch or theirs).

In this case you can save yourself a lot of time and effort by specifying a particular merge strategy option.

Some things that got my attention this week:

• Titan Image Generator in AWS Bedrock
• AWS Transcribe Supports 100+ Languages
• cron Jobs in Vercel
• R 4.3.2
• Spark 3.4.2
• Keras 3.0.0 and
• Oceanography Gift.

My information highlights for the week:

• CodeWhisperer on the Command Line
• Jobs with Higher Income and Fewer Hours and
• httr2 v1.0.0.

By default Gatsby will embed CSS into the <head> of each HTML page. This is not ideal. In this post I take a look at how to move that CSS into an external file and how the contents of that file can be optimised to remove unused CSS.

A short week for me since I’m travelling. A small sample of highlights:

• AWS Bedrock Models
• Pathfinding and
• Development Containers.

In a previous post we deployed our Gatsby site on Netlify. Now let’s take a look at another platform: Vercel.

Suppose that you want to make your site routing a little more flexible. For example, rather than just going straight to a 404 page if the path is not found, you might want to try and guess an appropriate (and valid!) path. This is where dynamic routing comes into play.

Highlights from this week:

• Custom GPTs
• Immortal Python Objects
• Hidden Gems in VS Code and
• Parameterball.

Setting up a custom 404 page can add something special to your site. It provides you with the opportunity to do something memorable in the unfortunate event that a user asks for an unknown page.