Blog Posts by Andrew B. Collier / @datawookie


How to Harvest RSS Feeds

At Fathom Data we tend to do quite a lot of web scraping. At the moment I’m working on a small project which requires assembling a large selection of RSS feeds. Aggregator sites (like Feedspot and Feedly) have extensive, carefully curated lists of RSS feeds. As we’ll see below, the underlying lists are not entirely trivial to access.

Read More →

{emayili} Message Templates

Services like Mailchimp and MailerLite make it easy to create stylish email campaigns. Their templating tools allow you to create elegant HTML messages which are personalised to the recipient.

Wouldn’t it be cool if you could do something similar when sending emails from R? Well, with the latest version of {emayili}, that’s now possible (although this feature is definitely in its infancy!).

Read More →

Translating QCT (Quick Chart) Map Files

I’ve got a stash of old (2004 vintage) UK Ordnance Survey maps. They are really works of art and the the folk at the Ordnance Survey should be commended on the level of detail embedded in these maps. There’s just one small snag: the maps are in a rather obscure format. The proprietary Quick Chart (.qct extension) format is intended for use with Memory Map navigation software. If you want to use these maps for other purposes then you are stuck.

Read More →

{filebin} Quick & Easy File Sharing

At Fathom Data we have a number of workflows that require us to share various bits of data for a short time. The data are not sensitive, so we can freely share them. We have been doing this manually via platforms like Google Drive, Box or Dropbox. However we need to remember to go back and delete the file some time later. This is not ideal. What we needed was a simple “fire and forget” solution which would allow us to share the files and they would disappear automatically after some time. Well, this is precisely what Filebin does.

Read More →

{binance} Dealing with Dust

Dust refers to the fragments of coins which are too small to use for transactions. In the fiat world the equivalent would be those worthless coins with too little value to actually buy anything, that take up space in your wallet and end up scattered across parking areas. Binance allows you to convert dust into BNB. In this post I discuss the functions in {binance} which support this operation. Read More →

{binance} Tracking Total Account Balance

I started dabbling in crypto trading on Binance at the beginning of September 2021. I am really impressed with the interface, which is smooth and full featured (if perhaps a little complicated and confusing!). One of the things that has frustrated me though is not being able to get an idea of whether I’m making progress. There’s no view which shows me the overall status of my account and how this has evolved over time.

Read More →

HCRIS Field Labels

Fathom Data has been doing a lot of work with the HCRIS (Healthcare Cost Report Information System) data. The underlying reports are submitted as a spreadsheet with multiple sheets. The data are then extracted and recorded in a simple tabular format, with each field linked to a worksheet code (wksht_cd), column number (clmn_num) and line number (clmn_num). These three keys are then mapped to a single compound key. Read More →

{emayili} Message Threads

Being able to view related messages as threads is really useful. To make this possible, messages must use either the In-Reply-To or References header field to link to the Message-ID from another message.

This is now possible in {emayili}.

Read More →