A question posed by one of my colleagues: can a checksum be used to guess message length? My immediate response was negative and, as it turns out, a simple simulation supported this knee-jerk reaction.
Read More →Logarithmic Loss, or simply Log Loss, is a classification loss function often used as an evaluation metric in Kaggle competitions. Since success in these competitions hinges on effectively minimising the Log Loss, it makes sense to have some understanding of how this metric is calculated and how it should be interpreted.
Log Loss quantifies the accuracy of a classifier by penalising false classifications. Minimising the Log Loss is basically equivalent to maximising the accuracy of the classifier, but there is a subtle twist which we’ll get to in a moment.
Read More →The recently published 2015 Data Science Salary Survey conducted by O’Reilly takes a look at the salaries received, tools used and other interesting facts about Data Scientists around the World. Download the report as PDF. It’s based on a survey of over 600 respondents from a variety of industries. The entire report is well worth a read, but I’ve picked out some highlights below.
The majority (67%) of the respondents in the survey were from the United States. They also demanded the highest median salaries across the globe. At the other end of the spectrum (and relevant to me personally), only 1% of the respondents were from Africa. These represented only one country: South Africa. The lowest salaries overall were recorded in Africa, while the lowest median salaries were found in Latin America.
Read More →Last week I took a high level look at the trends in children’s names over the last century. Today I’ll dig a little deeper and examine the ebb and flow in popularity of some specific names.
Read More →I spent a decent chunk of my morning trying to figure out how to construct a sparse adjacency matrix for use with graph.adjacency(). I’d have thought that this would be rather straight forward, but I tripped over a few subtle issues with the Matrix package. My biggest problem (which in retrospect seems rather trivial) was that elements in my adjacency matrix were occupied by the pipe symbol.
In light of recent developments, a bit of work that I did almost two years ago has become rather relevant.
Read More →I’ve just been looking at the historical relationship between the London Interbank Offered Rate (LIBOR) and government bond yields. LIBOR data can be found at Quandl and comes in CSV format, so it’s pretty simple to digest. The bond data can be sourced from the US Department of the Treasury. It comes as XML and requires a little more work.
treasury.xml = xmlParse('data/treasury-yield.xml')
xml.field = function(name) {
xpathSApply(xmlRoot(treasury.xml), paste0('//ns:entry/ns:content//d:', name),
function(x) {xmlValue(x)},
namespaces = c(ns = 'https://www.w3.org/2005/Atom',
d = 'http://schemas.microsoft.com/ado/2007/08/dataservices'))
}
bonds = data.frame(
date = strptime(xml.field('NEW_DATE'), format = '%Y-%m-%dT%H:%M:%S', tz = 'GMT'),
yield_1m = as.numeric(xml.field('BC_1MONTH')),
yield_6m = as.numeric(xml.field('BC_6MONTH')),
yield_1y = as.numeric(xml.field('BC_1YEAR')),
yield_5y = as.numeric(xml.field('BC_5YEAR')),
yield_10y = as.numeric(xml.field('BC_10YEAR'))
)
Once I had a data frame for each time series, the next step was to convert them each to xts objects. With the data in xts format it was a simple matter to enforce temporal overlap and merge the data into a single time series object. The final step in the analysis was to calculate the linear coefficient, or beta, for a least squares fit of LIBOR on bond yield. This was to be done with both a 1 month and a 1 year moving window. Both of these could be achieved quite easily using rollapply() from the zoo package.
Kelsey Jones of Search Engine Journal interviews Guy Kawasaki of Canva. The key take-home message is that maintaining a personal brand is vital even if you are permanently employed. Specifically, it’s important to keep a visible record of who you have worked for and your personal successes.
I'm living proof. I did one thing right for Apple thirty years ago. I've been coasting ever since. Just need to do one thing really right. Guy Kawasaki
The quote above is, of course, tongue in cheek, but it bears a nugget of truth: showcase your achievements on LinkedIn and other social media because they all contribute to your personal brand.
Read More →
Markdown is a lightweight format specification language developed by John Gruber. Markdown can be converted to HTML, LaTeX or other document formats. You probably knew all that already. The syntax is pretty simple. Check out this useful cheatsheet.
Read More →I’ve just finished reading Beautiful Data (published by O’Reilly in 2009), a collection of essays edited by Toby Segaran and Jeff Hammerbacher. The 20 essays from 39 contributors address a diverse array of topics relating to data and how it’s collected, analysed and interpreted.
Read More →The recipe below works for Light Table v. 0.7.2 and Julia v. 0.4.0. It might work for other versions too, but these are the ones I can vouch for.
Read More →
