There’s a skulk of updates in the new 145 version of Firefox. I’ll only be looking at changes that are relevant to web scraping.
Firefox Releases
I’m aiming to publish an update with each new stable release. Changes in version 145 can be found in the release notes and release notes for developers.
There’s a new release of Firefox every month (see the release calendar). Firefox supports four release channels:
- Extended Support Release (ESR)
- Release (the current stable version)
- Beta (a pre-release of the next stable version; also called the “Developer Edition”) and
- Nightly (the cutting edge, updated daily).
You can see which version is supported in each channel here. At present the stable version is 145, while beta and nightly are 146 and 147 respectively. Extended support is available for version 140.
Useful Links
Since this is the first post in the series I’m going to dump some (hopefully) useful general information here too.
The code for Firefox is Open Source and can be found in the official GitHub repository. Contributions are encouraged. There’s extensive and detailed technical documentation that should be mandatory reading for power users and anybody considering contributing.
Firefox is a great browser, but as a web scraper I’m even more interested in its Developer Tools. These are also well documented here.
Browser Fingerprinting
Browser Fingerprinting is a way to uniquely identify a browser and device based on characteristics like operating system, installed fonts, screen resolution, time zone, browser version and even subtle rendering differences. It doesn’t require storing anything on the user’s device and can be applied even when cookies are blocked or in a browser private window.
Because it uses a collection of characteristics, browser fingerprinting can provide an extremely granular signature that can be used to uniquely identify you (versus, for example, all users of Firefox on Linux). This signature can then be used to track you across sites and sessions.
If you’re concerned about privacy then you should be aware of browser fingerprinting. It’s also a persistent challenge for web scraping.
The latest version of Firefox implements a second phase of fingerprinting defenses which aim to reduce the uniqueness of each user’s browser environment, potentially halving the number of trackable user fingerprints.
At present these defenses are only enabled in private browsing mode and with strict Enhanced Tracking Protection (ETP), however they will be enabled by default in future. You can check your current protection settings by clicking on the little shield icon to the left of the URL in Firefox. Unless you have previously modified them then you’re likely to find that you have Standard settings enabled. You can find the settings under Privacy & Security. 🚨 As noted in the settings this might cause some sites to break.
You can use Am I Unique? or Cover Your Tracks to check your current browser signature and whether it can be used to uniquely identify you.
Conclusion
While this release includes just one feature of immediate relevance to me, I’m encouraged by Firefox’s continued work on browser fingerprinting. Future enhancements could make it an even more capable tool for evading anti-bot detection, which will be great for those of us building responsible automation.