In a previous post I wrote about an Undetected ChromeDriver Docker image. A container derived from that image exposed a view of the Chrome session via VNC on port 5900. This worked really well. However, it meant having yet another app (the VNC client) running on my already cluttered desktop. I have extended the Docker image to use noVNC which means that I can now view the Chrome session via a web browser. This is very convenient since I always have a browser running.
TL;DR
If you just want to try this out, the quickly launch a container.
docker run -it -p 7900:7900 datawookie/undetected-chromedriver:latest
The fire up Chrome.
import undetected_chromedriver as uc
driver = uc.Chrome()
You can view the Chrome session by visiting http://127.0.0.1:7900/ in your browser.
Before I dig into the details, here’s a quick recap of some of the key concepts.
What is ChromeDriver?
ChromeDriver is a driver for the Chrome web browser (it actually works for all browsers based on the Chromium web engine including Microsoft Edge, Opera, Brave, and Vivaldi). It allows you to automate the behaviour of a browser. Effectively it’s a remote control for the browser. It makes it possible to write code to make the browser navigate to an URL, interact with elements on the web page and extract specific content from the web page.
There are similar drivers for other browsers:
- GeckoDriver: a driver for Firefox;
- EdgeDriver: a driver for Microsoft Edge; and
- SafariDriver: a driver for Safari.
ChromeDriver is a separate application. A typical browser automation setup would consist of three components:
- the browser (for example, Chrome or Brave)
- the browser driver (for example, ChromeDriver)
- the client code (for example, a script written in Python).
What’s different about Undetected ChromeDriver?
Undetected ChromeDriver uses a patched version of ChromeDriver that is designed to avoid being detected by anti-bot systems. It modifies features of the browser which are commonly used by anti-bot systems, thereby making it less likely to be detected.
What is VNC?
When you run ChromeDriver directly it will pop up a Chrome window on your desktop. You will be able to see the results of your automation immediately in this window.
However, when ChromeDriver is running in a Docker container you do not have direct access to the Chrome window. This can make writing your client code rather challenging because you are effectively unable to see what’s happening.
Fortunately, VNC provides a solution to this problem. VNC (Virtual Network Computing) is a graphical desktop sharing system. It’s typically used to remotely view the desktop on another computer. But it can also be used to view the (virtual) desktop inside a Docker container.
The Undetected ChromeDriver Docker image I wrote about previously renders the Chrome browser onto a virtual framebuffer. This framebuffer exists within the container and is not visible from the outside world. A VNC server running in the container makes the contents of the framebuffer accessible.
Let’s see how this works. Launch a container and share port 5900 with the host.
docker run -it -p 5900:5900 datawookie/undetected-chromedriver:latest
Now load the undetected_chromedriver package and start the browser.
import undetected_chromedriver as uc
driver = uc.Chrome()
Now the browser is still locked up inside the container. We need to use a VNC client to see it. Open 127.0.0.1:5900 in the client. In the image below you can see the Chrome browser in a Vinagre remote desktop window.
What is noVNC?
noVNC is a VNC client which uses WebSockets to interact with the VNC server and renders the remote desktop as HTML5 so that it can be viewed in a browser.
Let’s see how this works. Launch a container and share port 5900 with the host.
docker run -it -p 7900:7900 datawookie/undetected-chromedriver:latest
Run the same bit of Python code and then go to http://127.0.0.1:7900/ in your favourite browser.
If for some obscure reason you want to use both noVNC and VNC, then this is absolutely possible. Simply expose both ports 5900 and 7900.
The symbiotic union of Undetected ChromeDriver and noVNC eliminates the need for a separate VNC viewer. It enables a cleaner, browser-centric approach. Now you can stay below the radar, while working with efficiency and simplicity. Get the new Undetected ChromeDriver Docker image here.