One day your Gatsby site is building fine and the next it’s breaking with a
My Gatsby site builds via a GitHub Actions pipeline. This is what the workflow looks like.
jobs: build-and-test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Node.js uses: actions/setup-node@v3 with: node-version: '18.x' cache: 'npm' - name: Install dependencies run: | # Dependencies not in package.json. npm -g install gatsby-cli npm -g install asciidoctor # Dependencies in package.json. yarn install --frozen-lockfile - name: Compile TypeScript and run Jest tests run: yarn tsc && yarn test - name: Gatsby Build run: yarn build
Everything runs fine until the last step. And TBH that step runs almost to completion. Then it seems to pause for a while… before breaking with a
So what’s going wrong?
There are a couple of things that are worth understanding:
- What is V8?
- What is the heap and why is it causing a problem?
The Gatsby site is written in Typescript. The
A Heap of Trouble
There are two common approaches to memory allocation: the stack and the heap.
The stack stores data using last-in-first-out (LIFO) order. It is typically used for quick access to temporary data. For example, when a function is called, its data is pushed onto the stack, and when the function exits, the data is cleared from the stack. The stack has a finite size and exceeding it will result in a stack overflow error.
The heap is more flexible than the stack. Memory on the heap can be allocated and freed in any order. The size of the heap adjusts dynamically at runtime according to requirements of the program. However, if the size of the heap exceeds that allocated to the program or is bigger than what’s available on the system, then the heap is exhausted, resulting in an out of memory error.
In this case the problem arises on a GitHub Actions runner, which typically has 7 GB of RAM. However, not all of that memory is available for the heap. The default heap size for Node is 2 GB on 64-bit machines. This limit is set by Node, not GitHub Actions.
The heap space allocated to Node can be adjusted via the
--max-old-space-size command-line parameter. For example, to compile
script.js with 4 GB of heap space you would use
node --max-old-space-size=4096 script.js
Another option is to use the
NODE_OPTIONS environment variable.
export NODE_OPTIONS="--max_old_space_size=4096" node script.js
This is the best approach if you are not running Node directly, for example if Node is being invoked via a shell script. In our case we’re running Node via Yarn. We could pass the command-line parameter through to Node like this:
yarn build -- --max-old-space-size=4096
Or we could use an environment variable. I prefer the latter approach. So this is what the modified GitHub Actions workflow looks like:
jobs: build-and-test: runs-on: ubuntu-latest env: NODE_OPTIONS: "--max_old_space_size=4096" steps: - uses: actions/checkout@v3 - name: Set up Node.js uses: actions/setup-node@v3 with: node-version: '18.x' cache: 'npm' - name: Install dependencies run: | # Dependencies not in package.json. npm -g install gatsby-cli npm -g install asciidoctor # Dependencies in package.json. yarn install --frozen-lockfile - name: Compile TypeScript and run Jest tests run: yarn tsc && yarn test - name: Gatsby Build run: yarn build
NODE_OPTIONS environment variable is set via the
env section. Problem solved. 🚀
I build this site locally using a Docker image. To simulate limited memory availability I used the
--memory command-line parameter to