Optimizing Jest Runs with Dynamic Roots
At Salesloft, our primary frontend application has nearly 3,000 test suites containing over 20,000 tests. That’s a lot of time spent running tests in CI and the opportunity was ripe for optimization. We were already using Jest’s
--shard CLI argument to run tests across a number of runners in GitHub actions. Moving to multiple runners a few years ago sped things up significantly. In fact, we wrote own sharding script before it was built in to Jest. But given we were now in a monorepo setup using PNPM, we knew we could be more efficient with our CI runs.
In a monorepo, we don’t need to run tests for everything. We only need to run tests for packages that have changed plus their dependents. Thankfully, PNPM makes determining this very easy1. Its
--filter flag allows filtering packages based on changes between the HEAD and a given git ref. This filtering can include dependent packages as well. By limiting the scope to specific packages, we can be much more intelligent about what tests we run.
Let’s take a moment to talk about our monorepo structure. If you were to peak inside our
pnpm-workspace.yaml file, it would look something like this:
Apps are organized as packages in the
infrastructure/ holds all the packages needed to run, build, test, lint, format, and ship the repo. The scripts we’ll be looking at below are in an infrastructure package.
shared/ contain shared code teams can use when building out their apps. In typical monorepo fashion, each package manages its own dependencies2 and many of these package depend on others from within the monorepo.
So, for example, a PR may make changes to only
apps/accounts. In this case, because nothing depends on
apps/accounts, we should only run the tests for
apps/accounts. Conversely, if we make a change to
platform/packages/permissions, we will need to run tests for many other packages because so many depend on the permissions package.
You may be wondering “Ray, why doesn’t each app have it’s own
test script that you run with something like Rush, Lerna, or Turborepo?” That’s a legitimate question. For our situation, it makes more sense to keep all tests under a single Jest instance for a number of reasons:
- Sharding is much easier when all tests are running under a single Jest instance. If each package had its own
testscript, we would either have to add a significant amount of complexity to get package-level sharding working across runners in GitHub actions or be locked in to a monorepo manager that supports it out-of-the-box. We’ve been very happy with vanilla PNPM as our monorepo manager and don’t want to change that.
- We keep our testing infrastructure very regulated. Every package is on the same version of Jest, uses the same setup file, etc. This regulation would require more management overhead if we specified a
testscript for each package.
One option we do want to explore is the use of
projects with Jest. They’re not incredibly well documented, but we think they may give us the right balance of centralized management while also decentralizing some configuration we want to give individual teams for their test runs. For now, our current approach serves us well and allows for high levels of optimization.
#Making it happen
Configuring Jest to limit its scope turned out to be fairly straightforward. It has a
roots option to define the root directories in which Jest should look for tests. We can pass in an array of paths that point just to the packages that have changed. In our
jest.config.js, you’d find this line:
getRoots comes from another script, which I’ve included here in its entirety. Below, we’ll break it down section by section to see how it works.
A few overarching things to note:
- We’ve added extra logging at every step in the process. Both locally and in CI, this gives us good visibility into why and what is happening.
- The script is tied to GitHub Actions environment variables (
process.env.GITHUB_BASE_REF), but these parts could be easily swapped out for another CI provider.
- Our whole script is wrapped in a
try/catchand we clearly surface underlying error messages if things go sideways.
Now, let’s start with the constants at the top of the file.
First, we determine if we’re running in a CI environment and if we’re working with a pull request. Next up, we keep mocks for certain platform packages in the
platform/mocks directory so regardless of which test suites we’re running, we always want those mocks included in the list of roots. We also have a list of default roots for when every test needs to be run. These roots encompass all our workspace packages. Lastly, there are certain packages that should trigger a full test run if they are changed. Currently, only the Jest package is in that list. If we change anything with our testing infrastructure, we want to run all tests to make sure we haven’t unintentionally broken anything.
The next thing we do is check to see if we’re running in CI:
If we’re running Jest locally, we don’t want to limit the roots at all. We want engineers to run Jest as they normally would, perhaps invoking watch mode or providing the path to a specific test file they need to run. By passing
DEFAULT_ROOTS when running locally, all of these typical Jest use-cases still work as they should.
The next piece of information we need is which git ref to compare against to find packages that have changed.
When we’re working with a pull request, GitHub already provides the base ref we’re merging into. We can use the value directly in that case. If we’re running via an actual merge commit (i.e. when we’ve merged the pull request), we need to do more work to determine the last merge commit.
getLastMergeCommit function invokes
git log looking for the first merge commit (
--merges -n 1) prior to the current HEAD commit (
HEAD~1) and returns the full git SHA
--format=format:%H. With our
gitRef in hand, we can now get which packages have changed.
pnpm ls to list out repo packages. It uses the
--filter "...[GIT_REF]" syntax which is built into PNPM to find any packages changed since the given
GIT_REF plus their dependents. We use the
--json flag to output JSON from PNPM and
If any of the changed packages are in
PACKAGES_FORCING_ALL_TESTS, we log the result and return
Then, we further filter down the list to remove any packages from the
infrastructure directories as these packages are tested outside of Jest. We also get the absolute paths to each package from the PNPM JSON output.
At this point, if there aren’t any
roots left (e.g. we made a change to the README.md file at the repo root), we return an empty array.
Otherwise, we log out the roots we’re running tests in and return the list along with the path to our mocks directory.
The returned array is what is finally passed in to Jest’s
And there is it! Using dynamic roots with Jest has drastically improved our CI runtimes. It saves us 10s of 1000s of CI run minutes per month! Very little I’ve shared here is specific to our setup though. If you’re running Jest in a PNPM monorepo, you can do all of this as well. And I hope you do. Please reach out to me on Twitter or LinkedIn if you’d like to discuss it further.
NPM and Yarn may have built-in utilities for this as well, but I haven’t checked recently. Either way, with a little
gitwork, one could probably emulate this feature without too much code. ↩
To a point. We do lock certain dependencies down to specific versions. And all parts of the app use the same build/test/lint infrastructure which is maintained by a single team. ↩