How we migrated our Rush.js monorepo to Node type stripping

By Stuart Dotson

Introduction

Sometimes an epiphany comes after years of hard work.

Other times, it's more like an asteroid crashing into you from the cosmos.

My epiphany came shortly after a coworker posted an article about Node type stripping, which is a way to skip transpilation and run TypeScript files natively in Node. We started chatting about the significant impact this change could have on developer productivity and experience at Calm: Faster builds. Faster tests. Shorter feedback loops for local development. Less wasted time waiting for local tasks to complete.

After the initial excitement, we talked about the migration challenges, and he mentioned one of the biggest was our reliance on Sinon stubbed modules.

I shuddered as the memories came flooding back. Ah, the return of my old nemesis. A few years prior, I had looked at the possibility of migrating our repository to swc, and stubbed modules were the biggest challenge then as well.

What do I mean by "stubbed modules"? This:

import sinon from 'sinon'
import thing from './thing'

sinon.stub(thing, 'functiontoBeStubbed')

Our codebase had stubbed modules everywhere. I had scoured the internet for possible solutions, even coming across this Sinon blog post on the subject, and failed to find a solution that worked without massive refactors or forcing our fellow engineers to write tests in some new, unfamiliar way. I had given up, thinking it was impractical with our current reality.

My coworker nonchalantly said something I hadn't seen anywhere on the internet: Stub classes, not modules. Yes, you're reading that right, the solution to using type stripping with Sinon can be as simple as changing how we group code.

That was my asteroid moment, the moment when the incremental migration of our Rush.js monorepo to Node type stripping felt possible. After picking myself up off the floor and dusting off some cosmic dust, I got to work. What follows is some background on Node type stripping and how I implemented the migration.

Why Node type stripping

Type stripping enables Node to execute TypeScript files, without any kind of third party transpilation tool. Since type stripping replaces type annotations with white space, the line and column references to code do not change, thus removing the need for source maps.

This is at the cost of not supporting TypeScript features that are not a core part of Node and cannot be replaced with whitespace. Node type stripping does not look at TypeScript tsconfig files, although keeping a tsconfig around still enhances the developer experience and determines what happens in the tsc type-checking step.

Type stripping increases the speed of your CI pipeline and local development because it allows you to skip creating transpiled files and source maps.

In addition, starting and hot reloading your application will become faster because of the use of asynchronous ES modules (CJS modules are synchronous). Speaking of ES module benefits, you'll also get enhanced tree-shaking. Those two things combined mean that the bigger the dependency tree of your app, the bigger the benefit.

Prerequisites for type stripping

Type stripping requires you to think more deeply about your module systems. There are two main module systems in Node: ESM and CJS.

ESM, short for ECMAScript Module, is the emerging standard for supporting modules in JavaScript environments, both on the web and the server. ESM is asynchronous, uses import and export for modules, requires file extensions for relative imports, does not allow directory imports, and has globals like import.meta.dirname.

ESM adds the additional requirement that you must not mutate or overwrite any modules. "What kind of careless engineer would do that?" you might ask yourself, imagining a mischievous engineer going one step further and overwriting things like Array.prototype.map. Well, let's list a few: sinon.stub, sinon.spy, testdouble, rewire. I'll stop now so you don't drown in your tears. You'll need to refactor all instances of mutated modules so they don't throw errors in an ESM context.

Node was created before the ESM standard and came up with its own scheme, known as CommonJS (CJS), which uses require and module.exports. CJS does not require file extensions for relative imports, allows directory imports, and has globals like __dirname.

If you're using TypeScript, you're likely using a mixture of ESM imports/exports with CJS features and transpiling to CJS. Which means you must decide between doing the work to be fully ESM compliant or refactoring all imports/exports to CJS require/module.exports.

The very thought of moving everything back to require and module.exports in our TypeScript files felt like a giant leap backwards, so we decided to move to ESM.

Aside from module system requirements, there are TypeScript features that don't exist in Node, like enums, parameter properties, and decorators. You'll need to refactor those to migrate to type stripping as well.

Our migration process

Don't get scared. Although this was a big task, it was a task I achieved incrementally over time. I'll share some strategies that helped me get across the finish line. A common pattern I followed was to refactor a single problem set and defend the new standard with an ESLint or homemade lint rule.

The phases in the migration process were:

• Make the case
• Strategize
• Upgrade Node if necessary
• Get ESM ready
• Migrate to ESM
• Migrate to type stripping

Make the case

I didn't have a case study that I could point to in my meetings with my manager. All I had was the reasonable premise that doing one less thing (transpilation) and using a more efficient module system (ESM) would make things faster.

Thanks to this blog post, you do not have this challenge.

You can tell your manager, "A charismatic engineer on the internet said we can expect a 30-40% decrease in local development tasks, as well as at least a few minutes in our CI pipelines. Migrating to Node type stripping will help us ship features, address bugs faster, and improve many developer productivity metrics that influence overall software engineer happiness. We can put guardrails in place to prevent the risk of excessive engineer happiness. I hereby pledge to throw sand in the eyes of any engineer guilty of gratuitous happiness."

Strategize

Every situation is different. You should first evaluate whether you'll be able to migrate module by module or all at once.

In our case, because of how our local development and application bootstrapping worked, we could not migrate module by module. So our strategy became incremental "progress and defend" steps.

Upgrade Node if necessary

CJS and ES modules can import each other. If you're using Node.js 20.12.0 or later, you can require an ES module into a CJS module. For earlier versions, you will need to use the dynamic await import('./module'), which has some obvious drawbacks.

But really, if you're serious about this at all, you should upgrade to at least 22.6 to use the --experimental-strip-types flag, 22.7 to use --experimental-transform-types, which will decrease the surface area of the problem, or 23.6.0 to skip needing any kind of flag at all.

At Calm, we were already running Node 22, so no upgrade was necessary.

Get ESM ready

We were already using TypeScript and ESM import/export syntax, so we were really close. However, we still needed to fix import file extensions, mutated/stubbed/spied/replaced modules, CJS third-party imports, JSON import attributes, and Mocha tests. This section outlines all the steps we took before flipping the switch to ESM.

Import file extensions

We needed to add file extensions to all relative imports, Rush.js workspace imports, and deep third-party imports. In addition, we needed to refactor directory imports like import thing from '.../thing' or ../, where the presence of index.ts is inferred. While we could have added .js to all of the relative imports, despite the mixture of JS and TS files in our respository, that idea was dismissed early on as confusing.

You might be wondering how Node made sense of seeing .ts files in the imports.

Node didn't.

At all.

It could if we sent the --experimental-strip-types flag or ran at least Node 23.6.0 for type stripping to be enabled by default. But we weren't there yet, because we were still working on the ESM prerequisite.

Thankfully, since we were still transpiling into JavaScript, we updated our tsconfig to include rewriteRelativeImportExtensions: true, which rewrote all the file extensions of the relative imported .ts files to .js for the production build. For local development configuration, we also added allowImportingTsExtensions: true.

Keep in mind that importing TS files from folders within node_modules is disallowed when using the type stripping feature. If you're working with a TypeScript package that is then consumed by other packages and projects, you still might need to build and transpile.

For purposes of the import file extension requirement, individual projects in a Rush.js monorepo are considered relative imports, even though they might not look like it. Rush.js uses symlinks to reference internal project imports in node_modules and so are exempt from the Node rule. At Calm, we used prefixes like @calm to differentiate Rush.js project imports from third-party imports.

TypeScript doesn't know anything about Rush.js, so it won't treat project imports as relative imports.

You might see a looming train wreck. There's a handful of engineers playing ping pong on the train tracks off in the distance, blissfully unaware of the impending doom. Yep, the TypeScript setting rewriteRelativeImportExtensions won't rewrite Rush.js project imports. Yep, the production Node build won't be able to make sense of .ts extensions.

But never fear. There are always solutions. We can still save lives. I ended up using scoped custom conditions and conditional exports to route imports to the correct files depending on the build environment. I got this idea from the TypeScript documentation for path-rewriting

If we pass the --conditions=production flag to the Node command that starts our app in production, we can start using creative package.json exports like the following, which effectively route .ts imports to route to .ts in development and .js in production:

"exports": {
  ".": {
    "production": "./index.js",
    "default": "./index.ts"
  },
  "./*.ts": {
    "production": "./*.js",
    "default": "./*.ts"
  },
  "./*": "./*"
},

The word production can be any word you want, even puppytails or crawdad.

While I considered using a clever regex find/replace to fix the import file extensions issue, fixing directory imports and things like ../.. require resolving the module path, which led me to create a jscodeshift codemod to make the changes.

I looked into different ESLint plugins to defend the changes made. eslint-plugin-import has an extension rule you can try and use, but I honestly had trouble getting it to work in an environment where we expected ts and js extensions to show up in either file type. So I abandoned that to create a custom ESLint rule to enforce this new expectation.

JSON import attributes

In an ESM context, imports of json files require a special import attribute:

import fooData from './foo.json' with { type: 'json' };

We could have added this before transition to ESM if our tsconfig module setting were set to 'esnext', 'node18', 'nodenext', or 'preserve'. I saved it for the migration pull request, as there weren't that many.

Stubbed/Spied/Mutated modules

Alas, my old nemesis. ES modules are read-only and do not permit mutations to the module object. Attempts to mutate the module object will throw errors. Unfortunately, a lot of testing libraries will happily mutate a module if you give them one. Some even explicitly depend upon this functionality.

I'll throw out some examples that existed in our codebase:

• sinon.stub and sinon.spy
• testdouble.replace
• rewire

There are testing approaches that avoid mutating modules and other dependencies. One alternative approach is the "stub classes, not modules" idea which convinced me that migrating to type stripping was achievable. Unfortunately, our codebase had thousands of uses of sinon.stub and sinon.spy. It was the testing approach our engineers used. I needed a realistic, achievable path forward that allowed engineers to continue to test the way they always had and minimized the complexity of the refactor to get us there.

Our codebase was ten years old. There were JS files, TS files, and a mixture of object-oriented and functional styles implemented in a variety of ways.

I realized early on that I could not realistically write a jscodeshift codemod that could handle all the variations. I instead wrote a custom lint rule that tried to detect instances where modules were being mutated in tests. The custom lint rule accomplished this in the following way:

1. Look for all test files where we use sinon or testdouble.
2. For each test file, find every import where we are importing the entire module, e.g., import foo from 'foo' or import * as foo from 'foo'.
3. For every module import, see if we are using sinon.stub(${module}, sinon.spy(${module}, td.replace(${module} in the file
4. Save a reference to the file and the mutated module.
5. Check if this count is more than the currently saved count at .stubbed_modules_count.

I then started refactoring the stubbed modules and their tests, one by one, depending on the situation. The strategies I followed included:

• If the module functions were already grouped together into an object or class but were simply exported as the default export, I refactored the default export to a named export.
• If the module was a logical collection of interdependent functions that invoked each other, I refactored the functions into a class, translating exported functions to public methods and non-exported to private. I then exported the class or an instance of the class as a named export.
• If the module was a logical collection of functions that did not reference each other, I grouped them as an exported named object.
• On occasion, I refactored to use dependency injection, passing the dependency I wanted to stub as an argument to the function or the class constructor. I provided the expected behavior as the default for this argument and provided a sinon.stub() or an instance from sinon.createInstance in the unit test files.
• If we were spying on a module that produced side effects, I would replace it with one of the strategies above.
• If we were spying on a module that produced no side effects, I generally removed the spy altogether, with the reason that it was not testing something valuable.

In this way, I was able to make incremental progress and defend the gains made across several months.

CJS third-party imports

Some CJS third-party imports must be refactored once they're imported into an ESM context. It all depends on how the package is structured and published.

If the package maintainer exports everything as module.exports = { every, little, thing }, you will not be able to use named imports like import { every, little, thing } in an ESM context. Instead, you'll have to namespace the CJS module like so: import module from 'module'.

On the other hand, if the maintainer uses module.exports.every = every, you'll still be able to use the named import approach.

The most modern technique for a NPM package maintainer is to publish both a CJS and ESM version. This was the rarest example that I saw in our codebase.

Some package maintainers decide to move to exclusively ESM with a major version bump. Another solid reason to migrate to ESM, so you get the latest updates and bug fixes. You'll have to upgrade the package. ES modules can be required into CJS starting in Node version 20.12.0.

Other packages publish an ESM variant. One notable example of this is lodash, which publishes the ESM variant as lodash-es.

Still others do not have an ESM option at all.

I could have exhaustively cataloged all our third-party imports, see which ones published an ESM variant, which ones could be upgraded to an ESM variant, and which ones are the CJS-only variant that needed to be refactored.

That sounded tedious.

After most of the other ESM work was done, I created an ESM branch, ran the app, ran the tests, ran lint, and looked to see what broke.

This was one of the least automated tasks of the project, but thankfully, there weren't that many third-party CJS imports that needed to be refactored. The biggest one in our codebase was lodash and the use of lodash named imports, e.g. import { sortBy } from 'lodash'. lodash is CJS and uses the module.exports = {} style in their package.json. As a result, I was faced with the choice of refactoring all named imports to be import _ from 'lodash' or migrating to lodash-es so that the named import syntax can just "work". I wrote a jscodeshift codemod to migrate over to lodash-es and use the named import approach exclusively.

Mocha tests

Mocha only supports CJS for their .mocharc.js files. I updated those file extensions to .cjs so that they would not throw lint errors in an ESM context.

Get Type stripping ready

ESM is a prerequisite for Node type stripping if you want to use the import/export module syntax.

The first decision we made was how to handle the TypeScript features that had a runtime impact. These features require a transformation and cannot be stripped out of the code. These features included enums, namespaces, parameter properties, and decorators.

We had two choices: Refactor all TypeScript runtime features to alternatives, or pass the —experimental-transforms-types, hope it predictably transforms things, and only refactor the TypeScript features that are not standard, like decorators. A notable downside to —experimental-transforms-types is that since we are now transforming code, we have to enable source maps.

I decided not to transform types and created a pull request for each unsupported feature. For each, I created a jscodeshift codemod to perform the task, ran the codemod, and then implemented a new ESLint rule to defend my progress.

I did a pull request for each of the following:

1. Refactored TypeScript enums to a nifty internal Calm Enum utility
2. We skipped namespaces because they did not exist in our codebase
3. Refactored parameter properties to their native JS equivalent
4. Added the tsconfig eraseableSyntaxOnly flag, which codifies that no parameter properties, enums, and namespaces are allowed.

Move to ESM

I added type: 'module' to all the package.json files. This illuminated any problems as I tried to run things. If quirky CJS modules or stubbed modules were still lurking, I saw errors.

I experimented with trying to get ESM to work with our transpilation solution. I had enormous difficulties getting ts-node to work with ESM in a Rush.js monorepo that had a mixture of JS and TS files, which inevitably led me to combine this step with the migration to Node type stripping.

One really important thing to point out is that some libraries have hidden ESM complexity. dd-trace-js, for example, needs you to pass along a special flag to your Node command to enable automated tracing and custom hooks in ESM. In our case, the --import dd-trace/register.js flag wasn't enough; we needed to init at the same time with --import dd-trace/initialize.mjs and upgrade to the latest patch version of the package.

Another issue uncovered at this stage was that mocha --watch did not officially support ESM out of the box. But if you pass the --parallel flag in local development as suggested in this open issue, you can re-enable --watch with the added benefit that the tests will be much, much faster since they'll be run in parallel.

Generally, these challenges are solvable, but they do increase the unknowns and risk to the whole process.

Migrate to type stripping

I replaced all instances of our transpiler ts-node with node --experimental-strip-types.

In addition, I added the --experimental-strip-types flag wherever we ran Mocha tests and removed some transpiler flags that were no longer necessary.

Since I also updated some package.json files with the conditional exports magic to support Rush.js workspace import file extensions before the migration to ESM, I simplified that to:

"exports": {
  ".":  "./index.ts",
  "./*": "./*"
},

Finally, I deleted all tsconfig files that dealt with deployment or production, added noEmit: true so that JS files were not emitted, and added "sourceMap": false to disable source maps. I also removed all transpilation steps from our build pipeline, but kept our non-emitting type-checking tsc step.

Results

Common local development tasks ran 30-40% faster. Feature branch commit jobs that included steps to build, type-check, and test the changes decreased by 3 minutes. Master branch jobs that included the previous tasks and deployment decreased by 6 minutes.

Local development

• Type-check tsc: 25% faster (30 seconds faster)
• Starting the app: 40% faster (27 seconds faster)
• Running a single successful unit test file: 35% faster (4.4s down from 6.6s)
• Other common development scripts saw similar gains, except for lint, which saw no benefit

If we collected local dev stats, I could do a little math like the following:

• n is the number of times the task is performed in a month
• w is our guestimate of the percentage of that time that is wasted time from engineers getting impatient, context-switching, doing a little Duolingo, listening to a meditation, etc. For local development, I'd guess about 80% of that time is wasted.
• t is the time to perform that task once

From there, I could calculate the decrease in wasted time:

wasted time = t * w percentage drop in wasted time = (wasted time before - wasted time after)/wasted time before quantity of time gained from improvement = n * t * w and then converted to minutes/hours/days, whatever is most impressive

Example

Let's say that the unit test file I benchmarked is representative of the average unit test file.

If a unit test file is executed 50 times per week by one engineer, that's 2.2 s * 50 = 110 s a week. Multiply that by however many engineers work in that repo. Let's say there are 30 engineers. 30 * 110 s = 330 s, about 2.016 minutes a week for the engineering team.

I think there's a solid case to be made that 100% of that waiting time is wasted because 6.6s, the original benchmark, isn't enough time to do anything else. You're just waiting. But let's imagine we only hire hyperproductive 10x engineer unicorns, who are capable of such a feat. 2.016 minutes * 0.8 (80% time wasted) = 1.612 minutes a week for the lower bound.

That's 1.6-2 minutes a week. 6.4-8 minutes a month. 76.8-96 minutes a year. For just one task. Extrapolate this same concept to other local tasks, and the impact grows.

CI/CD

• Feature branch build/test job: 20% faster, saving about 3 minutes on average
• Master branch build/test/deploy job: 15% faster, saving 6 minutes on average

If we multiply the savings by the number of times these steps were run in the last month, we can get a sense of impact.

1009 feature branch commit jobs = 1529 x 3 min = 4587 min = 76.45 hrs = 3.18 days of time savings a month 193 master branch commit jobs = 218 x 6 min = 1308 min = 21.8 hrs of time savings a month

We could try to get a sense of wasted time prevented here, but it's a little more fuzzy, as CI processes are much longer – in our case, at least 15 minutes before the ESM type-stripping migration. That's enough time to context switch and do something meaningful. You could argue that engineers aren't actively waiting for their master branch commit to pass or their feature branch commit tests to run, but if something goes wrong, like a failed test or a failed build, the notification represents a disruption to their flow, and is an even more impactful disruption if they've started something new.

Decreasing the duration of CI processes improve an engineering team's ability to rapidly pivot and address issues in the codebase.

Reflections

I hope this case study encourages others to take the leap and migrate their repositories to ESM and Node type stripping. You can expect common local development tasks like starting the app, running tests, and type checking to be about 30-40% faster. For your CI pipeline, it's however long it takes your app to build and emit JS files, as well as the gains from the asynchronous ES module imports. In our case, it was about 3 minutes for feature branch jobs and 6 minutes for master branch jobs.

While I designed a very careful and incremental approach, there were still some unexpected issues along the way. There is a level of risk and unknown with any large refactoring project.

For example, I initially did not add file extensions to Rush.js project imports. Unfortunately, IDE auto-complete features and GitHub Copilot would consistently suggest that they should be there, leading to all sorts of unexpected and un-rewritable ts file extensions being committed, merged, and showing up on prod, leading to errors and other unnecessary excitement. We solved this first with a custom build process and lint rule that mandated that ts file extensions were not in the emitted JS files. Shortly after, I learned the scoped custom conditions and conditional imports approach to allow ts extensions on Rush.js project imports before full ESM migration, and added a custom lint rule that enforced import file extensions across the board.

The dd-trace-js issue with ESM posed an existential threat to the entire effort two days after I migrated everything to Node type stripping, when we discovered that trace.express.request was no longer showing up in our DataDog logs for many of our services. Thankfully, that was resolved with the --import dd-trace/initialize.mjs flag mentioned earlier.