The Nine Node Pillars

The Nine Node Pillars

9 Principles for Doing Node.js Right in Enterprise Environments

Powering over 6.3 million websites and countless APIs, Node.js is an effective cornerstone for modern applications including Walmart and Netflix. With over 2 billion annual downloads, it is one of the most used web development tools today (OpenJS Foundation). It is an excellent choice for building high-performance applications, however, we know from experience that it takes significant expertise to manage, operate and scale an application.

Over the years, Platformatic's co-founders Luca Maraschi and Matteo Collina have worked with dozens of engineering teams at Fortune 500 companies to architect their most important Node.js applications. As part of the Node.js Technical Steering Committee, Matteo is immensely privileged to have a front row seat to the Node community and its evolving needs. If you’ve built Node.js applications before, you’ve probably run his code. The modules he maintains are downloaded 2.6 billion times per month, making him solely responsible for approximately 1/100 of all npm traffic.

To share our insights, we've created the Nine Node Pillars– nine guiding principles for creating robust, scalable, and maintainable Node applications in enterprise environments.This guide can be used as a checklist to identify gaps in your current practices and prioritize areas for improvement.

To create the most comprehensive guide possible, we wanted to tap into the collective wisdom of the Node.js community. With that being said, thank you to our incredible contributors who have taken the time to work on this piece with us. Their insights and experiences have been invaluable in refining the Nine Node Pillars.

Key contributors:

James Snell

James Snell

Principal Systems Engineer - Cloudflare

Natalia Venditto

Natalia Venditto

Lead Owner JavaScript DX and DevTools - Azure

Michael Dawson

Michael Dawson

Node.js Lead for Red Hat and IBM - Red Hat

Matteo Collina

Matteo Collina

Co-Founder & CTO - Platformatic

1. Do not block the event loop

Node.js's event-driven architecture is a cornerstone of its performance and scalability. Central to this architecture is the event loop, a mechanism that handles asynchronous operations and ensures that the application remains responsive.

The event loop is what allows Node.js to perform non-blocking I/O operations — despite the fact that a single JavaScript thread is used by default — by offloading operations to the system kernel whenever possible.

icon

The event loop consists of the following phases, each responsible for handling different types of events:

The event loop iterates through these phases in a continuous loop. If there are no events to handle in a particular phase, the loop moves to the next phase. This process continues until there are no more events to process.

Blocking the event loop

The event loop operates in a cyclical fashion, continuously checking for new events and executing corresponding callbacks. When a blocking operation occurs, the event loop is unable to process other events until the operation completes, resulting in a backlog of pending tasks. Inefficient code can inadvertently block the event loop, leading to performance degradation and potential application instability.

Consequences of blocking the event loop:

  • Reduced performance: As the event loop is stalled, the application's ability to handle concurrent requests decreases significantly, impacting response times.


  • Increased latency: Users experience delays as requests queue up while the event loop is blocked.


  • Unresponsiveness: In extreme cases, the application may appear to freeze or become unresponsive.

Best Practices:

To prevent event loop blocking and maintain optimal performance:

  • Decompose complex tasks: Break down complex tasks into smaller, asynchronous steps to prevent long-running operations from blocking the event loop. This can involve using promises, async/await, or callbacks effectively.


  • Offload CPU-intensive tasks: For computationally heavy operations, consider using worker threads to isolate them from the main event loop. This can be important even if your process only handles one type of request and processes them sequentially as they arrive. As an example, when running under Kubernetes, liveness checks may fail if the event loop is blocked on a long running request

Libraries like piscina can streamline the process of offloading CPU-intensive tasks to worker threads.

-Matteo Collina

  • Implement caching and deduplication: There is often a CPU-bound operation after each I/O operation (like a database or API call). Reduce it or reduce API calls by caching results after processing; try to deduplicate them. Libraries like async-cache-dedupe can assist with this.

  • Monitor Event Loop Utilization: Use tools like perf_hooks or the under-pressure plugin to track event loop performance and identify potential bottlenecks.

  • Understand where certain operations occur: Understanding where operations occur in Node.js is crucial for performance. For example, JSON parsing happens on the main thread, while asynchronous file reading occurs on a separate thread. This distinction helps you optimize your code to avoid blocking the event loop.


2. Monitor Node specific metrics and
act on them

Standard metrics for Node applications often lack the context needed for effective troubleshooting. As such, monitoring Node.js performance often requires combining insights from multiple tools and custom dashboards, juggling metrics like CPU usage, memory consumption, and latency. 

This fragmented view makes it difficult to pinpoint the root cause of slowdowns and unresponsive applications.

Imagine a platform with numerous microservices; a fragmented view might reveal high CPU usage, but identifying the culprit remains a guessing game without context. This hinders timely troubleshooting, leading to persistent performance bottlenecks that cost companies millions in lost revenue.

So, what should you monitor?


  • Memory usage:

    • Heap used vs heap total: Track the proportion of heap memory in use to identify potential memory leaks or inefficient data structures.

    • RSS (Resident Set Size): Measure the total amount of physical memory occupied by the process.


  • CPU usage: Monitor CPU utilization, but correlate it with other metrics like ELU to avoid premature scaling decisions.


  • Event Loop Utilization (ELU): This metric measures the proportion of time the Node.js event loop is actively processing events to the total elapsed time. It quantifies how busy the event loop is.


Acting on metrics
There are two keys to ensure that you are acting in the best interest of your application based on metrics:


  1. Do not scale or kill an instance on a limited memory consumption. Node.js will use all the memory that is made available to the process. Therefore, killing or scaling the process when it has 60-80% of that used is a waste of resources.

  2. Always consider the ELU when scaling on the CPU metric. Node.js consuming 100% of the available CPUs does not mean the process is not responsive.

Have a contingency plan in production

Even with best practices, unexpected issues can and will arise.

A well-thought-out contingency plan is essential for minimizing downtime and ensuring a smooth recovery in case of incidents. As the baseline of your contingency plan, you should have Node-specific monitoring systems to detect issues early and set up alerts to notify relevant teams.

Following this, your plan should include:


  • Automated rollbacks and canary deployments: Configure your deployment pipelines to automatically rollback to a previous stable version in case of issues. Gradually introduce new versions to a subset of users to identify and address potential problems before a full rollout.

  • Containerization and orchestration: Package your application and its dependencies into portable containers.Tools like Kubernetes can simplify the management and scaling of containers, making it easier to recover from failures.

  • Incident response procedures: Establish clear guidelines for responding to incidents, including roles and responsibilities, communication protocols, and escalation paths.


3. Use Node LTS versions in production

Node.js LTS (Long-Term Support) release lines are specifically maintained for extended periods, providing stability and predictability for production environments. LTS lines receive critical bug fixes and security patches for 3 years, as opposed to 7 months for non-LTS lines. 

Benefits of using LTS release lines:


  • Reduced risk of breaking changes: LTS versions prioritize compatibility with existing packages and modules, minimizing the chance of unexpected disruptions.


  • Enhanced security: By staying on LTS versions, you benefit from timely security updates, protecting your applications from potential threats.

  • Improved stability: LTS versions undergo rigorous testing and maintenance, ensuring a more predictable and reliable runtime environment.



How LTS versions work
Node.js maintains a strict release schedule for LTS versions, with each version receiving active support for a defined period, followed by maintenance support for an additional period. This structured approach guarantees that you have sufficient time to update your applications to newer LTS versions without compromising stability.



Best practices for using LTS versions


  • Monitor LTS release schedules: Stay informed about upcoming LTS releases and plan your migration strategies accordingly.

  • Test thoroughly: Before migrating to a new LTS version, conduct thorough testing to ensure compatibility with your applications and dependencies.


  • Utilize dependency management tools: Employ tools like npm audit or yarn audit to identify and address vulnerabilities in your dependencies and ensure they are compatible with your LTS version.



The dangers of using end-of-life release lines

As shown in the diagram, Node.js v14 and v16  have been end-of-life for 2 and 1 years respectively, and they are still popular. This underlines the problem we are seeing–companies are not updating their Node.js runtime. 


4. Automate testing, code review and conformance as much as possible

Write automated tests
Automated testing is not only crucial for building reliable and maintainable Node.js applications but also for accelerating development. By establishing a comprehensive testing strategy, organizations can confidently make changes, reduce the risk of defects, and ultimately move faster.

Testing best practices.

  • Define test cases: Once a test case is in place, developers can begin writing the tests. The tests may need configurations or fixtures specific to individual tests or large-scale environments, including data and network infrastructure. 


  • Prioritize live testing: Interact with live systems whenever feasible instead of mocking components to uncover potential integration issues.


  • Focus on behavior: Tests should primarily validate code's external behavior, treating components as black boxes.


  • Eliminate test flakiness: Consistently investigate and resolve test failures to maintain test reliability. Keep the CI green.

  • Avoid global state: Reducing global state reduces test flakiness, therefore improving test stability via isolation.


  • Focus on defensive in-depth testing: While code coverage is a valuable metric, it's essential to focus on defensive in-depth testing to ensure the reliability of your Node.js applications. This involves testing edge cases, negative testing and security testing. 


  • Test as close as possible to production: To accurately assess your application's behavior in real-world conditions, it's crucial to test as close to production environments as possible. To do so, set up staging environments, and conduct performance testing and integration testing. 


Comprehensive test coverage
A well-rounded testing strategy encompasses multiple test types:


  • Unit tests to verify the correct behavior of individual code units in isolation.


  • Integration tests to validate how different components interact.


  • End-to-end tests to simulate real-world user scenarios to ensure the application functions as expected.


Usually, we consider the database part of the unit. Therefore, we do not recommend mocking database drivers but rather accessing the database directly, at least for the “Happy Path”. Mocking database drivers can be very useful for testing error conditions.

         
            
Choose the right testing framework
The right choice of testing framework is essential. While Jest has gained popularity, its limitations in terms of global state management and error handling make it less suitable for Node.js testing. Specifically, Jest overrides the globals of Node.js causing all sorts of issues when dealing with networks and databases. If you have a large Jest test suite, we recommend switching to jest-light-runner, which avoids all monkey-patching of globals.

Consider alternatives like Vitest, Node-tap, jest-light-runner, import('node:test'), tape, or Mocha, which offer better support for Node.js-specific testing requirements

We recommend playwright for E2E tests that involve a browser.

TypeScript and testing
TypeScript, while providing valuable static type checking, does not eliminate the need for thorough testing. Tests are essential to ensure runtime correctness and to catch potential issues that TypeScript's type system may miss. 

In case you are writing JavaScript with separate types, don’t forget to test them too– we recommend using tsd for testing the types.

Mocking
Mocking involves replacing real dependencies with simulated versions to isolate components and simplify testing. It is also useful when you want to enable tests to be always reproducible under the same conditions. A good example of this is a component that downloads an external resource from a remote server out of your control.


Common mocking libraries include:


Some test frameworks, like Jest or node:test, offer integrated mocking capabilities.

Code quality and security

Ensuring the quality and security of your Node.js applications is key for delivering reliable and trustworthy software. By incorporating robust testing practices, leveraging static analysis tools, and conducting regular vulnerability scans, you can significantly enhance the overall health of your codebase.

Key practices:

  • Regular vulnerability scanning: Employ tools like Snyk, SonarQube or npm audit to identify and address potential security threats proactively.

  • Static analysis and linting: Use tools like ESLint to detect syntax errors, code style violations, and potential security vulnerabilities early in development.

By engaging in these practices, you stand to benefit through:


  • Improved code quality: Enhanced readability, maintainability, and adherence to best practices.


  • Reduced security risks: Proactive identification and mitigation of vulnerabilities.


  • Faster development: Early detection of defects reduces the number of defects reaching production and the amount of rework needed, effectively increasing the velocity of the team.


5. Avoid dependency creep

Dependency creep can significantly impact the complexity, maintainability, and performance of your Node.js applications. Understanding the risks associated with excessive dependencies and implementing effective management strategies is crucial for building robust and efficient software.

The dependency tree
Each new package you introduce into your project can introduce additional dependencies down the line. This can lead to a complex dependency tree, making it difficult to manage and understand the potential risks associated with each package.

Key considerations to avoid dependency creep:

  • Intentionality: Be intentional about the dependencies you introduce, carefully evaluating their benefits and potential risks.

  • Quality and reliability: Prioritize packages that are well-tested, maintained, and have maintainers with a proven track record.

  • Size and scope: Opt for smaller, more atomic packages focusing on specific functionalities, reducing the likelihood of introducing unnecessary dependencies.

You can effectively manage dependencies and avoid unnecessary bloat with the following strategies.

Default to using native Node APIs and understanding where they differ from web standard APIs

To build high-performance and maintainable Node.js applications, it's crucial to use the underlying runtime. By prioritizing the use of native Node.js APIs, developers can mitigate risk from unnecessary dependencies and align with established standards.

Leveraging native capabilities:

  • Core modules:  Utilize the built-in modules provided by Node.js (e.g., fs, http, path) whenever possible. These modules are highly optimized, battle-tested, and offer a familiar interface for common operations.

  • API Adoption: Embrace new Node.js APIs as they become stable. For instance, using fetch() API provides a more modern alternative to the traditional http.request API. By staying up-to-date with the latest advancements, you can benefit from performance improvements and enhanced features.

Native APIs offer several advantages. They are meticulously optimized for Node.js, resulting in faster performance compared to third-party libraries. Additionally, core modules are extensively tested and maintained by the Node.js core team, ensuring stability and reducing the risk of unexpected issues. Lastly, codebases that rely heavily on native APIs tend to be more straightforward to understand and maintain due to their familiarity and consistency.

Understanding native Node.js APIs vs. web standard APIs

While many Node.js APIs align with web standards, there are notable differences, especially in newer Node.js versions. Here are some key distinctions:

  • Streams: Node.js streams offer more granular control over data flow compared to web standard streams. Node.js streams can be paused, resumed, and piped directly to other streams, providing flexibility for various use cases.


  • HTTP: Node.js's http module provides low-level control over HTTP requests and responses, allowing for fine-grained customization. Web standard APIs like fetch offer a higher-level abstraction for simpler HTTP interactions. However, the best HTTP client for Node.js is undici, which is built to get past all the legacy issues of the http module.

  • File System: Node.js' fs module provides a rich set of APIs for working with files and directories, offering more granular control than web standard file APIs.

By understanding these differences, you can make informed decisions about when to use native Node.js APIs and when to leverage web standard APIs for your specific needs.

Reuse modules as much as possible

Node_modules is the heaviest object in the universe for a reason.

Together, Node.js and NPM solved the critical problem of all previous package managers: they allowed multiple versions of the same library to be loaded in the same process. This sparked a Cambrian explosion of modules on the registry, allowing developers to reuse code without worrying about API conflicts. This pattern of module reutilization critically reduces efforts and helps organizations go to market faster. It also helps to adopt industry standard patterns instead of reinventing the wheel with higher maintenance costs.

A look at NPM

NPM, the Node Package Manager, underpins efficient and scalable Node.js development. 

At its core, NPM is designed to facilitate code reuse on a massive scale. By providing a centralized repository for reusable code packages, NPM empowers developers to share and leverage common functionalities, accelerating development cycles and reducing redundancy.


Key benefits of leveraging NPM for code reuse include:

  • Accelerated development: By reusing existing modules, developers can focus on building new features rather than reinventing the wheel.


  • Improved code quality: Shared modules undergo more rigorous testing and refinement, leading to higher-quality code across the organization.


  • Enhanced maintainability: Centralized code management simplifies updates and bug fixes.


  • Increased collaboration: NPM promotes knowledge sharing and collaboration among teams. Particularly, by using private registries and collaborating with standard bodies, open-source communities, and a diverse group of developers, organizations can benefit from shared knowledge, code reuse, and the overall strength of the open-source ecosystem.

An organization fully embracing Node.js should also embrace NPM as a tool for fostering internal code sharing. By creating and publishing private NPM packages, teams can effectively share reusable modules within the company, preventing duplicate efforts and ensuring project consistency. This approach cultivates a culture of collaboration and knowledge sharing, increasing developer productivity and improved code quality.


To get the most out of using NPM, ensure that you:


  • Standardize dependencies: Establish a common set of dependencies within your organization to reduce redundancy and simplify maintenance.


  • Share your code internally: Consider creating and publishing private NPM packages to share reusable modules within your organization. This promotes collaboration, prevents duplicate efforts, and ensures consistency across projects.

Monorepos: Scaling code reuse
For larger projects with multiple interconnected components, monorepos can be a game-changer, providing efficient dependency management, atomic commits, simplified code sharing, and an improved developer experience. 

However, successful adoption requires careful consideration and planning.

Key considerations:


  • Governance: Implementing a monorepo necessitates clear governance guidelines to manage dependencies, ensure code quality, and maintain consistency across the project.

  • Dependency management: Effective dependency management strategies are crucial to prevent conflicts and ensure compatibility between modules. This might involve versioning policies, dependency locking, and automated testing.


  • Tooling: Choose appropriate tooling to streamline monorepo management. These tools can automate tasks like versioning, publishing, and testing across multiple packages.


While the advantages of monorepos are substantial, it's essential to weigh them against potential drawbacks:


  • Increased complexity: Managing a single repository with multiple packages can be more complex than managing separate repositories.


  • Performance overhead: Building and testing the entire project can be time-consuming, especially for large codebases.


  • Potential for conflicts: Changes in one package might inadvertently affect others, leading to increased debugging efforts.



    So, which registry client should you use?

    There are several options:

  • Npm (as shipped with Node), but kept up to date

  • Pnpm

  • Yarn

My personal recommendation is to use npm for simple modules, and use pnpm for a monorepo scenario.

-Matteo Collina


6. De-risk your dependencies

De-risking dependencies is crucial for security, performance, and maintainability. By regularly scanning for vulnerabilities, updating promptly, and choosing well-maintained dependencies, you can reduce the risks associated with dependency management and ensure your application remains secure, efficient, and adaptable.

Scan your dependencies for Common Vulnerabilities and Exposures (CVEs)

Optimizing Node.js applications for security and performance is crucial. 
A cornerstone of this security posture is conducting regular dependency scans to identify and address vulnerabilities early in the development lifecycle. Ignoring this practice can lead to disastrous consequences.

What are CVEs?

CVE is a list of publicly disclosed computer security flaws which has been assigned a CVE ID number. The CVE system was launched in 1999 and is operated by the MITRE Corporation.

A flaw must meet the following criteria to be assigned a CVE ID:

  1. It can be fixed independently

  2. It has been acknowledged by the affected vendor or has been documented by said vendor

  3. It affects one codebase

icon

Scanning best practices:

  • Perform automated scans as part of a CI/CD pipeline to automatically check for vulnerabilities in new and existing dependencies.


  • Implement a vulnerability management process to prioritize remediation based on severity and potential impact.

Use tools like npm audit, Socket.dev or Snyk to streamline the vulnerability management process.

-Matteo Collina

By proactively addressing vulnerabilities, organizations can significantly reduce the risk of security breaches and protect their applications from malicious attacks.

Remember, choosing the right dependencies is crucial to building secure and maintainable applications. Carefully evaluate dependencies before integrating them into your project.

In order to mitigate the risks introduced by your supply chain, you’ll need to prioritize dependency management tools that efficiently update dependencies, adhering to dependency graphs for proper ordering and weighting. This helps minimize the risk of cyclical dependencies and other issues that can degrade application performance or cause crashes.

Keep your dependencies up to date

Beyond security considerations, it's essential to optimize dependency usage to avoid performance bottlenecks and improve overall application health.

Key practices:


  • Dependency minimization: Reduce the number of dependencies to minimize potential conflicts and improve build times.

  • Dependency versioning: Carefully manage dependency versions to ensure compatibility and avoid breaking changes.


  • Dependency tree optimization: Analyze the dependency tree to identify potential performance impacts and optimize package installations.


  • Dependency updates: Regularly review and update dependencies to benefit from bug fixes, performance improvements, and new features.



    The dangers of neglecting dependency management
    Failing to maintain up-to-date dependencies will lead to significant challenges.


    Node.js' Long-Term Support (LTS) release model emphasizes stability, but it's crucial to recognize that these versions are not automatically updated. Relying on outdated dependencies will expose your application to security vulnerabilities, performance regressions, and compatibility issues. 

    Furthermore, when libraries reach end-of-life, you risk encountering critical bugs or missing out on essential features. This can render your entire automated pipeline useless, hindering development efforts, increasing maintenance costs, and potentially compromising your entire application.

    Proactive dependency management is essential to mitigate these risks and ensure the long-term health of your Node.js project.

Manage your dependency risk by supporting key projects and dependencies

The open-source ecosystem is the cornerstone of Node.js development. Failing to support key projects and dependencies can introduce significant business risks, including vulnerability exposure, performance degradation, feature limitations, vendor lock-in, and community fragmentation. To foster a sustainable and thriving community, it's essential for organizations to actively contribute and support key projects and dependencies.

Giving back to the community

  • Open Source contributions: Participate in open source projects by submitting bug reports, code improvements, or new features. Your contributions help enhance the quality and reliability of the tools used by the entire community.


  • Financial support: Consider sponsoring open source projects that are critical to your development efforts. This support can help maintainers allocate more time and resources to project development and improvement.


  • Dependency management: Actively manage your project's dependencies. Keep them up-to-date with the latest versions, report issues promptly, and contribute to bug fixes or feature enhancements when possible.


Supporting the OpenJS Foundation
The OpenJS Foundation is a non-profit organization dedicated to fostering the growth of JavaScript and related technologies. 


By sponsoring the OpenJS Foundation, you can directly support the development and maintenance of key Node.js projects and initiatives.


Your support also contributes to the long-term health and quality of the dependencies your applications rely on. Additionally, active engagement with project maintainers leads to quicker issue resolution and faster response times. Finally, contributing to open source projects fosters a strong, collaborative community, benefiting everyone involved.


7. Avoid global variables, config or singletons

In the context of enterprise development, building well-structured and maintainable applications requires a conscious effort to minimize the use of global variables, configurations, and singleton patterns.

Use the dependency injection pattern to structure your code

Minimizing the use of global variables is crucial in enterprise settings. 

Global variables introduce tight coupling, making code harder to test, reason about, and refactor. By adopting dependency injection and promoting modular design, developers can create more isolated and reusable code components.

Dependency injection involves passing dependencies explicitly as function arguments or constructor parameters, rather than relying on implicit global access. This approach enhances code testability, promotes better code organization, and reduces the risk of unintended side effects. To effectively implement this principle, organizations should establish coding standards that discourage global variable usage and provide developers with training and resources on dependency injection and modular design.

Enterprise environments require a focus on efficient module management. One crucial principle is maximizing code reuse – a practice with significant benefits for long-term development and maintenance.

The pitfalls of singletons
A common misstep is using module singletons to hold state variables. This creates tight coupling between modules, making refactoring and updates challenging. Singletons can also lead to version conflicts when different parts of your application depend on separate instances of the same module.

For instance, consider a scenario where multiple modules import a shared module containing a random number generator implemented as a singleton. While convenient at first, updating the random number generation logic in the shared module can lead to unexpected behavior if one module uses an outdated version.

Dependency injection to the rescue
Dependency injection provides a solution by promoting loose coupling between modules. Instead of modules creating or requiring their dependencies directly, dependencies are passed as arguments when a module is instantiated or called. This allows for:

  1. Increased flexibility: Modules become independent, facilitating easier testing, swapping implementations, and mocking behavior.


  2. Improved maintainability: Changes to one module have a minimal impact on others as long as the dependency interface remains consistent.



Example: Decoupling with dependency injection
Imagine a module responsible for retrieving an access token and another module performing an operation requiring the token. With dependency injection, the token retrieval logic can be decoupled:


  • The access token retrieval module exposes a function (getToken) to retrieve the token.


  • The operation module receives the getToken function as a dependency during its creation.


This approach allows for independent testing and updating of each module without affecting the other.

The One-Module-One-Feature Principle

Key benefits:


  • Isolation and reusability: Modules become independent, facilitating easier testing, swapping implementations, and mocking behavior.


  • Scalability: Modules can be developed and deployed independently, allowing for easier scaling and maintenance.


  • Microservices foundation: A module-based design lays the groundwork for transitioning to a microservices architecture if needed.

Focus on high cohesion, loose coupling

To create modular, testable, and maintainable Node.js applications, adhere to the principles of high cohesion and loose coupling.


  • High cohesion: A module should have a single, well-defined responsibility and focus on related functionalities. This makes modules easier to understand, test, and modify.


  • Loose coupling: Modules should have minimal dependencies on other modules, reducing the risk of unintended side effects and improving maintainability.


In order to do so:

  • Define clear responsibilities: Each module should have a well-defined purpose and focus on a single task.


  • Avoid overly large modules: Break down large modules into smaller, more focused components.


  • Use descriptive naming: Choose meaningful names for classes, variables, and methods.


  • Minimize global variables: Avoid excessive use of global variables, as they can introduce tight coupling.

Always set NODE_ENV=production

While NODE_ENV=production is often used to enable optimizations and specific behaviors in Node.js applications, its indiscriminate use can lead to inconsistencies and challenges in development, testing, and staging environments.

Why NODE_ENV is a flawed approach
An environment is a digital platform or a system where engineers can build, test, deploy, and manage software products. Conventionally, there are four stages or types of environments where our application is run:


  • Development: A local or shared environment used for building, testing, and debugging the application.

  • Testing: An environment dedicated to testing the application in a controlled setting, often with automated tests.


  • Staging: A production-like environment used for final testing and quality assurance before deployment to production.


  • Production: The live environment where the application is accessible to end-users.

    The fundamental problem with NODE_ENV stems from developers combining optimizations and software behavior with the environment their software is running on. The result is code like the following:

if (process.env.NODE_ENV === 'development') {
  // ...
}

if (process.env.NODE_ENV === 'production') {
  // ...
}

if (['production', 'staging'].includes(process.env.NODE_ENV)) {
  // ...
}

While this might look harmless, it makes the production and staging environments different, thus making reliable testing impossible. 

For example a test and thus a functionality of your product could pass when NODE_ENV is set to development but fail when setting NODE_ENV to production

Therefore, setting NODE_ENV to anything but production is considered an antipattern.

Best practices for enterprise Node.js configuration


  • Feature flags: Enable or disable features based on dedicated environment variables, allowing for granular control over application behavior.


  • Dedicated environment variables: Create separate environment variables for each deployment stage (development, testing, staging, production) to manage configurations specific to each.


  • Secret management tools: Utilize dedicated secret management solutions to store and securely access sensitive information like API keys and database credentials. These tools can integrate with your deployment process to inject secrets at runtime.It’s important here to have a deep understanding of how environment variables work in Node.js applications– this may help. 


  • Environment-agnostic configuration: Strive for code that adapts to different environments based on configuration loaded at runtime, minimizing the need for environment-specific logic.


  • Be mindful of CI/CD pipeline evolution: Adapt your environment configuration accordingly. Remember that while staging and production should ideally be identical, development environments can often have different configurations to facilitate local development and testing.


Why follow this principle?

  • Improved reliability: Consistent behavior across all environments reduces the risk of errors during deployments.


  • Enhanced Security: Sensitive data is protected through dedicated management tools.

  • Reduced maintenance efforts: Code becomes more flexible and easier to manage as environment-specific logic is minimized.


8. Handle errors and provide meaningful logs

Ensuring the robustness and stability of your Node.js applications hinges on implementing effective error handling and logging practices. To do so, you will need to capture and address unexpected errors gracefully, prevent application crashes, and gain valuable insights into your application's behavior through detailed logging. 

Let’s take a look in more detail:

Adopt consistent error handling patterns

Error handling is a critical aspect of Node.js development. It involves capturing and managing errors that may occur during the execution of your application, preventing crashes and providing meaningful feedback to users. By implementing effective error handling mechanisms, you can enhance the reliability and maintainability of your Node.js applications.

Handling global uncaught exceptions:
To ensure your application doesn't crash unexpectedly, it's essential to handle global uncaught exceptions. This involves using the process.on('uncaughtException') event listener to capture and log these exceptions. You can then take appropriate actions, such as gracefully shutting down the application or sending an alert to a monitoring system.

Graceful shutdown:
When handling uncaught exceptions, consider implementing a graceful shutdown process to ensure proper resource clean-up and avoid abrupt termination. You can use libraries like close-with-grace to register shutdown handlers and perform tasks like closing database connections or handling in-flight requests before the application exits.

Provide meaningful logs

Logging in Node.js development is important because it helps developers monitor the events of their application, track performance, and get alerted of bugs or system malfunctions in their apps.

Logging also comes in handy in production environments for quickly detecting and fixing the source of errors. Moreover, logging offers developers enhanced visibility by providing a clear view of an application’s behavior and performance.

What to Log:
When logging in Node.js, it's essential to strike a balance between providing sufficient information for troubleshooting and avoiding excessive logging that can impact performance

Here are some key considerations:

  • Error logging: Log errors with appropriate levels of detail (e.g., debug, info, warn, error) to track and address issues effectively.

  • Request and response information: Log incoming requests and outgoing responses, including HTTP methods, URLs, request parameters, and response status codes. This helps identify and troubleshoot potential problems.


  • Performance metrics: Log performance metrics such as response times, request durations, and resource utilization to monitor application performance and identify bottlenecks.


  • Application state: Log important application state changes or events to track the application's behavior and understand the sequence of events leading to errors or unexpected outcomes.


  • Custom log messages: Create custom log messages that provide meaningful context and information specific to your application.


How much to log:
The appropriate level of logging depends on your specific needs and the complexity of your application. Consider these guidelines:


  • Production environment: In production environments, log at a lower level (e.g., info, warn, error) to avoid excessive logging that can impact performance. Focus on logging errors, critical events, and performance metrics.


  • Development and testing: In development and testing environments, you can log at higher levels (e.g., debug) to gather more detailed information for troubleshooting and debugging purposes.

Logging libraries

My personal recommendation for logging is Pino, a very low-overhead Node.js logger, first published in 2014. Since then, it has gained popularity due to its speed, efficiency, and flexible configuration.

Pino has first-class support for other web frameworks like Express, Fastify, Nest, Restify, Hapi, Koa, and core Node, and is the default logger for Fastify.

Why do developers choose Pino?

First of all, Pino runs asynchronously; meaning it accumulates the logs if the destination is busy and sends them immediately when the destination becomes available.

Secondly, Pino is about five times faster than most other alternatives. To gain this speed, Pino utilizes minimal resources for logging, and as a result, log messages tend to get added over time, leading to a throttling effect on the application, like reduced requests per second.

Additionally, Pino is not a stand-alone tool as it supports being bundled with tools such as Webpack and esbuild, providing developers with more integrational options and subsequently, opportunities for creativity.

-Matteo Collina


9. Use API specifications and automatically generate clients

In a multi-team environment, establishing a clear and consistent API specification is crucial for effective collaboration and integration. By defining your API's structure, data types, and operations, you can ensure that different teams can work independently while maintaining compatibility and avoiding misunderstandings. Additionally, automating the generation of client code from API specifications can significantly streamline development and reduce the risk of errors.

API First vs. Code First

When designing and developing APIs, there are two primary approaches: API First and Code First. Each approach has its own advantages and considerations.


  • API First: Define the API specification before writing the implementation code. This approach promotes a contract-driven development style, ensuring that both the client and server teams have a clear understanding of the API's capabilities and expectations.


  • Code First: Develop the implementation code first and then generate the API specification based on the existing code. While this approach can be faster to start, it may introduce inconsistencies and make it more challenging to maintain alignment between the API and the implementation.

OpenAPI vs. GraphQL

Both OpenAPI and GraphQL are powerful tools for defining and interacting with APIs, but they serve different purposes and have different strengths.

OpenAPI:

  • A widely used specification language for RESTful APIs.


  • Defines the API's structure, data types, and operations.


  • Well-suited for scenarios with a predefined API contract.

GraphQL:

  • A query language for APIs that offers more flexibility and efficiency.


  • Allows clients to specify exactly the data they need, reducing over-fetching and under-fetching.

Generating Clients and types

Automating the generation of clients and types from API specifications can significantly streamline development and reduce the risk of errors. By leveraging tools like OpenAPI Generator or Swagger Codegen for OpenAPI specifications, and graphql-codegen for GraphQL, you can automate tasks that were previously time-consuming and error-prone, speed up development cycles, reduce the risk of type-related errors, and enhanced code quality and maintainability. Additionally, automated type generation can help ensure consistency between the API definition and the client code, reducing the likelihood of compatibility issues.

By adopting API specifications and automating client generation, you can streamline development, improve code quality, and foster collaboration in your multi-team environment.


Wrapping up

By adhering to these guiding principles, you'll be able to set up your Node.js applications for long-term success. From mastering the event loop to implementing robust dependency management, these pillars were written to help you to build scalable, secure, and maintainable solutions.

Remember, the Node.js community thrives on collaboration – actively contribute and support key projects to ensure a flourishing ecosystem for all.

With this knowledge in hand, we hope to see you elevate your Node.js development and build brilliant applications that stand the test of time.

Hear more about related content & events

Does your organization use Node.js in its tech stack

The recommendations and insights provided in this document are based on personal experiences and opinions of the contributors and should not be interpreted as definitive guidance or professional advice.

Copyright:
All product and company names mentioned in this document are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Platformatic

Platformatic Inc.

548 Market St PMB 85033 San Francisco,

California 94104-5401 US

Copyright © | All rights reserved

Follow us on:

Fastify™ & Node.js© are trademarks of OpenJS Foundation