diff --git a/docs/shared/mental-model.md b/docs/shared/mental-model.md index cf724fc814..fecc6fa3e6 100644 --- a/docs/shared/mental-model.md +++ b/docs/shared/mental-model.md @@ -1,14 +1,20 @@ # Mental Model -Nx is a VSCode of build tools, with a powerful core, driven by metadata, and extensible through plugins. Nx works with a few concepts to drive your monorepo efficiently, and effectively. This guide covers the mental model around how Nx works with project graphs, task graphs, affected commands, computation hashing and caching. +Nx is a VSCode of build tools, with a powerful core, driven by metadata, and extensible through plugins. Nx works with a +few concepts to drive your monorepo efficiently, and effectively. This guide covers the mental model around how Nx works +with project graphs, task graphs, affected commands, computation hashing and caching. ## The project graph -A project graph is used to reflect the source code in your repository and all the external dependencies that aren’t authored in your repository, such as Webpack, React, Angular, and so forth. +A project graph is used to reflect the source code in your repository and all the external dependencies that aren’t +authored in your repository, such as Webpack, React, Angular, and so forth. ![project-graph](/shared/mental-model/project-graph.png) -With Nx, nodes in the project graph are defined in `workspace.json`. You can manually define dependencies between the nodes, but you don’t have to do it very often. Nx analyzes files’ source code, your installed dependencies, TypeScript files, and others figuring out these dependencies for you. Nx also stores the cached project graph, so it only reanalyzes the files you have changed. +With Nx, nodes in the project graph are defined in `workspace.json`. You can manually define dependencies between the +nodes, but you don’t have to do it very often. Nx analyzes files’ source code, your installed dependencies, TypeScript +files, and others figuring out these dependencies for you. Nx also stores the cached project graph, so it only +reanalyzes the files you have changed. ![project-graph-updated](/shared/mental-model/project-graph-updated.png) @@ -16,7 +22,8 @@ Nx provides an updated graph after each analysis is done. ## Metadata-driven -Everything in Nx comes with metadata to enable toolability. The default values, validations, autocompletion work, and more are all defined in a schema, instead of in code. +Everything in Nx comes with metadata to enable toolability. The default values, validations, autocompletion work, and +more are all defined in a schema, instead of in code. The following sample schema shows inputs, prompts, and validations for adding a new application. @@ -52,7 +59,10 @@ The following sample schema shows inputs, prompts, and validations for adding a "message": "Which stylesheet format would you like to use?", "type": "list", "items": [ - { "value": "css", "label": "CSS" }, + { + "value": "css", + "label": "CSS" + }, { "value": "scss", "label": "SASS(.scss) [ http://sass-lang.com ]" @@ -116,7 +126,8 @@ The following sample schema shows inputs, prompts, and validations for adding a } ``` -This metadata is used by Nx itself, by VSCode and WebStorm integrations, by GitHub integration, and by third-party tools. +This metadata is used by Nx itself, by VSCode and WebStorm integrations, by GitHub integration, and by third-party +tools. ![metadata](/shared/mental-model/metadata.png) @@ -124,7 +135,8 @@ These tools are able to implement richer experiences with Nx using this metadata ## The task graph -Nx uses the project graph to create a task graph. Any time you run anything, Nx creates a task graph from the project graph and then executes the tasks in that graph. +Nx uses the project graph to create a task graph. Any time you run anything, Nx creates a task graph from the project +graph and then executes the tasks in that graph. For instance `nx test lib` creates a task graph with a single node: @@ -132,11 +144,14 @@ For instance `nx test lib` creates a task graph with a single node: A task is an invocation of a target. If you invoke the same target twice, you create two tasks. -Nx uses the [project graph](#the-project-graph), but the task graph and project graph aren’t isomorphic, meaning they aren’t directly connected. In the case above, app1 and app2 depend on lib, but running `nx run-many --target=test --projects=app1,app2,lib`, the created task graph will look like this: +Nx uses the [project graph](#the-project-graph), but the task graph and project graph aren’t isomorphic, meaning they +aren’t directly connected. In the case above, app1 and app2 depend on lib, but +running `nx run-many --target=test --projects=app1,app2,lib`, the created task graph will look like this: ![task-graph-creation](/shared/mental-model/task-graph-creation.png) -Even though the apps depend on lib, testing `app1` doesn’t depend on the testing lib. This means that the two tasks can run in parallel. +Even though the apps depend on lib, testing `app1` doesn’t depend on the testing lib. This means that the two tasks can +run in parallel. Let’s look at the test target relying on its dependencies. @@ -145,7 +160,12 @@ Let’s look at the test target relying on its dependencies. "test": { "executor": "@nrwl/jest:jest", "outputs": ["coverage/apps/app1"], - "dependsOn": [{ "target": "test", "projects": "dependencies" }], + "dependsOn": [ + { + "target": "test", + "projects": "dependencies" + } + ], "options": { "jestConfig": "apps/app1/jest.config.js", "passWithNoTests": true @@ -158,35 +178,46 @@ With this, running the same test command creates the following task graph: ![task-graph-run](/shared/mental-model/task-graph-run.png) -This often makes more sense for builds, where to build app1, you want to build lib first. You can also define similar relationships between targets of the same project, including a test target that depends on the build. +This often makes more sense for builds, where to build app1, you want to build lib first. You can also define similar +relationships between targets of the same project, including a test target that depends on the build. -A task graph can contain different targets, and those can run in parallel. For instance, as Nx is building `app2`, it can be testing `app1` at the same time. Learn more about configuring targets in the [configuration guide](/{{framework}}/core-concepts/configuration) +A task graph can contain different targets, and those can run in parallel. For instance, as Nx is building `app2`, it +can be testing `app1` at the same time. Learn more about configuring targets in +the [configuration guide](/{{framework}}/core-concepts/configuration) ![task-graph-execution](/shared/mental-model/task-graph-execution.png) -Nx also runs the tasks in the task graph in the right order. Nx executing tasks being executed speeds up your overall execution time. +Nx also runs the tasks in the task graph in the right order. Nx executing tasks being executed speeds up your overall +execution time. ## Affected commands When you run `nx test app1`, you are telling Nx to run the app1:test task plus all the tasks it depends on. -When you run `nx run-many --target=test --projects=app1,lib`, you are telling Nx to do the same for two tasks app1:test and lib:test. +When you run `nx run-many --target=test --projects=app1,lib`, you are telling Nx to do the same for two tasks app1:test +and lib:test. When you run `nx run-many --target=test --all`, you are telling Nx to do this for all the projects. -As your workspace grows, retesting all projects becomes too slow. To address this Nx implements code change analysis to get the min set of projects that need to be retested. How does it work? +As your workspace grows, retesting all projects becomes too slow. To address this Nx implements code change analysis to +get the min set of projects that need to be retested. How does it work? -When you run `nx affected --target=test`, Nx looks at the files you changed in your PR, it will look at the nature of change (what exactly did you update in those files), and it uses this to figure the list of projects in the workspace that can be affected by this change. It then runs the `run-many` command with that list. +When you run `nx affected --target=test`, Nx looks at the files you changed in your PR, it will look at the nature of +change (what exactly did you update in those files), and it uses this to figure the list of projects in the workspace +that can be affected by this change. It then runs the `run-many` command with that list. -For instance, if my PR changes `lib`, and I then run `nx affected --target=test`, Nx figures out that `app1` and `app2` depend on `lib`, so it will invoke `nx run-many --target=test --projects=app1,app2,lib`. +For instance, if my PR changes `lib`, and I then run `nx affected --target=test`, Nx figures out that `app1` and `app2` +depend on `lib`, so it will invoke `nx run-many --target=test --projects=app1,app2,lib`. ![affected](/shared/mental-model/affected.png) -Nx analyzes the nature of the changes. For example, if you change the version of Next.js in the package.json, Nx knows that `app2` cannot be affected by it, so it only retests `app1`. +Nx analyzes the nature of the changes. For example, if you change the version of Next.js in the package.json, Nx knows +that `app2` cannot be affected by it, so it only retests `app1`. ## Computation hashing and caching -Nx runs the tasks in the task graph in the right order. Before running the task, Nx computes its computation hash. As long as the computation hash is the same, the output of running the task is the same. +Nx runs the tasks in the task graph in the right order. Before running the task, Nx computes its computation hash. As +long as the computation hash is the same, the output of running the task is the same. How does Nx do it? @@ -200,36 +231,70 @@ By default, the computation hash for say `nx test app1` includes: ![computation-hashing](/shared/mental-model/computation-hashing.png) -This behavior is customizable. For instance, lint checks may only depend on the source code of the project and global configs. Builds can depend on the dts files of the compiled libs instead of their source. +This behavior is customizable. For instance, lint checks may only depend on the source code of the project and global +configs. Builds can depend on the dts files of the compiled libs instead of their source. -After Nx computes the hash for a task, it then checks if it ran this exact computation before. First, it checks locally, and then if it is missing, and if a remote cache is configured, it checks remotely. +After Nx computes the hash for a task, it then checks if it ran this exact computation before. First, it checks locally, +and then if it is missing, and if a remote cache is configured, it checks remotely. -If Nx finds the computation, Nx retrieves it and replay it. Nx places the right files in the right folders and prints the terminal output. So from the user’s point of view, the command ran the same, just a lot faster. +If Nx finds the computation, Nx retrieves it and replay it. Nx places the right files in the right folders and prints +the terminal output. So from the user’s point of view, the command ran the same, just a lot faster. ![cache](/shared/mental-model/cache.png) -If Nx doesn’t find this computation, Nx runs the task, and after it completes, it takes the outputs and the terminal output and stores it locally (and if configured remotely). All of this happens transparently, so you don’t have to worry about it. +If Nx doesn’t find this computation, Nx runs the task, and after it completes, it takes the outputs and the terminal +output and stores it locally (and if configured remotely). All of this happens transparently, so you don’t have to worry +about it. -Although conceptually this is fairly straightforward, Nx optimizes this to make this experience good for you. For instance, Nx: +Although conceptually this is fairly straightforward, Nx optimizes this to make this experience good for you. For +instance, Nx: - Captures stdout and stderr to make sure the replayed output looks the same, including on Windows. - Minimizes the IO by remembering what files are replayed where. - Only shows relevant output when processing a large task graph. -- Provides affordances for troubleshooting cache misses. - And many other optimizations. +- Provides affordances for troubleshooting cache misses. And many other optimizations. As your workspace grows, the task graph looks more like this: ![cache](/shared/mental-model/task-graph-big.png) -All of these optimizations are crucial for making Nx usable for any non-trivial workspace. Only the minimum amount of work happens. The rest is either left as is or restored from the cache. +All of these optimizations are crucial for making Nx usable for any non-trivial workspace. Only the minimum amount of +work happens. The rest is either left as is or restored from the cache. -In summary: +## Distributed task execution + +Nx supports running commands across multiple machines. You can either set it up by hand ( +see [here](/ci/distributed-builds)) or use Nx +CLoud. [Read the comparison of the two approaches.](https://blog.nrwl.io/distributing-ci-binning-and-distributed-task-execution-632fe31a8953?source=friends_link&sk=5120b7ff982730854ed22becfe7a640a) + +Nx Cloud is a cloud companion for Nx (which is free and MIT-licensed). Most features of Nx Cloud are free, but some are +paid. One of them is the distributed computation cache, which allows you to share cache with your team members and CI +agents. + +Another one is config-free distributed task execution (DTE). When using the distributed task execution, Nx is able to +run any task graph on a many agents instead of locally. + +When using this, `nx affected --build`, won't run the build locally (which for large workspace can take hours). Instead, +it will send the Task Graph to Nx Cloud. Nx Cloud Agents will then pick up the task they can run and execute them. + +Note this happens transparently. If an agent builds `app1`, it will fetch the outputs for `lib` if it doesn't have it +already. + +As agents complete tasks, the main job where you invoked `nx affected --build` will start receiving created files and +terminal outputs. + +After `nx affected --build` completes, the machine will have the build files and all the terminal outputs as if it ran +it locally. + +![DTE](/shared/mental-model/dte.png) + +## In summary - Nx is able to analyze your source code to create a Project Graph. - Nx can use the project graph and information about projects’ targets to create a Task Graph. - Nx is able to perform code-change analysis to create the smallest task graph for your PR. -- Nx supports computation caching to never execute the same computation twice. This computation cache is pluggable and can be distributed. +- Nx supports computation caching to never execute the same computation twice. This computation cache is pluggable and + can be distributed. ## Learn more: diff --git a/docs/shared/mental-model/dte.png b/docs/shared/mental-model/dte.png new file mode 100644 index 0000000000..eec3627e7f Binary files /dev/null and b/docs/shared/mental-model/dte.png differ diff --git a/nx-dev/nx-dev/pages/angular.tsx b/nx-dev/nx-dev/pages/angular.tsx index aa2898cd83..8163a0db85 100644 --- a/nx-dev/nx-dev/pages/angular.tsx +++ b/nx-dev/nx-dev/pages/angular.tsx @@ -3,14 +3,10 @@ import { useRouter } from 'next/router'; import Image from 'next/image'; import Link from 'next/link'; import Head from 'next/head'; -import { - Footer, - Header, - InlineCommand, - NxUsersShowcase, -} from '@nrwl/nx-dev/ui/common'; +import { Footer, Header, NxUsersShowcase } from '@nrwl/nx-dev/ui/common'; import { sendCustomEvent } from '@nrwl/nx-dev/feature-analytics'; import { useStorage } from '@nrwl/nx-dev/feature-storage'; +import { InlineCommand } from '@nrwl/nx-dev/ui-commands'; export function AngularPage() { const router = useRouter(); diff --git a/nx-dev/nx-dev/pages/index.tsx b/nx-dev/nx-dev/pages/index.tsx index 375a2ba0ba..ab27063df1 100644 --- a/nx-dev/nx-dev/pages/index.tsx +++ b/nx-dev/nx-dev/pages/index.tsx @@ -1,16 +1,32 @@ import React from 'react'; import { useRouter } from 'next/router'; -import Image from 'next/image'; import Link from 'next/link'; import Head from 'next/head'; import { Footer, Header, - InlineCommand, + NpxCreateNxWorkspace, NxUsersShowcase, - FeatureList, + Testimonials, } from '@nrwl/nx-dev/ui/common'; import { sendCustomEvent } from '@nrwl/nx-dev/feature-analytics'; +import { + AffectedCommand, + CloudSupport, + DependencyGraph, + EcosystemFeatures, + EggheadCourses, + ExperienceFeatures, + GettingStarted, + MonorepoFeatures, + NxPlaybook, + OpenPlatform, + OpenSourceProjects, + Performance, + VscodePlugin, + YoutubeChannel, +} from '@nrwl/nx-dev/ui-home'; +import { InlineCommand } from '@nrwl/nx-dev/ui-commands'; export function Index() { const router = useRouter(); @@ -60,16 +76,17 @@ export function Index() { {/*INTRO COMPONENT*/}
-
-
+
+
-

- Smart, Extensible Build Framework +

+ Smart, Extensible{' '} + Build Framework

Nx helps architect, test, and build at any scale: @@ -82,12 +99,18 @@ export function Index() {

-