From f98ab3f8206d752ca8601c40d1fb4ecd65ccda1b Mon Sep 17 00:00:00 2001 From: Craigory Coppola Date: Wed, 18 Oct 2023 18:33:09 -0400 Subject: [PATCH] docs(core): update docs for new tasks runner opts (#19647) Co-authored-by: Isaac Mann --- docs/nx-cloud/account/access-tokens.md | 32 +++++++---- docs/nx-cloud/reference/config.md | 53 +++++++++++++------ docs/shared/concepts/how-caching-works.md | 16 +++++- .../concepts/task-pipeline-configuration.md | 16 +++++- .../core-features/cache-task-results.md | 24 +++++++++ .../migration/adding-to-existing-project.md | 24 ++++++++- docs/shared/migration/adding-to-monorepo.md | 26 +++++++++ docs/shared/migration/from-turborepo.md | 38 +++++++++++++ docs/shared/migration/lerna-and-nx.md | 22 +++++++- .../recipes/troubleshoot-cache-misses.md | 2 +- docs/shared/reference/nx-json.md | 26 ++++++++- .../shared/reference/project-configuration.md | 20 +++++++ 12 files changed, 266 insertions(+), 33 deletions(-) diff --git a/docs/nx-cloud/account/access-tokens.md b/docs/nx-cloud/account/access-tokens.md index 66f9693f4d..57fc2a5f3d 100644 --- a/docs/nx-cloud/account/access-tokens.md +++ b/docs/nx-cloud/account/access-tokens.md @@ -23,21 +23,33 @@ Let's see how access tokens work. If you open your `nx.json`, you will see something like this: +{% tabs %} +{% tab label="Nx >= 17" %} + ```json { - "tasksRunnerOptions": { - "default": { - "runner": "nx-cloud", - "options": { - "accessToken": "SOMETOKEN", - "cacheableOperations": ["build", "test", "lint", "e2e"] - } - } - } + "nxCloudAccessToken": "SOMETOKEN" } ``` -If you remove the `accessToken` property from the configuration, the runner will run all commands as if you were not connected to Nx Cloud. This essentially turns off Nx Cloud. +{% /tab %} +{% tab label="Nx < 17" %} + +```json +"tasksRunnerOptions": { + "default": { + "runner": "nx-cloud", + "options": { + "accessToken": "SOMETOKEN" + } + } + } +``` + +{% /tab %} +{% /tabs %} + +If you remove the `nxCloudAccessToken` or `accessToken` property from the configuration, the runner will run all commands as if you were not connected to Nx Cloud. This essentially turns off Nx Cloud. ## Setting a Different Access Token in CI diff --git a/docs/nx-cloud/reference/config.md b/docs/nx-cloud/reference/config.md index 4f07607f20..c5d668d353 100644 --- a/docs/nx-cloud/reference/config.md +++ b/docs/nx-cloud/reference/config.md @@ -2,23 +2,35 @@ The Nx Cloud runner is configured in `nx.json`. +{% tabs %} +{% tab label="Nx >= 17" %} + ```json { - "tasksRunnerOptions": { - "default": { - "runner": "nx-cloud", - "options": { - "accessToken": "SOMETOKEN", - "cacheableOperations": ["build", "test", "lint", "e2e"] - } - } - } + "nxCloudAccessToken": "SOMETOKEN" } ``` +{% /tab %} +{% tab label="Nx < 17" %} + +```json +"tasksRunnerOptions": { + "default": { + "runner": "nx-cloud", + "options": { + "accessToken": "SOMETOKEN" + } + } + } +``` + +{% /tab %} +{% /tabs %} + ## Cacheable Operations -Only operations listed in `cacheableOperations` can be cached using Nx Cloud and distributed using the distributed task execution (DTE). You can add new targets to that list. +Targets can be marked as cacheable either in the `targetDefaults` in `nx.json` or in the project configuration by setting `"cache": true`. With this option enabled they can be cached using Nx Cloud and distributed using distributed task execution (DTE). ## Timeouts @@ -46,7 +58,19 @@ This can be useful for debugging unexpected cache misses, and issues with on-pre All communication with Nx Cloud’s API and cache is completed over HTTPS, but you can optionally enable e2e encryption by providing a secret key through `nx.json` or the `NX_CLOUD_ENCRYPTION_KEY` environment variable. -In `nx.json`, locate the `taskRunnerOptions` property. It will look something like this: +{% tabs %} +{% tab label="Nx >= 17" %} +In `nx.json`, add the `nxCloudEncryptionKey` property. It will look something like this: + +```json +{ + "nxCloudEncryptionKey": "cheddar" +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} +In `nx.json`, locate the `taskRunnerOptions` property. Under its "options" property, you can add another property called `encryptionKey`. This is what will be used to encrypt your artifacts. It will look something like this: ```json { @@ -55,7 +79,6 @@ In `nx.json`, locate the `taskRunnerOptions` property. It will look something li "runner": "nx-cloud", "options": { "accessToken": "SOMETOKEN", - "cacheableOperations": ["build", "test", "lint", "e2e"], // Add the following property with your secret key "encryptionKey": "cheddar" } @@ -64,7 +87,8 @@ In `nx.json`, locate the `taskRunnerOptions` property. It will look something li } ``` -Under the "options" property, you can add another property called `encryptionKey`. This is what will be used to encrypt your artifacts. +{% /tab %} +{% /tabs %} To instead use an environment variable to provide your secret key, run any Nx command as follows: @@ -90,9 +114,6 @@ You must be on version `16.0.4` or later of `nx-cloud` or `@nrwl/nx-cloud` for t ```json { - "tasksRunnerOptions": { - ... - }, // The following will cause all attempts to connect your workspace to Nx Cloud to fail "neverConnectToCloud": true } diff --git a/docs/shared/concepts/how-caching-works.md b/docs/shared/concepts/how-caching-works.md index c652c23472..40a43db920 100644 --- a/docs/shared/concepts/how-caching-works.md +++ b/docs/shared/concepts/how-caching-works.md @@ -231,12 +231,23 @@ npx nx run test --skip-nx-cache The cache is stored in `node_modules/.cache/nx` by default. To change the cache location, update the `cacheDirectory` option for the task runner in `nx.json`: +{% tabs %} +{% tab label="Nx >= 17" %} + +```json {% fileName="nx.json"%} +{ + "cacheDirectory": "/tmp/mycache" +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} + ```json {% fileName="nx.json"%} { "tasksRunnerOptions": { "default": { "options": { - "cacheableOperations": ["build", "test"], "cacheDirectory": "/tmp/mycache" } } @@ -244,6 +255,9 @@ The cache is stored in `node_modules/.cache/nx` by default. To change the cache } ``` +{% /tab %} +{% /tabs %} + ## Outputs vs Output Path Several executors have a property in `options` called `outputPath`. On its own, this property does not influence caching or what is stored at the end of a run. Frequently though, this property would point to your build artifacts. In these cases, you can include `"{options.outputPath}"` in the `outputs` array for your target to avoid duplicating the value. diff --git a/docs/shared/concepts/task-pipeline-configuration.md b/docs/shared/concepts/task-pipeline-configuration.md index 970908675c..245900f4d3 100644 --- a/docs/shared/concepts/task-pipeline-configuration.md +++ b/docs/shared/concepts/task-pipeline-configuration.md @@ -13,13 +13,24 @@ npx nx build myapp --parallel=5 Note, you can also change the default in `nx.json`, like this: +{% tabs %} +{% tab label="Nx >= 17" %} + +```json {% fileName="nx.json"%} +{ + "parallel": 5 +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} + ```json {% fileName="nx.json"%} { "tasksRunnerOptions": { "default": { "runner": "nx/tasks-runners/default", "options": { - "cacheableOperations": [], "parallel": 5 } } @@ -27,6 +38,9 @@ Note, you can also change the default in `nx.json`, like this: } ``` +{% /tab %} +{% /tabs %} + ## Define Task Dependencies (aka Task Pipelines) To ensure tasks run in the correct order, Nx needs to know how the tasks depend on each other. Add the following to `nx.json`: diff --git a/docs/shared/core-features/cache-task-results.md b/docs/shared/core-features/cache-task-results.md index bcb5dfe326..46c33aad75 100644 --- a/docs/shared/core-features/cache-task-results.md +++ b/docs/shared/core-features/cache-task-results.md @@ -8,6 +8,27 @@ same code twice. Nx has the most sophisticated and battle-tested computation caching system. It knows when the task you are about to run has been executed before, so it can use the cache to restore the results of running that task. +{% tabs %} +{% tab label="Nx >= 17" %} + +To enable caching for `build` and `test`, edit the `targetDefaults` property in `nx.json` to include the `build` and `test` tasks: + +```json {% fileName="nx.json" %} +{ + "targetDefaults": { + "build": { + "cache": true + }, + "test": { + "cache": true + } + } +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} + To enable caching for `build` and `test`, edit the `cacheableOperations` property in `nx.json` to include the `build` and `test` tasks: ```json {% fileName="nx.json" %} @@ -23,6 +44,9 @@ To enable caching for `build` and `test`, edit the `cacheableOperations` propert } ``` +{% /tab %} +{% /tabs %} + {% callout type="note" title="Cacheable operations need to be side effect free" %} This means that given the same input they should always result in the same output. As an example, e2e test runs that hit the backend API cannot be cached as the backend might influence diff --git a/docs/shared/migration/adding-to-existing-project.md b/docs/shared/migration/adding-to-existing-project.md index 5e6e924090..79f5eb9fdf 100644 --- a/docs/shared/migration/adding-to-existing-project.md +++ b/docs/shared/migration/adding-to-existing-project.md @@ -51,7 +51,26 @@ This process adds `nx` to your `package.json` at the root of your workspace: } ``` -In addition it generates a `nx.json` based on your answers during the setup process. This includes cacheable operations as well as some initial definition of the task pipeline. Here is an example: +In addition it generates a `nx.json` based on your answers during the setup process. This includes cacheable targets as well as some initial definition of the task pipeline. Here is an example: + +{% tabs %} +{% tab label="Nx >= 17" %} + +```json {% fileName="nx.json" %} +{ + "targetDefaults": { + "build": { + "cache": true + }, + "lint": { + "cache": true + } + } +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} ```json {% fileName="nx.json" %} { @@ -66,6 +85,9 @@ In addition it generates a `nx.json` based on your answers during the setup proc } ``` +{% /tab %} +{% /tabs %} + ## Wrapping Cacheable Scripts Nx also automatically wraps your cacheable scripts with the `nx exec` command. The main advantage here is that you can still keep using `npm start` or `npm run build` (or other package manager's alternatives) as you're accustomed to. But still get the benefits of making those operations cacheble. diff --git a/docs/shared/migration/adding-to-monorepo.md b/docs/shared/migration/adding-to-monorepo.md index 40912f8457..47b40d2380 100644 --- a/docs/shared/migration/adding-to-monorepo.md +++ b/docs/shared/migration/adding-to-monorepo.md @@ -49,6 +49,29 @@ This process adds `nx` to your `package.json` at the root of your workspace: It also creates a `nx.json` based on the answers given during the setup process. This includes cacheable operations as well as some initial definition of the task pipeline. Here is an example: +{% tabs %} +{% tab label="Nx >= 17" %} + +```json {% fileName="nx.json" %} +{ + "targetDefaults": { + "build": { + "cache": true, + "dependsOn": ["^build"] + }, + "test": { + "cache": true + }, + "lint": { + "cache": true + } + } +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} + ```json {% fileName="nx.json" %} { "tasksRunnerOptions": { @@ -67,6 +90,9 @@ It also creates a `nx.json` based on the answers given during the setup process. } ``` +{% /tab %} +{% /tabs %} + ## Incrementally Adopting Nx In a package-based monorepo, Nx only manages the scheduling and caching of your npm scripts. Hence, it can easily be adopted incrementally by initially using Nx just for a subset of your scripts and then gradually adding more. diff --git a/docs/shared/migration/from-turborepo.md b/docs/shared/migration/from-turborepo.md index bfdacbc2ba..7b7505f09c 100644 --- a/docs/shared/migration/from-turborepo.md +++ b/docs/shared/migration/from-turborepo.md @@ -80,6 +80,41 @@ Let's say you start with the following `turbo.json` file: Creating the equivalent configuration with Nx yields the following files: +{% tabs %} +{% tab label="Nx >= 17" %} + +```json {% fileName="/nx.json" %} +{ + "$schema": "./node_modules/nx/schemas/nx-schema.json", + "namedInputs": { + "sharedGlobals": ["babel.config.json"], + "default": ["{projectRoot}/**/*", "sharedGlobals"] + }, + "targetDefaults": { + "build": { + "dependsOn": ["^build"], + "inputs": ["default"], + "outputs": ["{projectRoot}/dist"], + "cache": true + }, + "test": { + "dependsOn": ["build"], + "inputs": ["default"], + "cache": true + }, + "e2e": { + "dependsOn": ["build"], + "inputs": ["default"], + "cache": true + } + }, + "nxCloudAccessToken": "..." +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} + ```json {% fileName="/nx.json" %} { "$schema": "./node_modules/nx/schemas/nx-schema.json", @@ -114,6 +149,9 @@ Creating the equivalent configuration with Nx yields the following files: } ``` +{% /tab %} +{% /tabs %} + ```jsonc {% fileName="/packages/docs/package.json" %} { "name": "docs", diff --git a/docs/shared/migration/lerna-and-nx.md b/docs/shared/migration/lerna-and-nx.md index 806c1b0b7f..67d882c048 100644 --- a/docs/shared/migration/lerna-and-nx.md +++ b/docs/shared/migration/lerna-and-nx.md @@ -63,7 +63,24 @@ By default `useNx` will be set to `false`, so you have to explicitly opt-in. **3. Create a nx.json (optional but recommended)** -Nx works even without `nx.json` but to configure some more details such as the `cacheableOperations` of your monorepo in particular, create a `nx.json` at the root of the monorepo. Alternatively you can also just run `npx nx@latest init` to have one generated. Specify the cacheable operations, usually something like `build`, `test`, `lint` etc, depending on your workspace setup: +Nx works even without `nx.json` but to configure some more details such as the `cacheableOperations` of your monorepo in particular, create a `nx.json` at the root of the monorepo. Alternatively you can also just run `npx nx@latest init` to have one generated. Specify the cacheable targets, usually something like `build`, `test`, `lint` etc, depending on your workspace setup: + +{% tabs %} +{% tab label="Nx >= 17" %} + +```json {% fileName="nx.json" %} +{ + "extends": "nx/presets/npm.json", + "targetDefaults": { + "build": { + "cache": true + } + } +} +``` + +{% /tab %} +{% tab label="Nx < 17" %} ```json {% fileName="nx.json" %} { @@ -79,6 +96,9 @@ Nx works even without `nx.json` but to configure some more details such as the ` } ``` +{% /tab %} +{% /tabs %} + Having done these steps, you can now keep using your Lerna repository as you did before. All the commands will work in a backwards compatible way but will be a lot faster. [Read our blog post for some benchmarks](https://blog.nrwl.io/lerna-used-to-walk-now-it-can-fly-eab7a0fe7700?source=friends_link&sk=6c827ec7c9adfc1c760ff2e3f3e05cc7). {% callout type="note" title="Enable remote caching?" %} diff --git a/docs/shared/recipes/troubleshoot-cache-misses.md b/docs/shared/recipes/troubleshoot-cache-misses.md index 2ba905e9e2..7c5915c112 100644 --- a/docs/shared/recipes/troubleshoot-cache-misses.md +++ b/docs/shared/recipes/troubleshoot-cache-misses.md @@ -2,7 +2,7 @@ Problem: A task is being executed when you expect it to be replayed from the cache. -1. Check if your task is listed in `cacheableOperations` defined in `nx.json#tasksRunnerOptions` +1. Check if your task is either marked with `"cache": true` in `nx.json#targetDefaults` or listed in `cacheableOperations` defined in `nx.json#tasksRunnerOptions` 1. Check if the output of your task is changing the inputs of your task diff --git a/docs/shared/reference/nx-json.md b/docs/shared/reference/nx-json.md index 97081730af..8a0162906e 100644 --- a/docs/shared/reference/nx-json.md +++ b/docs/shared/reference/nx-json.md @@ -37,7 +37,8 @@ The following is an expanded example showing all options. Your `nx.json` will li "executor": "@nrwl/js:tsc", "options": { "main": "{projectRoot}/src/index.ts" - } + }, + "cache": true } } } @@ -205,7 +206,8 @@ When defining any options or configurations inside of a target default, you may }, "build": { "inputs": ["prod"], - "outputs": ["{workspaceRoot}/{projectRoot}"] + "outputs": ["{workspaceRoot}/{projectRoot}"], + "cache": true } } } @@ -215,6 +217,26 @@ When defining any options or configurations inside of a target default, you may Note that the inputs and outputs are respecified on the @nx/js:tsc default configuration. This is **required**, as when reading target defaults Nx will only ever look at one key. If there is a default configuration based on the executor used, it will be read first. If not, Nx will fall back to looking at the configuration based on target name. For instance, running `nx build project` will read the options from `targetDefaults[@nx/js:tsc]` if the target configuration for build uses the @nx/js:tsc executor. It **would not** read any of the configuration from the `build` target default configuration unless the executor does not match. {% /callout %} +#### Cache + +In Nx 17 and higher, caching is configured by specifying `"cache": true` in a target's configuration. This will tell Nx that it's ok to cache the results of a given target. For instance, if you have a target that runs tests, you can specify `"cache": true` in the target default configuration for `test` and Nx will cache the results of running tests. + +```json {% fileName="nx.json" %} +{ + "targetDefaults": { + "test": { + "cache": true + } + } +} +``` + +{% callout type="warning" title="Per Project Caching + DTE" %} + +If you are using distributed task execution and disable caching for a given target, you will not be able to use distributed task execution for that target. This is because distributed task execution requires caching to be enabled. This means that the target you have disabled caching for, and any targets which depend on that target will fail the pipeline if you try to run them with DTE enabled. + +{% /callout %} + ### Generators Default generator options are configured in `nx.json` as well. For instance, the following tells Nx to always diff --git a/docs/shared/reference/project-configuration.md b/docs/shared/reference/project-configuration.md index a25debb238..7953d62f31 100644 --- a/docs/shared/reference/project-configuration.md +++ b/docs/shared/reference/project-configuration.md @@ -365,6 +365,26 @@ More advanced patterns can be used to exclude files and folders in a single line } ``` +#### Cache + +In Nx 17 and higher, caching is configured by specifying `"cache": true` in a target's configuration. This will tell Nx that it's ok to cache the results of a given target. For instance, if you have a target that runs tests, you can specify `"cache": true` in the target default configuration for `test` and Nx will cache the results of running tests. + +```json {% fileName="project.json" %} +{ + "targets": { + "test": { + "cache": true + } + } +} +``` + +{% callout type="warning" title="Per Project Caching + DTE" %} + +If you are using distributed task execution and disable caching for a given target, you will not be able to use distributed task execution for that target. This is because distributed task execution requires caching to be enabled. This means that the target you have disabled caching for, and any targets which depend on that target will fail the pipeline if you try to run them with DTE enabled. + +{% /callout %} + ### dependsOn Targets can depend on other targets. This is the relevant portion of the configuration file: