46 Commits

Author SHA1 Message Date
Jonathan Cammisuli
889dee0854
chore(core): update rust to 2024 edition (#31156)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->
Currently using 2021 edition of rust

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->
Updates to 2024 edition to support more features that are coming in
other PRs

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-05-13 18:11:05 -04:00
Jason Jean
0a2553aa2c
chore(core): add debug mode for tui (#31115)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

Debugging the TUI is a chore

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

Debugging the TUI is a little better..

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-05-09 11:57:32 -04:00
Jason Jean
0d53604b5a
fix(core): clearer tui colors on light themes (#31095)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

Light themes are not super clear with the new TUI.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

Light themes are much clearer with the new TUI.

Internal loom shared on slack for full details.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #

---------

Co-authored-by: JamesHenry <james@henry.sc>
2025-05-07 16:29:36 -04:00
James Henry
e2b27b849b
fix(core): switch to tui-term fork to support dimmed content (#31044) 2025-05-05 17:30:34 +00:00
Craigory Coppola
e29909e71f
fix(core): send signals when killing child process on unix (#30987)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
rspack and next do not get killed when using the tui. This appears to be
caused by them not responding correctly to SIGTERM, so we need a way to
pass signals with our `kill` call.

## Expected Behavior
We properly shutdown rspack and next servers

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-05-01 20:48:52 -04:00
James Henry
0f4c085297
chore(core): add tui layout manager (#30947) 2025-05-01 01:43:11 +04:00
Jason Jean
46888b294c fix(core): use dev tty for crossterm (#30696)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

`git push -u origin HEAD` causes the following panic when the TUI is
enabled on macos:

```
Backtrace (most recent call last):
  File "<unknown>:0", in __pthread_cond_wait
  File "<unknown>:0", in std::sys::pal::unix:🧵:Thread:🆕:thread_start
  File "<unknown>:0", in core::ops::function::FnOnce::call_once{{vtable.shim}}
  File "<unknown>:0", in std::sys::backtrace::__rust_begin_short_backtrace
  File "<unknown>:0", in tokio::runtime::task::raw::poll
  File "<unknown>:0", in tokio::runtime::scheduler::multi_thread::worker::Context::run_task
  File "<unknown>:0", in tokio::runtime::task::raw::poll
  File "<unknown>:0", in tokio::runtime::task::core::Core<T,S>::poll
  File "<unknown>:0", in nx::native::tui::tui::Tui::start::{{closure}}
  File "<unknown>:0", in core::option::expect_failed

The application panicked (crashed).
  reader source not set
in /Users/jason/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/crossterm-0.29.0/src/event/read.rs:39
thread: tokio-runtime-worker

```

This is due to the fact that crossterm until recently did not use
/dev/tty on mac and thus could not read events correctly when forwarded
from another program such as husky.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

This adds the `use-dev-tty` features so that `crossterm` utilizes
`/dev/tty` and works when forwarded from other programs such as husky on
macos.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-04-15 09:23:45 -04:00
Jason Jean
9359490326 chore(core): fix wasm build (#30684)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

Wasm build is broken.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

Wasm build is fixed.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-04-15 09:23:45 -04:00
James Henry
6541751aab feat(core): add the experimental Terminal UI for tasks (#30565) 2025-04-15 09:23:45 -04:00
Jason Jean
9fe73eaf31
chore(core): fix publish for linux (#30669)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

Publish is failing for linux workflows missing openssl.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

Publish passes as reqwest will use `rustls-tls`

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-04-10 11:55:29 -04:00
Craigory Coppola
0525426a51
feat(core): add support for http based caches (#30593)
Implements http based remote caches per the RFC here:
https://github.com/nrwl/nx/discussions/30548
2025-04-09 19:25:03 +00:00
Jonathan Cammisuli
17a543f2c4
chore(core): update cargo deps, fix warnings (#30570) 2025-04-08 15:21:41 +04:00
Craigory Coppola
fe49308c78
feat(core): provide default value for max cache size (#30351)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
The max cache size is disabled by default

## Expected Behavior
Max cache size is set to 10% of the current disk by default, and can be
disabled by specifying 0. Information about this shows up in `nx
report`.

<img width="331" alt="image"
src="https://github.com/user-attachments/assets/ee937101-9915-49d1-b3f1-c2f0d929c140"
/>


## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2025-03-13 19:07:26 +00:00
Craigory Coppola
e4f5224e9e
feat(core): add maxCacheSize option to limit local artifact size (#29654)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
Cache artifacts are removed based on age at a random interval. There is
not a way to set a max size for the cache, so it can grow quite large in
certain repos

## Expected Behavior
Cache size can be controlled via `maxCacheSize` in `nx.json`. Cache
artifacts are removed based on usage until the limit has been reached.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #

---------

Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
2025-03-11 18:41:29 -04:00
Craigory Coppola
5721ea3c21
feat(core): lock graph creation when running in another process (#29408)
## Current Behavior
Running Nx in multiple processes at the same time with the daemon
disabled can cripple a system due to excess memory usage when creating
the graph. This is due to plugin workers being started per-parent
process when there is no daemon. This change enables a file lock to
prevent the simultaneous processing, and read from the cache when the
first run completes.

Currently, running `nx show projects` 30 times in parallel looks
something like this:

30 processes exited within 37535ms

## Expected Behavior
30 processes exited within 6435ms

## Test Script
```js
//@ts-check

const { spawn } = require('child_process');

let alive = new Set();

let start = Date.now();
let iterations = 30;

for (let i = 0; i < iterations; i++) {
  const cp = spawn('npx nx show projects', [], {
    shell: true,
    env: {
      ...process.env,
      NX_DAEMON: 'false',
      NX_VERBOSE_LOGGING: 'true',
    },
  });
  alive.add(i);
  //   cp.stdout.on('data', (data) => {
  //     console.log(`stdout [${i}]: ${data}`);
  //   });
  cp.stderr.on('data', (data) => {
    console.error(`stderr [${i}]: ${data}`);
  });
  cp.on('exit', (code) => {
    console.log(`child process ${i} exited with code ${code}`);
    alive.delete(i);
  });
}

const i = setInterval(() => {
  if (alive.size > 0) {
  } else {
    clearInterval(i);
    console.log(
      `${iterations} processes exited within ${Date.now() - start}ms`
    );
  }
}, 1);

```
2025-01-28 09:46:52 -05:00
Craigory Coppola
facb6dbb95
chore(core): bump fs4 (#29611)
The version we relied on was yanked from cargo for some reason 🤷
2025-01-14 08:36:29 -05:00
Jason Jean
7f39dc1852
fix(core): do not depend on ci info crate (#28850)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

The way VS Code / Nx Console starts the daemon makes the ci info rust
crate believe that the daemon process is running in CI. This starts the
database using the unix dotfile... which causes issues

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

The ci info rust crate is not used, we have our own logic which is
identical to the JS logic we have.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2024-11-08 15:28:13 -05:00
Jonathan Cammisuli
63b745e8c3
fix(core): retry more db operations (#28667) 2024-10-30 11:45:34 -04:00
Jonathan Cammisuli
d3df76f2f4
fix(core): handle concurrent db connections better (#28544) 2024-10-23 23:33:33 +04:00
Emily Xiong
9c4092de8d
feat(core): use durations from task history to schedule tasks (#27783)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #

Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
2024-09-12 15:53:37 -04:00
Jason Jean
cade5bc671
feat(core): add sqlite db for nx (#26891)
## Current Behavior
<!-- This is the behavior we have today -->

Nx has some persistent storage managed as separate files on disk. For
example, the local cache queries the file system for existing
directories.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

Nx has a new more performant persistent storage via SQLite database. The
db is used for the following purposes now:
1. Storing task details of different hashes (This serves as reference
for other tables to get more information about a hash)
2. Storing a record of cached artifacts
3. Storing a history of tasks which have run

The cache in particular has the following benefits:
* It's faster, YMMV but it's definitely faster because it writes and
reads less from disk.
* It's able to track access of different cached artifacts
* It purges cached artifacts more intelligently by looking at when
artifacts were last ACCESSED rather than when they were CREATED. This
will also eliminate cache misses due to the cached artifacts being
purged simply because they were CREATED.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2024-08-21 13:12:20 -04:00
Jonathan Cammisuli
c81493dffc
fix(core): update napi-build dependency to 2.1.3 (#27341)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->
Current `napi-build` dependency causes some functions not to export
properly for wasm.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->
`napi-build` exports all functions properly. 

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2024-08-08 16:19:49 +00:00
Jason Jean
37cc8ab747
fix(core): bump the time crate so compilation succeeds (#27117)
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

<!-- If this is a particularly complex change or feature addition, you
can request a dedicated Nx release for this pull request branch. Mention
someone from the Nx team or the `@nrwl/nx-pipelines-reviewers` and they
will confirm if the PR warrants its own release for testing purposes,
and generate it for you if appropriate. -->

## Current Behavior
<!-- This is the behavior we have today -->

`time` crate is locked to a version which fails to compile.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

`time` crate is updated to a version which does not fail to compile.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes #
2024-07-25 10:45:56 -04:00
LongYinan
981eb30a0f
feat(core): support compile to wasi target (#22870)
This pull request is trying to add wasm32-wasi target support for the
nx/native

To test the build, you can run the following commands:

- `rustup target add wasm32-wasip1-threads`
- `pnpm exec napi build --release --platform --package-json-path
packages/nx/package.json --manifest-path packages/nx/Cargo.toml --js
./native-bindings.js -o packages/nx/src/native --target
wasm32-wasip1-threads`

And the wasm file will be built at
packages/nx/src/native/nx.wasm32-wasi.wasm

Blocked by:

- Support @napi-rs/cli 3.0  Cammisuli/monodon#48
- https://github.com/napi-rs/napi-rs/issues/2009

The pseudo_terminal mod is excluded on the wasm32 targets, which is as
expected.

The watch mod is excluded because of the upstream `watchexec` deps
introduced by ignore-files don't support the wasi target at this moment
(but we can improve it).

## Related Issues
Fixes https://github.com/nrwl/nx/issues/21860
Fixes https://github.com/nrwl/nx/issues/23821

---------

Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
2024-07-05 15:55:35 -04:00
Craigory Coppola
32144bb702
fix(core): update pty version to add windows specific flags (#22711) 2024-04-08 17:25:32 -04:00
Jonathan Cammisuli
1dd363706b
fix(core): handle blocking stdin (#21672) 2024-02-07 17:44:42 -05:00
Jonathan Cammisuli
e0d8eab0f5
fix(core): temporary use forked portable_pty to inherit cursor position for windows (#21683) 2024-02-07 16:31:30 -05:00
Jonathan Cammisuli
7623471bd9
fix(core): handle spaces when launching nxFork (#21571) 2024-02-02 21:22:47 -05:00
Craigory Coppola
cb5eeb7475
feat(core): forward stdin to commands started via rust (#21195)
Co-authored-by: Jonathan Cammisuli <jon@cammisuli.ca>
2024-01-22 12:25:52 -05:00
Jason Jean
d4f3e63a4c
feat(core): create a new function to run child processes via rust (#21070)
Co-authored-by: Jonathan Cammisuli <jon@cammisuli.ca>
Co-authored-by: Emily Xiong <xiongemi@gmail.com>
2024-01-17 02:56:59 -05:00
Jonathan Cammisuli
a2f7ae7f22
fix(core): prioritize nxignore for watcher updates (#20975) 2024-01-04 12:41:20 +00:00
Jonathan Cammisuli
15c2181664
feat(core): introduce workspace file archive (#20471) 2023-11-30 15:54:28 +00:00
Jonathan Cammisuli
cc8dbef25e
fix(core): improve file gathering performance (#20377) 2023-11-27 09:38:27 -05:00
Jonathan Cammisuli
5d82a2aab2
feat(core): rust task hasher (#19617) 2023-11-21 08:55:41 -05:00
Jonathan Cammisuli
dc7c3dbfe4
feat(core): extglob to standard glob parser (#20089) 2023-11-10 12:09:40 -05:00
Jonathan Cammisuli
a7e0abd3e4
fix(core): canonicalize watch event paths on all linux distros (#19848)
Co-authored-by: Jason Jean <jasonjean1993@gmail.com>
2023-10-25 18:51:31 -04:00
Jonathan Cammisuli
537d7eb8be
feat(core): add WorkspaceContext class (#18999) 2023-09-13 09:56:36 -04:00
Jonathan Cammisuli
cf0b2fd4f4
fix(core): handle symlinks on debian for watch events (#18636) 2023-08-16 14:02:02 -04:00
Jonathan Cammisuli
b6db266c9e
fix(core): handle complex glob patterns within rust (#18242) 2023-07-25 11:11:22 -04:00
Jason Jean
e2f1c92e03
feat(js): introduce rust-based typescript dependency processor (#17976)
Co-authored-by: Jack Hsu <jack.hsu@gmail.com>
2023-07-18 18:26:43 -04:00
Jason Jean
c04053b4e9
fix(core): parse project configs only in js (#18009) 2023-07-07 16:06:44 -04:00
Jonathan Cammisuli
d3272108c9
feat(core): use rust utilities for caching operations (#17638) 2023-06-27 11:08:22 -04:00
Jonathan Cammisuli
18f95a99ad
feat(core): build project file map with rust (#17472)
Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
2023-06-14 13:42:34 -04:00
Jason Jean
83895e4573
fix(core): properly hash tasks that depend on outputs of other tasks (#17512) 2023-06-09 16:46:11 -04:00
Jonathan Cammisuli
a978ad3094
feat(core): rust based watcher (#16915) 2023-06-01 14:00:27 -04:00
Jonathan Cammisuli
fecf7fb82f
feat(core): add native node bindings for hashing (#14476)
Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
2023-02-01 14:43:40 -05:00