miel.truyen/nx
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

fix(js): show lifecycle script contents in publish executor, scrub version in dry-run (#23850)

Browse Source 
<!-- Please make sure you have read the submission guidelines before
posting an PR -->
<!--
https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md#-submitting-a-pr
-->

<!-- Please make sure that your commit message follows our format -->
<!-- Example: `fix(nx): must begin with lowercase` -->

## Current Behavior
<!-- This is the behavior we have today -->

Lifecycle scripts related to publishing change the output of `npm
publish`, which mixes JSON with non-JSON content despite the `--json`
flag being set. Currently, we will error when attempting to parse the
whole thing as JSON.

The lifecycle scripts contents themselves are not shown to the user.

Additionally, during dry-run we have no choice but to print the version
that currently exists on disk, which can be confusing.

## Expected Behavior
<!-- This is the behavior we should expect with the changes in this PR
-->

We extract and parse the JSON from the `npm publish` output, even when
it is mixed with other output. We also show the lifecycle script outputs
to the user, where applicable.

During dry-run, we replace the version in the publish output with a
placeholder to avoid confusion around what would be published.

## Related Issue(s)
<!-- Please link the issue being fixed so it gets closed when this is
merged. -->

Fixes https://github.com/nrwl/nx/issues/22925
This commit is contained in:
James Henry 2024-05-27 18:20:01 +04:00 committed by GitHub
parent 4cbb0f0988
commit aa2519ff62
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 482 additions and 31 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
e2e/release/src/independent-projects.test.ts
View File

@ -637,7 +637,7 @@ describe('nx release - independent projects', () => {
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@999.9.9-version-git-operations-test.3
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXXB CHANGELOG.md
@ -646,8 +646,8 @@ describe('nx release - independent projects', () => {
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 999.9.9-version-git-operations-test.3
filename: proj-{project-name}-999.9.9-version-git-operations-test.3.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -686,7 +686,7 @@ describe('nx release - independent projects', () => {
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@999.9.9-version-git-operations-test.3
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXXB CHANGELOG.md
@ -695,8 +695,8 @@ describe('nx release - independent projects', () => {
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 999.9.9-version-git-operations-test.3
filename: proj-{project-name}-999.9.9-version-git-operations-test.3.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -723,7 +723,7 @@ describe('nx release - independent projects', () => {
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@999.9.9-version-git-operations-test.3
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXXB CHANGELOG.md
@ -732,8 +732,8 @@ describe('nx release - independent projects', () => {
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 999.9.9-version-git-operations-test.3
filename: proj-{project-name}-999.9.9-version-git-operations-test.3.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -768,7 +768,7 @@ describe('nx release - independent projects', () => {
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@999.9.9-version-git-operations-test.3
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXXB CHANGELOG.md
@ -777,8 +777,8 @@ describe('nx release - independent projects', () => {
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 999.9.9-version-git-operations-test.3
filename: proj-{project-name}-999.9.9-version-git-operations-test.3.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -790,7 +790,7 @@ describe('nx release - independent projects', () => {
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@999.9.9-version-git-operations-test.3
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXXB CHANGELOG.md
@ -799,8 +799,8 @@ describe('nx release - independent projects', () => {
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 999.9.9-version-git-operations-test.3
filename: proj-{project-name}-999.9.9-version-git-operations-test.3.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -832,7 +832,7 @@ describe('nx release - independent projects', () => {
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@999.9.9-version-git-operations-test.3
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXXB CHANGELOG.md
@ -841,8 +841,8 @@ describe('nx release - independent projects', () => {
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 999.9.9-version-git-operations-test.3
filename: proj-{project-name}-999.9.9-version-git-operations-test.3.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}

18
e2e/release/src/release.test.ts
View File

@ -407,7 +407,7 @@ ${JSON.stringify(
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@1000.0.0-next.0
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXB index.js
@ -415,8 +415,8 @@ ${JSON.stringify(
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 1000.0.0-next.0
filename: proj-{project-name}-1000.0.0-next.0.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -428,7 +428,7 @@ ${JSON.stringify(
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@1000.0.0-next.0
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXB index.js
@ -436,8 +436,8 @@ ${JSON.stringify(
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 1000.0.0-next.0
filename: proj-{project-name}-1000.0.0-next.0.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}
@ -449,7 +449,7 @@ ${JSON.stringify(
> nx run {project-name}:nx-release-publish
📦 @proj/{project-name}@1000.0.0-next.0
📦 @proj/{project-name}@X.X.X-dry-run
=== Tarball Contents ===
XXB index.js
@ -457,8 +457,8 @@ ${JSON.stringify(
XXB project.json
=== Tarball Details ===
name: @proj/{project-name}
version: 1000.0.0-next.0
filename: proj-{project-name}-1000.0.0-next.0.tgz
version: X.X.X-dry-run
filename: proj-{project-name}-X.X.X-dry-run.tgz
package size: XXXB
unpacked size: XXXB
shasum: {SHASUM}

348
packages/js/src/executors/release-publish/extract-npm-publish-json-data.spec.ts Normal file
View File

@ -0,0 +1,348 @@
import { extractNpmPublishJsonData } from './extract-npm-publish-json-data';
describe('extractNpmPublishJsonData()', () => {
describe('only unrelated JSON data', () => {
// Does not match expected npm publish JSON data
const data = {
foo: true,
bar: [1, 2, 3],
};
it('should safely ignore unrelated formatted JSON data', () => {
const formattedJsonStr = JSON.stringify(data, null, 2);
const res = extractNpmPublishJsonData(formattedJsonStr);
expect(res.beforeJsonData).toMatchInlineSnapshot(`
"{
"foo": true,
"bar": [
1,
2,
3
]
}"
`);
expect(res.jsonData).toEqual(null);
expect(res.afterJsonData).toMatchInlineSnapshot(`""`);
});
it('should safely ignore unrelated unformatted JSON data', () => {
const unformattedJsonStr = JSON.stringify(data);
const res = extractNpmPublishJsonData(unformattedJsonStr);
expect(res.beforeJsonData).toMatchInlineSnapshot(
`"{"foo":true,"bar":[1,2,3]}"`
);
expect(res.jsonData).toEqual(null);
expect(res.afterJsonData).toMatchInlineSnapshot(`""`);
});
});
describe('mixed unrelated JSON and non-JSON data', () => {
// Does not match expected npm publish JSON data
const data = {
foo: true,
bar: [1, 2, 3],
};
const extraContentBefore = 'Some random text';
const extraContentAfter = 'More random text';
it('should safely ignore unrelated mixed data containing formatted JSON', () => {
const formattedJsonStr = JSON.stringify(data, null, 2);
const res = extractNpmPublishJsonData(`${extraContentBefore}
${formattedJsonStr}
${extraContentAfter}`);
expect(res.beforeJsonData).toMatchInlineSnapshot(`
"Some random text
{
"foo": true,
"bar": [
1,
2,
3
]
}
More random text"
`);
expect(res.jsonData).toEqual(null);
expect(res.afterJsonData).toMatchInlineSnapshot(`""`);
});
it('should safely ignore unrelated mixed data containing unformatted JSON', () => {
const unformattedJsonStr = JSON.stringify(data);
const res = extractNpmPublishJsonData(`${extraContentBefore}
${unformattedJsonStr}
${extraContentAfter}`);
expect(res.beforeJsonData).toMatchInlineSnapshot(`
"Some random text
{"foo":true,"bar":[1,2,3]}
More random text"
`);
expect(res.jsonData).toEqual(null);
expect(res.afterJsonData).toMatchInlineSnapshot(`""`);
});
});
describe('output containing npm publish JSON data', () => {
it('should extract the relevant JSON data from a simple publish output string containing only the data', () => {
const commandOutput = `{
"id": "package-a@1.0.0",
"name": "package-a",
"version": "1.0.0",
"size": 251,
"unpackedSize": 233,
"shasum": "cf4a6657f230ddf5375102bafc8f5184002a620a",
"integrity": "sha512-Qra/YIkAxVavs3tumB/svugHLY5CISujdeUcMd2FfvtVkjEEsVAEYbqZTq0ixnkvjVrLr27mAvH94GjjMKWzIg==",
"filename": "package-a-1.0.0.tgz",
"files": [
{
"path": "package.json",
"size": 233,
"mode": 420
}
],
"entryCount": 1,
"bundled": []
}`;
const res = extractNpmPublishJsonData(commandOutput);
expect(res.beforeJsonData).toMatchInlineSnapshot(`""`);
expect(res.jsonData).toMatchInlineSnapshot(`
{
"bundled": [],
"entryCount": 1,
"filename": "package-a-1.0.0.tgz",
"files": [
{
"mode": 420,
"path": "package.json",
"size": 233,
},
],
"id": "package-a@1.0.0",
"integrity": "sha512-Qra/YIkAxVavs3tumB/svugHLY5CISujdeUcMd2FfvtVkjEEsVAEYbqZTq0ixnkvjVrLr27mAvH94GjjMKWzIg==",
"name": "package-a",
"shasum": "cf4a6657f230ddf5375102bafc8f5184002a620a",
"size": 251,
"unpackedSize": 233,
"version": "1.0.0",
}
`);
expect(res.afterJsonData).toMatchInlineSnapshot(`""`);
});
it('should extract the relevant JSON data from a publish output string containing lifecycle script outputs', () => {
const exampleCommandOutputWithLifecycleScripts = `
> package-a@1.0.0 prepublishOnly
> echo 'prepublishOnly from package-a'
prepublishOnly from package-a
{
"id": "package-a@1.0.0",
"name": "package-a",
"version": "1.0.0",
"size": 206,
"unpackedSize": 179,
"shasum": "f01c6f5c8d72ed33e70c1c1b1258f46c92360e57",
"integrity": "sha512-24/pgfxiTiNB/dw7ZbBZ+I1vidq09KU6n/QgXCtx1y4+ezYpEBSncdrEpDxuMD6YaP8twg3H8zQBLoG8xwygcA==",
"filename": "package-a-1.0.0.tgz",
"files": [
{
"path": "package.json",
"size": 179,
"mode": 420
}
],
"entryCount": 1,
"bundled": []
}
`;
const res = extractNpmPublishJsonData(
exampleCommandOutputWithLifecycleScripts
);
expect(res.beforeJsonData).toMatchInlineSnapshot(`
"
> package-a@1.0.0 prepublishOnly
> echo 'prepublishOnly from package-a'
prepublishOnly from package-a
"
`);
expect(res.jsonData).toMatchInlineSnapshot(`
{
"bundled": [],
"entryCount": 1,
"filename": "package-a-1.0.0.tgz",
"files": [
{
"mode": 420,
"path": "package.json",
"size": 179,
},
],
"id": "package-a@1.0.0",
"integrity": "sha512-24/pgfxiTiNB/dw7ZbBZ+I1vidq09KU6n/QgXCtx1y4+ezYpEBSncdrEpDxuMD6YaP8twg3H8zQBLoG8xwygcA==",
"name": "package-a",
"shasum": "f01c6f5c8d72ed33e70c1c1b1258f46c92360e57",
"size": 206,
"unpackedSize": 179,
"version": "1.0.0",
}
`);
expect(res.afterJsonData).toMatchInlineSnapshot(`
"
"
`);
});
it('should work when a user lifecycle script adds custom, unformatted JSON data to the output', () => {
const exampleCommandOutputWithLifecycleScripts = `
> package-a@1.0.0 prepublishOnly
> node -e 'console.log(JSON.stringify({"name": "package-a", "version": "1.0.0"}));'
{"name":"package-a","version":"1.0.0"}
{
"id": "package-a@1.0.0",
"name": "package-a",
"version": "1.0.0",
"size": 249,
"unpackedSize": 232,
"shasum": "63caa58603b8f9b76a5151ad4e965c3ac0b83c71",
"integrity": "sha512-mXgusXuPfyvqNpnHY3F0TwLiitKzt98hcAxgEq6/uueEM53haisRQx+tf5FEE6uNRhE+9U0A2y9//KD2OPnSBQ==",
"filename": "package-a-1.0.0.tgz",
"files": [
{
"path": "package.json",
"size": 232,
"mode": 420
}
],
"entryCount": 1,
"bundled": []
}`;
const res = extractNpmPublishJsonData(
exampleCommandOutputWithLifecycleScripts
);
expect(res.beforeJsonData).toMatchInlineSnapshot(`
"
> package-a@1.0.0 prepublishOnly
> node -e 'console.log(JSON.stringify({"name": "package-a", "version": "1.0.0"}));'
{"name":"package-a","version":"1.0.0"}
"
`);
expect(res.jsonData).toMatchInlineSnapshot(`
{
"bundled": [],
"entryCount": 1,
"filename": "package-a-1.0.0.tgz",
"files": [
{
"mode": 420,
"path": "package.json",
"size": 232,
},
],
"id": "package-a@1.0.0",
"integrity": "sha512-mXgusXuPfyvqNpnHY3F0TwLiitKzt98hcAxgEq6/uueEM53haisRQx+tf5FEE6uNRhE+9U0A2y9//KD2OPnSBQ==",
"name": "package-a",
"shasum": "63caa58603b8f9b76a5151ad4e965c3ac0b83c71",
"size": 249,
"unpackedSize": 232,
"version": "1.0.0",
}
`);
expect(res.afterJsonData).toMatchInlineSnapshot(`""`);
});
it('should extract the relevant JSON data when formatted JSON data is present alongside the expected npm publish JSON data', () => {
const exampleCommandOutputWithFormattedJSON = `
{
"unrelated": true,
"data": [
1,
2,
3
]
}
{
"id": "package-a@1.0.0",
"name": "package-a",
"version": "1.0.0",
"size": 249,
"unpackedSize": 232,
"shasum": "63caa58603b8f9b76a5151ad4e965c3ac0b83c71",
"integrity": "sha512-mXgusXuPfyvqNpnHY3F0TwLiitKzt98hcAxgEq6/uueEM53haisRQx+tf5FEE6uNRhE+9U0A2y9//KD2OPnSBQ==",
"filename": "package-a-1.0.0.tgz",
"files": [
{
"path": "package.json",
"size": 232,
"mode": 420
}
],
"entryCount": 1,
"bundled": []
}
{
"extra": "data",
"foo": "bar"
}`;
const res = extractNpmPublishJsonData(
exampleCommandOutputWithFormattedJSON
);
expect(res.beforeJsonData).toMatchInlineSnapshot(`
"
{
"unrelated": true,
"data": [
1,
2,
3
]
}
"
`);
expect(res.jsonData).toMatchInlineSnapshot(`
{
"bundled": [],
"entryCount": 1,
"filename": "package-a-1.0.0.tgz",
"files": [
{
"mode": 420,
"path": "package.json",
"size": 232,
},
],
"id": "package-a@1.0.0",
"integrity": "sha512-mXgusXuPfyvqNpnHY3F0TwLiitKzt98hcAxgEq6/uueEM53haisRQx+tf5FEE6uNRhE+9U0A2y9//KD2OPnSBQ==",
"name": "package-a",
"shasum": "63caa58603b8f9b76a5151ad4e965c3ac0b83c71",
"size": 249,
"unpackedSize": 232,
"version": "1.0.0",
}
`);
expect(res.afterJsonData).toMatchInlineSnapshot(`
"
{
"extra": "data",
"foo": "bar"
}"
`);
});
});
});

61
packages/js/src/executors/release-publish/extract-npm-publish-json-data.ts Normal file
View File

@ -0,0 +1,61 @@
const expectedNpmPublishJsonKeys = [
'id',
'name',
'version',
'size',
'filename',
];
// Regular expression to match JSON-like objects, including nested objects (which the expected npm publish output will have, e.g. in its "files" array)
// /{(?:[^{}]|{[^{}]*})*}/g
// /{ : Matches the opening brace of a JSON object
// (?: ) : Non-capturing group to apply quantifiers
// [^{}] : Matches any character except for braces
// | : OR
// {[^{}]*} : Matches nested JSON objects
// * : The non-capturing group (i.e. any character except for braces OR nested JSON objects) can repeat zero or more times
// } : Matches the closing brace of a JSON object
// /g : Global flag to match all occurrences in the string
const jsonRegex = /{(?:[^{}]|{[^{}]*})*}/g;
export function extractNpmPublishJsonData(str: string): {
beforeJsonData: string;
jsonData: Record<string, unknown> | null;
afterJsonData: string;
} {
const jsonMatches = str.match(jsonRegex);
if (jsonMatches) {
for (const match of jsonMatches) {
// Cheap upfront check to see if the stringified JSON data has the expected keys as substrings
if (!expectedNpmPublishJsonKeys.every((key) => str.includes(key))) {
continue;
}
// Full JSON parsing to identify the JSON object
try {
const parsedJson = JSON.parse(match);
if (
!expectedNpmPublishJsonKeys.every(
(key) => parsedJson[key] !== undefined
)
) {
continue;
}
const jsonStartIndex = str.indexOf(match);
return {
beforeJsonData: str.slice(0, jsonStartIndex),
jsonData: parsedJson,
afterJsonData: str.slice(jsonStartIndex + match.length),
};
} catch {
// Ignore parsing errors for unrelated JSON blocks
}
}
}
// No applicable jsonData detected, the whole contents is the beforeJsonData
return {
beforeJsonData: str,
jsonData: null,
afterJsonData: '',
};
}

50
packages/js/src/executors/release-publish/release-publish.impl.ts
View File

@ -6,6 +6,7 @@ import { parseRegistryOptions } from '../../utils/npm-config';
import { logTar } from './log-tar';
import { PublishExecutorSchema } from './schema';
import chalk = require('chalk');
import { extractNpmPublishJsonData } from './extract-npm-publish-json-data';
const LARGE_BUFFER = 1024 * 1000000;
@ -200,6 +201,11 @@ export default async function runExecutor(
console.log('Skipped npm view because --first-release was set');
}
/**
* NOTE: If this is ever changed away from running the command at the workspace root and pointing at the package root (e.g. back
* to running from the package root directly), then special attention should be paid to the fact that npm publish will nest its
* JSON output under the name of the package in that case (and it would need to be handled below).
*/
const npmPublishCommandSegments = [
`npm publish "${packageRoot}" --json --"${registryConfigKey}=${registry}" --tag=${tag}`,
];
@ -220,11 +226,47 @@ export default async function runExecutor(
stdio: ['ignore', 'pipe', 'pipe'],
});
const stdoutData = JSON.parse(output.toString());
/**
* We cannot JSON.parse the output directly because if the user is using lifecycle scripts, npm will mix its publish output with the JSON output all on stdout.
* Additionally, we want to capture and show the lifecycle script outputs as beforeJsonData and afterJsonData and print them accordingly below.
*/
const { beforeJsonData, jsonData, afterJsonData } =
extractNpmPublishJsonData(output.toString());
if (!jsonData) {
console.error(
'The npm publish output data could not be extracted. Please report this issue on https://github.com/nrwl/nx'
);
return {
success: false,
};
}
// If npm workspaces are in use, the publish output will nest the data under the package name, so we normalize it first
const normalizedStdoutData = stdoutData[packageName] ?? stdoutData;
logTar(normalizedStdoutData);
// If in dry-run mode, the version on disk will not represent the version that would be published, so we scrub it from the output to avoid confusion.
const dryRunVersionPlaceholder = 'X.X.X-dry-run';
if (isDryRun) {
for (const [key, val] of Object.entries(jsonData)) {
if (typeof val !== 'string') {
continue;
}
jsonData[key] = val.replace(
new RegExp(packageJson.version, 'g'),
dryRunVersionPlaceholder
);
}
}
if (
typeof beforeJsonData === 'string' &&
beforeJsonData.trim().length > 0
) {
console.log(beforeJsonData);
}
logTar(jsonData);
if (typeof afterJsonData === 'string' && afterJsonData.trim().length > 0) {
console.log(afterJsonData);
}
if (isDryRun) {
console.log(