Skip to content

feat: publish the minimum files to npm#243

Merged
marionebl merged 8 commits intoconventional-changelog:masterfrom
ybiquitous:add-files-to-all-package-json
Jan 31, 2018
Merged

feat: publish the minimum files to npm#243
marionebl merged 8 commits intoconventional-changelog:masterfrom
ybiquitous:add-files-to-all-package-json

Conversation

@ybiquitous
Copy link
Copy Markdown
Contributor

Hi! 😄
I intend to fix #144 by this PR, but I have no confidence whether the new E2E test (npm.testPackingFiles function) is valid. 😓

So would you please review this PR especially about new test code?
After reviewing and fixing, I will add .npmignore to other packages than @commilint/cli. 💪

Description

  • Add new test function to assert publishing files: npm.testPackingFiles
  • At first, add .npmignore to @commitlint/cli to review whether testing is valid

Motivation and Context

N/A

Usage examples

N/A

How Has This Been Tested?

By adding new E2E test.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@marionebl
Copy link
Copy Markdown
Contributor

Thanks a lot! Will have look at this later today!

Copy link
Copy Markdown
Contributor

@marionebl marionebl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got some ideas on how to improve this. We'd need some additional tests checking basic functionality (as in: "can the contents of the tarball be imported / executed as advertised")

Comment thread @packages/test/src/npm.js Outdated
return cwd;
}

async function testPackingFiles(t, expectedFiles) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not pass in the testing context - the methods in @packages/test should only provide required setup / side effects. Please check the actual assertions in the test itself.

Comment thread @packages/test/src/npm.js Outdated

// Install pack file
packFile = path.resolve(await execa.stdout('npm', ['pack']));
await execa('npm', ['install', '--no-progress', packFile], {cwd});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer reading the file list this from the tarball directly, e.g. via tar-fs.

Comment thread @commitlint/cli/src/publish.test.js Outdated
import test from 'ava';

test('should publish the minimum files', async t => {
await npm.testPackingFiles(t, [
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of specifying the expected bundle files manually I'd like to derive the absolute minimum from package.json entries: main: string = "index.js" and bin: string || { key: string }. This could be done via a helper fn that should live in @commitlint/test

import pkgDir from 'pkg-dir';
import {npm} from '@commitlint/test';

test('should publish required files', async t => {
  const cwd = pkgDir(__dirname);
  const expected = await npm.getPackageFiles({ cwd });
  const actual = await npm.getTarballFiles();
  t.deepEquals(actual, expected);
});

@ybiquitous
Copy link
Copy Markdown
Contributor Author

Thanks for the feedbacks! I try to fix them. 😃 💪

@ybiquitous ybiquitous changed the title feat: publish the minimum files to npm [WIP] feat: publish the minimum files to npm Jan 27, 2018
Copy link
Copy Markdown
Contributor

@marionebl marionebl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job on improving the test setup, thank you. I have some additional feedback - tell me if I am nitpicking.

Comment thread @packages/test/src/npm.js Outdated
fs
.createReadStream(tarballFile)
.pipe(zlib.createGunzip())
.pipe(tar.extract(workDir))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could use the ignore options of tar.extract to get a file list without actually writing to disk:

const files = [];

fs.createReadStream(tarballFile)
  .pipe(zlib.createGunzip())
  .pipe(tar.extract(workDir, {
    ignore(name) {
        files.push(name);
       return true;
    }
   }))
 .once('error', err => reject(err))
 .once('finish', () => resolve());

Might be elaborate enough to factor it out to a listArchive function.

Comment thread @packages/test/src/npm.js Outdated
});

// List all files
return (await globby(['**'], {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be obsolete if we implement listArchive

Comment thread @packages/test/src/npm.js Outdated
sander.unlink(packFile);
}
sander.rimraf(tarballFile);
sander.rimraf(workDir);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be obsolete if we implement listArchive

Comment thread @packages/test/src/npm.js Outdated
const tarballFile = path.resolve(cwd, tarballFilename);
const workDir = await fix.bootstrap();

try {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can unwrap this from

try {
  // 1
} finally {
  // 2
}

to

  // 1
  // 2

now the assertions happens in the test body.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we put clean up code in finally clause?

try {
  // do something
finally {
  sander.rimraf(tarballFile)
  sander.rimraf(workDir)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could use the ignore options of tar.extract to get a file list without actually writing to disk:

Sorry. Please ignore my question. We don't need writing to disk 😓

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for many times...
The following code (npm pack) writes tarball file (e.g. commitlint-cli-6.0.2.tgz) to disk actually.
Should we clean-up this file in finally clause?

const tarballFilename = await execa.stdout('npm', ['pack'], {cwd});
const tarballFile = path.resolve(cwd, tarballFilename);
try {
  // do something
} finally {
  sander.unlink(tarballFile);
}

Comment thread @packages/test/src/npm.js Outdated

async function getPackageFiles({cwd, npmignore}) {
return (await packlist({path: cwd}))
.concat(npmignore ? '.npmignore' : [])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because npm-packlist module ignores .npmignore even if it exists. 😓

Comment thread @commitlint/cli/src/publish.test.js Outdated
const cwd = await pkgDir(__dirname);
const actual = await npm.getTarballFiles({cwd});
const expected = await npm.getPackageFiles({cwd, npmignore: true});
t.deepEqual(actual, expected);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should change this to a less strict check, e.g.

// for each file we expect
t.true(includes(actual, 'some-file.js'))

I am rolling around the idea to introduce unexpected to the testing setup. Should make things like this easier.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That idea makes test code easier, because we don't need implementing code such as npm.packageFiles()! 👍
Is that idea is like the following code, correct?

  t.true(actual.includes('package.json')) //<= unnecessary? (because it is obvious)
  t.true(actual.includes('README.md'))
  t.true(actual.includes('CHANGELOG.md'))
  t.true(actual.includes('index.js'))
  t.true(actual.includes('lib/cli.js'))
  t.true(actual.includes('lib/help.js'))

@marionebl
Copy link
Copy Markdown
Contributor

@ybiquitous Thanks for starting this. I pushed some changes on top, implementing the more intricate parts of my ideas. Do you want to implement the publish tests for all packages?

@ybiquitous
Copy link
Copy Markdown
Contributor Author

I understand well from your pushed code! 😄
I think that you will do this PR better, so I will leave you the rest of the implementation, please. 🙏

Comment thread @packages/test/src/npm.js Outdated
return files;
}

function getArchveFiles(filePath) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitspick] I found typo: getArchveFiles -> getArchiveFiles.

@marionebl marionebl merged commit 9028aa6 into conventional-changelog:master Jan 31, 2018
@ybiquitous ybiquitous changed the title [WIP] feat: publish the minimum files to npm feat: publish the minimum files to npm Feb 1, 2018
@ybiquitous ybiquitous deleted the add-files-to-all-package-json branch February 1, 2018 01:07
@ybiquitous
Copy link
Copy Markdown
Contributor Author

@marionebl Great work! 🎉 🎉 🎉

I appreciate so much that you have done the rest of my work. 😊 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Do not publish src files and tests

2 participants