Skip to content

Filename can be a nested path#355

Merged
sebdeckers merged 1 commit intoform-data:masterfrom
sebdeckers:patch-1
Jun 11, 2017
Merged

Filename can be a nested path#355
sebdeckers merged 1 commit intoform-data:masterfrom
sebdeckers:patch-1

Conversation

@sebdeckers
Copy link
Copy Markdown
Collaborator

@sebdeckers sebdeckers commented Jun 5, 2017

TL;DR This patch allows setting of arbitrary filename to support uploads of directories. This matches current spec and behaviour of Chrome.

Example

for (const filepath of ['foo/bar.txt', 'foo/baz.txt']) {
  form.append('uploads', createReadStream(filepath), {filepath})
}
// Content-Disposition: form-data; name="uploads"; filename="foo/bar.txt"
// Content-Disposition: form-data; name="uploads"; filename="foo/baz.txt"

Edit: Updated to reflect latest patch.

Description

The FormData .append spec defers to the File API spec which states:

There are numerous file name variations and conventions used by different underlying OS file systems; this is merely the name of the file, without path information.

However, the HTMLInputElement.webkitdirectory spec supports relative paths, and Chrome sets them as the filename=... in the form field.

This is also supported by Multer, for example.

Caveat Emptor

Breaking change for anyone manually specifying filename as a full path. Does not change behaviour on auto-detecting of filename from stream, http, etc.

Edit: The filepath option overrides filename. This PR is no longer a breaking change.

See also

@sebdeckers
Copy link
Copy Markdown
Collaborator Author

😱 What's wrong with CI tests?! Worked fine for me. 🤔

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 10, 2017

Coverage Status

Coverage remained the same at 97.949% when pulling 0cd101c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@sebdeckers
Copy link
Copy Markdown
Collaborator Author

sebdeckers commented Jun 10, 2017

All unit tests pass now. The CodeFactor issues seem to be unrelated to this patch, AFAICT.

@alexindigo
Copy link
Copy Markdown
Member

Thanks for the PR @sebdeckers. Do you mind to add a test for the use case?
Also, how strong do you feel about using filename for the full path?
If we'd add something like filepath – it won't be breaking change.

Thanks.

PS. I'll check what's wrong with CodeFactor.

@alexindigo
Copy link
Copy Markdown
Member

I removed CodeFactor (it was weird and inconsistent) – it should be better now.

@sebdeckers
Copy link
Copy Markdown
Collaborator Author

Ooh, that's a great idea! 💡 How did I not think of just adding a new property. 😅

Will refactor this PR and add testing.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.02%) to 97.97% when pulling 04f9746 on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.02%) to 97.97% when pulling 8e49db0 on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.02%) to 97.97% when pulling ecc572a on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.02%) to 97.97% when pulling 26ecb81 on sebdeckers:patch-1 into d7398c3 on form-data:master.

@sebdeckers
Copy link
Copy Markdown
Collaborator Author

All right. Took me a few tries to get Windows support working but this should do the trick. 😅 Appveyor is very useful.

Comment thread lib/form_data.js Outdated

if (filename) {
contentDisposition = 'filename="' + path.basename(filename) + '"';
filename = options.filepath ? normalize(filename) : path.basename(filename);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using path.normalize?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Different purpose:

  • path.normalise cleans up .. and de-dupes path separators, but retains platform-specific path separators.
  • normalize-path converts Windows \ path separator to forward slash.

TBH I didn't check if the spec calls for forward slashes but seems consistent with all web-related paths.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.

Probably we still want to clean up don't, './photos/toys/unicycle.jpg' -> 'photos/toys/unicycle.jpg', as for win -> unix, we could do something like:

filename = filename.replace('\\', '/');

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay!

Comment thread lib/form_data.js Outdated
// fs- and request- streams have path property
// formidable and the browser add a name property.
var filename = options.filename || value.name || value.path;
var filename = options.filepath || options.filename || value.name || value.path;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think doing it in "one-stroke" fashion?

Something like:

var filename = options.filepath ? path.normalize(options.filepath) : path.basename(options.filename || value.name || value.path);

And add basename call to the streams.

The main reason is to handle options here and don't "leak" them further into the function.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree. Will clean this up.

@alexindigo
Copy link
Copy Markdown
Member

Thanks for the update. Looks good in general, just want to discuss implementation details.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.04%) to 97.99% when pulling a8f6303 on sebdeckers:patch-1 into d7398c3 on form-data:master.

@sebdeckers
Copy link
Copy Markdown
Collaborator Author

sebdeckers commented Jun 11, 2017

(Sorry for the coveralls spam ... It seems to leave a comment on every rebase.)

Fixed as per review:

  • Removed normalize-path dependency in favour of .replace('\\', '/')
  • Refactored filename assignment to avoid mutating the variable.

Also fixed:

  • contentType detection based on both options.filepath and options.filename

Edit: Oops, broke again in Windows. Hang on.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 7f0fc80 on sebdeckers:patch-1 into d7398c3 on form-data:master.

@alexindigo
Copy link
Copy Markdown
Member

Don't worry about coveralls :) With greenkeeper gone from this repo, it's all nice and quiet :)
And thank you for walking extra mile :)

Comment thread lib/form_data.js Outdated
filename = value.client._httpMessage.path;
if (typeof options.filepath === 'string') {
// custom filepath for relative paths
filename = path.normalize(options.filepath.replace(/\\/g, '/'));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha :) My bad. Sorry 👍

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling b9f38fb on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

2 similar comments
@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

3 similar comments
@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 125046c on sebdeckers:patch-1 into d7398c3 on form-data:master.

assert.strictEqual(files['custom_filename'].type, mime.lookup(knownFile), 'Expects original content-type');

assert('custom_filepath' in files);
assert.strictEqual(files['custom_filepath'].name, relativeFile.replace(/\\/g, '/'), 'Expects custom filename');
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember struggles like that, with no local access to windows :)

You could try something like:

assert.strictEqual(files['custom_filepath'].name, relativeFile.replace(/\\/g, '/'), 'Expects custom filename:' + files['custom_filepath'].name + ' -vs- ' + relativeFile.replace(/\\/g, '/') + ' -from- ' + relativeFile + ' -end-');

:)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah this is too funny. 😂

Anyway I think it's because I did path.resolve after the backslash replace. Seems on Windows path.resolve expects to still see the native OS separator. Makes sense, I suppose.

Should be fixed now. Also cleaned up some path concatenation in text/common.js, just to be on the safe side.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Native nodejs methods do convert slashes for windows, so it should be fine.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, do you mind to change 'Expects custom filename' to 'Expects custom filepath', it will be more clear which line is failed.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling d38e932 on sebdeckers:patch-1 into d7398c3 on form-data:master.

@sebdeckers sebdeckers force-pushed the patch-1 branch 2 times, most recently from 92096d4 to 517081a Compare June 11, 2017 06:26
@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 517081a on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 517081a on sebdeckers:patch-1 into d7398c3 on form-data:master.

var unknownFile = common.dir.fixture + '/unknown_file_type';
var knownFile = path.join(common.dir.fixture, 'unicycle.jpg');
var unknownFile = path.join(common.dir.fixture, 'unknown_file_type');
var relativeFile = path.relative(path.join(__dirname, '..'), knownFile);
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops this should have been relative to knownFile, not __dirname. One more patch incoming...

Comment thread test/common.js Outdated
tmp: path.join(rootDir, '/test/tmp')
lib: path.join(rootDir, 'lib'),
fixture: path.join(rootDir, 'test', 'fixture'),
tmp: path.join(rootDir, 'test', 'tmp')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

path.join updates all the slashes on windows.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, will roll this back.

var unknownFile = common.dir.fixture + '/unknown_file_type';
var knownFile = path.join(common.dir.fixture, 'unicycle.jpg');
var unknownFile = path.join(common.dir.fixture, 'unknown_file_type');
var relativeFile = path.relative(path.join(__dirname, '..'), knownFile);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to pass rootDir to the common.dir.root, so we stay more DRY here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this to be a relative path to knownFile. So it will have the OS-specific separator. This makes the test more meaningful than a hardcoded string. LMK if you prefer a change.

@alexindigo
Copy link
Copy Markdown
Member

🤞

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 9512d0c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 11, 2017

Coverage Status

Coverage increased (+0.03%) to 97.98% when pulling 9512d0c on sebdeckers:patch-1 into d7398c3 on form-data:master.

@alexindigo
Copy link
Copy Markdown
Member

Looks like it's in good working state. I invited you as a Collaborator, you can merge when it's passes. Thank you.

@sebdeckers sebdeckers merged commit 561dece into form-data:master Jun 11, 2017
@sebdeckers
Copy link
Copy Markdown
Collaborator Author

@alexindigo Thanks! It's an honour. Appreciate your guidance & patience. 🤝

@alexindigo
Copy link
Copy Markdown
Member

Thank you. I'll publish it shortly.

@sebdeckers sebdeckers deleted the patch-1 branch June 11, 2017 07:32
@alexindigo
Copy link
Copy Markdown
Member

Published v2.2.0. Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants