Skip to content

Slack large import fail #15659

@tuxity

Description

@tuxity

Description:

When importing a large ZIP file (1.3G) downloaded from Slack, from either an http url or local path, server trigger an exception.

Steps to reproduce:

Try to import a large Slack backup with a remote http url or from a local path

Expected behavior:

Start parsing json files

Actual behavior:

RocketChat console print an error

Server Setup Information:

  • Version of Rocket.Chat Server: 2.1.1
  • Operating System: Debian 9
  • Deployment Method: docker(-compose)
  • Number of Running Instances: 1
  • DB Replicaset Oplog:
  • NodeJS Version: v8.15.1
  • MongoDB Version: 4.0.13

Client Setup Information

  • Desktop App or Browser Version: Safari
  • Operating System: MacOS

Relevant logs:

API ➔ debug POST: /api/v1/downloadPublicImportFile
Meteor ➔ method downloadPublicImportFile -> userId: J5fEjToAAvEcD9iG9 , arguments:  [ 'http://xxx.xxx.xxx.xxx/slack.zip', 'slack' ]
Slack Importer ➔ debug Slack is now at importer_uploading.
Slack Importer ➔ debug Slack is now at downloading_file_url.
API ➔ debug Success { statusCode: 200, body: { success: true } }
API ➔ debug Success { statusCode: 200, body: { success: true } }
API ➔ debug GET: /api/v1/getImportFileData?importerKey=slack
Meteor ➔ method getImportFileData -> userId: J5fEjToAAvEcD9iG9 , arguments:  [ 'slack' ]
API ➔ debug Success { statusCode: 200, body: { waiting: true, success: true } }
API ➔ debug GET: /api/v1/getImportFileData?importerKey=slack
[...]
API ➔ debug GET: /api/v1/getImportFileData?importerKey=slack
Meteor ➔ method getImportFileData -> userId: J5fEjToAAvEcD9iG9 , arguments:  [ 'slack' ]
API ➔ debug Success { statusCode: 200, body: { waiting: true, success: true } }
Slack Importer ➔ debug Slack is now at download_complete.
API ➔ debug GET: /api/v1/getImportFileData?importerKey=slack
Meteor ➔ method getImportFileData -> userId: J5fEjToAAvEcD9iG9 , arguments:  [ 'slack' ]
API ➔ debug get getImportFileData threw an error: Error: "toString()" failed
    at stringSlice (buffer.js:578:45)
    at Buffer.toString (buffer.js:633:10)
    at SlackImporter.prepareUsingLocalFile (app/importer/server/classes/ImporterBase.js:151:23)
    at MethodInvocation.getImportFileData (app/importer/server/methods/getImportFileData.js:64:37)
    at MethodInvocation.methodsMap.(anonymous function) (app/lib/server/lib/debug.js:62:34)
    at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1771:12)
    at DDP._CurrentMethodInvocation.withValue (packages/ddp-server/livedata_server.js:1689:15)
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12)
    at resolve (packages/ddp-server/livedata_server.js:1687:36)
    at new Promise (<anonymous>)
    at Server.applyAsync (packages/ddp-server/livedata_server.js:1686:12)
    at Server.apply (packages/ddp-server/livedata_server.js:1625:26)
    at Server.call (packages/ddp-server/livedata_server.js:1607:17)
    at DDPCommon.MethodInvocation.Meteor.runAsUser (app/api/server/v1/import.js:36:20)
    at packages/dispatch_run-as-user.js:211:14
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12)
    at Object.Meteor.runAsUser (packages/dispatch_run-as-user.js:210:33)
    at Object.get (app/api/server/v1/import.js:35:10)
    at Object._internalRouteActionHandler [as action] (app/api/server/api.js:324:31)
    at Route.share.Route.Route._callEndpoint (packages/nimble_restivus/lib/route.coffee:150:32)
    at packages/nimble_restivus/lib/route.coffee:59:33
    at packages/simple_json-routes.js:98:9
 => awaited here:
    at Promise.await (/app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:60:12)
    at Server.apply (packages/ddp-server/livedata_server.js:1638:14)
    at Server.call (packages/ddp-server/livedata_server.js:1607:17)
    at DDPCommon.MethodInvocation.Meteor.runAsUser (app/api/server/v1/import.js:36:20)
    at packages/dispatch_run-as-user.js:211:14
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12)
    at Object.Meteor.runAsUser (packages/dispatch_run-as-user.js:210:33)
    at Object.get (app/api/server/v1/import.js:35:10)
    at Object._internalRouteActionHandler [as action] (app/api/server/api.js:324:31)
    at Route.share.Route.Route._callEndpoint (packages/nimble_restivus/lib/route.coffee:150:32)
    at packages/nimble_restivus/lib/route.coffee:59:33
    at packages/simple_json-routes.js:98:9
API ➔ debug Failure { statusCode: 400,
  body:
   { success: false,
     error: '"toString()" failed',
     stack: undefined } }

Note:

Using FileSytem storage on /app/uploads, chmod are 777 for testing purpose

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions