Simulate external servers with Node.js+Javascript instead of (Apache|mongoose|…)+PHP #1999

gibson042 · 2015-01-08T15:24:53Z

This will allow us to set up and tear down the server with grunt and track all dependencies in package.json, but requires reimplementing our PHP scripts and probably (hopefully?) also establishing a mechanism for communicating the listening port to QUnit.

markelog · 2015-01-08T15:42:47Z

There is complication about that - testswarm requires ability to execute tests on any previous commit not just the current one. Otherwise it would done a long time ago.

@Krinkle have more info about that.

markelog · 2015-01-08T15:55:36Z

Or just move away from testswarm, as it was suggested by the infra team, browserstack-tunnel or karma our candidates for the replacement.

But those requires some good amount of time to do it, at least karma does, which is my preference.

ChrisAntaki · 2015-01-08T17:49:02Z

Intern looks like an option as well jquery/testswarm@dd6002f

markelog · 2015-01-08T18:49:54Z

Does intern offers qunit support? I was under impression it has its own framework.

/cc @csnover

csnover · 2015-01-08T19:17:40Z

You could certainly load QUnit into it if you wanted (I have heard of some people dumping Mocha in there), but it’s a kind of inception if you do since it comes with quite robust unit testing built-in :)

I’ve actually got a mostly-working branch with a full* QUnit API compatible interface so, with only a small amount of work to address the TODOs (probably only a day or two of work?), you should be able to drop existing QUnit tests into Intern by wrapping them with define([ 'intern!qunit'], function (QUnit) { }) and have them Just Work.

(And then, fingers crossed!, I can convince team QUnit to come over & help me get cool new stuff into Intern more quickly :)))

* Well it was, prior to the most recent QUnit release. Now it is slightly outdated, but it should be easy to update, since from what I could see the QUnit update was adding some features that Intern already has so it’s easy to map them

Krinkle · 2015-01-08T19:50:05Z

Brain dump:

Assumption: This dynamic server would be auto-started by Grunt, and uses automatic port choosing in order to run concurrently with other services on the same computer (e.g. projects developers run locally, as well as concurrent builds in Jenkins). Alternatively, we'd block this on migrating to Travis CI (no port clash) and/or pick unique ports for each git branch in each git repo.

As long as TestSwarm is used (unless you're okay with ajax tests not running in TestSwarm), we can't have entry points not automatically served by common installations of Apache and nginx (hence PHP was used).

Any server set up by Grunt won't exist by the time TestSwarm distributes the test urls to BrowserStack. These test urls point at the not-so-static "static" web server serving build artefacts from Jenkins jobs for upto 30 days at unique subdirectories (html/js/css and php, served by nginx). It can also be re-ran from TestSwarm at any point in the future (within 30 days).

Even if we'd remove the re-run feature in TestSwarm, QUnit would still have to know about the hostname and port the server is on. And the url would have to be publicly exposed to BrowserStack. Though of course, this wouldn't be an issue if it was using an ssh tunnel to BrowserStack. But I'm not sure that is feasible in the current testswarm-browserstack setup.

gibson042 · 2015-01-09T07:02:23Z

For the purposes of CI generally (as opposed to local development) and TestSwarm in particular, I imagine we'd still use the stable servers for builds.jenkins.jquery.com/jquery/{build}/… URLs, they'd just defer to grunt-managed node.js servers for dynamic responses. There are many ways to skin this cat, but here's one example:

On inbound request to a build URL, the PHP handler verifies that the corresponding port is known
- ...and if not, spawns the build's grunt test-server and then checks output after a little while to discover it (with suitable failure handling, perhaps kill the spawned process and respond 404)
PHP gets its responses from forwarding the request to that port (with suitable failure-to-connect handling, perhaps decrement available tries and replay the original request)
The test-server grunt task leverages grunt-contrib-connect and possibly also node-proper-lockfile, something like:

// Connect to already-running server listening at port specified in the runfile
tryConnecting();
function tryConnecting() {
  try {
    port = fs.readFileSync( runfile );
    req = http.get({ port: port || "invalid" }, function() {
      console.log( "Server running at port " + port );
    });

    // On failure, try to _start_ a server
    req.on( "error", tryStarting );
  } catch ( err ) {
    tryStarting( err );
  }
}

function tryStarting( err ) {
  console.warn( err.message );

  // Lock the runfile for our use
  lockfile.lock(
    runfile,
    {
      // Consider existing locks stale after 7 seconds
      stale: 7000,

      // Try acquisition for up to 7 seconds (2x backoff per retry)
      { retries: 3, minTimeout: 1000 }
    },
    function( err, release ) {
      // Got the lock
      if ( !err ) {
        var keepAlive = grunt.config( "connect.testServer.keepAlive" ),
          onCreate = grunt.config( "connect.testServer.onCreateServer" );

        // Server creation listener
        onCreate.push(function( server, connect, options ) {
          console.log( "Server running at port " + options.port );

          // Write out the runfile and register a shutdown listener to clear it
          fs.writeFileSync( runfile, options.port + "" );
          server.on( "close", function() {
            fs.writeFile( runfile, "" );
            release();
          });

          // Schedule a deferable future shutdown
          if ( keepAlive ) {
            grunt.config( "connect.testServer.shutdown" ) =
              setTimeout( server.close.bind( server ), keepAlive );
          }
        });

        // Start the server daemon (only self-terminating by the above)
        grunt.task.run( "connect:testServer:keepalive" );

      // If we lost a race for the lock, try connecting to the winner
      } else if ( err.code === "ELOCKED" ) {
        console.warn( "Couldn't get lock on " + runfile );
        tryConnecting();

      // Bail on any other error
      } else {
        throw err;
      }
    }
  );
}

csnover · 2015-01-09T07:30:34Z

Since I was summoned already and so am seeing replies on this, I would just also point out that Dojo core has a JSGI server for mock services set up with its Intern test suite, so you don’t need to do much work again, you could just take that code and adapt it for your own use. It’s used to do the same thing the jQuery PHP test scripts are used for (testing Dojo’s I/O code). The services are in the tests/services directory.

Krinkle · 2017-07-31T22:22:40Z

Regarding incompatibility between TestSwarm (which independently and outside the Grunt/Jenkins run context) and Travis CI / local development, we can use the isSwarm variable that already exists in jQuery core unit tests to make any decisions that need to vary. (E.g. to change the url paths slightly, in case we don't want the .php suffix for the Node.js version of these endpoints)

More likely though, from what I can tell there isn't any need for the urls to vary. We could easily implement the same urls both ways.

The Ajax tests are already written with the expectation that they are served over HTTP (e.g. any static server will work, and if served over file:/// the Ajax tests are already skipped). The tests make requests to the PHP files using relative urls.

We'd configure out Node.js server to similarly serve all files statically, with a non-static router (that has higher precedence) for our Ajax server mocks. E.g. route /data/ajax/:script.php to a JavaScript based handler that will vary according to the script name and query parameters accordingly.

Once that is fixed, we can run the unit tests normally with QUnit/Karma/Grunt, e.g. defaulting to Headless Chrome, and we can run them Travis CI in at least Headless Chrome and Firefox (pre-installed). In addition, once they pass in e.g. Headless Chrome, we can go run them in various other browsers using Saucelabs or BrowserStack as various other jQuery projects do already.

All this can happen whilst the tests will continue to work in TestSwarm as well. (If desired)

* Add 'grunt karma:main' task that runs tests via Karma. - Karma uses its own server and uses its own HTML page by default. All files to be served are registered, and for the most part also loaded ("included") through Karma. - Use the 'customContextFile' option, because: * Preserve the qunit-fixture we have. * Workaround karma-qunit bug. - Use the 'customDebugFile' option, because (above reasons) and: * Create 'qunit' element and load QUnit CSS styles for easier debugging. * Base URI - Karma opens at /context.html and serves test/ files from /base/test/. This is incompatible with our manual run which is at /test/index.html. - Using a <base href> tag with "/base/test/" fixed most tests when trying it out, but still caused three problems: 1. While <base href> fixes native expansion (e.g. in HTML attributes, in CSS, and when passed to XHR), it did affect in-JS expansion in our tests, which use hacks based on location.href (e.g. for <a> property tests in unit/basic.js, and creation of absolute urls by unit/ajax.js). Fix those by introducing a prependUrlBase() function that prepends document.baseURI instead of location.href 2. A test in unit/manipulation.js clicks on an anchor link. This broke the test entirely by navigating away from the test page since href="jquery#2" would now expand to /base/test/jquery#2. Fix those by: - Remove the <base> tag again. - Revert the logic in prependUrlBase() to how it used to be inlined. - Remove prependUrlBase() from basic/attributes test. - Instead, fix url() to have baseURL be "/base/test/" when in Karma. - Apply url() to bare urls: - unit/core.js (globalEval test) - unit/support.js (csp test) * In addition to all the above, the most time-consuming part still was making the Ajax tests pass because they currently assume being served from Apache with mod_php installed. I originally planned on addressing this by starting another Node server before Karma starts, somehow injecting its url, and stopping it after Karma. Instead, let's try to use Karma's support for middleware to (hopefully) make this prettier. Fixes jquery#1999.

* Add 'grunt karma:main' task that runs tests via Karma. - Karma uses its own server and uses its own HTML page by default. All files to be served are registered, and for the most part also loaded ("included") through Karma. - Use the 'customContextFile' option, because: * Preserve the qunit-fixture we have. * Workaround karma-qunit bug. - Use the 'customDebugFile' option, because (above reasons) and: * Create 'qunit' element and load QUnit CSS styles for easier debugging. * Base URI - Karma opens at /context.html and serves test/ files from /base/test/. This is incompatible with our manual run which is at /test/index.html. - Using a <base href> tag with "/base/test/" fixed most tests when trying it out, but still caused three problems: 1. While <base href> fixes native expansion (e.g. in HTML attributes, in CSS, and when passed to XHR), it did affect in-JS expansion in our tests, which use hacks based on location.href (e.g. for <a> property tests in unit/basic.js, and creation of absolute urls by unit/ajax.js). Fix those by introducing a prependUrlBase() function that prepends document.baseURI instead of location.href 2. A test in unit/manipulation.js clicks on an anchor link. This broke the test entirely by navigating away from the test page since href="jquery#2" would now expand to /base/test/jquery#2. Fix those by: - Remove the <base> tag again. - Revert the logic in prependUrlBase() to how it used to be inlined. - Remove prependUrlBase() from basic/attributes test. - Instead, fix url() to have baseURL be "/base/test/" when in Karma. - Apply url() to bare urls: - unit/core.js (globalEval test) - unit/support.js (csp test) * Move definition of 'isSwarm' and 'basicTests' out of loadTests(), given Karma will run without that function. * In addition to all the above, the most time-consuming part still was making the Ajax tests pass because they currently assume being served from Apache with mod_php installed. I originally planned on addressing this by starting another Node server before Karma starts, somehow injecting its url, and stopping it after Karma. Instead, let's try to use Karma's support for middleware to (hopefully) make this prettier. Fixes jquery#1999.

* Add 'grunt karma:main' task that runs tests via Karma. - Karma uses its own server and uses its own HTML page by default. All files to be served are registered, and for the most part also loaded ("included") through Karma. - Use the 'customContextFile' option, because: * Preserve the qunit-fixture we have. * Workaround karma-qunit bug. - Use the 'customDebugFile' option, because (above reasons) and: * Create 'qunit' element and load QUnit CSS styles for easier debugging. * Move definition of 'isSwarm' and 'basicTests' out of loadTests(), given Karma will run without that function. * In addition to all the above, the most time-consuming part still was making the Ajax tests pass because they currently assume being served from Apache with mod_php installed. I originally planned on addressing this by starting another Node server before Karma starts, somehow injecting its url, and stopping it after Karma. Instead, let's try to use Karma's support for middleware to (hopefully) make this prettier. Fixes jquery#1999.

* Add 'grunt karma:main' task that runs tests via Karma. - Karma uses its own server and uses its own HTML page by default. All files to be served are registered, and for the most part also loaded ("included") through Karma. - Use the 'customContextFile' option, because: * Preserve the qunit-fixture we have. * Workaround karma-qunit bug. - Use the 'customDebugFile' option, because (above reasons) and: * Create 'qunit' element and load QUnit CSS styles for easier debugging. * Move definition of 'isSwarm' and 'basicTests' out of loadTests(), given Karma will run without that function. * In addition to all the above, start porting some of the PHP mocks to Node.js. (The Ajax tests currently assume being served from Apache with mod_php.). I originally planned on addressing this by starting another Node server before Karma starts, somehow injecting its url, and stopping it after Karma. Instead, let's try to use Karma's support for middleware to (hopefully) make this prettier. Ref issue jquerygh-1999, pull jquerygh-3744.

* Merge all PHP stubs into one MockServer class. * Complete the middleware-mockserver.js port (Node.js equivalent). Notes: * Merge action=script, and action=evalScript. (Similar) * Merge action=wait, action=longLoadScript, action=dont_return. (Similar) * Merge action=status, and action=nocontent. (Similar) * Renamed text.php to text.txt. (No PHP code) * Change 404.php reference to 404.txt. (Neither exists) * Remove data/test.php. (Unused) Now that all tests pass, also add 'grunt karma:main' to npm test. Fixes jquery#1999.

* Merge all PHP stubs into one MockServer class. * Complete the middleware-mockserver.js port (Node.js equivalent). Notes: * Merge action=script, and action=evalScript. (Similar) * Merge action=wait, action=longLoadScript, action=dont_return. (Similar) * Merge action=status, and action=nocontent. (Similar) * Renamed text.php to text.txt. (No PHP code) * Change 404.php reference to 404.txt. (Neither exists) * Remove data/test.php. (Unused) Now that all tests pass, also add 'grunt karma:main' to npm test (for use in local development and on Travis CI). Also add a separate 'npm run jenkins' for use on jenkins.jquery.com where we won't run Chrome (runs TestSwarm instead). Fixes jquery#1999.

* Add 'grunt karma:main' task that runs tests via Karma. - Karma uses its own server and uses its own HTML page by default. All files to be served are registered, and for the most part also loaded ("included") through Karma. - Use the 'customContextFile' option, because: * Preserve the qunit-fixture we have. * Workaround karma-qunit bug. - Use the 'customDebugFile' option, because (above reasons) and: * Create 'qunit' element and load QUnit CSS styles for easier debugging. * Move definition of 'isSwarm' and 'basicTests' out of loadTests(), given Karma will run without that function. * In addition to all the above, start porting some of the PHP mocks to Node.js. (The Ajax tests currently assume being served from Apache with mod_php.). I originally planned on addressing this by starting another Node server before Karma starts, somehow injecting its url, and stopping it after Karma. Instead, let's try to use Karma's support for middleware to (hopefully) make this prettier. Ref issue jquerygh-1999, pull jquerygh-3744.

* Merge all PHP stubs into one MockServer class. * Complete the middleware-mockserver.js port (Node.js equivalent). Notes: * Merge action=script, and action=evalScript. (Similar) * Merge action=wait, action=longLoadScript, action=dont_return. (Similar) * Merge action=status, and action=nocontent. (Similar) * Renamed text.php to text.txt. (No PHP code) * Change 404.php reference to 404.txt. (Neither exists) * Remove data/test.php. (Unused) Now that all tests pass, also add 'grunt karma:main' to npm test (for use in local development and on Travis CI). Also add a separate 'npm run jenkins' for use on jenkins.jquery.com where we won't run Chrome (runs TestSwarm instead). Fixes jquery#1999.

* Merge all PHP stubs into one MockServer class. * Complete the middleware-mockserver.js port (Node.js equivalent). Notes: * Merge action=script, and action=evalScript. (Similar) * Merge action=wait, action=longLoadScript, action=dont_return. (Similar) * Merge action=status, and action=nocontent. (Similar) * Renamed text.php to text.txt. (No PHP code) * Merge references to 404.php, 404.html, and someFileThatDoesNotExist.html to 404.txt, and add a handler for the latter to test middleware to avoid warnings in the Karma server output. * Remove data/test.php. (Unused) Now that all tests pass, also add 'grunt karma:main' to npm test (for use in local development and on Travis CI). Also add a separate 'npm run jenkins' for use on jenkins.jquery.com where we won't run Chrome (runs TestSwarm instead). Fixes jquery#1999.

gibson042 added the Build label Jan 8, 2015

gibson042 mentioned this issue Jan 8, 2015

Support passing url/data object to $.post and $.get #1986

Closed

togakangaroo mentioned this issue Jan 9, 2015

ajax: documented new post and get config object interfaces jquery/api.jquery.com#622

Closed

gibson042 changed the title ~~Simulate external servers with node.js+Javascript instead of (Apache|mongoose|…)+PHP~~ Simulate external servers with Node.js+Javascript instead of (Apache|mongoose|…)+PHP Jan 12, 2015

arschmitz mentioned this issue Mar 3, 2015

Better Issue Tracking jquery/content#4

Open

6 tasks

timmywil added this to the Future milestone Jan 15, 2016

mgol mentioned this issue Jan 15, 2016

Add Vagrant Support or other lightweight server #1755

Closed

timmywil assigned jaubourg Jan 15, 2016

Krinkle mentioned this issue Aug 1, 2017

Existing fixture should be preserved karma-runner/karma-qunit#92

Closed

Krinkle mentioned this issue Aug 1, 2017

Tests: Add support for running unit test via Grunt with Karma #3744

Closed

3 tasks

Krinkle referenced this issue Aug 1, 2017

Tests: Fix failing ajax tests in oldIE

d89c278

timmywil closed this as completed in ecd8dde Dec 18, 2017

timmywil modified the milestones: Future, 3.3.0 Dec 18, 2017

GulajavaMinistudio mentioned this issue Dec 19, 2017

Tests: Add support for running unit tests via grunt with karma GulajavaMinistudio/jquery#33

Merged

4 tasks

lock bot locked as resolved and limited conversation to collaborators Jun 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simulate external servers with Node.js+Javascript instead of (Apache|mongoose|…)+PHP #1999

Simulate external servers with Node.js+Javascript instead of (Apache|mongoose|…)+PHP #1999

gibson042 commented Jan 8, 2015

markelog commented Jan 8, 2015

markelog commented Jan 8, 2015

ChrisAntaki commented Jan 8, 2015

markelog commented Jan 8, 2015

csnover commented Jan 8, 2015

Krinkle commented Jan 8, 2015

gibson042 commented Jan 9, 2015

csnover commented Jan 9, 2015

Krinkle commented Jul 31, 2017 •

edited

Loading

Simulate external servers with Node.js+Javascript instead of (Apache|mongoose|…)+PHP #1999

Simulate external servers with Node.js+Javascript instead of (Apache|mongoose|…)+PHP #1999

Comments

gibson042 commented Jan 8, 2015

markelog commented Jan 8, 2015

markelog commented Jan 8, 2015

ChrisAntaki commented Jan 8, 2015

markelog commented Jan 8, 2015

csnover commented Jan 8, 2015

Krinkle commented Jan 8, 2015

gibson042 commented Jan 9, 2015

csnover commented Jan 9, 2015

Krinkle commented Jul 31, 2017 • edited Loading

Krinkle commented Jul 31, 2017 •

edited

Loading