Skip to content

Comments

Do not cache available engines early#1092

Closed
xsawyerx wants to merge 3 commits intomasterfrom
fix/gh-1013
Closed

Do not cache available engines early#1092
xsawyerx wants to merge 3 commits intomasterfrom
fix/gh-1013

Conversation

@xsawyerx
Copy link
Member

@xsawyerx xsawyerx commented Jan 4, 2016

GH #1013 discovered a problem which can be summarized to: When changing an engine (template, logger, session, serializer) after declaring a hook, the engine will not maintain the current request object during request-time.

package App;
use Dancer2;
hook before => sub {1};
set template => 'simple';
...

# template cannot be rendered because there is no request

What happens is a simple matter of race condition which I'll try to explain.

Normally, when a request is run through the dispatcher, it will instantiate a lazy attribute (defined_engines) which lists the available engines for the App. It will then ask them to set
their current active request to the one it received.

That value is then stored in the attribute and is never recalculated.

So far this is just by design.

However, if that value gets calculated ahead of time, by some other action taking place, when the dispatching process reaches that code, it will use the available engines that were created previously. These engines might not be relevant anymore. This is what's happening.

When calling a hook (hook $name => sub {...}), the application will call any hook candidates in order to have them add a hook as well. This in turn reaches for the currently available engines, as they are possible hook candidates too. If that value wasn't populated before, it will be now. This means that all the currently-defined engines will be caluclates as the available ones (never to be recalculated).

Enter the race condition manifestation:

When the hook is called, it creates the defined_engines, which will include whatever engines are configured as of that moment (such as the template engine). Later the template is reconfigured. However, the defined_engines attribute stays the same.

During a request, the Dancer dispatcher tries to set the current request on all available engines, including the original template engine that was available when the hook inflates the value the
dispatcher is using to decide on the available engines.

Thus, the dispatcher sets the old value.

Fixing it:

In order to fix it, we turn the attribute to a subroutine. We call it every time we need to dispatch and keep it for the remainder of the request dispatching. This means that every dispatching now calls this subroutine, which only creates an arrayref of all engine objects, so it's negligible, IMHO.

Additional comments:

  • I'm not sure this applies to the serializer engine too. If it does, it will solve it for the serializer engine too.
  • Theoretically, all engines could use the request variable $Dancer2::Core::Route::REQUEST to reach the current request at any given time, but that would be an implementation detail. Perhaps we should introduce that in the role. That might allow removing an attribute and a few ugly methods for setting the current one. However, I'm not inclined to do that at the moment, since it will incur more depth of knowledge in maintaining the core, and that's never a good thing.

@veryrusty
Copy link
Member

@xsawyerx++ Looks good. ( But damn; one failing test case :( )
If you need to sleep, I'll try and sort that out!

GH #1013 discovered a problem which can be summarized to: When
changing an engine (template, logger, session, serializer)
*after* declaring a hook, the engine will not maintain the
current request object during request-time.

    package App;
    use Dancer2;
    hook before => sub {1};
    set template => 'simple';
    ...

    # template cannot be rendered because there is no request

What happens is a simple matter of race condition which I'll try
to explain.

Normally, when a request is run through the dispatcher, it will
instantiate a lazy attribute (`defined_engines`) which lists the
available engines for the App. It will then ask them to set
their current active request to the one it received.

That value is then stored in the attribute and is never
recalculated.

So far this is just by design.

However, if that value gets calculated ahead of time, by some
other action taking place, when the dispatching process reaches
that code, it will use the available engines that were created
previously. These engines might not be relevant anymore. This is
what's happening.

When calling a hook (`hook $name => sub {...}`), the application
will call any hook candidates in order to have them add a hook
as well. This in turn reaches for the currently available
engines, as they are possible hook candidates too. If that value
wasn't populated before, it will be now. This means that all the
currently-defined engines will be caluclates as the available
ones (never to be recalculated).

Enter the race condition manifestation:

When the hook is called, it creates the `defined_engines`,
which will include whatever engines are configured as of that
moment (such as the template engine). Later the template is
reconfigured. However, the `defined_engines` attribute stays
the same.

During a request, the Dancer dispatcher tries to set the current
request on all available engines, including the original template
engine that was available when the hook inflates the value the
dispatcher is using to decide on the available engines.

Thus, the dispatcher sets the old value.

Fixing it:

In order to fix it, we turn the attribute to a subroutine. We
call it every time we need to dispatch and keep it for the
remainder of the request dispatching. This means that every
dispatching now calls this subroutine, which only creates an
arrayref of all engine objects, so it's negligible, IMHO.

Additional comments:

* I'm not sure this applies to the serializer engine too. If it
  does, it will solve it for the serializer engine too.

* Theoretically, all engines could use the request variable
  `$Dancer2::Core::Route::REQUEST` to reach the current request
  at any given time, but that would be an implementation detail.
  Perhaps we should introduce that in the role. That might allow
  removing an attribute and a few ugly methods for setting the
  current one. However, I'm not inclined to do that at the moment,
  since it will incur more depth of knowledge in maintaining the
  core, and that's never a good thing.
xsawyerx and others added 2 commits January 5, 2016 12:00
Allows `set_request` to be called outside of the dispatch loop, such as
when generating a 404 response when no routes match.
@veryrusty
Copy link
Member

👍 that was one nasty race condition to chase down.

@xsawyerx
Copy link
Member Author

xsawyerx commented Jan 5, 2016

👍 @veryrusty for always covering my ass. :)

@SysPete
Copy link
Member

SysPete commented Jan 5, 2016

👍 @xsawyerx great work! I cherry-picked these commits into my local plugins-yanick branch and can now get more plugins working with plugin2. Any chance of getting this into a release soon so I can starting pushing PRs for the affected plugins?

@xsawyerx
Copy link
Member Author

xsawyerx commented Jan 5, 2016

Yeah, I can merge this and push a new release today.

I can also rebase the plugins branch to refresh it. (I'd rather do that than cherry-pick or merge.)

@veryrusty veryrusty added this to the 0.166000 milestone Jan 5, 2016
@veryrusty
Copy link
Member

Merged as 56d38c1.

@xsawyerx++ Thanks for chasing that race condition down!

@veryrusty veryrusty closed this Jan 5, 2016
@veryrusty veryrusty deleted the fix/gh-1013 branch January 5, 2016 11:39
xsawyerx added a commit that referenced this pull request Jan 12, 2016
    [ BUG FIXES ]
    * GH #1013, #1092: Remove race condition caused by caching available
      engines. (Sawyer X, Menno Blom, Russell Jenkins)
    * GH #1089: Exact macthing of route regex comments for tokens/splats.
      (Sawyer X)
    * GH #1079, #1082: Allow routes to return '0' as response content,
      and serializer hooks are called when default response content is
      to be returned. (Alberto Simões, Russell Jenkins)
    * GH #1093, 1095: Use a dynamic TT2 INCLUDE_PATH to allow relative
      views with relative includes; fixing regression introduced by #1037.
      (Russell Jenkins)
    * GH #1096, #1097: Return compatibility on Perl 5.8.x!
      (Peter Mottram - @SysPete)

    [ DOCUMENTATION ]
    * GH #1076: Typo in Dancer2::Core::Hook POD. (Jonathan Scott Duff)

    [ ENHANCEMENTS ]
    * GH #1074: Add sample session engine config to skeleton app.
      (Peter Mottram - @SysPete)
    * GH #1088: Return route objects when defining new routes.
      (Sawyer X)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants