Patch apache#18095 replaced various previously-existing worker cancellation
mechanisms with an interrupt of the worker thread. Unfortunately, in
debugging some stuck tests, it has been revealed that there is at least
one scenario where an InterruptedException could be swallowed when the
main worker thread handles a new work order. This defeated the
cancellation mechanism, leading the worker to run forever.

This patch does three things to improve robustness:

1) Fix the specific code path in RunAllFullyWidget that was swallowing
   InterruptedException in the problematic test.

2) Add a non-interrupt-based cancellation mechanism: a stop() call that
   throws an exception into the main thread's kernel manipulation queue.
   This is useful as a failsafe in case the interrupt gets lost in some
   other code path that may not have been discovered yet.

3) Update MSQTestBase to wait for workers to exit before moving on to
   the next test, and fail if they don't exit within 10 seconds. This is
   preferable to the tests running forever and possibly polluting the
   shared executor.

Patch apache#18095 replaced various previously-existing worker cancellation
mechanisms with an interrupt of the worker thread. Unfortunately, in
debugging some stuck tests, it has been revealed that there is at least
one scenario where an InterruptedException could be swallowed when the
main worker thread handles a new work order. This defeated the
cancellation mechanism, leading the worker to run forever.

This patch does three things to improve robustness:

1) Fix the specific code path in RunAllFullyWidget that was swallowing
   InterruptedException in the problematic test.

2) Add a non-interrupt-based cancellation mechanism: a stop() call that
   throws an exception into the main thread's kernel manipulation queue.
   This is useful as a failsafe in case the interrupt gets lost in some
   other code path that may not have been discovered yet.

3) Update MSQTestBase to wait for workers to exit before moving on to
   the next test, and fail if they don't exit within 10 seconds. This is
   preferable to the tests running forever and possibly polluting the
   shared executor.

Patch #18095 replaced various previously-existing worker cancellation
mechanisms with an interrupt of the worker thread. Unfortunately, in
debugging some stuck tests, it has been revealed that there is at least
one scenario where an InterruptedException could be swallowed when the
main worker thread handles a new work order. This defeated the
cancellation mechanism, leading the worker to run forever.

This patch does three things to improve robustness:

1) Fix the specific code path in RunAllFullyWidget that was swallowing
   InterruptedException in the problematic test.

2) Add a non-interrupt-based cancellation mechanism: a stop() call that
   throws an exception into the main thread's kernel manipulation queue.
   This is useful as a failsafe in case the interrupt gets lost in some
   other code path that may not have been discovered yet.

3) Update MSQTestBase to wait for workers to exit before moving on to
   the next test, and fail if they don't exit within 10 seconds. This is
   preferable to the tests running forever and possibly polluting the
   shared executor.

In PRs apache#18095 and apache#18931, worker cancellation was switched to use
interrupts with a lightweight non-interrupt-based failsafe. This patch
implements a similar idea for controllers, to aid in more prompt
cancellation in cases where the controller is blocking on something.

The main change is to track the controller thread in ControllerHolder
and interrupt it on cancel(), in addition to calling controller.stop().
ControllerHolder is also moved from dart.controller to msq.exec, since
it is now a shared class, no longer Dart-specific. In addition, the
"workerOffline" logic is moved to Dart's ControllerMessageListener,
since that really is Dart-specific.

In PRs apache#18095 and apache#18931, worker cancellation was switched to use
interrupts with a lightweight non-interrupt-based failsafe. This patch
implements a similar idea for controllers, to aid in more prompt
cancellation in cases where the controller is blocking on something.

The main change is to track the controller thread in ControllerHolder
and interrupt it on cancel(), in addition to calling controller.stop().
ControllerHolder is also moved from dart.controller to msq.exec, since
it is now a shared class, no longer Dart-specific. In addition, the
"workerOffline" logic is moved to Dart's ControllerMessageListener,
since that really is Dart-specific.

In PRs #18095 and #18931, worker cancellation was switched to use
interrupts with a lightweight non-interrupt-based failsafe. This patch
implements a similar idea for controllers, to aid in more prompt
cancellation in cases where the controller is blocking on something.

The main change is to track the controller thread in ControllerHolder
and interrupt it on cancel(), in addition to calling controller.stop().
ControllerHolder is also moved from dart.controller to msq.exec, since
it is now a shared class, no longer Dart-specific. In addition, the
"workerOffline" logic is moved to Dart's ControllerMessageListener,
since that really is Dart-specific.

* MSQ: Use interrupts for worker cancellation.

This patch simplifies the worker lifecycle and focuses on using interruption
instead of various flags to manage cancellation. This should enable workers to
cancel themselves more quickly, and is particularly important when they may
be waiting for a resource to become available (such as merge buffers as in apache#18075).

Changes:

1) Introduce WorkerRunRef to manage worker execution and cancellation. This class
   encapsulates the logic for running a worker in an executor and canceling it
   via thread interruption.

2) Move periodic controller status checking out of IndexerWorkerContext and into
   a dedicated PeriodicControllerChecker class.

3) Remove stop(), awaitStop(), and controllerFailed() from the Worker interface.
   Workers are now canceled via interruption only.

4) Remove controllerAlive tracking from WorkerImpl. It should no longer be necessary
   now that workers are interrupted on controller failure.

* Style fixes.