-
-
Notifications
You must be signed in to change notification settings - Fork 1k
Unhandled error when shutting down from client #270
Copy link
Copy link
Closed
Labels
Description
Example
If you start a cluster manually and then run:
>c = Client(profile='default')
>c.shutdown(hub=True) # no errors thrown here
The controller throws an error:
> ipcontroller --profile=default
2017-06-15 13:11:05.160 [IPControllerApp] Hub listening on tcp://127.0.0.1:50566 for registration.
2017-06-15 13:11:05.166 [IPControllerApp] Hub using DB backend: 'DictDB'
2017-06-15 13:11:05.419 [IPControllerApp] hub::created hub
2017-06-15 13:11:05.420 [IPControllerApp] writing connection info to /groups/turaga/home/bergera/.ipython/profile_default/security/ipcontroller-client.json
2017-06-15 13:11:05.427 [IPControllerApp] writing connection info to /groups/turaga/home/bergera/.ipython/profile_default/security/ipcontroller-engine.json
2017-06-15 13:11:05.432 [IPControllerApp] task::using Python leastload Task scheduler
2017-06-15 13:11:05.432 [IPControllerApp] Heartmonitor started
2017-06-15 13:11:05.447 [IPControllerApp] Creating pid file: /groups/turaga/home/bergera/.ipython/profile_default/pid/ipcontroller.pid
2017-06-15 13:11:05.465 [scheduler] Scheduler started [leastload]
2017-06-15 13:11:05.467 [IPControllerApp] client::client b'\x00k\x8bEg' requested 'connection_request'
2017-06-15 13:11:05.468 [IPControllerApp] client::client [b'\x00k\x8bEg'] connected
2017-06-15 13:11:19.363 [IPControllerApp] client::client b'aa3e92a5-1e34-4708-b364-abe2a0c6830a' requested 'registration_request'
2017-06-15 13:11:23.435 [IPControllerApp] registration::finished registering engine 0:aa3e92a5-1e34-4708-b364-abe2a0c6830a
2017-06-15 13:11:23.436 [IPControllerApp] engine::Engine Connected: 0
2017-06-15 13:11:31.529 [IPControllerApp] client::client b'\x00k\x8bEh' requested 'connection_request'
2017-06-15 13:11:31.529 [IPControllerApp] client::client [b'\x00k\x8bEh'] connected
2017-06-15 13:11:37.322 [IPControllerApp] ERROR | Uncaught exception in <function Hub.dispatch_monitor_traffic at 0x7fab85862a60>
Traceback (most recent call last):
File "/groups/turaga/home/bergera/anaconda3/envs/tf_gpu2/lib/python3.5/site-packages/ipyparallel/util.py", line 110, in log_errors
return f(self, *args, **kwargs)
File "/groups/turaga/home/bergera/anaconda3/envs/tf_gpu2/lib/python3.5/site-packages/ipyparallel/controller/hub.py", line 513, in dispatch_monitor_traffic
handler(idents, msg)
File "/groups/turaga/home/bergera/anaconda3/envs/tf_gpu2/lib/python3.5/site-packages/ipyparallel/controller/hub.py", line 839, in monitor_iopub_message
uuid = self.engines[eid].uuid
KeyError
2017-06-15 13:11:37.329 [IPControllerApp] client::client b'\x00k\x8bEh' requested 'shutdown_request'
2017-06-15 13:11:37.332 [scheduler] Unhandled message type: 'shutdown_notification'
ERROR:scheduler:Unhandled message type: 'shutdown_notification'
2017-06-15 13:11:38.331 [IPControllerApp] hub::hub shutting down.
Engine session:
>ipengine --profile=default
2017-06-15 13:11:19.349 [IPEngineApp] Loading url_file '/groups/turaga/home/bergera/.ipython/profile_default/security/ipcontroller-engine.json'
2017-06-15 13:11:19.361 [IPEngineApp] Registering with controller at tcp://127.0.0.1:50566
2017-06-15 13:11:19.469 [IPEngineApp] Starting to monitor the heartbeat signal from the hub every 3010 ms.
2017-06-15 13:11:19.476 [IPEngineApp] Completed registration with id 0
Environment info
- python: 3.5.2
- ipyparallel: 6.0.1
Reactions are currently unavailable