Skip to content

Process tree structure refactor#149

Closed
guilledk wants to merge 1 commit into
pikers:masterfrom
guilledk:daemonization
Closed

Process tree structure refactor#149
guilledk wants to merge 1 commit into
pikers:masterfrom
guilledk:daemonization

Conversation

@guilledk

Copy link
Copy Markdown

This started trying to add the marketstore supervisor #143, but in the end it turned to a general refactor of the piker process tree.

…okerd to adapt to new process tree structure.
Comment thread piker/_daemon.py
log.info(f'Spawning {brokername} broker daemon')
tractor_kwargs = getattr(brokermod, '_spawnkwargs', {})

# TODO: raise exception when root_nursery == None?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if root_nursery == None here, calling this function doesn't make any sense, maybe we could do an if check and raise an exception but not sure which one would be apropiate.

Comment thread piker/_daemon.py

# init root nursery
async with tractor.open_nursery() as nursery:
root_nursery = nursery

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just put a comment here saying that this assigns the reference to the global module variable so that new calls to pikerd can access this same singleton nursery instance.

Comment thread piker/_daemon.py
brokername,
loglevel: Optional[str] = None,
**tractor_kwargs
):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc string would be good.

Just explain that this func is used to start sub-daemons under pikerd from remote client actors er wtv.

Comment thread piker/_daemon.py

# TODO: raise exception when root_nursery == None?
global root_nursery
await root_nursery.start_actor(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm another thing that might be handy here is caching the output portals to each name and then on repeat lookups you just noop if the daemon is already spawned?

Comment thread piker/_daemon.py
@@ -0,0 +1,79 @@
"""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the license header.

We're being pedantic about this since the sw is likely to have a lot of attention in the future.

Comment thread piker/_daemon.py
]

@asynccontextmanager
async def maybe_spawn_pikerd(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm looking at this further I wonder if we should choose a better name like maybe_open_pikerd?

Technically this doesn't actually spawn anything, just might open a nursery.

Comment thread piker/data/__init__.py
yield portal

else: # no daemon has been spawned yet
async with maybe_spawn_pikerd(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm or maybe_start_pikerd()?

Comment thread piker/data/__init__.py
) as pikerd_portal:

if pikerd_portal is None:
await spawn_brokerd(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this is interesting. I wonder if we can avoid having to check for None here (because we're likely going to see this pattern often) and instead hand some kind of local portal thing kinda like what's already inside tractor - but maybe it's over the top?

it would be nice to have the polymorphism to avoid all this branch logic.

@goodboy goodboy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks great!

I left a variety of questions, docs suggestions, and a bit of naming changes.
The one thing i wonder if we should consider is some kind of local portal shim to avoid all the branch logic for when we're "spawning" things locally. Not sure i'm convinced it's that big a deal yet tho.

@goodboy

goodboy commented Mar 20, 2021

Copy link
Copy Markdown
Contributor

@guilledk I made a PR into your branch to update to the new clearing / order systems stuff.

I'm not sure if it's more of a hassle to get your branch updated and merge it or I'll just close this PR and make a new one with that updates branch?

@goodboy goodboy mentioned this pull request Mar 20, 2021
@goodboy

goodboy commented Mar 20, 2021

Copy link
Copy Markdown
Contributor

Moved to #151.

@goodboy goodboy closed this Mar 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants