orca: create ORCA producer for LB policies to use to receive OOB load reports by dfawley · Pull Request #5669 · grpc/grpc-go

dfawley · 2022-09-23T18:13:28Z

RELEASE NOTES:

balancer: add experimental Producer types and methods and * orca: provide a way for LB policies to receive OOB load reports

easwars · 2022-10-05T17:46:06Z

 // avoid polluting the godoc of the top-level orca package.
 package internal

+import ibackoff "google.golang.org/grpc/internal/backoff"


Why do we need this import to be renamed?

We don't; I just like to keep things consistent, and we have a backoff package and an internal/backoff package, and it's confusing if we reference both using the same name in different files.

Package names should ideally be self contained and self explanatory. If we are always going to rename this import, we should ideally rename the package itself.

easwars · 2022-10-05T18:00:05Z

+// A Producer is a type shared among potentially many consumers.  It is
+// associated with a SubConn, and an implementation will typically contain
+// other methods to provide additional functionality, e.g. configuration or
+// subscription registration.


This description feels very vague to me, in the sense that I can't get a good idea of when to use a Producer, how to use one etc.

Could you suggest something better after our discussion today?

Ack. The first sentence is fine, but I don't get much technical understanding from: "and an implementation will typically contain other methods to provide additional functionality, e.g. configuration or subscription registration." Additional functionality with respect to what? There's no functionality defined before this. Do we expect this to ever be implemented by another concrete type? This comment is very specific to type producer in orca/producer. If so, why have this interface?

Additional functionality with respect to what? There's no functionality defined before this.

In addition to the builder that created it, I suppose, and the close function the builder returns.

Do we expect this to ever be implemented by another concrete type?

Yes, definitely. This is intended to be generic and able to be implemented by anything. The other example we already have is the health checks, which are implemented in the other languages as a generic wrapper, but hard-coded into gRPC in Go. I'd like to move to a similar model there by using this mechanism.

This comment is very specific to type producer in orca/producer.

Not at all. If it was, it would talk about ORCA configuration. ORCA should not be part of grpc-go directly, but instead should only be a plugin which uses this.

easwars

I think you might have missed pushing a commit. I have more comments which I will add soon.

zasweq

Cool PR bossman. Some comments.

zasweq · 2022-10-13T18:27:40Z

+// A Producer is a type shared among potentially many consumers.  It is
+// associated with a SubConn, and an implementation will typically contain
+// other methods to provide additional functionality, e.g. configuration or
+// subscription registration.


Ack. The first sentence is fine, but I don't get much technical understanding from: "and an implementation will typically contain other methods to provide additional functionality, e.g. configuration or subscription registration." Additional functionality with respect to what? There's no functionality defined before this. Do we expect this to ever be implemented by another concrete type? This comment is very specific to type producer in orca/producer. If so, why have this interface?

dfawley

Updated; sorry for the force push; I messed up the synchronization between my laptop & workstation.

dfawley · 2022-10-17T21:06:42Z

+// A Producer is a type shared among potentially many consumers.  It is
+// associated with a SubConn, and an implementation will typically contain
+// other methods to provide additional functionality, e.g. configuration or
+// subscription registration.


Additional functionality with respect to what? There's no functionality defined before this.

In addition to the builder that created it, I suppose, and the close function the builder returns.

Do we expect this to ever be implemented by another concrete type?

Yes, definitely. This is intended to be generic and able to be implemented by anything. The other example we already have is the health checks, which are implemented in the other languages as a generic wrapper, but hard-coded into gRPC in Go. I'd like to move to a similar model there by using this mechanism.

This comment is very specific to type producer in orca/producer.

Not at all. If it was, it would talk about ORCA configuration. ORCA should not be part of grpc-go directly, but instead should only be a plugin which uses this.

zasweq · 2022-10-18T03:10:34Z

 	})
 	if err != nil {
-		return resetBackoff, err
+		return false, err


You liked this rather than using default value of bools more?

This was changed due to an earlier comment from @easwars. I said in my reply there I didn't feel strongly either way. I still don't.

zasweq · 2022-10-18T03:21:17Z

+	// to avoid overloading a server experiencing problems.  The attempt count
+	// is incremented when stream errors occur and is reset when the stream
+	// reports a result.


I don't think the last part of this comment is correct/like this comment. First of all, reset has two semantical meanings for this field, one for the timer, one for the attempt count, which gets plumbed into Exponential Backoff to determine what the backoff timer gets reset to I'm assuming your reset refers to attempt count. Also, there is no linkage or even local variable named request count. I see backoffAttempt, which isn't a field here, but the int in this function variable. The attempt count is not reset when the stream reports a result. Right now, the attempt count is reset on either a. the minInterval has changed after processing the receive from the stream compared to what was the stream was configured with. b. If the second+ stream.Recv() returns an error, this also feels wrong, I will leave a comment there.

zasweq · 2022-10-18T03:25:42Z

+			return resetBackoff, err
+		}
+		resetBackoff = true


Is this correct? I don't get why it matters whether the error comes from the second+ stream.Recv() or the first one with respect to resetting backoff. Shouldn't it be io.EOF the conditional for a successful stream where we can infer the server is working properly?

dfawley · 2022-10-25T20:52:43Z

Ping @easwars @zasweq this should be ready to review again.

easwars

LGTM.

Deep down though, I would have still preferred if we could have added the methods required to make RPCs on the SubChannel interface. I feel that would have been a more intuitive change.

Like, currently, the RegisterOOBListener function in the orca package calls GetOrBuildProducer on the SubConn and passes a builder defined locally. This seems quite roundabout. But, if we want the orca package to handle everything locally, then we would need the functionality to make RPCs on the subConns, and also the functionality to uniquely identify subConns based on their channelz ID. If we don't think adding this functionality to the subConn interface is a good long term option, I agree with the approach taken in this PR.

zasweq

All testing nits.

zasweq · 2022-11-02T00:36:53Z

+	// Stop listener 3.  This makes the interval longer, with stream recreation
+	// delayed until the next report is received.  Reports only go to listener
+	// 1 now.


Reports should only go to listener 1 now. Also explain why the interval is now longer, and perhaps tie that idea of a longer interval to stream recreation.

ping about stream recreation.

I think a very high level understanding of the feature (ORCA OOB reporting) plus this comment as-is is sufficient to understand what's happening here. I could make a longer comment to make things painfully obvious, but sometimes less is more.

zasweq · 2022-11-02T22:00:39Z

+
+	lisOpts := orca.OOBListenerOptions{ReportInterval: 50 * time.Millisecond}
+	lao := &listenerAndOptions{listener: oobLis, opts: lisOpts}
+	r.InitialState(resolver.State{Addresses: []resolver.Address{setListenerAndOptions(resolver.Address{Addr: lis.Addr().String()}, lao)}})


I don't see this split up still.

zasweq · 2022-11-02T22:08:09Z

 	// Provide a convenient way to expect backoff calls and return a minimal
 	// value.
-	expectedBackoff := -1 // -1 indicates any value is allowed.
+	const backoffShouldNotBeCalled = 9999 // Use to assert backoff function is not called.


Can you clarify why?

I do at the usage points: https://github.com/grpc/grpc-go/pull/5669/files#diff-6a71af69cb3159cbd1bbb20fb808da29d460ee437f134b485afc11df20079bb3R343-R345

I don't get that explanation, but it's probably just me.

I explain in comments why we don't want the backoff function to be called when backoffShouldNotBeCalled is assigned to expectedBackoff below. (It's because the last stream was successful.)

zasweq · 2022-11-02T22:08:34Z

+
+	// Register our fake ORCA service
+	s := grpc.NewServer()
+	fake := newFakeORCAService()


zasweq

Some comments.

zasweq · 2022-11-02T22:11:27Z

-	// Stop listener 2.  This does not affect the interval.  The next update
-	// goes to listeners 1 and 3.
+	// Stop listener 2.  This does not affect the interval as listener 3 is
+	// still the shortest.  The next update goes to listeners 1 and 3.


and still has the shortest interval

This level of review isn't important. The meaning of this comment is very clear, and it's an internal comment in a test, not in external documentation.

Wouldn't hurt though. But I see what you're saying

zasweq

LGTM outside my final responses, but again don't feel strongly about any of my nits.

zasweq · 2022-11-03T17:18:03Z

You can go ahead and merge btw.

… reports (grpc#5669)

dfawley added the Type: Feature New features or improvements in behavior label Sep 23, 2022

dfawley added this to the 1.50 Release milestone Sep 23, 2022

dfawley requested review from easwars and zasweq September 23, 2022 18:13

dfawley assigned easwars and zasweq Sep 23, 2022

dfawley force-pushed the orca_producer branch 3 times, most recently from 1b369f2 to 1cf3c60 Compare September 27, 2022 21:16

dfawley modified the milestones: 1.50 Release, 1.51 Release Oct 4, 2022

easwars reviewed Oct 5, 2022

View reviewed changes

easwars assigned dfawley and unassigned easwars Oct 5, 2022

easwars reviewed Oct 5, 2022

View reviewed changes

Comment thread orca/producer.go Outdated

Comment thread balancer_conn_wrappers.go Outdated

Comment thread orca/producer.go

Comment thread orca/producer.go Outdated

Comment thread orca/producer.go Outdated

Comment thread orca/producer.go

Comment thread balancer_conn_wrappers.go Outdated

dfawley assigned easwars and unassigned dfawley Oct 6, 2022

easwars reviewed Oct 6, 2022

View reviewed changes

Comment thread balancer/balancer.go

Comment thread balancer/balancer.go Outdated

easwars assigned dfawley and unassigned easwars Oct 6, 2022

zasweq suggested changes Oct 13, 2022

View reviewed changes

zasweq removed their assignment Oct 13, 2022

dfawley added 3 commits October 17, 2022 14:41

balancer: add Producer types and methods

ed0eb16

orca: create ORCA producer for LB policy use

e1c66fa

review comments

4971070

dfawley force-pushed the orca_producer branch from 1cf3c60 to 4971070 Compare October 17, 2022 21:42

dfawley commented Oct 17, 2022

View reviewed changes

zasweq reviewed Oct 18, 2022

View reviewed changes

dfawley added 2 commits October 18, 2022 09:51

don't log ctx.Err

31dd4a6

fix min computation

f6c6b4c

dfawley assigned easwars and zasweq and unassigned dfawley Oct 18, 2022

dfawley mentioned this pull request Oct 18, 2022

client: synchronously verify server preface in newClientTransport #5731

Merged

dfawley added 2 commits October 18, 2022 15:57

add grpcsync.OnceFunc

0961db3

use OnceFunc

b259326

easwars approved these changes Oct 27, 2022

View reviewed changes

Comment thread orca/producer.go

easwars assigned dfawley and unassigned easwars Oct 27, 2022

zasweq suggested changes Nov 2, 2022

View reviewed changes

zasweq removed their assignment Nov 2, 2022

test updates

65653a4

zasweq reviewed Nov 2, 2022

View reviewed changes

Comment thread orca/producer_test.go Outdated

Comment thread orca/producer_test.go

zasweq reviewed Nov 2, 2022

View reviewed changes

zasweq suggested changes Nov 2, 2022

View reviewed changes

more test changes

8aa79d6

dfawley assigned zasweq and unassigned dfawley Nov 2, 2022

zasweq approved these changes Nov 3, 2022

View reviewed changes

zasweq assigned dfawley and unassigned zasweq Nov 3, 2022

dfawley merged commit e41e894 into grpc:master Nov 3, 2022

dfawley deleted the orca_producer branch November 3, 2022 17:27

jronak pushed a commit to jronak/grpc-go that referenced this pull request Nov 21, 2022

orca: create ORCA producer for LB policies to use to receive OOB load…

faf2776

… reports (grpc#5669)

github-actions Bot locked as resolved and limited conversation to collaborators May 3, 2023

Conversation

dfawley commented Sep 23, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfawley Oct 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

easwars left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zasweq left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dfawley left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zasweq Oct 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfawley commented Oct 25, 2022

Uh oh!

easwars left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zasweq left a comment

Choose a reason for hiding this comment

dfawley Oct 17, 2022 •

edited

Loading

zasweq Oct 18, 2022 •

edited

Loading