Skip to content
This repository was archived by the owner on Jan 8, 2020. It is now read-only.

V3#234

Closed
dim wants to merge 5 commits intomasterfrom
feature/v3
Closed

V3#234
dim wants to merge 5 commits intomasterfrom
feature/v3

Conversation

@dim
Copy link
Copy Markdown
Member

@dim dim commented Apr 27, 2018

No description provided.

@dim dim force-pushed the feature/v3 branch 4 times, most recently from c382986 to 208ebf2 Compare April 27, 2018 17:43
@steve-gray
Copy link
Copy Markdown

@dim - What's the current status and timeline on this - are there open areas for PR/contributions?

I'm encountering issues with shutdown race conditions inside the sarama-cluster Close methods, and the other issue report indicated it won't be fixed in the as-is code pending this rewrite?

@dim
Copy link
Copy Markdown
Member Author

dim commented Aug 13, 2018

@steve-gray sorry was travelling, I will resume work on the IBM/sarama#1099 PR this week

@steve-gray
Copy link
Copy Markdown

@dim - No worries - if there's anything to help with, I'm reasonably open to throwing time at it. In the meantime, I've mitigated the shutdown lockups/memory leaks with a drain routine I call in a goroutine immediately before stopping the consumer, forcing it to drain out completely - probably need something like this in the mainline - as I can't imagine there's a use case for Stop() that isn't going to need to deal with this manually:

func drainConsumer(c *cluster.Consumer) {
	if c == nil {
		return
	}

	// Errors
	go func() {
		for range c.Errors() {
		}
	}()

	// Notifications
	go func() {
		for range c.Notifications() {
		}
	}()

	// Messages
	go func() {
		for range c.Messages() {
		}
	}()

	// Partitions
	go func() {
		for p := range c.Partitions() {
			errClose := p.Close()
			log.WithError(errClose).Warn("kafka_draining_partition_aquired")
		}
	}()
}

@dim
Copy link
Copy Markdown
Member Author

dim commented Aug 14, 2018

Closing in favour of #256

@dim dim closed this Aug 14, 2018
@dim dim deleted the feature/v3 branch August 14, 2018 13:39
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants