Skip to content

Conversation

@srh
Copy link
Contributor

@srh srh commented Feb 7, 2019

…ffecting changefeed limit queries

Description

Fixes a bug where a so-called "superblock" gets released too early. Affects behavior of limit change feeds defined on secondary indexes.

This might fix issue #6695.

@srh srh added this to the 2.4 milestone Feb 7, 2019
@srh srh merged commit aba4bdb into rethinkdb:v2.4.x Feb 7, 2019
@srh
Copy link
Contributor Author

srh commented Feb 7, 2019

Cherry-picked fix to next with 8f69b1c.

@atassis
Copy link

atassis commented Feb 8, 2019

Hello there, @srh! Is it possible to make a 2.3.7 version of rethinkdb with this fix? We have a pretty bad issue because of this, it is not so easy to rewrite queries because of our own data model provider

@srh
Copy link
Contributor Author

srh commented Feb 11, 2019

Hi @atassis. It would be possible, though I think people are more interested in spending their energy releasing 2.4 right now.

You could cherry-pick the commit yourself onto the v2.3.x branch and build it yourself (using these build instructions), and test whether it fixes the issue.

@srh srh deleted the sam/24x/limit branch February 11, 2019 18:09
@atassis
Copy link

atassis commented Feb 11, 2019

Yep, we are going to build our own docker image for that after a little discussing. Thanks for your answer.

@srh
Copy link
Contributor Author

srh commented Feb 11, 2019

Please let us know if it fixes the issue.

@atassis
Copy link

atassis commented Feb 11, 2019

Of course I will.
Also, I have forgotten to ask you about how to definitely crash the db. For us it happens randomly, I am still unable to track it.

@srh
Copy link
Contributor Author

srh commented Feb 12, 2019

The bug I fixed was when creating a new limit change feed. So you would have to create new limit change feeds, while writes are being performed.

@stellanhaglund
Copy link

@atassis hi, is there any way that you could share the docker image with the fix compiled?

@srh
Copy link
Contributor Author

srh commented Nov 22, 2019

@stellanhaglund You can compile your own RethinkDB 2.4 package using the setup.rb script (and Dockerfiles) at https://github.com/srh/rethinkdb-package-builder/

E.g. ./setup.rb --distro bionic --v24support --commit HASH [--builds] [--packages]

There is a list of distros in the setup.rb script. You will probably want --packages, not --builds, unless you want to copy out the raw binary.

Note that the --copy option is a WIP and hasn't actually been run yet (I'm working on this today), so if that breaks you'd have to fix it, but in any case you can figure out how to copy the package files out from the image at /platform/rethinkdb/build/packages/ yourself.

HASH has to be a 9-digit commit hash prefix (exactly 9 digits). Probably the tip of v2.4.x is what you want.

The Dockerfiles and such have -j7 hardcoded, which is probably fine.

If you want to build a docker image and run RethinkDB inside that, well, you've got .deb packages for you, and you can figure out how best to create that.

You could also download a bionic package at https://github.com/srh/rethinkdb/releases/tag/v2.4.0-srh-special-3 , if you're running Bionic.

Edit: Changed link to v2.4.0-srh-special-3, now that that's out.

@srh
Copy link
Contributor Author

srh commented Nov 22, 2019

E.g. base your dockerfile off of https://github.com/rethinkdb/rethinkdb-dockerfiles/blob/master/xenial/2.3.6/Dockerfile , just COPY in the .deb and install it instead of using apt-get.

@srh
Copy link
Contributor Author

srh commented Nov 23, 2019

Okay, if you pull the latest master commit in rethinkdb-package-builder and pass --packages --copy-pkgs, things will work.

@atassis
Copy link

atassis commented Nov 23, 2019

@stellanhaglund I'll try to find our private registries image, but you can try to check the atassis/rethinkdb image on docker hub. I have only cherry picked the commit, should work more stable. There are troubles anyway with query queuing and performance, but at least it doesn't break and leak heavily. Still, recommend to mine on another database, because ours is using a lot of memory. Seemless changefeeds is a fairy tale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants