Skip to content

Create v8Schema with series index#430

Closed
aaron7 wants to merge 4 commits intomasterfrom
416-create-new-index
Closed

Create v8Schema with series index#430
aaron7 wants to merge 4 commits intomasterfrom
416-create-new-index

Conversation

@aaron7
Copy link
Copy Markdown
Contributor

@aaron7 aaron7 commented May 18, 2017

First step for #416.

This index will be used to fetch series from the index. It will replace the metric name index and we will make the switch after the next steps are complete.

Decided to encode the model.Metric as JSON because it already implements the json.Unmarshaler interface.

Used sha256 for identifying the series as the variant of fnv64a which if used for fingerprints is not unique enough for indexing and using sha256 keeps it simple (we don't have to deal with collision logic).

Test plan

  • Tests pass

@aaron7 aaron7 requested a review from tomwilkie May 18, 2017 14:36
@tomwilkie
Copy link
Copy Markdown
Contributor

Used the metric fingerprint for identifying the series. It uses a variant of fnv64a https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function. Is this suitable for use in our index?

As discussed, we're going to go with a sha256.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should change this to error if its a duplicate within the batch, to match server behaviour.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already doing so a few lines above. I didn't remove the Dupe write check across all writes as it's still useful for checking if chunks are no duplicated.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we decode the string back into model.Metric?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find a convenient function, no.

I don't like the idea of putting JSON in here, its a PITA to deal with backwards compatibility etc. But I guess it'll have to do for now.

@aaron7
Copy link
Copy Markdown
Contributor Author

aaron7 commented Jun 9, 2017

#442

@aaron7 aaron7 closed this Jun 9, 2017
@tomwilkie tomwilkie deleted the 416-create-new-index branch September 24, 2018 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants