feat(cells): introduce locality concept in configuration#108410
feat(cells): introduce locality concept in configuration#108410
Conversation
This is a small initial PR to introduce the locality concept. Locality is a group of cells (which represent the geographic region, or tenant in the case of single-tenant deployments). The existing region_name / sentry-region terminology is deprecated in favor of locality + cells. This naming will contiue to be migrated in subsequent PRs / database migrations. This PR includes the following: - introduce SENTRY_LOCALITIES setting to configure the cell -> locality mapping - when SENTRY_LOCALITIES is not present (rollout phase), synthesize a 1:1 locality per cell as a fallback - update the OrgCellMappingsEndpoint to return the real cell to locality map from config instead of the hardcoded value - update the OrgCellMappingsEndpoint to support multiple localities being requested - add TODO(cells) comments next to model fields where region_name should be later updated to cell_name in the DB
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| ) | ||
|
|
||
| # TODO(cells): rename to cell_name | ||
| region_name = models.CharField(max_length=REGION_NAME_LENGTH) |
There was a problem hiding this comment.
I wouldn't attempt a rename. Instead we would have to add a column, dual write, shift reads and then drop the current column.
Changing the names on these columns will be a non-trivial amount of work for no benefit. Given the current scope and staffing levels on this workstream I don't see how we would be able to prioritize this work.
There was a problem hiding this comment.
seems like rename is a metadata-only change though https://www.postgresql.org/docs/9.6/sql-altertable.html
why would we need so many steps?
There was a problem hiding this comment.
Because of django and release boundaries. When we do deploys we apply migrations while the old code is running. When the column is renamed, the 'old' version of the application will still be using the old column name, and all queries will fail until application instances are restarted with new code which can can take several minutes.
There was a problem hiding this comment.
can't we handle that with aliasing rather than the full data migration though?
i've just written the task as a todo here since i think we could determine the exact strategy later but wanted to call it out now. long term i think there's benefit in names that are easy for future contributors to understand, and region_name will be very confusing in a few months time when it doesn't map to anything else in the codebase anymore.
There was a problem hiding this comment.
Yes we could alias in the django model. Then folks have to remember two different names between the schema and code. I agree that having region around long term isn't ideal, and if we're going to make a change we should do it across schema and code.
| query = OrganizationMapping.objects.all() | ||
| if request.GET.get("locale"): | ||
| query = query.filter(region_name=request.GET.get("locale")) | ||
| localities = request.GET.getlist("locality") |
There was a problem hiding this comment.
Which term are we using? Locality is longer and harder to say, and the words are rough synonyms.
When would synapse need more than a single locale?
There was a problem hiding this comment.
synapse only ever uses the word locality... locale is never used
locale is fine with me if we want to change the term everywhere, but i stuck to the ones previously documented
synapse's design tries not make any assumptions about the shape of our infrastructure, even though our final deployments will probably only have one synapse per locality. so it can proxy for one locality as easily as for multiple.
There was a problem hiding this comment.
synapse's design tries not make any assumptions about the shape of our infrastructure, even though our final deployments will probably only have one synapse per locality. so it can proxy for one locality as easily as for multiple.
👍 If supporting multiple doesn't add too much complexity it is nice to have.
locale is fine with me if we want to change the term everywhere, but i stuck to the ones previously documented
I agree we should be consistent. I took a look through the ops repo and it has locality already. I don't think 'locale' is significantly more clear that it justifies making changes. Lets stick with locality in the code going forward.
src/sentry/types/region.py
Outdated
| for region in self.regions: | ||
| region.validate() | ||
|
|
||
| # Ensure that a cell cannot be registed to more than one locality |
There was a problem hiding this comment.
| # Ensure that a cell cannot be registed to more than one locality | |
| # Ensure that a cell cannot be registered to more than one locality |
|
Refs INFRENG-219 |
| # Mapping of localities (e.g. "us", "de") to their constituent cells (e.g. "us1", "us2") | ||
| SENTRY_LOCALITIES: list[LocalityConfig] = [] |
There was a problem hiding this comment.
If each Region/Cell had a locality attribute assigned to them would we need this? We could infer the list of localities from the Cell configuration. Doing that would avoid any drift between REGION_CONFIG and LOCALITIES, which could arise during tests when we have to stub out the region lists, and would make generating config data for the application simpler.
There was a problem hiding this comment.
Locality might be an important concept we expand on later though. For example we will likely need to mark which ones are saas vs single-tenants, which default cell of a locality to send new organizations to, and other things.
So i think introducing the top level locality concept may make sense here, rather than burying it inside the cell configuration, even if it's not strictly needed yet.
The test cases are mostly stubbed correctly by the TestEnvRegionDirectory class so other test code won't have to handle it
There was a problem hiding this comment.
Good point about needing a place to denote the default cell for a locality, and hidden localities.
This is a small initial PR to introduce the locality concept. Locality is a group of cells (which represent the geographic region, or tenant in the case of single-tenant deployments). The existing region_name / sentry-region terminology is deprecated in favor of locality + cells. This naming will contiue to be migrated in subsequent PRs / database migrations. This PR includes the following: - introduce SENTRY_LOCALITIES setting to configure the cell -> locality mapping - when SENTRY_LOCALITIES is not present (rollout phase), synthesize a 1:1 locality per cell as a fallback - update the OrgCellMappingsEndpoint to return the real cell to locality map from config instead of the hardcoded value - update the OrgCellMappingsEndpoint to support multiple localities being requested - add TODO(cells) comments next to model fields where region_name should be later updated to cell_name in the DB
This is a small initial PR to introduce the locality concept. Locality is a group of cells (which represent the geographic region, or tenant in the case of single-tenant deployments). The existing region_name / sentry-region terminology is deprecated in favor of locality + cells. This naming will contiue to be migrated in subsequent PRs / database migrations. This PR includes the following: - introduce SENTRY_LOCALITIES setting to configure the cell -> locality mapping - when SENTRY_LOCALITIES is not present (rollout phase), synthesize a 1:1 locality per cell as a fallback - update the OrgCellMappingsEndpoint to return the real cell to locality map from config instead of the hardcoded value - update the OrgCellMappingsEndpoint to support multiple localities being requested - add TODO(cells) comments next to model fields where region_name should be later updated to cell_name in the DB
This is a small initial PR to introduce the locality concept. Locality is a group of cells (which represent the geographic region, or tenant in the case of single-tenant deployments). The existing region_name / sentry-region terminology is deprecated in favor of locality + cells. This naming will contiue to be migrated in subsequent PRs / database migrations.
This PR includes the following: