…nto release/1.16.x (#18229)

* [OSS] Post Consul 1.16 updates (#17606)

* chore: update dev build to 1.17

* chore(ci): add nightly 1.16 test

Drop the oldest and add the newest running release branch to nightly
builds.

* Add writeAuditRPCEvent to agent_oss (#17607)

* Add writeAuditRPCEvent to agent_oss

* fix the other diffs

* backport change log

* Add Envoy and Consul version constraints to Envoy extensions (#17612)

* [API Gateway] Fix trust domain for external peered services in synthesis code (#17609)

* [API Gateway] Fix trust domain for external peered services in synthesis code

* Add changelog

* backport ent changes to oss (#17614)

* backport ent changes to oss

* Update .changelog/_5669.txt

Co-authored-by: Michael Zalimeni <[email protected]>

---------

Co-authored-by: Michael Zalimeni <[email protected]>

* Update intentions.mdx (#17619)

Make behaviour of L7 intentions clearer

* enterprise changelog update for audit (#17625)

* Update list of Envoy versions (#17546)

* [API Gateway] Fix rate limiting for API gateways (#17631)

* [API Gateway] Fix rate limiting for API gateways

* Add changelog

* Fix failing unit tests

* Fix operator usage tests for api package

* sort some imports that are wonky between oss and ent (#17637)

* PmTLS and tproxy improvements with failover and L7 traffic mgmt for k8s (#17624)

* porting over changes from enterprise repo to oss

* applied feedback on service mesh for k8s overview

* fixed typo

* removed ent-only build script file

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <[email protected]>

* Apply suggestions from code review

Co-authored-by: David Yu <[email protected]>
Co-authored-by: Jeff Boruszak <[email protected]>

---------

Co-authored-by: Jeff Boruszak <[email protected]>
Co-authored-by: David Yu <[email protected]>

* Delete check-legacy-links-format.yml (#17647)

* docs: Reference doc updates for permissive mTLS settings (#17371)

* Reference doc updates for permissive mTLS settings
* Document config entry filtering
* Fix minor doc errors (double slashes in link url paths)

---------

Co-authored-by: trujillo-adam <[email protected]>

* Add generic experiments configuration and use it to enable catalog v2 resources (#17604)

* Add generic experiments configuration and use it to enable catalog v2 resources

* Run formatting with -s as CI will validate that this has been done

* api-gateway: stop adding all header filters to virtual host when generating xDS (#17644)

* Add header filter to api-gateway xDS golden test

* Stop adding all header filters to virtual host when generating xDS for api-gateway

* Regenerate xDS golden file for api-gateway w/ header filter

* fix: add agent info reporting log (#17654)

* Add new Consul 1.16 docs (#17651)

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16

updated docs for rate limiting for IP addresses - 1.16

* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting

Add docs for enterprise utilization reporting

* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16

Docs/envoy ext 1.16

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16

updated docs for rate limiting for IP addresses - 1.16

* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting

Add docs for enterprise utilization reporting

* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16

Docs/envoy ext 1.16

* fix build errors

---------

Co-authored-by: trujillo-adam <[email protected]>

* Default `ProxyType` for builtin extensions (#17657)

* Post 1.16.0-rc1 updates (#17663)

- Update changelog to include new entries from release
- Update submodule versions to latest published

* Update service-defaults.mdx (#17656)

* docs: Sameness Groups (#17628)

* port from enterprise branch

* Apply suggestions from code review

Co-authored-by: shanafarkas <[email protected]>

* Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx

* next steps

* Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx

Co-authored-by: trujillo-adam <[email protected]>

---------

Co-authored-by: shanafarkas <[email protected]>
Co-authored-by: trujillo-adam <[email protected]>

* Remove "BETA" marker from config entries (#17670)

* CAPIgw for K8s installation updates for 1.16 (#17627)

* trimmed CRD step and reqs from installation

* updated tech specs

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <[email protected]>
Co-authored-by: Jeff Apple <[email protected]>

* added upgrade instruction

* removed tcp port req

* described downtime and DT-less upgrades

* applied additional review feedback

---------

Co-authored-by: Jeff Boruszak <[email protected]>
Co-authored-by: Jeff Apple <[email protected]>

* additional feedback on API gateway upgrades (#17677)

* additional feedback

* Update website/content/docs/api-gateway/upgrades.mdx

Co-authored-by: Jeff Apple <[email protected]>

---------

Co-authored-by: Jeff Apple <[email protected]>

* docs: JWT Authorization for intentions (#17643)

* Initial page/nav creation

* configuration entry reference page

* Usage + fixes

* service intentions page

* usage

* description

* config entry updates

* formatting fixes

* Update website/content/docs/connect/config-entries/service-intentions.mdx

Co-authored-by: Paul Glass <[email protected]>

* service intentions review fixes

* Overview page review fixes

* Apply suggestions from code review

Co-authored-by: trujillo-adam <[email protected]>

---------

Co-authored-by: Paul Glass <[email protected]>
Co-authored-by: trujillo-adam <[email protected]>

* docs: minor fixes to JWT auth docs (#17680)

* Fixes

* service intentions fixes

* Fix two WAL metrics in docs/agent/telemetry.mdx (#17593)

* updated failover for k8s w-tproxy page title (#17683)

* Add release notes 1.16 rc (#17665)

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16

updated docs for rate limiting for IP addresses - 1.16

* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting

Add docs for enterprise utilization reporting

* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16

Docs/envoy ext 1.16

* Add release notes for 1.16-rc

* Add consul-e license utlization reporting

* Update with rc absolute links

* Update with rc absolute links

* fix typo

* Apply suggestions from code review

Co-authored-by: trujillo-adam <[email protected]>

* Update to use callout component

* address typo

* docs: FIPS 140-2 Compliance (#17668)

* Page + nav + formatting

* link fix

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <[email protected]>

* link fix

* Apply suggestions from code review

Co-authored-by: Jeff Apple <[email protected]>

* Update website/content/docs/enterprise/fips.mdx

---------

Co-authored-by: trujillo-adam <[email protected]>
Co-authored-by: Jeff Apple <[email protected]>

* fix apigw install values file

* fix typos in release notes

---------

Co-authored-by: trujillo-adam <[email protected]>
Co-authored-by: Jeff Boruszak <[email protected]>
Co-authored-by: Jeff Apple <[email protected]>

* fix release notes links (#17687)

* adding redirects for tproxy and envoy extensions (#17688)

* adding redirects

* Apply suggestions from code review

* Fix FIPS copy (#17691)

* fix release notes links

* fix typos on fips docs

* [NET-4107][Supportability] Log Level set to TRACE and duration set to 5m for consul-debug (#17596)

* changed duration to 5 mins and log level to trace

* documentation update

* change log

* ENT merge of ext-authz extension updates (#17684)

* docs: Update default values for Envoy extension proxy types (#17676)

* fix: stop peering delete routine on leader loss (#17483)

* Refactor disco chain prioritize by locality structs (#17696)

This includes prioritize by localities on disco chain targets rather than
resolvers, allowing different targets within the same partition to have
different policies.

* agent: remove agent cache dependency from service mesh leaf certificate management (#17075)

* agent: remove agent cache dependency from service mesh leaf certificate management

This extracts the leaf cert management from within the agent cache.

This code was produced by the following process:

1. All tests in agent/cache, agent/cache-types, agent/auto-config,
   agent/consul/servercert were run at each stage.

    - The tests in agent matching .*Leaf were run at each stage.

    - The tests in agent/leafcert were run at each stage after they
      existed.

2. The former leaf cert Fetch implementation was extracted into a new
   package behind a "fake RPC" endpoint to make it look almost like all
   other cache type internals.

3. The old cache type was shimmed to use the fake RPC endpoint and
   generally cleaned up.

4. I selectively duplicated all of Get/Notify/NotifyCallback/Prepopulate
   from the agent/cache.Cache implementation over into the new package.
   This was renamed as leafcert.Manager.

    - Code that was irrelevant to the leaf cert type was deleted
      (inlining blocking=true, refresh=false)

5. Everything that used the leaf cert cache type (including proxycfg
   stuff) was shifted to use the leafcert.Manager instead.

6. agent/cache-types tests were moved and gently replumbed to execute
   as-is against a leafcert.Manager.

7. Inspired by some of the locking changes from derek's branch I split
   the fat lock into N+1 locks.

8. The waiter chan struct{} was eventually replaced with a
   singleflight.Group around cache updates, which was likely the biggest
   net structural change.

9. The awkward two layers or logic produced as a byproduct of marrying
   the agent cache management code with the leaf cert type code was
   slowly coalesced and flattened to remove confusion.

10. The .*Leaf tests from the agent package were copied and made to work
    directly against a leafcert.Manager to increase direct coverage.

I have done a best effort attempt to port the previous leaf-cert cache
type's tests over in spirit, as well as to take the e2e-ish tests in the
agent package with Leaf in the test name and copy those into the
agent/leafcert package to get more direct coverage, rather than coverage
tangled up in the agent logic.

There is no net-new test coverage, just coverage that was pushed around
from elsewhere.

* [core]: Pin github action workflows (#17695)

* docs: missing changelog for _5517 (#17706)

* add enterprise notes for IP-based rate limits (#17711)

* add enterprise notes for IP-based rate limits

* Apply suggestions from code review

Co-authored-by: Tu Nguyen <[email protected]>
Co-authored-by: David Yu <[email protected]>

* added bolded 'Enterprise' in list items.

---------

Co-authored-by: Tu Nguyen <[email protected]>
Co-authored-by: David Yu <[email protected]>

* Update compatibility.mdx (#17713)

* Remove extraneous version info for Config entries (#17716)

* Update terminating-gateway.mdx
* Update exported-services.mdx
* Update mesh.mdx

* fix: typo in link to section (#17527)

Co-authored-by: trujillo-adam <[email protected]>

* Bump Alpine to 3.18 (#17719)

* Update Dockerfile

* Create 17719.txt

* NET-1825: New ACL token creation docs (#16465)

Co-authored-by: trujillo-adam <[email protected]>
Co-authored-by: Jared Kirschner <[email protected]>

* [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <[email protected]>

* nits

* updated docs

---------

Co-authored-by: Semir Patel <[email protected]>

* OSS merge: Update error handling login when applying extensions (#17740)

* Bump atlassian/gajira-transition from 3.0.0 to 3.0.1 (#17741)

Bumps [atlassian/gajira-transition](https://github.com/atlassian/gajira-transition) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/atlassian/gajira-transition/releases)
- [Commits](atlassian/gajira-transition@4749176...38fc9cd)

---
updated-dependencies:
- dependency-name: atlassian/gajira-transition
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add truncation to body (#17723)

* docs: Failover overview minor fix (#17743)

* Incorrect symbol

* Clarification

* slight edit for clarity

* docs - update Envoy and Dataplane compat matrix (#17752)

* Update envoy.mdx

added more detail around default versus other compatible versions

* validate localities on agent configs and registration endpoints (#17712)

* Updated docs added explanation. (#17751)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <[email protected]>

* nits

* updated docs

* explanation added

---------

Co-authored-by: Semir Patel <[email protected]>

* Update index.mdx (#17749)

* added redirects and updated links (#17764)

* Add transparent proxy enhancements changelog (#17757)

* docs - remove use of consul leave during upgrade instructions (#17758)

* Fix issue with streaming service health watches. (#17775)

Fix issue with streaming service health watches.

This commit fixes an issue where the health streams were unaware of service
export changes. Whenever an exported-services config entry is modified, it is
effectively an ACL change.

The bug would be triggered by the following situation:

- no services are exported
- an upstream watch to service X is spawned
- the streaming backend filters out data for service X (due to lack of exports)
- service X is finally exported

In the situation above, the streaming backend does not trigger a refresh of its
data.  This means that any events that were supposed to have been received prior
to the export are NOT backfilled, and the watches never see service X spawning.

We currently have decided to not trigger a stream refresh in this situation due
to the potential for a thundering herd effect (touching exports would cause a
re-fetch of all watches for that partition, potentially).  Therefore, a local
blocking-query approach was added by this commit for agentless.

It's also worth noting that the streaming subscription is currently bypassed
most of the time with agentful, because proxycfg has a `req.Source.Node != ""`
which prevents the `streamingEnabled` check from passing.  This means that while
agents should technically have this same issue, they don't experience it with
mesh health watches.

Note that this is a temporary fix that solves the issue for proxycfg, but not
service-discovery use cases.

* Property Override validation improvements (#17759)

* Reject inbound Prop Override patch with Services

Services filtering is only supported for outbound TrafficDirection patches.

* Improve Prop Override unexpected type validation

- Guard against additional invalid parent and target types
- Add specific error handling for Any fields (unsupported)

* Fixes (#17765)

* Update license get explanation (#17782)

This PR is to clarify what happens if the license get command is run on a follower if the leader hasn't been updated with a newer license.

* Add Patch index to Prop Override validation errors (#17777)

When a patch is found invalid, include its index for easier debugging
when multiple patches are provided.

* Stop referenced jwt providers from being deleted (#17755)

* Stop referenced jwt providers from being deleted

* Implement a Catalog Controllers Lifecycle Integration Test (#17435)

* Implement a Catalog Controllers Lifecycle Integration Test

* Prevent triggering the race detector.

This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.

* HCP Add node id/name to config (#17750)

* Catalog V2 Container Based Integration Test (#17674)

* Implement the C