…Version / Datacenter into release/1.15.x (#18639)

* Reference hashicorp/consul instead of consul for Docker image (#17914)

* Reference hashicorp/consul instead of consul for Docker image

* Update Make targets that pull consul directly

* Update Consul K8s Upgrade Doc Updates (#17921)

Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x.

* Update sameness-group.mdx (#17915)

* Update create-sameness-groups.mdx (#17927)

* deps: coredns v1.10.1 (#17912)

* Ensure RSA keys are at least 2048 bits in length (#17911)

* Ensure RSA keys are at least 2048 bits in length

* Add changelog

* update key length check for FIPS compliance

* Fix no new variables error and failing to return when error exists from
validating

* clean up code for better readability

* actually return value

* tlsutil: Fix check TLS configuration (#17481)

* tlsutil: Fix check TLS configuration
* Rewording docs.
* Update website/content/docs/services/configuration/checks-configuration-reference.mdx
Co-authored-by: trujillo-adam <[email protected]>
* Fix typos and add changelog entry.
---------

Co-authored-by: trujillo-adam <[email protected]>

* docs: Deprecations for connect-native SDK and specific connect native APIs (#17937)

* Update v1_16_x.mdx
* Update connect native golang page

---------

Co-authored-by: trujillo-adam <[email protected]>

* Revert "Add workflow to verify linux release packages (#17904)" (#17942)

This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5.

* Fixes Secondary ConnectCA update (#17846)

This fixes a bug that was identified which resulted in subsequent
ConnectCA configuration update not to persist in the cluster.

* fixing typo in link to jwt-validations-with-intentions doc (#17955)

* Fix streaming backend link (#17958)

* Fix streaming backend link
* Update health.mdx

* Dynamically create jwks clusters for jwt-providers (#17944)

* website: remove deprecated agent rpc docs (#17962)

* Fix missing BalanceOutboundConnections in v2 catalog. (#17964)

* feature - [NET - 4005]  - [Supportability] Reloadable Configuration - enable_debug (#17565)

* # This is a combination of 9 commits.
# This is the 1st commit message:

init without tests

# This is the commit message #2:

change log

# This is the commit message #3:

fix tests

# This is the commit message #4:

fix tests

# This is the commit message #5:

added tests

# This is the commit message #6:

change log breaking change

# This is the commit message #7:

removed breaking change

# This is the commit message #8:

fix test

# This is the commit message #9:

keeping the test behaviour same

* # This is a combination of 12 commits.
# This is the 1st commit message:

init without tests

# This is the commit message #2:

change log

# This is the commit message #3:

fix tests

# This is the commit message #4:

fix tests

# This is the commit message #5:

added tests

# This is the commit message #6:

change log breaking change

# This is the commit message #7:

removed breaking change

# This is the commit message #8:

fix test

# This is the commit message #9:

keeping the test behaviour same

# This is the commit message #10:

made enable debug atomic bool

# This is the commit message #11:

fix lint

# This is the commit message #12:

fix test true enable debug

* parent 10f500e895d92cc3691ade7b74a33db755d22039
author absolutelightning <[email protected]> 1687352587 +0530
committer absolutelightning <[email protected]> 1687352592 +0530

init without tests

change log

fix tests

fix tests

added tests

change log breaking change

removed breaking change

fix test

keeping the test behaviour same

made enable debug atomic bool

fix lint

fix test true enable debug

using enable debug in agent as atomic bool

test fixes

fix tests

fix tests

added update on correct locaiton

fix tests

fix reloadable config enable debug

fix tests

fix init and acl 403

* revert commit

* Fix formatting codeblocks on APIgw docs (#17970)

* fix formatting codeblocks

* remove unnecessary indents

* Remove POC code (#17974)

* update doc (#17910)

* update doc

* update link

* Remove duplicate and unused newDecodeConfigEntry func (#17979)

* docs: samenessGroup YAML examples (#17984)

* configuration entry syntax

* Example config

* Add changelog entry for 1.16.0 (#17987)

* Fix typo (#17198)

servcies => services

* Expose JWKS cluster config through JWTProviderConfigEntry (#17978)

* Expose JWKS cluster config through JWTProviderConfigEntry

* fix typos, rename trustedCa to trustedCA

* Integration test for ext-authz Envoy extension (#17980)

* Fix incorrect protocol for transparent proxy upstreams. (#17894)

This PR fixes a bug that was introduced in:
https://github.com/hashicorp/consul/pull/16021

A user setting a protocol in proxy-defaults would cause tproxy implicit
upstreams to not honor the upstream service's protocol set in its
`ServiceDefaults.Protocol` field, and would instead always use the
proxy-defaults value.

Due to the fact that upstreams configured with "tcp" can successfully contact
upstream "http" services, this issue was not recognized until recently (a
proxy-defaults with "tcp" and a listening service with "http" would make
successful requests, but not the opposite).

As a temporary work-around, users experiencing this issue can explicitly set
the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should
take precedence.

The fix in this PR removes the proxy-defaults protocol from the wildcard
upstream that tproxy uses to configure implicit upstreams. When the protocol
was included, it would always overwrite the value during discovery chain
compilation, which was not correct. The discovery chain compiler also consumes
proxy defaults to determine the protocol, so simply excluding it from the
wildcard upstream config map resolves the issue.

* feat: include nodes count in operator usage endpoint and cli command (#17939)

* feat: update operator usage api endpoint to include nodes count

* feat: update operator usange cli command to includes nodes count

* [OSS] Improve Gateway Test Coverage of Catalog Health (#18011)

* fix(cli): remove failing check from 'connect envoy' registration for api gateway

* test(integration): add tests to check catalog statsus of gateways on startup

* remove extra sleep comment

* Update test/integration/consul-container/libs/assert/service.go

* changelog

* Fixes Traffic rate limitting docs (#17997)

* Fix removed service-to-service peering links (#17221)

* docs: fix removed service-to-service peering links

* docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam)

---------

Co-authored-by: trujillo-adam <[email protected]>

* docs: Sameness "beta" warning (#18017)

* Warning updates

* .x

* updated typo in tab heading (#18022)

* updated typo in tab heading

* updated tab group typo, too

* Document that DNS lookups can target cluster peers (#17990)

Static DNS lookups, in addition to explicitly targeting a datacenter,
can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation.

The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25

The biggest change in this is to adjust the standard lookup syntax to tie
`.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups.

Co-authored-by: David Yu <[email protected]>

* Add first integration test for jwt auth with intention (#18005)

* fix stand-in text for name field (#18030)

* removed sameness conf entry from failover nav (#18033)

* docs - add service sync annotations and k8s service weight annotation (#18032)

* Docs for https://github.com/hashicorp/consul-k8s/pull/2293
* remove versions for enterprise features since they are old

---------

Co-authored-by: Tu Nguyen <[email protected]>

* docs - add jobs use case for service mesh k8s (#18037)

* docs - add jobs use case for service mesh k8s
* add code blocks

* address feedback (#18045)

* Add verify server hostname to tls default (#17155)

* [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024)

* fix(connect): set initial_fetch_time to wait indefinitely

* changelog

* PR feedback 1

* ui: fix typos for peer service imports (#17999)

* test: fix FIPS inline cert test message (#18076)

* Fix a couple typos in Agent Telemetry Metrics docs (#18080)

* Fix metrics docs

* Add changelog

Signed-off-by: josh <[email protected]>

---------

Signed-off-by: josh <[email protected]>

* docs updates - cluster peering and virtual services (#18069)

* Update route-to-virtual-services.mdx
* Update establish-peering.mdx

* Update service-mesh-compare.mdx (#17279)

grammar change

* Update helm docs on main (#18085)

* ci: use gotestsum v1.10.1 [NET-4042] (#18088)

* Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075)

* Update proxy lifecycle annotations and consul-dataplane flags

* Pass configured role name to Vault for AWS auth in Connect CA (#17885)

* Docs for dataplane upgrade on k8s (#18051)

* Docs for dataplane upgrade on k8s

---------

Co-authored-by: David Yu <[email protected]>
Co-authored-by: trujillo-adam <[email protected]>

* docs - update upgrade index page to not recommend consul leave. (#18100)

* Displays Consul version of each nodes in UI nodes section (#17754)

* update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members

* update test cases TestUINodes, TestUINodeInfo

* added nil check for map

* add consul-version in local agent node metadata

* get consul version from serf member and add this as node meta in catalog register request

* updated ui mock response to include consul versions as node meta

* updated ui trans and added version as query param to node list route

* updates in ui templates to display consul version with filter and sorts

* updates in ui - model class, serializers,comparators,predicates for consul version feature

* added change log for Consul Version Feature

* updated to get version from consul service, if for some reason not available from serf

* updated changelog text

* updated dependent testcases

* multiselection version filter

* Update agent/consul/state/catalog.go

comments updated

Co-authored-by: Jared Kirschner <[email protected]>

---------

Co-authored-by: Jared Kirschner <[email protected]>

* api gw 1.16 updates (#18081)

* api gw 1.16 updates

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <[email protected]>

* update CodeBlockConfig filename

* Apply suggestions from code review

Co-authored-by: trujillo-adam <[email protected]>
Co-authored-by: Jeff Boruszak <[email protected]>

* remove non-standard intentions page

* Update website/content/docs/api-gateway/configuration/index.mdx

Co-authored-by: trujillo-adam <[email protected]>

---------

Co-authored-by: Jeff Boruszak <[email protected]>
Co-authored-by: trujillo-adam <[email protected]>

* [NET-4103] ci: build s390x (#18067)

* ci: build s390x

* ci: test s390x

* ci: dev build s390x

* no GOOS

* ent only

* build: publish s390x

* fix syntax error

* fix syntax error again

* fix syntax error again x2

* test branch

* Move s390x conditionals to step level

* remove test branch

---------

Co-authored-by: emilymianeil <[email protected]>

* :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116)

Just a typo in the docs.

* Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991)

Configuration that previously was inlined into the Upstreams resource
applies to both explicit and implicit upstreams and so it makes sense to
split it out into its own resource.

It also has other minor changes:
- Renames `proxy.proto` proxy_configuration.proto`
- Changes the type of `Upstream.destination_ref` from `pbresource.ID` to
`pbresource.Reference`
- Adds comments to fields that didn't have them

* [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6.  Pin go to 1.20.5. (#18124)

### Description
The following jobs started failing when go 1.20.6 was released:
- `go-test-api-1-19`
- `go-test-api-1-20`
- `compatibility-integration-tests`
- `upgrade-integration-tests`

`compatibility-integration-tests` and `compatibility-integration-tests`
to this testcontainers issue:
https://github.com/testcontainers/testcontainers-go/issues/1359. This
issue calls for testcontainers to release a new version when one of
their dependencies is fixed. When that is done, we will unpin the go
versions in `compatibility-integration-tests` and
`compatibility-integration-tests`.

### Testing & Reproduction steps

See these jobs broken in CI and then see them work with this PR.

---------

Co-authored-by: Chris Thain <[email protected]>

* Add ingress gateway deprecation notices to docs (#18102)

### Description

This adds notices, that ingress gateway is deprecated, to several places
in the product docs where ingress gateway is the topic.

### Testing & Reproduction steps

Tested with a local copy of the website.

### Links

Deprecation of ingress gateway was announced in the Release Notes for
Consul 1.16 and Consul-K8s 1.2. See:

[https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated
)

[https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated)

### PR Checklist

* [N/A] updated test coverage
* [X] external facing docs updated
* [X] appropriate backport labels added
* [X] not a security concern

---------

Co-authored-by: trujillo-adam <[email protected]>

* Add docs for jwt cluster configuration (#18004)

### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Add jwt-provider docs for jwks cluster configuration. The
configuration was added here:
https://github.com/hashicorp/consul/pull/17978

* Docs: fix unmatched bracket for health checks page (#18134)

* NET-4657/add resource service client (#18053)

### Description

<!-- Please describe why you're making this change, in plain English.
-->
Dan had already started on this
[task](https://github.com/hashicorp/consul/pull/17849) which is needed
to start building the HTTP APIs. This just needed some cleanup to get it
ready for review.

Overview:

- Rename `internalResourceServiceClient` to
`insecureResourceServiceClient` for name consistency
- Configure a `secureResourceServiceClient` with auth enabled

### PR Checklist

* [ ] ~updated test coverage~
* [ ] ~external facing docs updated~
* [x] appropriate backport labels added
* [ ] ~not a security concern~

* Fix bug with Vault CA provider (#18112)

Updating RootPKIPath but not IntermediatePKIPath would not update 
leaf signing certs with the new root. Unsure if this happens in practice 
but manual testing showed it is a bug that would break mesh and agent 
connections once the old root is pruned.

* [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129)

### Description

This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.

go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.

For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.

[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)

### Testing & Reproduction steps

Check CI tests.

### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern

* add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139)

### Description

This is to correct a code problem because this assumes all segments, but
when you get to Enterprise, you can be in partition that is not the
default partition, in which case specifying all segments does not
validate and fails. This is to correct the setting of this filter with
`AllSegments` to `true` to only occur when in the the `default`
partition.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

### Links

<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

### PR Checklist

* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern

* chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130)

* re org resource type registry (#18133)

* fix: update delegateMock used in ENT (#18149)

### Description

<!-- Please describe why you're making this change, in plain English.
-->
The mock is used in `http_ent_test` file which caused lint failures. For
OSS->ENT parity adding the same change here.

### Links

<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

Identified in OSS->ENT [merge
PR](https://github.com/hashicorp/consul-enterprise/pull/6328)

### PR Checklist

* [ ] ~updated test coverage~
* [ ] ~external facing docs updated~
* [x] appropriate backport labels added
* [ ] ~not a security concern~

* Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062)

### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.

This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.

- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
  protocol = "http"
}
```
- Create two providers 
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"

JSONWebKeySet = {
    Local = {
     JWKS = "eyJrZXlzIjog....."
    }
}
```

```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"

JSONWebKeySet = {
   Local = {
     JWKS = "eyJr