Resolves: #259

Adds an option for the storage type of the provided ZK cluster. Ephemeral or Persistent.

If the user does not provide either option, then the Solr Operator will use whatever type is used by the Solr pods. (which defaults to ephemeral if none is provided).

Since pravega/zookeeper-operator#215 was merged, the Zookeeper Operator has had the ability to deploy Zookeeper with Ephemeral Storage. All we have to do is provide this as an option through the SolrCloud CRD, and solr helm chart.

This is a breaking change that requires a note in the Upgrade Notes, because the ZK storage now defaults to ephemeral if a user is using ephemeral storage for Solr. In previous versions of the Solr Operator, only persistent ZK storage is supported, so this is clearly a change to default deployments.

New options are found in:

SolrCloud CRD

spec:
  zookeeperRef:
    provided:
      persistence: {} # Always an option
      ephemeral: {} # New option introduced in this PR

Solr Helm Chart

zk:
  provided:
    storageType: "" # Explicitly set the type, since empty objects are ignored in helm. Not necessary to use
    persistence: {} # Always an option
    ephemeral: {} # New option introduced in this PR