Skip to content

perf setup update: add grafana, misc fixes #5028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ldemailly merged 7 commits into from
Apr 24, 2018
Merged

perf setup update: add grafana, misc fixes #5028

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e823105
need git pull --tags to get latest_release movement, use DUR variable…
ldemailly Apr 17, 2018
38779eb
Merge branch 'master' into perf_update2
ldemailly Apr 17, 2018
dfcc7d6
Merge branch 'master' into perf_update2
ldemailly Apr 20, 2018
5ac4115
Add grafana ingress
ldemailly Apr 21, 2018
1fb0e08
Add annotation for no mtls in helm template
ldemailly Apr 23, 2018
55124db
Merge branch 'master' into perf_update2
ldemailly Apr 23, 2018
079eef3
From 0.8 prometheus is already in the yaml
ldemailly Apr 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions install/kubernetes/helm/istio/charts/grafana/templates/service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ kind: Service
metadata:
name: grafana
namespace: {{ .Release.Namespace }}
annotations:
auth.istio.io/{{ .Values.service.externalPort }}: NONE
Copy link
Copy Markdown
Member Author

@ldemailly ldemailly Apr 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@douglas-reid : here, PTAL

labels:
app: {{ template "grafana.name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
Expand Down
2 changes: 2 additions & 0 deletions install/kubernetes/templates/addons/grafana.yaml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ kind: Service
metadata:
name: grafana
namespace: {ISTIO_NAMESPACE}
annotations:
Copy link
Copy Markdown
Contributor

@douglas-reid douglas-reid Apr 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: this is the wrong place to edit the templates.

you want to edit: https://github.com/istio/istio/blob/master/install/kubernetes/helm/istio/charts/grafana/templates/deployment.yaml

Copy link
Copy Markdown
Member Author

@ldemailly ldemailly Apr 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I know, I commented in #5091 - added the other one above, thanks for the pointer
(while we have 2 I prefer to make both changes to increase the chances that it does work)

Copy link
Copy Markdown
Contributor

@douglas-reid douglas-reid Apr 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. This sounds reasonable. We desperately need to untangle this mess.

auth.istio.io/3000: NONE
spec:
ports:
- port: 3000
Expand Down
63 changes: 52 additions & 11 deletions tools/setup_perf_cluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@ else
fi
fi


function update_gcp_opts() {
export GCP_OPTS="--project $PROJECT --zone $ZONE"
}
Expand Down Expand Up @@ -101,7 +100,7 @@ function run_on_vm() {

function setup_vm() {
Execute gcloud compute instances add-tags $VM_NAME $GCP_OPTS --tags https-server
run_on_vm '(sudo add-apt-repository ppa:gophers/archive > /dev/null && sudo apt-get update > /dev/null && sudo apt-get upgrade --no-install-recommends -y && sudo apt-get install --no-install-recommends -y golang-1.9-go make && mv .bashrc .bashrc.orig && (echo "export PATH=/usr/lib/go-1.9/bin:\$PATH:~/go/bin"; cat .bashrc.orig) > ~/.bashrc ) < /dev/null'
run_on_vm '(sudo add-apt-repository ppa:gophers/archive > /dev/null && sudo apt-get update > /dev/null && sudo apt-get upgrade --no-install-recommends -y && sudo apt-get install --no-install-recommends -y golang-1.10-go make && mv .bashrc .bashrc.orig && (echo "export PATH=/usr/lib/go-1.9/bin:\$PATH:~/go/bin"; cat .bashrc.orig) > ~/.bashrc ) < /dev/null'
}

function setup_vm_firewall() {
Expand All @@ -113,7 +112,7 @@ function delete_vm_firewall() {
}

function update_fortio_on_vm() {
run_on_vm 'go get istio.io/fortio && cd go/src/istio.io/fortio && git fetch && git checkout latest_release && go build -o ~/go/bin/fortio -ldflags "-X istio.io/fortio/version.tag=$(git describe --tag --match v\*) -X istio.io/fortio/version.buildInfo=$(git rev-parse HEAD)" . && sudo setcap 'cap_net_bind_service=+ep' `which fortio` && fortio version'
run_on_vm 'go get istio.io/fortio && cd go/src/istio.io/fortio && git fetch --tags && git checkout latest_release && make submodule-sync && go build -o ~/go/bin/fortio -ldflags "-X istio.io/fortio/version.tag=$(git describe --tag --match v\*) -X istio.io/fortio/version.buildInfo=$(git rev-parse HEAD)" . && sudo setcap 'cap_net_bind_service=+ep' `which fortio` && fortio version'
Copy link
Copy Markdown
Contributor

@rkpagadala rkpagadala Apr 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make submodule-sync ??? I thought this is no longer needed.

Copy link
Copy Markdown
Member Author

@ldemailly ldemailly Apr 23, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, on fortio

I restored this that got broken by the global remove of "submodule-sync"

}

function run_fortio_on_vm() {
Expand All @@ -134,6 +133,12 @@ function install_istio() {
Execute sh -c 'sed -e "s/_debug//g" install/kubernetes/istio-auth.yaml | egrep -v -e "- (-v|\"2\")" | kubectl apply -f -'
}

function install_istio_addons() {
Copy link
Copy Markdown
Contributor

@Tahler Tahler Apr 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function isn't called, are you sure you meant to add it?

Copy link
Copy Markdown
Member Author

@ldemailly ldemailly Apr 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's the wip part, I will add it to setup all but need to manually test first, if you have time you could get that branch and give it a shot

Copy link
Copy Markdown
Member Author

@ldemailly ldemailly Apr 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed now

# Starting in 0.8, prometheus is already in istio-auth.yaml
# Execute sh -c 'kubectl apply -f install/kubernetes/addons/prometheus.yaml'
Execute sh -c 'kubectl apply -f install/kubernetes/addons/grafana.yaml'
}

# assumes run from istio/ (or release) directory
function delete_istio() {
# Use the non debug ingress and remove the -v "2"
Expand Down Expand Up @@ -180,6 +185,33 @@ function get_fortio_k8s_ip() {
echo "+++ In k8s fortio external ip: http://$FORTIO_K8S_IP:8080/fortio/"
}

function setup_istio_addons_ingress() {
cat <<_EOF_ | kubectl apply -n istio-system -f -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: istio
name: istio-ingress
spec:
rules:
- http:
paths:
- path: /d/.*
backend:
serviceName: grafana
servicePort: http
- path: /public/.*
backend:
serviceName: grafana
servicePort: http
- path: /api/.*
backend:
serviceName: grafana
servicePort: http
_EOF_
}

# Doesn't work somehow...
function setup_non_istio_ingress2() {
cat <<_EOF_ | kubectl apply -n fortio -f -
Expand Down Expand Up @@ -239,13 +271,20 @@ function get_istio_ingress_ip() {
done

echo "+++ In k8s istio ingress: http://$ISTIO_INGRESS_IP/fortio1/fortio/ and fortio2"
echo "+++ In k8s grafana: http://$ISTIO_INGRESS_IP/d/1/"
}

# Set default QPS to max qps
if [[ $QPS == "" ]]; then
if [ -z ${QPS+x} ] || [ $QPS == "" ]; then
echo "Setting default qps"
QPS=-1
fi


# Set default run duration to 30s
if [ -z ${DUR+x} ] || [ $DUR == "" ]; then
DUR="30s"
fi

function get_istio_version() {
kubectl describe pods -n istio|grep /proxy:|head -1 | awk -F: '{print $3}'
}
Expand Down Expand Up @@ -279,32 +318,32 @@ function label_escape() {

function run_fortio_test1() {
echo "Using default loadbalancer, no istio:"
Execute curl "$VM_URL?json=on&save=on&qps=$QPS&t=30s&c=48&load=Start&url=http://$FORTIO_K8S_IP:8080/echo"
Execute curl "$VM_URL?json=on&save=on&qps=$QPS&t=$DUR&c=48&load=Start&url=http://$FORTIO_K8S_IP:8080/echo"
}
function run_fortio_test2() {
echo "Using default ingress, no istio:"
Execute curl "$VM_URL?json=on&save=on&qps=$QPS&t=30s&c=48&load=Start&url=http://$K8S_INGRESS_IP/echo"
Execute curl "$VM_URL?json=on&save=on&qps=$QPS&t=$DUR&c=48&load=Start&url=http://$K8S_INGRESS_IP/echo"
}

function run_fortio_test_istio_ingress1() {
get_json_file_name "ingress to s1"
echo "Using istio ingress to fortio1, saving to $FNAME"
ExecuteEval curl -s "$VM_URL?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=30s\&c=48\&load=Start\&url=http://$ISTIO_INGRESS_IP/fortio1/echo" \| tee $FNAME.json \| grep ActualQPS
ExecuteEval curl -s "$VM_URL?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=$DUR\&c=48\&load=Start\&url=http://$ISTIO_INGRESS_IP/fortio1/echo" \| tee $FNAME.json \| grep ActualQPS
}
function run_fortio_test_istio_ingress2() {
get_json_file_name "ingress to s2"
echo "Using istio ingress to fortio2, saving to $FNAME"
ExecuteEval curl -s "$VM_URL?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=30s\&c=48\&load=Start\&url=http://$ISTIO_INGRESS_IP/fortio2/echo" \| tee $FNAME.json \| grep ActualQPS
ExecuteEval curl -s "$VM_URL?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=$DUR\&c=48\&load=Start\&url=http://$ISTIO_INGRESS_IP/fortio2/echo" \| tee $FNAME.json \| grep ActualQPS
}
function run_fortio_test_istio_1_2() {
get_json_file_name "s1 to s2"
echo "Using istio f1 to f2, saving to $FNAME"
ExecuteEval curl -s "http://$ISTIO_INGRESS_IP/fortio1/fortio/?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=30s\&c=48\&load=Start\&url=http://echosrv2:8080/echo" \| tee $FNAME.json \| grep ActualQPS
ExecuteEval curl -s "http://$ISTIO_INGRESS_IP/fortio1/fortio/?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=$DUR\&c=48\&load=Start\&url=http://echosrv2:8080/echo" \| tee $FNAME.json \| grep ActualQPS
}
function run_fortio_test_istio_2_1() {
get_json_file_name "s2 to s1"
echo "Using istio f2 to f1, saving to $FNAME"
ExecuteEval curl -s "http://$ISTIO_INGRESS_IP/fortio2/fortio/?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=30s\&c=48\&load=Start\&url=http://echosrv1:8080/echo" \| tee $FNAME.json \| grep ActualQPS
ExecuteEval curl -s "http://$ISTIO_INGRESS_IP/fortio2/fortio/?labels=$LABELS\&json=on\&save=on\&qps=$QPS\&t=$DUR\&c=48\&load=Start\&url=http://echosrv1:8080/echo" \| tee $FNAME.json \| grep ActualQPS
}

# Run canonical perf tests.
Expand Down Expand Up @@ -410,6 +449,8 @@ function setup_istio_all() {
install_istio_svc
install_istio_ingress_rules
install_istio_cache_busting_rule
install_istio_addons
setup_istio_addons_ingress
}

function setup_cluster_all() {
Expand Down