In what area(s)?

/area autoscale

Describe the feature

Currently we un Activator as a deployment with an HPA autoscaler.
For Node Local Scheduling, we need to run Activator as a DaemonSet.
The work here is consisted of several steps:

1. Find the appropriate instance requirements.
   • Currently we're requesting 300m of CPU and 400MB of memory for each instance. This presumes running just a few instances.
   • Our integration tests explicitly ask for 2 instances.
   • For benchmarks we are asking for 10-15 depending on the test, though they are strongly over-provisioned there). On my cluster I have reliably achieved desired results with 4-5 instances per 1000RPS and 150 target pods with CC=1 on 15 node cluster.
   • If we're going to switch to a DaemonSet, we probably need to lower these number somewhat.
   • We need to measure against the benchmarks to make sure the current performance levels do no degrade.
2. Provide optional template that permits running Activator as DS in v0.13.
3. Switch DaemonSet as default and set HPA min limits to 0 and deployment replicas to 0 in v0.14.
4. [possibly] Remove the existing configuration in v0.15

/assign @vagababov
/cc mattmoor @markusthoemmes