emptyDir with medium: Memory ignores sizeLimit #63126
Description
Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
What happened:
The tmpfs volume that's created with emptyDir doesn't have the correct size limit. In fact, the tmpfs volume size is always half of the available RAM on the Kubernetes node, which is the default behavior on a Linux kernel (https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt). This implies that the size option was likely not provided when mounting a tmpfs. This is similar to the issue here: #48912.
Note: All the Kubernetes components have the necessary feature gates enabled.
What you expected to happen:
The tmpfs volume should respect the sizeLimit parameter, unless the sizeLimit parameter is not meant for tmpfs.
How to reproduce it (as minimally and precisely as possible):
Step 1: Create a deployment that with an emptyDir volume with medium set to Memory and sizeLimit set to 1Gi.
$ cat sample.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: test
spec:
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- mountPath: /scratch
name: scratch
volumes:
- name: scratch
emptyDir:
medium: Memory
sizeLimit: "1Gi"
$ kubectl apply -f sample.yaml
deployment "test" configured
Step 2: Inspect the volume mount and the size is not 1GB. Instead, the size is 126GB, because the Kubernetes node has 256GB of available RAM.
kubectl exec -it test-56f7bb87d5-j68n5 bash
root@test-56f7bb87d5-j68n5:/# mount -l | grep scratch
tmpfs on /scratch type tmpfs (rw,relatime)
root@test-56f7bb87d5-j68n5:/# df -h /scratch
Filesystem Size Used Avail Use% Mounted on
tmpfs 126G 0 126G 0% /scratch
Step 3: When exporting the deployment manifest file, the sizeLimit option is retained, demonstrating that the necessary feature gates were enabled.
kubectl get pods test-56f7bb87d5-j68n5 -o yaml --export
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
generateName: test-56f7bb87d5-
labels:
app: test
pod-template-hash: "1293664381"
ownerReferences:
- apiVersion: extensions/v1beta1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: test-56f7bb87d5
uid: 63b2a2d3-4815-11e8-aacf-ec0d9a749964
selfLink: /api/v1/namespaces/default/pods/test-56f7bb87d5-j68n5
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /scratch
name: scratch
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-qb9lt
readOnly: true
dnsPolicy: ClusterFirst
nodeName: <HOST>
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir:
medium: Memory
sizeLimit: 1Gi
name: scratch
- name: default-token-qb9lt
secret:
defaultMode: 420
secretName: default-token-qb9lt
status:
phase: Pending
qosClass: BestEffort
Anything else we need to know?:
Environment:
- Kubernetes version (use
kubectl version):
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
- Cloud provider or hardware configuration:
The api server, controller manager, scheduler, and kubelets have the following feature gates enabled:
--feature-gates=PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true,LocalStorageCapacityIsolation=true,DevicePlugins=true
The only feature gate that is needed is LocalStorageCapacityIsolation, but the other features are enabled for other reasons.
The Kubernetes nodes in this cluster have 256GB of RAM.
$ free -g
total used free shared buff/cache available
Mem: 251 2 238 2 10 245
- OS (e.g. from /etc/os-release):
cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.4 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.4 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
- Kernel (e.g.
uname -a):
uname -a
Linux <host> 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
- Install tools:
kubeadm.
- Others: