support quota set on overlay snapshot #5859

yylt · 2021-08-11T03:22:59Z

support set quota on overlay snapshot

as discussed in issue #3329

enable quota set by follows:

Make sure the root directory of containerd is mounted with 'pquota'
Set 'overlay' as the default snapshot and enable quota in 'overlay' config
Set the default quota size in CRI

config.toml like this

  [plugins.'io.containerd.cri.v1.runtime']
    enable_selinux = false
    default_snapshot_quota_size = '2M'   #+
...
  [plugins.'io.containerd.snapshotter.v1.overlayfs']
    root_path = ''
    upperdir_label = false
    sync_remove = false
    slow_chown = false
    mount_options = []
    enable_quota = true  #+

check containerd root mount info

# /etc/fstab
...
/dev/mapper/containerd_root  /var/lib/containerd  xfs     defaults,pquota        0 0

# mount ...
/dev/mapper/containerd_root on /var/lib/containerd type xfs (rw,relatime,prjquota)

check container had been set quota

xfs_quota -x -c "report -h" /var/lib/containerd/

Signed-off-by: Yang Yang [email protected]

k8s-ci-robot · 2021-08-11T03:23:08Z

Hi @yylt. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

theopenlab-ci · 2021-08-11T04:04:56Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 4m 52s (non-voting)
containerd-test-arm64 : SUCCESS in 6m 12s (non-voting)
containerd-integration-test-arm64 : SUCCESS in 24m 31s (non-voting)

AkihiroSuda · 2021-08-11T13:01:26Z

snapshots/overlay/overlayutils/fs_linux.go

+   See the License for the specific language governing permissions and
+   limitations under the License.
+*/
+


This file seems copied from Moby. Please clarify the origin of the code and the license in code comments.

AkihiroSuda · 2021-08-11T13:01:53Z

snapshots/quota/projectquota.go

+#define Q_XGETPQUOTA QCMD(Q_XGETQUOTA, PRJQUOTA)
+#endif
+*/
+import "C"


Can we avoid cgo

Can we avoid cgo

Done!

dmcgowan · 2021-08-11T18:34:02Z

Quota handling is a topic we should discuss at a high level first and possibly a good topic for a future community meeting. However, getting library support for setting project quotas across different backing FS makes sense to do now.

As an end feature, we need to discuss how it will be integrated with Kubernetes and how we can use it from our own tooling and go libraries. It seems backwards to have the cri layer here enable the quota then the snapshot set a static quota. I would think the quota would be set by the client adding a quota and the snapshotter enabling it via configuration and erroring out if the backing FS couldn't support it when enabled. The difficult part is figuring out from the client perspective whether quota is enabled so it knows whether it can avoid expensive ephemeral storage accounting (such as in the Kubelet).

lining2020x · 2021-11-11T02:48:47Z

pkg/cri/config/config.go

@@ -266,6 +266,9 @@ type PluginConfig struct {
 	// of being placed under the hardcoded directory /var/run/netns. Changing this setting requires
 	// that all containers are deleted.
 	NetNSMountsUnderStateDir bool `toml:"netns_mounts_under_state_dir" json:"netnsMountsUnderStateDir"`
+	// SupportSetQuota indicate to set quota on container read-write snapshot, when snapshot do not
+	// support quota set, it will do nothing.
+	SupportSetQuota bool `toml:"support_set_quota" json:"supportSetQuota"`


Maybe EnableRootfsQuota or EnableSetQuotasounds better.

pacoxu · 2022-06-09T05:35:03Z

The difficult part is figuring out from the client perspective whether quota is enabled so it knows whether it can avoid expensive ephemeral storage accounting (such as in the Kubelet).

In Kubernetes, there is a feature gate and a check method SupportsQuotas, here is the KEP "Quotas for Ephemeral Storage" kubernetes/enhancements#1029 from Kubernetes.
https://github.com/kubernetes/kubernetes/blob/226323178e23b4c476001266beab1e2f116b3879/pkg/volume/util/fsquota/common/quota_linux_common_impl.go#L180-L190

// SupportsQuotas determines whether the filesystem supports quotas.
func SupportsQuotas(mountpoint string, qType QuotaType) (bool, error) {
	data, err := runXFSQuotaCommand(mountpoint, "state -p")
	if err != nil {
		return false, err
	}
	if qType == FSQuotaEnforcing {
		return strings.Contains(data, "Enforcement: ON"), nil
	}
	return strings.Contains(data, "Accounting: ON"), nil
}

yylt · 2022-06-16T07:23:22Z

sorry for so long time later.

As an end feature, we need to discuss how it will be integrated with Kubernetes and how we can use it from our own tooling and go libraries. It seems backwards to have the cri layer here enable the quota then the snapshot set a static quota. I would think the quota would be set by the client adding a quota and the snapshotter enabling it via configuration and erroring out if the backing FS couldn't support it when enabled. The difficult part is figuring out from the client perspective whether quota is enabled so it knows whether it can avoid expensive ephemeral storage accounting (such as in the Kubelet).

it is good suggestion. now quota setter focus on overlay, and with pure go.

and update snapshots litter, add new option WithQuotaSize. mostly if the option not define, will use default size in overlayfs configurtion

such as:

  [plugins."io.containerd.snapshotter.v1.overlayfs"]
    root_path = ""
    quota_size = "128MB"

pacoxu

I tried to build a containerd to do some test. Too many quota is created in my env.

[root@daocloud ~]# xfs_quota -xc "report -a -p -h"  | head -n 100
Project quota on /var/lib/kubelet (/dev/sdc)
                        Blocks
Project ID   Used   Soft   Hard Warn/Grace
---------- ---------------------------------
#0           820K      0      0  00 [------]
volume1048590      0     8E     8E  00 [------]
volume1048605      0     8E     8E  00 [------]
volume1048607      0     8E     8E  00 [------]
volume1048610      0     8E     8E  00 [------]

Project quota on /var/lib/containerd (/dev/sdb)
                        Blocks
Project ID   Used   Soft   Hard Warn/Grace
---------- ---------------------------------
#0          10.2G      0      0  00 [------]
#2              0      0      0  00 [------]
#3              0    15G    15G  00 [------]
#4            12K    15G    15G  00 [------]
#5              0    15G    15G  00 [------]
#6              0    15G    15G  00 [------]
#7              0    15G    15G  00 [------]
#8              0    15G    15G  00 [------]
#9              0    15G    15G  00 [------]
#10             0    15G    15G  00 [------]
#11             0    15G    15G  00 [------]
#12             0    15G    15G  00 [------]
#13             0    15G    15G  00 [------]
#14             0    15G    15G  00 [------]
#15             0    15G    15G  00 [------]
#16             0    15G    15G  00 [------]
#17             0    15G    15G  00 [------]
#18             0    15G    15G  00 [------]
#19             0    15G    15G  00 [------]
#20             0    15G    15G  00 [------]
#21             0    15G    15G  00 [------]
#22             0    15G    15G  00 [------]
#23             0    15G    15G  00 [------]
#24            4K    15G    15G  00 [------]
#25             0    15G    15G  00 [------]
#26             0    15G    15G  00 [------]
#27             0    15G    15G  00 [------]
#28             0    15G    15G  00 [------]
#29            4K    15G    15G  00 [------]
#30             0    15G    15G  00 [------]
#31             0    15G    15G  00 [------]
#32             0    15G    15G  00 [------]
#33             0    15G    15G  00 [------]
#34             0    15G    15G  00 [------]
#35            4K    15G    15G  00 [------]
#36            4K    15G    15G  00 [------]
#37             0    15G    15G  00 [------]
#38          540K    15G    15G  00 [------]
#39             0    15G    15G  00 [------]
#40             0    15G    15G  00 [------]
#41             0    15G    15G  00 [------]
#42             0    15G    15G  00 [------]
#43             0    10G    10G  00 [------]
#44             0    10G    10G  00 [------]
#45             0    10G    10G  00 [------]
#46             0    10G    10G  00 [------]
#47             0    10G    10G  00 [------]
#48             0    10G    10G  00 [------]
#49             0    10G    10G  00 [------]
#50             0    10G    10G  00 [------]
#51             0    10G    10G  00 [------]
#52             0    10G    10G  00 [------]
#53             0    10G    10G  00 [------]
#54             0    10G    10G  00 [------]
#55             0    10G    10G  00 [------]
#56             0    10G    10G  00 [------]
#57             0    10G    10G  00 [------]
#58             0    10G    10G  00 [------]
#59             0    10G    10G  00 [------]
#60             0    10G    10G  00 [------]
#61             0    10G    10G  00 [------]
#62             0    10G    10G  00 [------]
#63             0    10G    10G  00 [------]

I set the size to 10G at first and 15G later.

[root@daocloud ~]# xfs_quota -xc "report -a -p -h"  | wc -l
1956507

pacoxu · 2022-06-23T08:02:42Z

snapshots/overlay/plugin/plugin.go

@@ -57,6 +62,15 @@ func init() {
 				oOpts = append(oOpts, overlay.WithUpperdirLabel)
 			}

+			ic.Meta.Exports["root"] = root


dup with line 74.

pacoxu · 2022-06-23T08:03:39Z

snapshots/overlay/overlay.go

+	fsMagic, err := overlayutils.GetFSMagic(root)
+	if err != nil {
+		return nil, err
+	}
+	if config.quotaSize > 0 {


Suggested change

fsMagic, err := overlayutils.GetFSMagic(root)

if err != nil {

return nil, err

}

if config.quotaSize > 0 {

if config.quotaSize > 0 {

fsMagic, err := overlayutils.GetFSMagic(root)

if err != nil {

return nil, err

}

pacoxu · 2022-06-23T10:02:49Z

snapshots/overlay/overlay.go

+				} else {
+					size = config.quotaSize
+				}
+				return quotaCtl.SetAllQuota(size, targets...)


setting quota may need some logs. Probably, some debug log.

yylt · 2022-06-27T09:49:51Z

I tried to build a containerd to do some test. Too many quota is created in my env.

[root@daocloud ~]# xfs_quota -xc "report -a -p -h"  | head -n 100
Project quota on /var/lib/kubelet (/dev/sdc)
                        Blocks
Project ID   Used   Soft   Hard Warn/Grace
---------- ---------------------------------
#0           820K      0      0  00 [------]
volume1048590      0     8E     8E  00 [------]
volume1048605      0     8E     8E  00 [------]
volume1048607      0     8E     8E  00 [------]
volume1048610      0     8E     8E  00 [------]

Project quota on /var/lib/containerd (/dev/sdb)
                        Blocks
Project ID   Used   Soft   Hard Warn/Grace
---------- ---------------------------------
#0          10.2G      0      0  00 [------]
#2              0      0      0  00 [------]
#3              0    15G    15G  00 [------]
#4            12K    15G    15G  00 [------]
#5              0    15G    15G  00 [------]
#6              0    15G    15G  00 [------]
#7              0    15G    15G  00 [------]
#8              0    15G    15G  00 [------]
#9              0    15G    15G  00 [------]
#10             0    15G    15G  00 [------]
#11             0    15G    15G  00 [------]
#12             0    15G    15G  00 [------]
#13             0    15G    15G  00 [------]
#14             0    15G    15G  00 [------]
#15             0    15G    15G  00 [------]
#16             0    15G    15G  00 [------]
#17             0    15G    15G  00 [------]
#18             0    15G    15G  00 [------]
#19             0    15G    15G  00 [------]
#20             0    15G    15G  00 [------]
#21             0    15G    15G  00 [------]
#22             0    15G    15G  00 [------]
#23             0    15G    15G  00 [------]
#24            4K    15G    15G  00 [------]
#25             0    15G    15G  00 [------]
#26             0    15G    15G  00 [------]
#27             0    15G    15G  00 [------]
#28             0    15G    15G  00 [------]
#29            4K    15G    15G  00 [------]
#30             0    15G    15G  00 [------]
#31             0    15G    15G  00 [------]
#32             0    15G    15G  00 [------]
#33             0    15G    15G  00 [------]
#34             0    15G    15G  00 [------]
#35            4K    15G    15G  00 [------]
#36            4K    15G    15G  00 [------]
#37             0    15G    15G  00 [------]
#38          540K    15G    15G  00 [------]
#39             0    15G    15G  00 [------]
#40             0    15G    15G  00 [------]
#41             0    15G    15G  00 [------]
#42             0    15G    15G  00 [------]
#43             0    10G    10G  00 [------]
#44             0    10G    10G  00 [------]
#45             0    10G    10G  00 [------]
#46             0    10G    10G  00 [------]
#47             0    10G    10G  00 [------]
#48             0    10G    10G  00 [------]
#49             0    10G    10G  00 [------]
#50             0    10G    10G  00 [------]
#51             0    10G    10G  00 [------]
#52             0    10G    10G  00 [------]
#53             0    10G    10G  00 [------]
#54             0    10G    10G  00 [------]
#55             0    10G    10G  00 [------]
#56             0    10G    10G  00 [------]
#57             0    10G    10G  00 [------]
#58             0    10G    10G  00 [------]
#59             0    10G    10G  00 [------]
#60             0    10G    10G  00 [------]
#61             0    10G    10G  00 [------]
#62             0    10G    10G  00 [------]
#63             0    10G    10G  00 [------]

I set the size to 10G at first and 15G later.

[root@daocloud ~]# xfs_quota -xc "report -a -p -h"  | wc -l
1956507

yes, it becauce set quota committed layer too. but it is not expected

pacoxu · 2022-07-06T06:58:46Z

I got some new errors with latest code.

7月 06 14:57:03 paco containerd[10220]: time="2022-07-06T14:57:03.299383030+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
7月 06 14:57:03 paco containerd[10220]: time="2022-07-06T14:57:03.299968488+08:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.overlayfs" error="Failed to set quota limit for projid 1 on /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/backingFsBlockDev: function not implemented"
7月 06 14:57:03 paco containerd[10220]: time="2022-07-06T14:57:03.300169800+08:00" level=warning msg="could not use snapshotter overlayfs in metadata plugin" error="Failed to set quota limit for projid 1 on /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/backingFsBlockDev: function not implemented"

And the quota is not set as expected.

yylt · 2022-07-29T08:00:05Z

I try command GOGC=off containerd -c myconfig.toml, and it always successed, and use code below, but it does not work.

	_, _, errno := syscall.Syscall6(syscall.SYS_QUOTACTL, uintptr(qcmd(Q_XSETQLIM, XFS_PROJ_QUOTA)),
		(*[2]uintptr)(unsafe.Pointer(&backingFsBlockDev))[0], uintptr(projectID),
		uintptr(unsafe.Pointer(&fd)), 0, 0)

and use bpftrace to debug, and the syscall _sys_quotactl retval could be 0, -2 , -20.

bpftrace -e 'kretprobe:__x64_sys_quotactl {printf(%s, retval %d\n",kstack,retval)}'

finally, use below to translate string to uintptr could be work fine.

	devbyte := append([]byte(backingFsBlockDev), 0)

	_, _, errno := syscall.Syscall6(syscall.SYS_QUOTACTL, uintptr(qcmd(Q_XSETQLIM, XFS_PROJ_QUOTA)),
		uintptr(unsafe.Pointer(&devbyte[0])), uintptr(fd.id),
		uintptr(unsafe.Pointer(&fd)), 0, 0)

yylt · 2024-11-26T02:35:57Z

Currently, only the quota for overlayfs snapshotter has been implemented, and unit test cases, it is believed that the code in CRI should be discussed and implemented in a separate patch, therefore could not add more integration tests.

Could you please take another look? @dmcgowan @AkihiroSuda

lukasmrtvy · 2025-02-17T14:35:21Z

Any update?

AkihiroSuda · 2025-02-17T15:02:21Z

core/snapshots/snapshotter.go

Could you also update https://github.com/containerd/containerd/tree/main/docs/snapshotters to explain the usage?

At present, there is no direct way to use quota on overlayfs, here is to support quota-enabled for overlayfs only.

As mentioned at #5859 (comment), perhaps in the future, configuring quotas through config.toml, cri-api, or annotations will be added, and this should discussion more.

Therefore, I believe it is more appropriate to modify the document when it is supported configure.

core/snapshots/snapshotter.go

plugins/snapshots/overlay/quota/iotcl.go

Signed-off-by: Yang Yang <[email protected]>

mikebrow

see comments

mikebrow · 2025-03-07T19:41:50Z

core/snapshots/snapshotter.go

@@ -379,6 +382,24 @@ func WithLabels(labels map[string]string) Opt {
 	}
 }

+// WithLabels appends labels to a created snapshot


Suggested change

// WithLabels appends labels to a created snapshot

// WithQuotaSize appends a label to this snapshot identifying it's maximum quota size in bytes for the snapshot writable layer

mikebrow · 2025-03-07T19:42:54Z

core/snapshots/snapshotter.go

@@ -379,6 +382,24 @@ func WithLabels(labels map[string]string) Opt {
 	}
 }

+// WithLabels appends labels to a created snapshot
+func WithQuotaSize(bytesize uint64) Opt {
+	return func(info *Info) error {


call WithLabels

mikebrow · 2025-03-07T19:44:45Z

plugins/snapshots/overlay/overlay.go

@@ -71,6 +73,12 @@ func WithUpperdirLabel(config *SnapshotterConfig) error {
 	return nil
 }

+// WithEnableQuota define the enable quota


Suggested change

// WithEnableQuota define the enable quota

// WithEnableQuota set enable quota to true/on

mikebrow · 2025-03-07T19:47:20Z

core/snapshots/snapshotter.go

+	}
+}
+
+func QuotaSize(labels map[string]string) string {


Suggested change

func QuotaSize(labels map[string]string) string {

// QuotaSize returns a string comprising the maximum quota size in bytes as an unsigned integer or nil if unset

func QuotaSize(labels map[string]string) string {

I don't think we should add more helpers here. This helper doesn't do much either.

mikebrow · 2025-03-07T20:15:37Z

plugins/snapshots/overlay/plugin/plugin.go

@@ -48,6 +48,9 @@ type Config struct {

 	// MountOptions are options used for the overlay mount (not used on bind mounts)
 	MountOptions []string `toml:"mount_options"`
+
+	// EnableQuota is define the quota on the snapshot writable layer


Suggested change

// EnableQuota is define the quota on the snapshot writable layer

// EnableQuota turns on the quota max size in bytes feature for the snapshot writable layer

mikebrow · 2025-03-07T20:23:26Z

plugins/snapshots/overlay/overlayutils/check.go

+	case fs.MagicXfs:
+		quotaCtl, err := quota.NewControl(root)
+		if err != nil {
+			log.L.WithError(err).Warnf("could not initinal quota control")


Suggested change

log.L.WithError(err).Warnf("could not initinal quota control")

log.L.WithError(err).Warnf("could not initialize quota control")

mikebrow · 2025-03-07T20:28:42Z

plugins/snapshots/overlay/overlayutils/check.go

+			log.L.WithError(err).Warnf("could not initinal quota control")
+			return nil, fmt.Errorf("directory '%s' does not support set quota, make sure mount with 'pquota'", root)
+		}
+		return func(targets []string, size uint64) error {


noting we switched to a string... but are still using quota size here as a uint. thus no way to set "0" on size vs "" to mean unset return nil...

dmcgowan

I understand this is only the first step but it is defining an interface for how it will be used by other snapshotters and backing filesystems in the future. I am fine we we want to start by just defining the label but think we should have a more generalized solution or plan before we put it into the default snapshotter.

Is there a plan for how this would interact with CRI/k8s? Setting a default max in the CRI config seems more dangerous than helpful as these sort of limits should be defined by k8s. This should also be able to free up the costly Usage calculation when it is used, otherwise it is only a minor enhancement.

I would suggest not considering this for the 2.1 release since it would only be experimental and the limited backing store support and possible client use doesn't warrant changes to the default snapshotter. I do envision in 2.2 on beyond we can start thinking about more hybrid snapshotters, where a snapshotter may be using overlay as well as block devices (similar to what erofs is doing). That could provide a more generalized solution for quota but possibly not a part of the default overlay snapshotter.

dmcgowan · 2025-03-08T19:59:54Z

core/snapshots/snapshotter.go

@@ -38,6 +39,8 @@ const (
 	LabelSnapshotUIDMapping = "containerd.io/snapshot/uidmapping"
 	// LabelSnapshotGIDMapping is the label used for GID mappings
 	LabelSnapshotGIDMapping = "containerd.io/snapshot/gidmapping"
+	// LabelSnapshotQuotaSize is the quota size.


If we define the label here, we should clearly define the expected value

dmcgowan · 2025-03-08T20:00:49Z

core/snapshots/snapshotter.go

+	}
+}
+
+func QuotaSize(labels map[string]string) string {


I don't think we should add more helpers here. This helper doesn't do much either.

dmcgowan · 2025-03-08T20:05:28Z

plugins/snapshots/overlay/overlayutils/check.go

+// SupportQuota checks if overlay filesystem supports quota or not.
+//
+// This function returns quotaSetter or error if it fails to check the filesystem.
+func SupportQuota(root string) (QuotaSetter, error) {


This isn't overlay specific

yylt · 2025-03-10T08:45:21Z

@dmcgowan clear on the current concern, the current implementation is indeed limited to overlayfs. More detailed context can be found in this discussion: #11516.

wangguangju · 2025-03-11T12:58:39Z

plugins/snapshots/overlay/quota/projectquota.go

+	//
+	// Get project id of parent dir as minimal id to be used by driver
+	//
+	minProjectID, err := getProjectID(basePath)


when containerd restart , can not obtain an existing path with prjid set on the system. In this case, the value in q.quotas[] is empty. The newly started container will use the prjid already used by another container
(dlv) p q
("*github.com/containerd/containerd/snapshots/quota.Control")(0xc000456800)
github.com/containerd/containerd/snapshots/quota.Control {
RWMutex: sync.RWMutex {
w: (sync.Mutex)(0xc000456800),
writerSem: 0,
readerSem: 0,
readerCount: ("sync/atomic.Int32")(0xc000456810),
readerWait: ("sync/atomic.Int32")(0xc000456814),},
backingFsBlockDev: "/home/containerd_rt/root/io.containerd.snapshotter.v1.overlayfs/backingFsBlockDev",
nextProjectID: 2,
quotas: map[string]uint32 [],}
(dlv)

yylt · 2025-03-13T08:41:09Z

Compared to ephemeral storage, adding storage configuration to cri-API is completely unnecessary, and there are many file system restrictions, making it seem useless

mikebrow · 2025-03-13T14:16:56Z

are we on the same page? "Goals - Universal snapshot support for quota: To implement a unified approach for managing quotas across all snapshot types. ... " along with Crosby's orig. recommendation "this can be implemented with snapshot labels and then the snapshotters will have to be updated to support these things. However, we still need higher layers like CRI and kube to pass these labels" and this discussion thread: #759

It's doesn't feel like we should only care about quotas for overlayfs and it doesn't seem like quota requirements don't need to flow over the CRI pod/container/metrics apis, ... not following the idea that quotas would be completely unnecessary and seemingly useless for CRI clients.

yylt · 2025-03-13T15:15:42Z

are we on the same page? "Goals - Universal snapshot support for quota: To implement a unified approach for managing quotas across all snapshot types. ... " along with Crosby's orig. recommendation "this can be implemented with snapshot labels and then the snapshotters will have to be updated to support these things. However, we still need higher layers like CRI and kube to pass these labels" and this discussion thread: #759

sorry for that, it's possible that I didn't fully express my thoughts. Integrating with CRI is obviously necessary, but that would require submitting a KEP([Kubernetes Enhancement Proposals) and engaging in discussions. I anticipate this process to be quite time-consuming. Moreover, ephemeral storage generally doesn't require snapshot type, making it more convenient.

Only after the KEP had been merged, that would be necessary to modify the current patch, it's likely that widespread agreement will only be achieved at that point.

Vigilans · 2025-04-01T15:42:08Z

As mentioned at #5859 (comment), perhaps in the future, configuring quotas through config.toml, cri-api, or annotations will be added, and this should discussion more.

As an end feature, we need to discuss how it will be integrated with Kubernetes and how we can use it from our own tooling and go libraries.

It would be great if this feature could also be introduced to docker with containerd snapshotter enabled. Currently with containerd-snapshotter: true, docker will lose the ability to set quota which was available in overlay2 graph driver.

k8s-ci-robot added the needs-ok-to-test label Aug 11, 2021

yylt force-pushed the quota branch from e27997e to a90db5f Compare August 11, 2021 03:38

AkihiroSuda reviewed Aug 11, 2021

View reviewed changes

AkihiroSuda added the status/needs-discussion Needs discussion and decision from maintainers label Aug 11, 2021

AkihiroSuda added this to the 1.6 milestone Aug 11, 2021

AkihiroSuda added kind/enhancement kind/feature labels Aug 11, 2021

This was referenced Aug 16, 2021

Filesystem quotas #759

Open

Support disk usage stats for containerd #5882

Closed

lining2020x reviewed Nov 11, 2021

View reviewed changes

dmcgowan modified the milestones: 1.6, 1.7 Dec 9, 2021

yylt force-pushed the quota branch from a90db5f to 72e9222 Compare June 16, 2022 07:23

yylt force-pushed the quota branch from 72e9222 to d165339 Compare June 16, 2022 08:01

pacoxu reviewed Jun 23, 2022

View reviewed changes

yylt force-pushed the quota branch from d165339 to 07398d4 Compare June 27, 2022 09:45

yylt force-pushed the quota branch 2 times, most recently from f72d83c to ed16c4b Compare July 29, 2022 07:42

dmcgowan marked this pull request as draft November 30, 2022 01:53

dmcgowan removed kind/feature needs-ok-to-test labels Nov 30, 2022

yylt force-pushed the quota branch from 5f3bc7f to 4a3aad1 Compare November 25, 2024 06:11

k8s-ci-robot added size/XL and removed needs-rebase size/XXL labels Nov 25, 2024

yylt force-pushed the quota branch 3 times, most recently from 6e0ed18 to 333e1a4 Compare November 26, 2024 02:35

yylt force-pushed the quota branch 2 times, most recently from 368240f to 8ac1836 Compare November 26, 2024 03:06

yylt force-pushed the quota branch from 8ac1836 to 5b52576 Compare December 5, 2024 06:53

AkihiroSuda reviewed Feb 17, 2025

View reviewed changes

core/snapshots/snapshotter.go Outdated Show resolved Hide resolved

AkihiroSuda reviewed Feb 17, 2025

View reviewed changes

plugins/snapshots/overlay/quota/iotcl.go Outdated Show resolved Hide resolved

AkihiroSuda reviewed Feb 17, 2025

View reviewed changes

plugins/snapshots/overlay/quota/iotcl.go Outdated Show resolved Hide resolved

yylt force-pushed the quota branch from 5b52576 to 9b0197b Compare February 18, 2025 02:36

support quota enable in overlayfs

d9f54d5

Signed-off-by: Yang Yang <[email protected]>

yylt force-pushed the quota branch from 9b0197b to d9f54d5 Compare February 18, 2025 05:50

AkihiroSuda approved these changes Feb 18, 2025

View reviewed changes

mikebrow reviewed Mar 7, 2025

View reviewed changes

dmcgowan requested changes Mar 8, 2025

View reviewed changes

dmcgowan modified the milestones: 2.1, 2.2 Mar 8, 2025

wangguangju reviewed Mar 11, 2025

View reviewed changes

	// WithLabels appends labels to a created snapshot
	// WithQuotaSize appends a label to this snapshot identifying it's maximum quota size in bytes for the snapshot writable layer

	// WithEnableQuota define the enable quota
	// WithEnableQuota set enable quota to true/on

	func QuotaSize(labels map[string]string) string {
	// QuotaSize returns a string comprising the maximum quota size in bytes as an unsigned integer or nil if unset
	func QuotaSize(labels map[string]string) string {

	// EnableQuota is define the quota on the snapshot writable layer
	// EnableQuota turns on the quota max size in bytes feature for the snapshot writable layer

	log.L.WithError(err).Warnf("could not initinal quota control")
	log.L.WithError(err).Warnf("could not initialize quota control")

support quota set on overlay snapshot #5859

Are you sure you want to change the base?

support quota set on overlay snapshot #5859

Conversation

yylt commented Aug 11, 2021 • edited Loading

k8s-ci-robot commented Aug 11, 2021

theopenlab-ci bot commented Aug 11, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmcgowan commented Aug 11, 2021

lining2020x Nov 11, 2021 • edited Loading

Choose a reason for hiding this comment

pacoxu commented Jun 9, 2022

yylt commented Jun 16, 2022

pacoxu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yylt commented Jun 27, 2022

pacoxu commented Jul 6, 2022 • edited Loading

yylt commented Jul 29, 2022

yylt commented Nov 26, 2024

lukasmrtvy commented Feb 17, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikebrow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmcgowan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yylt commented Mar 10, 2025

Choose a reason for hiding this comment

yylt commented Mar 13, 2025

mikebrow commented Mar 13, 2025

yylt commented Mar 13, 2025

Vigilans commented Apr 1, 2025 • edited Loading

yylt commented Aug 11, 2021 •

edited

Loading

lining2020x Nov 11, 2021 •

edited

Loading

pacoxu commented Jul 6, 2022 •

edited

Loading

Vigilans commented Apr 1, 2025 •

edited

Loading