🚀 Feature

Improve error handling for empty directories in make_dataset().

Motivation

datasets.folder.make_dataset() requires the class_to_idx attribute that is then used to collect the instances:

Currently, we have four places where make_dataset() is used and in all cases class_to_idx is generated the same:

1. vision/torchvision/datasets/folder.py

   Line 126 in 945f3a8

   classes, class_to_idx = self._find_classes(self.root)

   with def _find_classes(dir):

   vision/torchvision/datasets/folder.py

   Lines 164 to 167 in 945f3a8

   	classes = [d.name for d in os.scandir(dir) if d.is_dir()]
   	classes.sort()
   	class_to_idx = {cls_name: i for i, cls_name in enumerate(classes)}
   	return classes, class_to_idx

2. vision/torchvision/datasets/hmdb51.py

   Lines 65 to 71 in 945f3a8

   	classes = sorted(list_dir(root))
   	class_to_idx = {class_: i for (i, class_) in enumerate(classes)}
   	self.samples = make_dataset(
   	self.root,
   	class_to_idx,
   	extensions,
   	)

3. vision/torchvision/datasets/kinetics.py

   Lines 58 to 60 in 945f3a8

   	classes = list(sorted(list_dir(root)))
   	class_to_idx = {classes[i]: i for i in range(len(classes))}
   	self.samples = make_dataset(self.root, class_to_idx, extensions, is_valid_file=None)

4. vision/torchvision/datasets/ucf101.py

   Lines 58 to 60 in 945f3a8

   	classes = list(sorted(list_dir(root)))
   	class_to_idx = {classes[i]: i for i in range(len(classes))}
   	self.samples = make_dataset(self.root, class_to_idx, extensions, is_valid_file=None)

Furthermore, only DatasetFolder has a builtin check if make_dataset found any samples:

While this is better than passing silently and failing somewhere else (#2903), it still misses the underlying issue in case of an directory without subfolders.

Pitch

I propose three things:

1. Factor out the implementation of the DatasetFolder._find_classes() method into a find_classes() function similar to what we did with make_dataset in 'make_dataset' as staticmethod of 'DatasetFolder' #3215.
2. Raise an expressive error in find_classes() if no classes were found.
3. Make the class_to_idx parameter optional in make_dataset and call find_classes if it is omitted.

With this we are as flexible as before while we remove duplicated code.

• If one does not want the default behavior, class_to_idx can still be passed explicitly
• If one needs the returned classes, e.g. the video datasets, a call could look like this

  self.classes, class_to_idx = find_classes(root)
  self.samples = make_dataset(root, class_to_idx, ...)

• If one only needs the samples calling self.samples = make_dataset(root, ...) is enough

cc @pmeier