Having an AttributeError while loading a pickled file may also be an indication of mismatched dependency versions.
I encountered the following when trying to load a 1.3.x dataframe with pandas 1.0.x installed:

Traceback (most recent call last):
  File "...../AppData/Roaming/JetBrains/PyCharm2020.1/scratches/scratch_3.py", line 8, in <module>
    X, y, _, _ = data.get_data(dataset_format='dataframe',
  File ".....\venv2100\lib\site-packages\openml\datasets\dataset.py", line 698, in get_data
    data, categorical, attribute_names = self._load_data()
  File ".....\venv2100\lib\site-packages\openml\datasets\dataset.py", line 544, in _load_data
    data, categorical, attribute_names = pickle.load(fh)
AttributeError: Can't get attribute 'new_block' on <module 'pandas.core.internals.blocks' from '.....\\venv2100\\lib\\site-packages\\pandas\\core\\internals\\blocks.py'>

Upgrading back to pandas 1.3.x solved the issue (confirming it was exclusively a version issue).
Clearing the cache also made it work (confirming the advice given by the error is sound for this issue).
Did not add a unit test since it's not done for the similar situation (ModuleNotFoundError) either and it would be somewhat complex/annoying (have to add "old" data to the repository and execute the unit test only for specific pandas versions and/or have multiple "old" data so there is always a predictable incompatible mismatch).