Skip to content

[BUG] ImportError: import vowpalwabbit failed. #4970

@albertcthomas

Description

@albertcthomas

Bug Report Checklist

  • I provided code that demonstrates a minimal reproducible example.
  • I confirmed bug exists on the latest mainline of AutoGluon via source install.
  • I confirmed bug exists on the latest stable version of AutoGluon.

Describe the bug
In a fresh python session, running

from autogluon.common.utils.try_import import try_import_vowpalwabbit
try_import_vowpalwabbit()

I have the following error:

----> 1 import vowpalwabbit

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/vowpalwabbit/__init__.py:35
      7 __all__ = [
      8     "AbstractLabel",
      9     "ActionScore",
   (...)     31     "Workspace",
     32 ]
     34 from .version import __version__
---> 35 from . import pyvw
     36 from .pyvw import (
     37     AbstractLabel,
     38     ActionScore,
   (...)     60     Workspace,
     61 )
     64 def __getattr__(name):

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/vowpalwabbit/pyvw.py:6
      4 from __future__ import division
      5 from typing import Any, Dict, Iterator, List, Optional, Tuple, Type, Union
----> 6 import pylibvw
      7 import warnings
      8 import inspect

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ~/miniconda3/envs/autogluon/lib/libboost_python311.so.1.82.0)

Running this in a fresh python session still gives me the same error

from autogluon.common.utils.try_import import try_import_vowpalwabbit
import vowpalwabbit

However doing the following I have no error...

import vowpalwabbit
from autogluon.common.utils.try_import import try_import_vowpalwabbit
import vowpalwabbit  # or try_import_vowpalwabbit()

Finally, if copying the try_import_vowpalwabbit function and running it locally I don't have the error either

In [1]: def my_try_import_vowpalwabbit():
   ...:     try:
   ...:         import vowpalwabbit
   ...:         from pkg_resources import parse_version  # pylint: disable=import-outside-toplevel
   ...:
   ...:         vowpalwabbit_version = parse_version(vowpalwabbit.__version__)
   ...:         assert vowpalwabbit_version >= parse_version("9.0.0") and vowpalwabbit_version < parse_version(
   ...:             "9.10.0"
   ...:         ), f"Currently, we only support vowpalwabbit version >=9.0 and <9.10. Found vowpalwabbit version: {vowpalwabbit_version}"
   ...:     except ImportError:
   ...:         raise ImportError("`import vowpalwabbit` failed.\n" "A quick tip is to install via `pip install vowpalwabbit>=9,<9.10")
   ...:

In [2]: my_try_import_vowpalwabbit()

In [3]:

I observed this while I was trying the multimodal feature for data table.
When doing the prediction on a test set I have the following error (I don't have the error at training...):

Predicting DataLoader 0: 100%|██████████████████████████████████████████████████████████████████████████████████████████| 3125/3125 [00:47<00:00, 66.18it/s]---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)                                                                                 

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/common/utils/try_import.py:165, in try_import_vowpalwabbit()
       164 try:                                                                                                                                                
--> 165     import vowpalwabbit                                                                                                                                 
       166     from pkg_resources import parse_version  # pylint: disable=import-outside-toplevel                                                                                                                                                                                                                          File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/vowpalwabbit/__init__.py:35                                                               
        34 from .version import __version__                                                                                                                    
---> 35 from . import pyvw                                                                                                                                       
        36 from .pyvw import (                                                                                                                                      
        37     AbstractLabel,                                                                                                                                       
        38     ActionScore,                                                                                                                                       
(...)     60     Workspace,                                                                                                                                   
61 )                                                                                                                                                                                                                                                                                                               

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/vowpalwabbit/pyvw.py:6                                                                     
          5 from typing import Any, Dict, Iterator, List, Optional, Tuple, Type, Union                                                                          
----> 6 import pylibvw                                                                                                                                            
          7 import warnings                                                                                                                                                                                                                                                                                                 
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ~/miniconda3/envs/autogluon/lib/libboost_python311.so.1.82.0)                            
During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
File ~/run_autogluon/autogluon_script.py:248
--> 248     y_pred = predictor.predict(test_data)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/tabular/predictor/predictor.py:2117, in TabularPredictor.predict(self, data, model, as_pandas, transform_features, decision_threshold)
   2115 if decision_threshold is None:
   2116     decision_threshold = self.decision_threshold
-> 2117 return self._learner.predict(X=data, model=model, as_pandas=as_pandas, transform_features=transform_features, decision_threshold=decision_threshold)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/tabular/learner/abstract_learner.py:208, in AbstractTabularLearner.predict(self, X, model, as_pandas, inverse_transform, transform_features, decision_threshold)
    206     decision_threshold = 0.5
    207 X_index = copy.deepcopy(X.index) if as_pandas else None
--> 208 y_pred_proba = self.predict_proba(
    209     X=X, model=model, as_pandas=False, as_multiclass=False, inverse_transform=False, transform_features=transform_features
    210 )
    211 problem_type = self.label_cleaner.problem_type_transform or self.problem_type
    212 y_pred = get_pred_from_proba(y_pred_proba=y_pred_proba, problem_type=problem_type, decision_threshold=decision_threshold)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/tabular/learner/abstract_learner.py:189, in AbstractTabularLearner.predict_proba(self, X, model, as_pandas, as_multiclass, inverse_transform, transform_features)
    187     if transform_features:
    188         X = self.transform_features(X)
--> 189     y_pred_proba = self.load_trainer().predict_proba(X, model=model)
    190 y_pred_proba = self._post_process_predict_proba(
    191     y_pred_proba=y_pred_proba, as_pandas=as_pandas, index=X_index, as_multiclass=as_multiclass, inverse_transform=inverse_transform
    192 )
    193 return y_pred_proba

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/trainer/abstract_trainer.py:837, in AbstractTrainer.predict_proba(self, X, model)
    835     model = self._get_best()
    836 cascade = isinstance(model, list)
--> 837 return self._predict_proba_model(X, model, cascade=cascade)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/trainer/abstract_trainer.py:2611, in AbstractTrainer._predict_proba_model(self, X, model, model_pred_proba_dict, cascade)
   2610 def _predict_proba_model(self, X, model, model_pred_proba_dict=None, cascade=False):
-> 2611     return self.get_pred_proba_from_model(model=model, X=X, model_pred_proba_dict=model_pred_proba_dict, cascade=cascade)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/trainer/abstract_trainer.py:851, in AbstractTrainer.get_pred_proba_from_model(self, model, X, model_pred_proba_dict, cascade)
    849 else:
    850     models = [model]
--> 851 model_pred_proba_dict = self.get_model_pred_proba_dict(X=X, models=models, model_pred_proba_dict=model_pred_proba_dict, cascade=cascade)
    852 if not isinstance(model, str):
    853     model = model.name

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/trainer/abstract_trainer.py:1100, in AbstractTrainer.get_model_pred_proba_dict(self, X, models, model_pred_proba_dict, model_pred_time_dict, record_pred_time, use_val_cache, cascade, cascade_threshold)
   1098     else:
   1099         preprocess_kwargs = dict(infer=False, model_pred_proba_dict=model_pred_proba_dict)
-> 1100     model_pred_proba_dict[model_name] = model.predict_proba(X, **preprocess_kwargs)
   1101 else:
   1102     model_pred_proba_dict[model_name] = model.predict_proba(X)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/models/abstract/abstract_model.py:968, in AbstractModel.predict_proba(self, X, normalize, record_time, **kwargs)
    943 """
    944 Returns class prediction probabilities of X.
    945 For binary problems, this returns the positive class label probability as a 1d numpy array.
   (...)    964     The prediction probabilities
    965 """
    966 time_start = time.time() if record_time else None
--> 968 y_pred_proba = self._predict_proba_internal(X=X, normalize=normalize, **kwargs)
    970 if self.params_aux.get("temperature_scalar", None) is not None:
    971     y_pred_proba = self._apply_temperature_scaling(y_pred_proba)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py:454, in BaggedEnsembleModel._predict_proba_internal(self, X, normalize, **kwargs)
    453 def _predict_proba_internal(self, X, *, normalize: bool | None = None, **kwargs):
--> 454     model = self.load_child(self.models[0])
    455     X = self.preprocess(X, model=model, **kwargs)
    456     y_pred_proba = model.predict_proba(X=X, preprocess_nonadaptive=False, normalize=normalize)

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py:911, in BaggedEnsembleModel.load_child(self, model, verbose)
    909 if isinstance(model, str):
    910     child_path = self.create_contexts(os.path.join(self.path, model))
--> 911     return self._child_type.load(path=child_path, verbose=verbose)
    912 else:
    913     return model
File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/tabular/models/vowpalwabbit/vowpalwabbit_model.py:199, in VowpalWabbitModel.load(cls, path, reset_paths, verbose)
    192 @classmethod
    193 def load(cls, path: str, reset_paths=True, verbose=True):
    194     """
    195     There are two files which needs to be loaded.
    196     First is the Abstract Model pickle dump and second is the internal model file.
    197     For VW, based on different problem_type/hyperparams, loading arguments will be different
    198     """
--> 199     try_import_vowpalwabbit()
    200     import vowpalwabbit
    202     # Load Abstract Model. This is without the internal model

File ~/miniconda3/envs/autogluon/lib/python3.11/site-packages/autogluon/common/utils/try_import.py:173, in try_import_vowpalwabbit()
    169     assert vowpalwabbit_version >= parse_version("9.0.0") and vowpalwabbit_version < parse_version(
    170         "9.10.0"
    171     ), f"Currently, we only support vowpalwabbit version >=9.0 and <9.10. Found vowpalwabbit version: {vowpalwabbit_version}"
    172 except ImportError:
--> 173     raise ImportError("`import vowpalwabbit` failed.\n" "A quick tip is to install via `pip install vowpalwabbit>=9,<9.10")

ImportError: `import vowpalwabbit` failed.
A quick tip is to install via `pip install vowpalwabbit>=9,<9.10

Expected behavior
No ImportError

To Reproduce

Screenshots / Logs

Installed Versions

Details
show_versions()

INSTALLED VERSIONS
------------------
date                   : 2025-03-10
time                   : 14:06:25.993833
python                 : 3.11.10.final.0
OS                     : Linux
OS-release             : 5.4.0-186-generic
Version                : #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024
machine                : x86_64
processor              : x86_64
num_cores              : 64
cpu_ram_mb             : 515973.59375
cuda version           : 12.530.30.02
num_gpus               : 8
gpu_ram_mb             : [32497, 32497, 32497, 32497, 32497, 32497, 32497, 32497]
avail_disk_size_mb     : 440183

accelerate             : 0.21.0
autogluon              : 1.1.1
autogluon.common       : 1.1.1
autogluon.core         : 1.1.1
autogluon.features     : 1.1.1
autogluon.multimodal   : 1.1.1
autogluon.tabular      : 1.1.1
autogluon.timeseries   : 1.1.1
boto3                  : 1.37.3
catboost               : 1.2.7
defusedxml             : 0.7.1
evaluate               : 0.4.3
fastai                 : 2.7.18
gluonts                : 0.15.1
hyperopt               : 0.2.7
imodels                : None
jinja2                 : 3.1.5
joblib                 : 1.4.2
jsonschema             : 4.21.1
lightgbm               : 4.3.0
lightning              : 2.3.3
matplotlib             : 3.10.1
mlforecast             : 0.10.0
networkx               : 3.4.2                                                                                                                                                                                   nlpaug                 : 1.1.11
nltk                   : 3.9.1                                                                                                                                                                                   nptyping               : 2.4.1
numpy                  : 1.26.4
nvidia-ml-py3          : 7.352.0
omegaconf              : 2.2.3
onnxruntime-gpu        : None
openmim                : 0.3.9
optimum                : 1.18.1
optimum-intel          : None
orjson                 : 3.10.15
pandas                 : 2.2.3
pdf2image              : 1.17.0
Pillow                 : 10.4.0
psutil                 : 5.9.8
pytesseract            : 0.3.10
pytorch-lightning      : 2.3.3
pytorch-metric-learning: 2.3.0
ray                    : 2.10.0
requests               : 2.32.3
scikit-image           : 0.20.0
scikit-learn           : 1.4.0
scikit-learn-intelex   : None
scipy                  : 1.12.0
seqeval                : 1.2.2
setuptools             : 75.8.0
skl2onnx               : None
statsforecast          : 1.4.0
tabpfn                 : None
tensorboard            : 2.19.0
text-unidecode         : 1.3
timm                   : 0.9.16
torch                  : 2.3.1
torchmetrics           : 1.2.1
torchvision            : 0.18.1
tqdm                   : 4.67.1
transformers           : 4.39.3
utilsforecast          : 0.0.10
vowpalwabbit           : 9.0.0
xgboost                : 2.0.3

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions