01:17:30  Running ./modules/learning_rate.ipynb
01:17:30  Checking PEP8 compliance...
01:17:31  Running notebook...
01:17:31  Before:
01:17:31      "max_epochs = 100\n",
01:17:31  After:
01:17:31      "max_epochs = 1\n",
01:17:31  Before:
01:17:31      "        val_interval = 1\n",
01:17:31  After:
01:17:31      "        val_interval = 1\n",
01:17:35  MONAI version: 0.9.1+22.ga9095d09
01:17:35  Numpy version: 1.22.4
01:17:35  Pytorch version: 1.13.0a0+08820cb
01:17:35  MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
01:17:35  MONAI rev id: a9095d09d919e94c2c07aa74fc46c1e4f9530ecd
01:17:35  MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
01:17:35  
01:17:35  Optional dependencies:
01:17:35  Pytorch Ignite version: 0.4.9
01:17:35  Nibabel version: 4.0.1
01:17:35  scikit-image version: 0.19.3
01:17:35  Pillow version: 9.0.1
01:17:35  Tensorboard version: 2.9.1
01:17:35  gdown version: 4.5.1
01:17:35  TorchVision version: 0.14.0a0
01:17:35  tqdm version: 4.64.0
01:17:35  lmdb version: 1.3.0
01:17:35  psutil version: 5.9.1
01:17:35  pandas version: 1.4.3
01:17:35  einops version: 0.4.1
01:17:35  transformers version: 4.21.1
01:17:35  mlflow version: 1.27.0
01:17:35  pynrrd version: 0.4.3
01:17:35  
01:17:35  For details about installing the optional dependencies, please visit:
01:17:35      https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
01:17:35  
01:17:36  /opt/conda/lib/python3.8/site-packages/papermill/iorw.py:58: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
01:17:36    from pyarrow import HadoopFileSystem
01:18:17  
Executing:   0%|          | 0/26 [00:00<?, ?cell/s]
Executing:   4%|▍         | 1/26 [00:01<00:31,  1.25s/cell]
Executing:  12%|█▏        | 3/26 [00:06<00:51,  2.22s/cell]
Executing:  19%|█▉        | 5/26 [00:09<00:41,  1.97s/cell]
Executing:  46%|████▌     | 12/26 [00:20<00:23,  1.68s/cell]
Executing:  54%|█████▍    | 14/26 [00:21<00:16,  1.34s/cell]
Executing:  62%|██████▏   | 16/26 [00:22<00:11,  1.15s/cell]
Executing:  69%|██████▉   | 18/26 [00:37<00:22,  2.81s/cell]
Executing:  88%|████████▊ | 23/26 [00:39<00:05,  1.69s/cell]
Executing:  88%|████████▊ | 23/26 [00:41<00:05,  1.79s/cell]
01:18:17  Traceback (most recent call last):
01:18:17    File "/opt/conda/bin/papermill", line 8, in <module>
01:18:17      sys.exit(papermill())
01:18:17    File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
01:18:17      return self.main(*args, **kwargs)
01:18:17    File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1053, in main
01:18:17      rv = self.invoke(ctx)
01:18:17    File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
01:18:17      return ctx.invoke(self.callback, **ctx.params)
01:18:17    File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 754, in invoke
01:18:17      return __callback(*args, **kwargs)
01:18:17    File "/opt/conda/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
01:18:17      return f(get_current_context(), *args, **kwargs)
01:18:17    File "/opt/conda/lib/python3.8/site-packages/papermill/cli.py", line 250, in papermill
01:18:17      execute_notebook(
01:18:17    File "/opt/conda/lib/python3.8/site-packages/papermill/execute.py", line 122, in execute_notebook
01:18:17      raise_for_execution_errors(nb, output_path)
01:18:17    File "/opt/conda/lib/python3.8/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
01:18:17      raise error
01:18:17  papermill.exceptions.PapermillExecutionError: 
01:18:17  ---------------------------------------------------------------------------
01:18:17  Exception encountered at "In [12]":
01:18:17  ---------------------------------------------------------------------------
01:18:17  RuntimeError                              Traceback (most recent call last)
01:18:17  Input In [12], in <cell line: 22>()
01:18:17       16 data["Steepest LR"] = {"lr_lim": steepest_lr}
01:18:17       17 data["Cyclical LR"] = {
01:18:17       18     "lr_lims": (0.8 * steepest_lr, 1.2 * steepest_lr),
01:18:17       19     "step": step_size,
01:18:17       20 }
01:18:17  ---> 22 train(max_epochs, axes, data)
01:18:17  
01:18:17  Input In [11], in train(max_epochs, axes, data)
01:18:17       74 y_pred_act = [y_pred_trans(i) for i in decollate_batch(data[d]["y_pred"])]
01:18:17       75 auc_metric(y_pred_act, y_onehot)
01:18:17  ---> 76 auc_result = auc_metric.aggregate()
01:18:17       77 auc_metric.reset()
01:18:17       78 del y_pred_act, y_onehot
01:18:17  
01:18:17  File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/metrics/rocauc.py:69, in ROCAUCMetric.aggregate(self, average)
01:18:17       66 if not isinstance(y_pred, torch.Tensor) or not isinstance(y, torch.Tensor):
01:18:17       67     raise ValueError("y_pred and y must be PyTorch Tensor.")
01:18:17  ---> 69 return compute_roc_auc(y_pred=y_pred, y=y, average=average or self.average)
01:18:17  
01:18:17  File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/metrics/rocauc.py:166, in compute_roc_auc(y_pred, y, average)
01:18:17      164     return _calculate(y_pred.flatten(), y.flatten())
01:18:17      165 y, y_pred = y.transpose(0, 1), y_pred.transpose(0, 1)
01:18:17  --> 166 auc_values = [_calculate(y_pred_, y_) for y_pred_, y_ in zip(y_pred, y)]
01:18:17      167 if average == Average.NONE:
01:18:17      168     return auc_values
01:18:17  
01:18:17  File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/metrics/rocauc.py:166, in <listcomp>(.0)
01:18:17      164     return _calculate(y_pred.flatten(), y.flatten())
01:18:17      165 y, y_pred = y.transpose(0, 1), y_pred.transpose(0, 1)
01:18:17  --> 166 auc_values = [_calculate(y_pred_, y_) for y_pred_, y_ in zip(y_pred, y)]
01:18:17      167 if average == Average.NONE:
01:18:17      168     return auc_values
01:18:17  
01:18:17  File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/metrics/rocauc.py:85, in _calculate(y_pred, y)
01:18:17       83 n = len(y)
01:18:17       84 indices = y_pred.argsort()
01:18:17  ---> 85 y = y[indices].cpu().numpy()
01:18:17       86 y_pred = y_pred[indices].cpu().numpy()
01:18:17       87 nneg = auc = tmp_pos = tmp_neg = 0.0
01:18:17  
01:18:17  RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)