HypModalAlign

Repository for the paper Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds.

Performance Highlights([TOS classification])

Performance on Cifar100 and Sun

K-Shot	Base Method	Variant	Cifar100			SUN
			LA	HCA	MTA	LA	HCA	MTA
1	MaPLe	vanilla	68.75	4.65	50.60	63.98	25.15	50.31
		+ProTeCt	69.33	48.10	83.36	64.29	50.45	76.73
		+Ours	71.37	53.19	85.29	67.57	57.92	80.55
	PromptSRC	vanilla	72.48	14.36	51.91	70.58	42.14	57.19
		+ProTeCt	73.07	49.54	85.16	70.61	55.52	78.73
		+Ours	73.54	51.91	85.76	70.64	57.79	79.94
16	MaPLe	vanilla	75.01	17.54	52.21	71.86	33.25	54.29
		+ProTeCt	75.34	61.15	88.04	72.17	59.71	82.27
		+Ours	77.92	69.38	90.89	75.47	68.67	86.02
	PromptSRC	vanilla	77.71	15.07	56.86	75.75	45.23	59.42
		+ProTeCt	78.76	66.74	90.79	75.54	66.01	84.75
		+Ours	78.90	68.47	91.12	76.54	69.18	86.20

Performance on ImageNet and Rare Species

K-Shot	Base Method	Variant	ImageNet			Rare Species
			LA	HCA	MTA	LA	HCA	MTA
1	MaPLe	vanilla	68.91	2.97	48.16	41.55	5.09	44.75
		+ProTeCt	66.16	20.44	85.18	39.92	13.22	70.04
		+Ours	66.33	25.56	85.98	46.77	20.94	76.83
	PromptSRC	vanilla	68.82	4.46	54.10	45.39	6.72	44.72
		+ProTeCt	68.43	20.36	85.63	44.56	20.36	74.42
		+Ours	68.86	25.13	86.45	46.98	23.03	77.32
16	MaPLe	vanilla	70.70	4.15	48.16	50.94	5.30	40.41
		+ProTeCt	69.52	31.24	87.87	48.14	24.82	78.79
		+Ours	71.41	43.79	88.78	69.96	53.65	87.27
	PromptSRC	vanilla	71.50	2.48	46.71	59.20	11.64	55.82
		+ProTeCt	70.98	32.89	88.31	56.40	33.92	82.47
		+Ours	71.67	42.26	89.64	67.38	50.77	87.60

Install environment

$ conda env create -f environment.yml
$ conda activate alitree

Download Dataset

• Cifar100(we use a hierachical version provided by ProTeCt)
• SUN
• ImageNet
• Rare Species

After downloading the datasets using the links provided above, you can place them directly into the ./prepro/raw/ directory or create symbolic links to their locations. For the Rare Species dataset, an additional preprocessing step is required, which can be executed by running:

$ bash python ./prepro/scripts/extract_rarespecies.py

The dataset annotations are organized under the directory ./data/{datasetname}. Specifically:

• The files gt_{split}.txt contain the data list and leaf-node level annotations.
• The files tree.pyand tree_{subsample}.npy record the hierarchical information.
• The files treecuts_{num}.pkl and treecuts_{num}_{subsample}.pkl are used for MTA evaluation.

Training

We have organized detailed training configurations in the ./configs/ directory, with the main configuration parameters explained in ./configs/few_shot/1-shot/cifar100/maple+ours.yml.

You can refer to the corrsponding training scripts provided in ./scripts/ to reproduce the results.For example, to reproduce the 1-shot Cifar100-100 results using MaPLe, you can execute the command

$ python train.py --config ./configs/few_shot/1-shot/cifar100/maple+ours.yml --trial 1

For experiments under different settings, simply specify the corresponding configuration file.

Evaluation

LA and HCA are automatically evaluated after training completes. To re-evaluate these metrics for a saved checkpoint, run reeval.py and specify the experiment directory, for example:

$ python reeval.py --folder ./runs/cifar100/maple/ViT-B_16/few_shot/1-shot/ours/trial_1 --bz {your_batch_size}

To evaluate MTA, use evalmta.py with the target experiment directory:

$ python evalmta.py --folder ./runs/cifar100/maple/ViT-B_16/few_shot/$1-shot/ours/trial_1 --bz ${your_batch_size}
               

Acknowledgements

Our work is based on the following codebases. Thanks for their brilliant contributions to the community!

• https://github.com/gina9726/ProTeCt
• https://github.com/muzairkhattak/multimodal-prompt-learning
• https://github.com/muzairkhattak/PromptSRC
• https://github.com/KaiyangZhou/CoOp

Cite

If you find this repository useful, please consider cite our paper.

@misc{wei2025modalityalignmenttreesheterogeneous,
      title={Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds}, 
      author={Wu Wei and Xiaomeng Fan and Yuwei Wu and Zhi Gao and Pengxiang Li and Yunde Jia and Mehrtash Harandi},
      year={2025},
      eprint={2510.27391},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2510.27391}, 
}