2024-04-22 简化
2023-10-24 initial semantic segmentation
- pip install -U -r requirements.txt
- 如果无法安装， 可以切换官方源 pip install -i https://pypi.org/simple -U -r requirements.txt
支持且不限于以下权重 , See all SegFormer models at https://huggingface.co/models?filter=segformer
- open_data https://github.com/ssbuild/open_data
- sidewalk-semantic https://huggingface.co/datasets/segments/sidewalk-semantic
# infer_finetuning.py 推理微调模型
# infer_lora_finetuning.py 推理微调模型
python infer_finetuning.py
# 制作数据
cd scripts
bash train_full.sh -m dataset
注: num_process_worker 为多进程制作数据 ， 如果数据量较大 ， 适当调大至cpu数量
dataHelper.make_dataset_with_args(data_args.train_file,mixed_data=False, shuffle=True,mode='train',num_process_worker=0)
# 全参数训练
bash train_full.sh -m train
- pytorch-task-example
- tf-task-example
- chatmoss_finetuning
- chatglm_finetuning
- chatglm2_finetuning
- chatglm3_finetuning
- t5_finetuning
- llm_finetuning
- llm_rlhf
- chatglm_rlhf
- t5_rlhf
- rwkv_finetuning
- baichuan_finetuning
纯粹而干净的代码