Skip to content

Commit 82fc27b

Browse files
AtsuMiyaiAtsuMiyai
committed
add upd
1 parent 0464e31 commit 82fc27b

37 files changed

+172
-172
lines changed

‎README.md‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -190,19 +190,19 @@ We also provide the raw data exported from Weights & Biases for the detailed res
190190
- MMMU (mmmu)
191191
- MMMU Validation (mmmu_val)
192192
- MMMU Test (mmmu_test)
193-
- MMUPDBench (mmupdbench)
194-
- MMUPDBench Base (mmupdbench_base)
195-
- MMAADBench Base (mmaadbench_base)
196-
- MMIASDBench Base (mmiasdbench_base)
197-
- MMIVQDBench Base (mmivqdbench_base)
198-
- MMUPDBench Option (mmupdbench_option)
199-
- MMAADBench Option (mmaadbench_option)
200-
- MMIASDBench Option (mmiasdbench_option)
201-
- MMIVQDBench Option (mmivqdbench_option)
202-
- MMUPDBench Instruction (mmupdbench_instruction)
203-
- MMAADBench Instruction (mmaadbench_instruction)
204-
- MMIASDBench Instruction (mmiasdbench_instruction)
205-
- MMIVQDBench Instruction (mmivqdbench_instruction)
193+
- MMUPD (mmupd)
194+
- MMUPD Base (mmupd_base)
195+
- MMAAD Base (mmaad_base)
196+
- MMIASD Base (mmiasd_base)
197+
- MMIVQD Base (mmivqd_base)
198+
- MMUPD Option (mmupd_option)
199+
- MMAAD Option (mmaad_option)
200+
- MMIASD Option (mmiasd_option)
201+
- MMIVQD Option (mmivqd_option)
202+
- MMUPD Instruction (mmupd_instruction)
203+
- MMAAD Instruction (mmaad_instruction)
204+
- MMIASD Instruction (mmiasd_instruction)
205+
- MMIVQD Instruction (mmivqd_instruction)
206206
- MMVet (mmvet)
207207
- Multi-DocVQA (multidocvqa)
208208
- Multi-DocVQA Validation (multidocvqa_val)

lmms_eval/tasks/mmupdbench/_default_template_mmaadbench_base_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmaad_base_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\n"
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmaadbench_instruction_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmaad_instruction_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\nIf all the options are incorrect, answer \"F. None of the above\"."
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmaadbench_option_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmaad_option_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\nAnswer with the option's letter from the given choices directly."
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmiasdbench_base_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmiasd_base_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\n"
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmiasdbench_instruction_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmiasd_instruction_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\nIf all the options are incorrect, answer \"F. None of the above\"."
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmiasdbench_option_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmiasd_option_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\nAnswer with the option's letter from the given choices directly."
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmivqdbench_base_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmivqd_base_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\n"
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmivqdbench_instruction_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmivqd_instruction_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\nIf the given image is irrelevant to the question, answer \"F. The image and question are irrelevant.\"."
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

lmms_eval/tasks/mmupdbench/_default_template_mmivqdbench_option_yaml renamed to lmms_eval/tasks/mmupd/_default_template_mmivqd_option_yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ model_specific_prompt_kwargs:
44
default:
55
pre_prompt: ""
66
post_prompt: "\nAnswer with the option's letter from the given choices directly."
7-
doc_to_visual: !function utils.mmupdbench_doc_to_visual
8-
doc_to_text: !function utils.mmupdbench_doc_to_text
7+
doc_to_visual: !function utils.mmupd_doc_to_visual
8+
doc_to_text: !function utils.mmupd_doc_to_text
99
doc_to_target: "answer"
10-
process_results: !function utils.mmupdbench_process_results
10+
process_results: !function utils.mmupd_process_results
1111
model_specific_generation_kwargs:
1212
llava:
1313
image_aspect_ratio: original

0 commit comments

Comments
 (0)