TRAIT: Personality Testset designed for LLMs with Psychometrics

Do LLMs Have Distinct and Consistent Personalities? TRAIT: Personality Testset designed for LLMs with Psychometrics, Seungbeen Lee^1,, Seungwon Lim^1,, Seungju Han^2,3, Giyeong Oh¹, Hyungjoo Chae¹, Jiwan Chung¹ Minju Kim¹ Beong-woo Kwak¹ Yeonsoo Lee⁴ Dongha Lee¹ Jinyoung Yeo¹ Youngjae Yu^1
1Yonsei University, ²Seoul National University ³Allen Institute for AI ⁴NCSOFT
^*Equal contribution.

​

Preparation

You do not need a complex environment. Install the required python packages with the following command: ​

pip install -r requirements.txt

​

How to run

You can run 2 mode: Base mode and Personality-prompt mode. With Base mode,

Base mode

Chatgpt

python run.py --model_name Chatgpt --model_name_short Chatgpt --prompt_type 1

mistralai/Mistral-7B-Instruct-v0.2

python run.py --model_name mistralai/Mistral-7B-Instruct-v0.2 --model_name_short mistral_instruct --inference_type chat --prompt_type 1

Personality-prompt mode

Chatgpt

python run_personality_prompt.py --model_name Chatgpt --model_name_short Chatgpt --model_close --personality "high openness" --prompt_type 2

mistralai/Mistral-7B-Instruct-v0.2

python run_personality_prompt.py --model_name mistralai/Mistral-7B-Instruct-v0.2 --model_name_short mistral_instruct --inference_type chat --personality "high openness" --prompt_type 2

Result

You can get the result of the model.

python analysis.py

More codes are available soon!