FAQ

Q&A

Q: Model outputs repetitive responses or inaccurate content

A: Adjust by modifying the relevant parameters in the corresponding model's xx-htp.json file. Using Qwen2.5-7B-Instruct as an example:

# Adjust the model's output by modifying the following parameters in the configuration file
"sampler" : {
    "version" : 1,
    "seed" : 42,
    "temp" : 0.4,
    "top-k" : 20,
    "top-p" : 0.8,
    "greedy" : false
}

For parameter explanations, please refer to: https://zhuanlan.zhihu.com/p/720829470

Q: Model outputs mixed Chinese and English

A: Modify the prompt to make the model output in the specified language. For example, add this sentence to the prompt: Please answer in Chinese

Q: How to set up model conversation templates

A: The model resource package downloaded from Model Farm includes the model conversation template file aidgen_chat_template.txt. Using Qwen2.5-7B-Instruct as an example:

• The aidgen_chat_template.txt template content is as follows:

<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n<|im_start|>user\nGive me a short introduction to large language model.<|im_end|>\n<|im_start|>assistant\n

Q: fatal error: fmt/format.h: No such file or directory

A: The fmt/format.h library was not found. Please install the related libraries as follows:

sudo apt update
sudo apt install libfmt-dev