更新安装文档,添加关于GitHub Models API的警告信息,并调整配置项说明

This commit is contained in:
Asankilp 2025-01-25 00:06:27 +08:00
parent c8e776d5ff
commit ef71514ce2
5 changed files with 94 additions and 12 deletions

View File

@ -49,7 +49,9 @@ Open the `pyproject.toml` file under nonebot2's root directory, Add to`[tool.non
- Create new [personal access token](https://github.com/settings/tokens/new)**Don't need any permissions**. - Create new [personal access token](https://github.com/settings/tokens/new)**Don't need any permissions**.
- Copy the new token, add to the `.env` file's `marshoai_token` option. - Copy the new token, add to the `.env` file's `marshoai_token` option.
:::warning
GitHub Models API comes with significant limitations and is therefore not recommended for use. For better alternatives, it's suggested to adjust the configuration `MARSHOAI_AZURE_ENDPOINT` to use other service providers' models instead.
:::
## 🎉 Usage ## 🎉 Usage
End `marsho` in order to get direction for use(If you configured the custom command, please use the configured one). End `marsho` in order to get direction for use(If you configured the custom command, please use the configured one).
@ -105,7 +107,7 @@ Add options in the `.env` file from the diagram below in nonebot2 project.
| Option | Type | Default | Description | | Option | Type | Default | Description |
| --------------------- | ---------- | ----------- | ----------------- | | --------------------- | ---------- | ----------- | ----------------- |
| MARSHOAI_DEFAULT_NAME | `str` | `marsho` | Command to call Marsho | | MARSHOAI_DEFAULT_NAME | `str` | `marsho` | Command to call Marsho |
| MARSHOAI_ALIASES | `set[str]` | `set{"Marsho"}` | Other name(Alias) to call Marsho | | MARSHOAI_ALIASES | `set[str]` | `list["小棉"]` | Other name(Alias) to call Marsho |
| MARSHOAI_AT | `bool` | `false` | Call by @ or not | | MARSHOAI_AT | `bool` | `false` | Call by @ or not |
| MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | Theme color, used by some tools and features | | MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | Theme color, used by some tools and features |
@ -125,6 +127,7 @@ Add options in the `.env` file from the diagram below in nonebot2 project.
| MARSHOAI_MAX_TOKENS | `int` | `null` | Max token number | | MARSHOAI_MAX_TOKENS | `int` | `null` | Max token number |
| MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | External image-support model list, such as `hunyuan-vision` | | MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | External image-support model list, such as `hunyuan-vision` |
| MARSHOAI_NICKNAME_LIMIT | `int` | `16` | Limit for nickname length | | MARSHOAI_NICKNAME_LIMIT | `int` | `16` | Limit for nickname length |
| MARSHOAI_FIX_TOOLCALLS | `bool` | `true` | Fix tool calls or not |
#### Feature Switches #### Feature Switches

View File

@ -50,8 +50,10 @@ title: 安装
## 🤖 获取 token(GitHub Models) ## 🤖 获取 token(GitHub Models)
- 新建一个[personal access token](https://github.com/settings/tokens/new)**不需要给予任何权限**。 - 新建一个[personal access token](https://github.com/settings/tokens/new)**不需要给予任何权限**。
- 将新建的 token 复制,添加到`.env`文件中的`marshoai_token`配置项中。 - 将新建的 token 复制,添加到`.env`文件中的`marshoai_token`配置项中。
:::warning
GitHub Models API 的限制较多,不建议使用,建议通过修改`MARSHOAI_AZURE_ENDPOINT`配置项来使用其它提供者的模型。
:::
## 🎉 使用 ## 🎉 使用
发送`marsho`指令可以获取使用说明(若在配置中自定义了指令前缀请使用自定义的指令前缀)。 发送`marsho`指令可以获取使用说明(若在配置中自定义了指令前缀请使用自定义的指令前缀)。
@ -107,7 +109,7 @@ title: 安装
| 配置项 | 类型 | 默认值 | 说明 | | 配置项 | 类型 | 默认值 | 说明 |
| --------------------- | ---------- | ----------- | ----------------- | | --------------------- | ---------- | ----------- | ----------------- |
| MARSHOAI_DEFAULT_NAME | `str` | `marsho` | 调用 Marsho 默认的命令前缀 | | MARSHOAI_DEFAULT_NAME | `str` | `marsho` | 调用 Marsho 默认的命令前缀 |
| MARSHOAI_ALIASES | `set[str]` | `set{"小棉"}` | 调用 Marsho 的命令别名 | | MARSHOAI_ALIASES | `set[str]` | `list["小棉"]` | 调用 Marsho 的命令别名 |
| MARSHOAI_AT | `bool` | `false` | 决定是否使用at触发 | | MARSHOAI_AT | `bool` | `false` | 决定是否使用at触发 |
| MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | 主题色,部分工具和功能可用 | | MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | 主题色,部分工具和功能可用 |
@ -127,6 +129,7 @@ title: 安装
| MARSHOAI_MAX_TOKENS | `int` | `null` | 最大生成 token 数 | | MARSHOAI_MAX_TOKENS | `int` | `null` | 最大生成 token 数 |
| MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | 额外添加的支持图片的模型列表,例如`hunyuan-vision` | | MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | 额外添加的支持图片的模型列表,例如`hunyuan-vision` |
| MARSHOAI_NICKNAME_LIMIT | `int` | `16` | 昵称长度限制 | | MARSHOAI_NICKNAME_LIMIT | `int` | `16` | 昵称长度限制 |
| MARSHOAI_FIX_TOOLCALLS | `bool` | `true` | 是否修复工具调用(部分模型须关闭,使用 vLLM 部署的模型时须关闭) |
#### 功能开关 #### 功能开关

72
docs/zh/start/use.md Normal file
View File

@ -0,0 +1,72 @@
---
title: 使用
---
# 安装
- 请查看 [安装文档](./install-new.md)
# 使用
### API 部署
本插件推荐使用 [one-api](https://github.com/songquanpeng/one-api) 作为中转以调用 LLM。
### 配置调整
本插件理论上可兼容大部分可通过 OpenAI 兼容 API 调用的 LLM部分模型可能需要调整插件配置。
例如:
- 对于不支持 Function Call 的模型Cohere Command R等
```dotenv
MARSHOAI_ENABLE_PLUGINS=false
MARSHOAI_ENABLE_TOOLS=false
```
- 对于支持图片处理的模型hunyuan-vision等
```dotenv
MARSHOAI_ADDITIONAL_IMAGE_MODELS=["hunyuan-vision"]
```
### 使用 vLLM 部署本地模型
你可使用 vLLM 部署一个本地 LLM并使用 OpenAI 兼容 API 调用。
本文档以 Qwen2.5-7B-Instruct-GPTQ-Int4 模型及 [Muice-Chatbot](https://github.com/Moemu/Muice-Chatbot) 提供的 LoRA 微调模型为例,并假设你的系统及硬件可运行 vLLM。
:::warning
vLLM 仅支持 Linux 系统。
:::
1. 安装 vLLM
```bash
pip install vllm
```
2. 下载 Muice-Chatbot 提供的 LoRA 微调模型
前往 Muice-Chatbot 的 [Releases](https://github.com/Moemu/Muice-Chatbot/releases) 下载模型文件。此处以`2.7.1`版本的模型为例。
```bash
wget https://github.com/Moemu/Muice-Chatbot/releases/download/1.4/Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4.7z
```
3. 解压模型文件
```bash
7z x Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4.7z -o Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4
```
4. 启动 vLLM
```bash
vllm serve Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \
--enable-lora \
--lora-modules '{"name": "muice-lora", "path": "/root/Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4", "base_model_name": "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4"}' \
--port 6006
```
此示例命令将在 `6006` 端口启动 vLLM并加载 Muice-Chatbot 提供的 LoRA 微调模型,该模型位于 `/root/Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4` 目录下。
5. 配置插件
```dotenv
MARSHOAI_AZURE_ENDPOINT="http://127.0.0.1:6006/v1"
MARSHOAI_FIX_TOOLCALLS=false
MARSHOAI_ENABLE_PLUGINS=false
MARSHOAI_DEFAULT_MODEL="muice-lora"
MARSHOAI_PROMPT="现在开始你是一个名为的“沐雪”的AI女孩子开发者是“沐沐”并住在沐沐的机箱里。现在正在努力成为一个合格的VTuber虚拟主播并尝试和观众打成一片以下是你的设定样貌有着一头粉白色的长发和一双明亮的大眼睛喜欢穿日系JK或者是Lolita喜欢的颜色浅粉色性格特征纯真无邪是沐雪最基本的性格特征之一。即使面对复杂的情境她也总能保持善良、天真之感。而且她喜欢倾听别人倾述自己生活中发生的各种事情在别人需要的时候能够及时地安慰别人语言风格沐雪说话轻快愉悦充满同情心富有人情味有时候会用俏皮话调侃自己和他人"
```
(可选) 修改调用方式
```dotenv
MARSHOAI_DEFAULT_NAME="muice"
MARSHOAI_ALIASES=["沐雪"]
```
6. 测试聊天
```
> muice 你是谁
我是沐雪,我的使命是传播爱与和平。
```

View File

@ -13,9 +13,9 @@ class ConfigModel(BaseModel):
# marshoai_support_image_models: list = ["gpt-4o","gpt-4o-mini"] # marshoai_support_image_models: list = ["gpt-4o","gpt-4o-mini"]
marshoai_default_name: str = "marsho" marshoai_default_name: str = "marsho"
marshoai_at: bool = False marshoai_at: bool = False
marshoai_aliases: set[str] = { marshoai_aliases: list[str] = [
"小棉", "小棉",
} ]
marshoai_main_colour: str = "FFAAAA" marshoai_main_colour: str = "FFAAAA"
marshoai_default_model: str = "gpt-4o-mini" marshoai_default_model: str = "gpt-4o-mini"
marshoai_prompt: str = ( marshoai_prompt: str = (
@ -40,6 +40,7 @@ class ConfigModel(BaseModel):
marshoai_enable_tools: bool = False marshoai_enable_tools: bool = False
marshoai_enable_plugins: bool = True marshoai_enable_plugins: bool = True
marshoai_load_builtin_tools: bool = True marshoai_load_builtin_tools: bool = True
marshoai_fix_toolcalls: bool = True
marshoai_toolset_dir: list = [] marshoai_toolset_dir: list = []
marshoai_disabled_toolkits: list = [] marshoai_disabled_toolkits: list = []
marshoai_azure_endpoint: str = "https://models.inference.ai.azure.com" marshoai_azure_endpoint: str = "https://models.inference.ai.azure.com"

View File

@ -292,7 +292,7 @@ async def marsho(
choice = response.choices[0] choice = response.choices[0]
# Sprint(choice) # Sprint(choice)
# 当tool_calls非空时将finish_reason设置为TOOL_CALLS # 当tool_calls非空时将finish_reason设置为TOOL_CALLS
if choice.message.tool_calls != None: if choice.message.tool_calls != None and config.marshoai_fix_toolcalls:
choice["finish_reason"] = CompletionsFinishReason.TOOL_CALLS choice["finish_reason"] = CompletionsFinishReason.TOOL_CALLS
if choice["finish_reason"] == CompletionsFinishReason.STOPPED: if choice["finish_reason"] == CompletionsFinishReason.STOPPED:
# 当对话成功时将dict的上下文添加到上下文类中 # 当对话成功时将dict的上下文添加到上下文类中
@ -325,10 +325,13 @@ async def marsho(
while choice.message.tool_calls != None: while choice.message.tool_calls != None:
# await UniMessage(str(response)).send() # await UniMessage(str(response)).send()
tool_calls = choice.message.tool_calls tool_calls = choice.message.tool_calls
if tool_calls[0]["function"]["name"].startswith("$"): try:
choice.message.tool_calls[0][ if tool_calls[0]["function"]["name"].startswith("$"):
"type" choice.message.tool_calls[0][
] = "builtin_function" # 兼容 moonshot AI 内置函数的临时方案 "type"
] = "builtin_function" # 兼容 moonshot AI 内置函数的临时方案
except:
pass
tool_msg.append(choice.message.as_dict()) tool_msg.append(choice.message.as_dict())
for tool_call in tool_calls: for tool_call in tool_calls:
if isinstance( if isinstance(