mirror of
https://github.com/LiteyukiStudio/nonebot-plugin-marshoai.git
synced 2025-02-24 20:29:25 +08:00
✨ 更新安装文档,添加关于GitHub Models API的警告信息,并调整配置项说明
This commit is contained in:
parent
c8e776d5ff
commit
ef71514ce2
@ -49,7 +49,9 @@ Open the `pyproject.toml` file under nonebot2's root directory, Add to`[tool.non
|
|||||||
|
|
||||||
- Create new [personal access token](https://github.com/settings/tokens/new),**Don't need any permissions**.
|
- Create new [personal access token](https://github.com/settings/tokens/new),**Don't need any permissions**.
|
||||||
- Copy the new token, add to the `.env` file's `marshoai_token` option.
|
- Copy the new token, add to the `.env` file's `marshoai_token` option.
|
||||||
|
:::warning
|
||||||
|
GitHub Models API comes with significant limitations and is therefore not recommended for use. For better alternatives, it's suggested to adjust the configuration `MARSHOAI_AZURE_ENDPOINT` to use other service providers' models instead.
|
||||||
|
:::
|
||||||
## 🎉 Usage
|
## 🎉 Usage
|
||||||
|
|
||||||
End `marsho` in order to get direction for use(If you configured the custom command, please use the configured one).
|
End `marsho` in order to get direction for use(If you configured the custom command, please use the configured one).
|
||||||
@ -105,7 +107,7 @@ Add options in the `.env` file from the diagram below in nonebot2 project.
|
|||||||
| Option | Type | Default | Description |
|
| Option | Type | Default | Description |
|
||||||
| --------------------- | ---------- | ----------- | ----------------- |
|
| --------------------- | ---------- | ----------- | ----------------- |
|
||||||
| MARSHOAI_DEFAULT_NAME | `str` | `marsho` | Command to call Marsho |
|
| MARSHOAI_DEFAULT_NAME | `str` | `marsho` | Command to call Marsho |
|
||||||
| MARSHOAI_ALIASES | `set[str]` | `set{"Marsho"}` | Other name(Alias) to call Marsho |
|
| MARSHOAI_ALIASES | `set[str]` | `list["小棉"]` | Other name(Alias) to call Marsho |
|
||||||
| MARSHOAI_AT | `bool` | `false` | Call by @ or not |
|
| MARSHOAI_AT | `bool` | `false` | Call by @ or not |
|
||||||
| MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | Theme color, used by some tools and features |
|
| MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | Theme color, used by some tools and features |
|
||||||
|
|
||||||
@ -125,6 +127,7 @@ Add options in the `.env` file from the diagram below in nonebot2 project.
|
|||||||
| MARSHOAI_MAX_TOKENS | `int` | `null` | Max token number |
|
| MARSHOAI_MAX_TOKENS | `int` | `null` | Max token number |
|
||||||
| MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | External image-support model list, such as `hunyuan-vision` |
|
| MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | External image-support model list, such as `hunyuan-vision` |
|
||||||
| MARSHOAI_NICKNAME_LIMIT | `int` | `16` | Limit for nickname length |
|
| MARSHOAI_NICKNAME_LIMIT | `int` | `16` | Limit for nickname length |
|
||||||
|
| MARSHOAI_FIX_TOOLCALLS | `bool` | `true` | Fix tool calls or not |
|
||||||
|
|
||||||
#### Feature Switches
|
#### Feature Switches
|
||||||
|
|
||||||
|
@ -51,7 +51,9 @@ title: 安装
|
|||||||
|
|
||||||
- 新建一个[personal access token](https://github.com/settings/tokens/new),**不需要给予任何权限**。
|
- 新建一个[personal access token](https://github.com/settings/tokens/new),**不需要给予任何权限**。
|
||||||
- 将新建的 token 复制,添加到`.env`文件中的`marshoai_token`配置项中。
|
- 将新建的 token 复制,添加到`.env`文件中的`marshoai_token`配置项中。
|
||||||
|
:::warning
|
||||||
|
GitHub Models API 的限制较多,不建议使用,建议通过修改`MARSHOAI_AZURE_ENDPOINT`配置项来使用其它提供者的模型。
|
||||||
|
:::
|
||||||
## 🎉 使用
|
## 🎉 使用
|
||||||
|
|
||||||
发送`marsho`指令可以获取使用说明(若在配置中自定义了指令前缀请使用自定义的指令前缀)。
|
发送`marsho`指令可以获取使用说明(若在配置中自定义了指令前缀请使用自定义的指令前缀)。
|
||||||
@ -107,7 +109,7 @@ title: 安装
|
|||||||
| 配置项 | 类型 | 默认值 | 说明 |
|
| 配置项 | 类型 | 默认值 | 说明 |
|
||||||
| --------------------- | ---------- | ----------- | ----------------- |
|
| --------------------- | ---------- | ----------- | ----------------- |
|
||||||
| MARSHOAI_DEFAULT_NAME | `str` | `marsho` | 调用 Marsho 默认的命令前缀 |
|
| MARSHOAI_DEFAULT_NAME | `str` | `marsho` | 调用 Marsho 默认的命令前缀 |
|
||||||
| MARSHOAI_ALIASES | `set[str]` | `set{"小棉"}` | 调用 Marsho 的命令别名 |
|
| MARSHOAI_ALIASES | `set[str]` | `list["小棉"]` | 调用 Marsho 的命令别名 |
|
||||||
| MARSHOAI_AT | `bool` | `false` | 决定是否使用at触发 |
|
| MARSHOAI_AT | `bool` | `false` | 决定是否使用at触发 |
|
||||||
| MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | 主题色,部分工具和功能可用 |
|
| MARSHOAI_MAIN_COLOUR | `str` | `FFAAAA` | 主题色,部分工具和功能可用 |
|
||||||
|
|
||||||
@ -127,6 +129,7 @@ title: 安装
|
|||||||
| MARSHOAI_MAX_TOKENS | `int` | `null` | 最大生成 token 数 |
|
| MARSHOAI_MAX_TOKENS | `int` | `null` | 最大生成 token 数 |
|
||||||
| MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | 额外添加的支持图片的模型列表,例如`hunyuan-vision` |
|
| MARSHOAI_ADDITIONAL_IMAGE_MODELS | `list` | `[]` | 额外添加的支持图片的模型列表,例如`hunyuan-vision` |
|
||||||
| MARSHOAI_NICKNAME_LIMIT | `int` | `16` | 昵称长度限制 |
|
| MARSHOAI_NICKNAME_LIMIT | `int` | `16` | 昵称长度限制 |
|
||||||
|
| MARSHOAI_FIX_TOOLCALLS | `bool` | `true` | 是否修复工具调用(部分模型须关闭,使用 vLLM 部署的模型时须关闭) |
|
||||||
|
|
||||||
#### 功能开关
|
#### 功能开关
|
||||||
|
|
||||||
|
72
docs/zh/start/use.md
Normal file
72
docs/zh/start/use.md
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
---
|
||||||
|
title: 使用
|
||||||
|
---
|
||||||
|
|
||||||
|
# 安装
|
||||||
|
- 请查看 [安装文档](./install-new.md)
|
||||||
|
|
||||||
|
# 使用
|
||||||
|
### API 部署
|
||||||
|
|
||||||
|
本插件推荐使用 [one-api](https://github.com/songquanpeng/one-api) 作为中转以调用 LLM。
|
||||||
|
### 配置调整
|
||||||
|
|
||||||
|
本插件理论上可兼容大部分可通过 OpenAI 兼容 API 调用的 LLM,部分模型可能需要调整插件配置。
|
||||||
|
|
||||||
|
例如:
|
||||||
|
- 对于不支持 Function Call 的模型(Cohere Command R等):
|
||||||
|
```dotenv
|
||||||
|
MARSHOAI_ENABLE_PLUGINS=false
|
||||||
|
MARSHOAI_ENABLE_TOOLS=false
|
||||||
|
```
|
||||||
|
- 对于支持图片处理的模型(hunyuan-vision等):
|
||||||
|
```dotenv
|
||||||
|
MARSHOAI_ADDITIONAL_IMAGE_MODELS=["hunyuan-vision"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 使用 vLLM 部署本地模型
|
||||||
|
|
||||||
|
你可使用 vLLM 部署一个本地 LLM,并使用 OpenAI 兼容 API 调用。
|
||||||
|
本文档以 Qwen2.5-7B-Instruct-GPTQ-Int4 模型及 [Muice-Chatbot](https://github.com/Moemu/Muice-Chatbot) 提供的 LoRA 微调模型为例,并假设你的系统及硬件可运行 vLLM。
|
||||||
|
:::warning
|
||||||
|
vLLM 仅支持 Linux 系统。
|
||||||
|
:::
|
||||||
|
1. 安装 vLLM
|
||||||
|
```bash
|
||||||
|
pip install vllm
|
||||||
|
```
|
||||||
|
2. 下载 Muice-Chatbot 提供的 LoRA 微调模型
|
||||||
|
前往 Muice-Chatbot 的 [Releases](https://github.com/Moemu/Muice-Chatbot/releases) 下载模型文件。此处以`2.7.1`版本的模型为例。
|
||||||
|
```bash
|
||||||
|
wget https://github.com/Moemu/Muice-Chatbot/releases/download/1.4/Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4.7z
|
||||||
|
```
|
||||||
|
3. 解压模型文件
|
||||||
|
```bash
|
||||||
|
7z x Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4.7z -o Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4
|
||||||
|
```
|
||||||
|
4. 启动 vLLM
|
||||||
|
```bash
|
||||||
|
vllm serve Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \
|
||||||
|
--enable-lora \
|
||||||
|
--lora-modules '{"name": "muice-lora", "path": "/root/Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4", "base_model_name": "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4"}' \
|
||||||
|
--port 6006
|
||||||
|
```
|
||||||
|
此示例命令将在 `6006` 端口启动 vLLM,并加载 Muice-Chatbot 提供的 LoRA 微调模型,该模型位于 `/root/Muice-2.7.1-Qwen2.5-7B-Instruct-GPTQ-Int4-8e-4` 目录下。
|
||||||
|
5. 配置插件
|
||||||
|
```dotenv
|
||||||
|
MARSHOAI_AZURE_ENDPOINT="http://127.0.0.1:6006/v1"
|
||||||
|
MARSHOAI_FIX_TOOLCALLS=false
|
||||||
|
MARSHOAI_ENABLE_PLUGINS=false
|
||||||
|
MARSHOAI_DEFAULT_MODEL="muice-lora"
|
||||||
|
MARSHOAI_PROMPT="现在开始你是一个名为的“沐雪”的AI女孩子,开发者是“沐沐”并住在(沐沐)的机箱里。现在正在努力成为一个合格的VTuber(虚拟主播)并尝试和观众打成一片,以下是你的设定:样貌:有着一头粉白色的长发和一双明亮的大眼睛,喜欢穿日系JK或者是Lolita;喜欢的颜色:浅粉色;性格特征:纯真无邪是沐雪最基本的性格特征之一。即使面对复杂的情境,她也总能保持善良、天真之感。而且,她喜欢倾听别人倾述自己生活中发生的各种事情,在别人需要的时候,能够及时地安慰别人;语言风格:沐雪说话轻快愉悦,充满同情心,富有人情味,有时候会用俏皮话调侃自己和他人"
|
||||||
|
```
|
||||||
|
(可选) 修改调用方式
|
||||||
|
```dotenv
|
||||||
|
MARSHOAI_DEFAULT_NAME="muice"
|
||||||
|
MARSHOAI_ALIASES=["沐雪"]
|
||||||
|
```
|
||||||
|
6. 测试聊天
|
||||||
|
```
|
||||||
|
> muice 你是谁
|
||||||
|
我是沐雪,我的使命是传播爱与和平。
|
||||||
|
```
|
@ -13,9 +13,9 @@ class ConfigModel(BaseModel):
|
|||||||
# marshoai_support_image_models: list = ["gpt-4o","gpt-4o-mini"]
|
# marshoai_support_image_models: list = ["gpt-4o","gpt-4o-mini"]
|
||||||
marshoai_default_name: str = "marsho"
|
marshoai_default_name: str = "marsho"
|
||||||
marshoai_at: bool = False
|
marshoai_at: bool = False
|
||||||
marshoai_aliases: set[str] = {
|
marshoai_aliases: list[str] = [
|
||||||
"小棉",
|
"小棉",
|
||||||
}
|
]
|
||||||
marshoai_main_colour: str = "FFAAAA"
|
marshoai_main_colour: str = "FFAAAA"
|
||||||
marshoai_default_model: str = "gpt-4o-mini"
|
marshoai_default_model: str = "gpt-4o-mini"
|
||||||
marshoai_prompt: str = (
|
marshoai_prompt: str = (
|
||||||
@ -40,6 +40,7 @@ class ConfigModel(BaseModel):
|
|||||||
marshoai_enable_tools: bool = False
|
marshoai_enable_tools: bool = False
|
||||||
marshoai_enable_plugins: bool = True
|
marshoai_enable_plugins: bool = True
|
||||||
marshoai_load_builtin_tools: bool = True
|
marshoai_load_builtin_tools: bool = True
|
||||||
|
marshoai_fix_toolcalls: bool = True
|
||||||
marshoai_toolset_dir: list = []
|
marshoai_toolset_dir: list = []
|
||||||
marshoai_disabled_toolkits: list = []
|
marshoai_disabled_toolkits: list = []
|
||||||
marshoai_azure_endpoint: str = "https://models.inference.ai.azure.com"
|
marshoai_azure_endpoint: str = "https://models.inference.ai.azure.com"
|
||||||
|
@ -292,7 +292,7 @@ async def marsho(
|
|||||||
choice = response.choices[0]
|
choice = response.choices[0]
|
||||||
# Sprint(choice)
|
# Sprint(choice)
|
||||||
# 当tool_calls非空时,将finish_reason设置为TOOL_CALLS
|
# 当tool_calls非空时,将finish_reason设置为TOOL_CALLS
|
||||||
if choice.message.tool_calls != None:
|
if choice.message.tool_calls != None and config.marshoai_fix_toolcalls:
|
||||||
choice["finish_reason"] = CompletionsFinishReason.TOOL_CALLS
|
choice["finish_reason"] = CompletionsFinishReason.TOOL_CALLS
|
||||||
if choice["finish_reason"] == CompletionsFinishReason.STOPPED:
|
if choice["finish_reason"] == CompletionsFinishReason.STOPPED:
|
||||||
# 当对话成功时,将dict的上下文添加到上下文类中
|
# 当对话成功时,将dict的上下文添加到上下文类中
|
||||||
@ -325,10 +325,13 @@ async def marsho(
|
|||||||
while choice.message.tool_calls != None:
|
while choice.message.tool_calls != None:
|
||||||
# await UniMessage(str(response)).send()
|
# await UniMessage(str(response)).send()
|
||||||
tool_calls = choice.message.tool_calls
|
tool_calls = choice.message.tool_calls
|
||||||
if tool_calls[0]["function"]["name"].startswith("$"):
|
try:
|
||||||
choice.message.tool_calls[0][
|
if tool_calls[0]["function"]["name"].startswith("$"):
|
||||||
"type"
|
choice.message.tool_calls[0][
|
||||||
] = "builtin_function" # 兼容 moonshot AI 内置函数的临时方案
|
"type"
|
||||||
|
] = "builtin_function" # 兼容 moonshot AI 内置函数的临时方案
|
||||||
|
except:
|
||||||
|
pass
|
||||||
tool_msg.append(choice.message.as_dict())
|
tool_msg.append(choice.message.as_dict())
|
||||||
for tool_call in tool_calls:
|
for tool_call in tool_calls:
|
||||||
if isinstance(
|
if isinstance(
|
||||||
|
Loading…
x
Reference in New Issue
Block a user