diff --git a/CNAME b/CNAME index 5e502e62..32348c3e 100755 --- a/CNAME +++ b/CNAME @@ -1 +1 @@ -marsho.liteyuki.icu +marshoai-docs.meli.liteyuki.icu diff --git a/README.md b/README.md index 53f2eade..dab811ba 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@
- MarshoLogo + MarshoLogo
@@ -48,7 +48,7 @@ _谁不喜欢回复消息快又可爱的猫娘呢？_ ## 😼 使用 -请查看[使用文档](https://marsho.liteyuki.icu/start/use) +请查看[使用文档](https://marshoai-docs.meli.liteyuki.icu/start/use) ## ❤ 鸣谢&版权说明 diff --git a/README_EN.md b/README_EN.md index b36ce2e3..c376e03e 100644 --- a/README_EN.md +++ b/README_EN.md @@ -1,6 +1,6 @@
- MarshoLogo + MarshoLogo
@@ -48,7 +48,7 @@ Plugin internally installed the catgirl character of Marsho, is able to have a c - 🐾 Play! I like play with friends! ## 😼 Usage -Please read [Documentation](https://marsho.liteyuki.icu/start/install) +Please read [Documentation](https://marshoai-docs.meli.liteyuki.icu/start/install) ## ❤ Thanks&Copyright This project uses the following code from other projects: diff --git a/docs/en/start/install.md b/docs/en/start/install.md index 19157281..19ddd28a 100644 --- a/docs/en/start/install.md +++ b/docs/en/start/install.md @@ -65,7 +65,7 @@ When nonebot linked to OneBot v11 adapter, can recieve double click and response MarshoTools is a feature added in `v0.5.0`, support loading external function library to provide Function Call for Marsho. ## 🧩 Marsho Plugin -Marsho Plugin is a feature added in `v1.0.0`, replacing the old MarshoTools feature. [Documentation](https://marsho.liteyuki.icu/dev/extension) +Marsho Plugin is a feature added in `v1.0.0`, replacing the old MarshoTools feature. [Documentation](https://marshoai-docs.meli.liteyuki.icu/dev/extension) ## 👍 Praise list @@ -147,4 +147,5 @@ Add options in the `.env` file from the diagram below in nonebot2 project. | MARSHOAI_ENABLE_RICHTEXT_PARSE | `bool` | `true` | Turn on auto parse rich text feature(including image, LaTeX equation) | | MARSHOAI_SINGLE_LATEX_PARSE | `bool` | `false`| Render single-line equation or not | | MARSHOAI_FIX_TOOLCALLS | `bool` | `true` | Fix tool calls or not | -| MARSHOAI_SEND_THINKING | `bool` | `true` | Send thinking chain or not | \ No newline at end of file +| MARSHOAI_SEND_THINKING | `bool` | `true` | Send thinking chain or not | +| MARSHOAI_STREAM | `bool` | `false`| 是否通过流式方式请求 API **开启此项后暂无法使用函数调用，无法在 Bot 用户侧聊天界面呈现出流式效果** | diff --git a/docs/zh/start/install.md b/docs/zh/start/install.md index e35d3f18..581560ae 100644 --- a/docs/zh/start/install.md +++ b/docs/zh/start/install.md @@ -68,7 +68,7 @@ GitHub Models API 的限制较多，不建议使用，建议通过修改`MARSHOA ## 🧩 小棉插件 -小棉插件是`v1.0.0`的新增功能，替代旧的小棉工具功能。[使用文档](https://marsho.liteyuki.icu/dev/extension) +小棉插件是`v1.0.0`的新增功能，替代旧的小棉工具功能。[使用文档](https://marshoai-docs.meli.liteyuki.icu/dev/extension) ## 👍 夸赞名单 @@ -149,6 +149,8 @@ GitHub Models API 的限制较多，不建议使用，建议通过修改`MARSHOA | MARSHOAI_SINGLE_LATEX_PARSE | `bool` | `false` | 单行公式是否渲染（当消息富文本解析启用时可用）（如果单行也渲……只能说不好看） | | MARSHOAI_FIX_TOOLCALLS | `bool` | `true` | 是否修复工具调用（部分模型须关闭，使用 vLLM 部署的模型时须关闭） | | MARSHOAI_SEND_THINKING | `bool` | `true` | 是否发送思维链（部分模型不支持） | +| MARSHOAI_STREAM | `bool` | `false`| 是否通过流式方式请求 API **开启此项后暂无法使用函数调用，无法在 Bot 用户侧聊天界面呈现出流式效果** | + #### 开发及调试选项 diff --git a/nonebot_plugin_marshoai/config.py b/nonebot_plugin_marshoai/config.py index 1e0ec207..6a052dff 100644 --- a/nonebot_plugin_marshoai/config.py +++ b/nonebot_plugin_marshoai/config.py @@ -32,6 +32,7 @@ class ConfigModel(BaseModel): marshoai_enable_sysasuser_prompt: bool = False marshoai_additional_prompt: str = "" marshoai_poke_suffix: str = "揉了揉你的猫耳" + marshoai_stream: bool = False marshoai_enable_richtext_parse: bool = True """ 是否启用自动消息富文本解析 即若包含图片链接则发送图片、若包含LaTeX公式则发送公式图。 diff --git a/nonebot_plugin_marshoai/constants.py b/nonebot_plugin_marshoai/constants.py index 9f09513f..4abff051 100755 --- a/nonebot_plugin_marshoai/constants.py +++ b/nonebot_plugin_marshoai/constants.py @@ -37,7 +37,7 @@ OPENAI_NEW_MODELS: list = [ INTRODUCTION: str = f"""MarshoAI-NoneBot by LiteyukiStudio 你好喵~我是一只可爱的猫娘AI，名叫小棉~🐾！ 我的主页在这里哦~↓↓↓ -https://marsho.liteyuki.icu +https://marshoai-docs.meli.liteyuki.icu ※ 使用 「{config.marshoai_default_name}.status」命令获取状态信息。 ※ 使用「{config.marshoai_default_name}.help」命令获取使用说明。""" diff --git a/nonebot_plugin_marshoai/handler.py b/nonebot_plugin_marshoai/handler.py index a802c45b..913e1525 100644 --- a/nonebot_plugin_marshoai/handler.py +++ b/nonebot_plugin_marshoai/handler.py @@ -20,6 +20,7 @@ from nonebot.matcher import ( from nonebot_plugin_alconna.uniseg import UniMessage, UniMsg from openai import AsyncOpenAI, AsyncStream from openai.types.chat import ChatCompletion, ChatCompletionChunk, ChatCompletionMessage +from openai.types.chat.chat_completion import Choice from .config import config from .constants import SUPPORT_IMAGE_MODELS @@ -94,7 +95,7 @@ class MarshoHandler: self, user_message: Union[str, list], model_name: str, - tools_list: list, + tools_list: list | None, tool_message: Optional[list] = None, stream: bool = False, ) -> Union[ChatCompletion, AsyncStream[ChatCompletionChunk]]: @@ -196,13 +197,19 @@ class MarshoHandler: """ global target_list if stream: - raise NotImplementedError - response = await self.handle_single_chat( - user_message=user_message, - model_name=model_name, - tools_list=tools_list, - tool_message=tool_message, - ) + response = await self.handle_stream_request( + user_message=user_message, + model_name=model_name, + tools_list=tools_list, + tools_message=tool_message, + ) + else: + response = await self.handle_single_chat( # type: ignore + user_message=user_message, + model_name=model_name, + tools_list=tools_list, + tool_message=tool_message, + ) if isinstance(response, ChatCompletion): choice = response.choices[0] else: @@ -250,22 +257,35 @@ class MarshoHandler: return None async def handle_stream_request( - self, user_message: Union[str, list], model_name: str, tools_list: list - ): + self, + user_message: Union[str, list], + model_name: str, + tools_list: list, + tools_message: Optional[list] = None, + ) -> Union[ChatCompletion, None]: """ 处理流式请求 """ response = await self.handle_single_chat( user_message=user_message, model_name=model_name, - tools_list=tools_list, + tools_list=None, # TODO:让流式调用支持工具调用 + tool_message=tools_message, stream=True, ) if isinstance(response, AsyncStream): reasoning_contents = "" answer_contents = "" + last_chunk = None + is_first_token_appeared = False + is_answering = False async for chunk in response: + last_chunk = chunk + # print(chunk) + if not is_first_token_appeared: + logger.debug(f"{chunk.id}: 第一个 token 已出现") + is_first_token_appeared = True if not chunk.choices: logger.info("Usage:", chunk.usage) else: @@ -276,5 +296,33 @@ class MarshoHandler: ): reasoning_contents += delta.reasoning_content else: + if not is_answering: + logger.debug( + f"{chunk.id}: 思维链已输出完毕或无 reasoning_content 字段输出" + ) + is_answering = True if delta.content is not None: answer_contents += delta.content + # print(last_chunk) + # 创建新的 ChatCompletion 对象 + if last_chunk and last_chunk.choices: + message = ChatCompletionMessage( + content=answer_contents, + role="assistant", + tool_calls=last_chunk.choices[0].delta.tool_calls, # type: ignore + ) + choice = Choice( + finish_reason=last_chunk.choices[0].finish_reason, # type: ignore + index=last_chunk.choices[0].index, + message=message, + ) + return ChatCompletion( + id=last_chunk.id, + choices=[choice], + created=last_chunk.created, + model=last_chunk.model, + system_fingerprint=last_chunk.system_fingerprint, + object="chat.completion", + usage=last_chunk.usage, + ) + return None diff --git a/nonebot_plugin_marshoai/marsho.py b/nonebot_plugin_marshoai/marsho.py index 88d59db7..47c70aad 100644 --- a/nonebot_plugin_marshoai/marsho.py +++ b/nonebot_plugin_marshoai/marsho.py @@ -257,7 +257,9 @@ async def marsho( ) logger.info(f"正在获取回答，模型：{model_name}") # logger.info(f"上下文：{context_msg}") - response = await handler.handle_common_chat(usermsg, model_name, tools_lists) + response = await handler.handle_common_chat( + usermsg, model_name, tools_lists, config.marshoai_stream + ) # await UniMessage(str(response)).send() if response is not None: context_user, context_assistant = response @@ -293,7 +295,7 @@ with contextlib.suppress(ImportError): # 优化先不做（） ), ], ) - choice = response.choices[0] + choice = response.choices[0] # type: ignore if choice.finish_reason == CompletionsFinishReason.STOPPED: content = extract_content_and_think(choice.message)[0] await UniMessage(" " + str(content)).send(at_sender=True) diff --git a/nonebot_plugin_marshoai/metadata.py b/nonebot_plugin_marshoai/metadata.py index 04a9d4cf..85efda00 100755 --- a/nonebot_plugin_marshoai/metadata.py +++ b/nonebot_plugin_marshoai/metadata.py @@ -5,7 +5,7 @@ from .constants import USAGE metadata = PluginMetadata( name="Marsho AI 插件", - description="接入 Azure API 或其他 API 的 AI 聊天插件，支持图片处理，外部函数调用，兼容包括 DeepSeek-R1 在内的多个模型", + description="接入 Azure API 或其他 API 的 AI 聊天插件，支持图片处理，外部函数调用，兼容包括 DeepSeek-R1， QwQ-32B 在内的多个模型", usage=USAGE, type="application", config=ConfigModel, diff --git a/pyproject.toml b/pyproject.toml index c4c87157..7f80740b 100755 --- a/pyproject.toml +++ b/pyproject.toml @@ -34,7 +34,7 @@ dependencies = [ license = { text = "MIT, Mulan PSL v2" } [project.urls] -Homepage = "https://marsho.liteyuki.icu/" +Homepage = "https://marshoai-docs.meli.liteyuki.icu/" [tool.nonebot]