流式调用 90%

CNAME

@@ -1 +1 @@
-marsho.liteyuki.icu
+marshoai-docs.meli.liteyuki.icu

README.md

@@ -1,6 +1,6 @@
 <!--suppress LongLine -->
 <div align="center">
-  <a href="https://marsho.liteyuki.icu"><img src="https://marsho.liteyuki.icu/marsho-full.svg" width="800" height="430" alt="MarshoLogo"></a>
+  <a href="https://marshoai-docs.meli.liteyuki.icu"><img src="https://marshoai-docs.meli.liteyuki.icu/marsho-full.svg" width="800" height="430" alt="MarshoLogo"></a>
   <br>
 </div>
 
@@ -48,7 +48,7 @@ _谁不喜欢回复消息快又可爱的猫娘呢？_
 
 ## 😼 使用
 
-请查看[使用文档](https://marsho.liteyuki.icu/start/use)
+请查看[使用文档](https://marshoai-docs.meli.liteyuki.icu/start/use)
 
 ## ❤ 鸣谢&版权说明
 

README_EN.md

@@ -1,6 +1,6 @@
 <!--suppress LongLine -->
 <div align="center">
-  <a href="https://marsho.liteyuki.icu"><img src="https://marsho.liteyuki.icu/marsho-full.svg" width="800" height="430" alt="MarshoLogo"></a>
+  <a href="https://marshoai-docs.meli.liteyuki.icu"><img src="https://marshoai-docs.meli.liteyuki.icu/marsho-full.svg" width="800" height="430" alt="MarshoLogo"></a>
   <br>
 </div>
 
@@ -48,7 +48,7 @@ Plugin internally installed the catgirl character of Marsho, is able to have a c
 - 🐾 Play! I like play with friends!
 
 ## 😼 Usage
-Please read [Documentation](https://marsho.liteyuki.icu/start/install)
+Please read [Documentation](https://marshoai-docs.meli.liteyuki.icu/start/install)
 
 ## ❤ Thanks&Copyright
 This project uses the following code from other projects:  

docs/en/start/install.md

@@ -65,7 +65,7 @@ When nonebot linked to OneBot v11 adapter, can recieve double click and response
 MarshoTools is a feature added in `v0.5.0`, support loading external function library to provide Function Call for Marsho. 
 
 ## 🧩 Marsho Plugin
-Marsho Plugin is a feature added in `v1.0.0`, replacing the old MarshoTools feature. [Documentation](https://marsho.liteyuki.icu/dev/extension)
+Marsho Plugin is a feature added in `v1.0.0`, replacing the old MarshoTools feature. [Documentation](https://marshoai-docs.meli.liteyuki.icu/dev/extension)
 
 ## 👍 Praise list
 
@@ -148,3 +148,4 @@ Add options in the `.env` file from the diagram below in nonebot2 project.
 | MARSHOAI_SINGLE_LATEX_PARSE       | `bool` | `false`| Render single-line equation or not    |
 | MARSHOAI_FIX_TOOLCALLS           | `bool`  | `true` | Fix tool calls or not |
 | MARSHOAI_SEND_THINKING           | `bool`  | `true` | Send thinking chain or not |
+| MARSHOAI_STREAM                  | `bool`  | `false`| 是否通过流式方式请求 API **开启此项后暂无法使用函数调用，无法在 Bot 用户侧聊天界面呈现出流式效果** |

docs/zh/start/install.md

@@ -68,7 +68,7 @@ GitHub Models API 的限制较多，不建议使用，建议通过修改`MARSHOA
 
 ## 🧩 小棉插件
 
-小棉插件是`v1.0.0`的新增功能，替代旧的小棉工具功能。[使用文档](https://marsho.liteyuki.icu/dev/extension)
+小棉插件是`v1.0.0`的新增功能，替代旧的小棉工具功能。[使用文档](https://marshoai-docs.meli.liteyuki.icu/dev/extension)
 
 ## 👍 夸赞名单
 
@@ -149,6 +149,8 @@ GitHub Models API 的限制较多，不建议使用，建议通过修改`MARSHOA
 | MARSHOAI_SINGLE_LATEX_PARSE    | `bool` | `false` |   单行公式是否渲染（当消息富文本解析启用时可用）（如果单行也渲……只能说不好看）           |
 | MARSHOAI_FIX_TOOLCALLS           | `bool`  | `true` | 是否修复工具调用（部分模型须关闭，使用 vLLM 部署的模型时须关闭） |
 | MARSHOAI_SEND_THINKING           | `bool`  | `true` | 是否发送思维链（部分模型不支持） |
+| MARSHOAI_STREAM                  | `bool`  | `false`| 是否通过流式方式请求 API **开启此项后暂无法使用函数调用，无法在 Bot 用户侧聊天界面呈现出流式效果** |
+
 
 #### 开发及调试选项
 

nonebot_plugin_marshoai/config.py

@@ -32,6 +32,7 @@ class ConfigModel(BaseModel):
     marshoai_enable_sysasuser_prompt: bool = False
     marshoai_additional_prompt: str = ""
     marshoai_poke_suffix: str = "揉了揉你的猫耳"
+    marshoai_stream: bool = False
     marshoai_enable_richtext_parse: bool = True
     """
     是否启用自动消息富文本解析 即若包含图片链接则发送图片、若包含LaTeX公式则发送公式图。

nonebot_plugin_marshoai/constants.py

@@ -37,7 +37,7 @@ OPENAI_NEW_MODELS: list = [
 INTRODUCTION: str = f"""MarshoAI-NoneBot by LiteyukiStudio
 你好喵~我是一只可爱的猫娘AI，名叫小棉~🐾！
 我的主页在这里哦~↓↓↓
-https://marsho.liteyuki.icu
+https://marshoai-docs.meli.liteyuki.icu
 
 ※ 使用 「{config.marshoai_default_name}.status」命令获取状态信息。
 ※ 使用「{config.marshoai_default_name}.help」命令获取使用说明。"""

nonebot_plugin_marshoai/handler.py

@@ -20,6 +20,7 @@ from nonebot.matcher import (
 from nonebot_plugin_alconna.uniseg import UniMessage, UniMsg
 from openai import AsyncOpenAI, AsyncStream
 from openai.types.chat import ChatCompletion, ChatCompletionChunk, ChatCompletionMessage
+from openai.types.chat.chat_completion import Choice
 
 from .config import config
 from .constants import SUPPORT_IMAGE_MODELS
@@ -94,7 +95,7 @@ class MarshoHandler:
         self,
         user_message: Union[str, list],
         model_name: str,
-        tools_list: list,
+        tools_list: list | None,
         tool_message: Optional[list] = None,
         stream: bool = False,
     ) -> Union[ChatCompletion, AsyncStream[ChatCompletionChunk]]:
@@ -196,8 +197,14 @@ class MarshoHandler:
         """
         global target_list
         if stream:
-            raise NotImplementedError
+            response = await self.handle_stream_request(
-        response = await self.handle_single_chat(
+                user_message=user_message,
+                model_name=model_name,
+                tools_list=tools_list,
+                tools_message=tool_message,
+            )
+        else:
+            response = await self.handle_single_chat(  # type: ignore
                 user_message=user_message,
                 model_name=model_name,
                 tools_list=tools_list,
@@ -250,22 +257,35 @@ class MarshoHandler:
             return None
 
     async def handle_stream_request(
-        self, user_message: Union[str, list], model_name: str, tools_list: list
+        self,
-    ):
+        user_message: Union[str, list],
+        model_name: str,
+        tools_list: list,
+        tools_message: Optional[list] = None,
+    ) -> Union[ChatCompletion, None]:
         """
         处理流式请求
         """
         response = await self.handle_single_chat(
             user_message=user_message,
             model_name=model_name,
-            tools_list=tools_list,
+            tools_list=None,  # TODO:让流式调用支持工具调用
+            tool_message=tools_message,
             stream=True,
         )
 
         if isinstance(response, AsyncStream):
             reasoning_contents = ""
             answer_contents = ""
+            last_chunk = None
+            is_first_token_appeared = False
+            is_answering = False
             async for chunk in response:
+                last_chunk = chunk
+                # print(chunk)
+                if not is_first_token_appeared:
+                    logger.debug(f"{chunk.id}: 第一个 token 已出现")
+                    is_first_token_appeared = True
                 if not chunk.choices:
                     logger.info("Usage:", chunk.usage)
                 else:
@@ -276,5 +296,33 @@ class MarshoHandler:
                     ):
                         reasoning_contents += delta.reasoning_content
                     else:
+                        if not is_answering:
+                            logger.debug(
+                                f"{chunk.id}: 思维链已输出完毕或无 reasoning_content 字段输出"
+                            )
+                            is_answering = True
                         if delta.content is not None:
                             answer_contents += delta.content
+            # print(last_chunk)
+            # 创建新的 ChatCompletion 对象
+            if last_chunk and last_chunk.choices:
+                message = ChatCompletionMessage(
+                    content=answer_contents,
+                    role="assistant",
+                    tool_calls=last_chunk.choices[0].delta.tool_calls,  # type: ignore
+                )
+                choice = Choice(
+                    finish_reason=last_chunk.choices[0].finish_reason,  # type: ignore
+                    index=last_chunk.choices[0].index,
+                    message=message,
+                )
+                return ChatCompletion(
+                    id=last_chunk.id,
+                    choices=[choice],
+                    created=last_chunk.created,
+                    model=last_chunk.model,
+                    system_fingerprint=last_chunk.system_fingerprint,
+                    object="chat.completion",
+                    usage=last_chunk.usage,
+                )
+        return None

nonebot_plugin_marshoai/marsho.py

@@ -257,7 +257,9 @@ async def marsho(
         )
         logger.info(f"正在获取回答，模型：{model_name}")
         # logger.info(f"上下文：{context_msg}")
-        response = await handler.handle_common_chat(usermsg, model_name, tools_lists)
+        response = await handler.handle_common_chat(
+            usermsg, model_name, tools_lists, config.marshoai_stream
+        )
         # await UniMessage(str(response)).send()
         if response is not None:
             context_user, context_assistant = response
@@ -293,7 +295,7 @@ with contextlib.suppress(ImportError):  # 优化先不做（）
                         ),
                     ],
                 )
-                choice = response.choices[0]
+                choice = response.choices[0]  # type: ignore
                 if choice.finish_reason == CompletionsFinishReason.STOPPED:
                     content = extract_content_and_think(choice.message)[0]
                     await UniMessage(" " + str(content)).send(at_sender=True)

nonebot_plugin_marshoai/metadata.py

@@ -5,7 +5,7 @@ from .constants import USAGE
 
 metadata = PluginMetadata(
     name="Marsho AI 插件",
-    description="接入 Azure API 或其他 API 的 AI 聊天插件，支持图片处理，外部函数调用，兼容包括 DeepSeek-R1 在内的多个模型",
+    description="接入 Azure API 或其他 API 的 AI 聊天插件，支持图片处理，外部函数调用，兼容包括 DeepSeek-R1， QwQ-32B 在内的多个模型",
     usage=USAGE,
     type="application",
     config=ConfigModel,

pyproject.toml

@@ -34,7 +34,7 @@ dependencies = [
 license = { text = "MIT, Mulan PSL v2" }
 
 [project.urls]
-Homepage = "https://marsho.liteyuki.icu/"
+Homepage = "https://marshoai-docs.meli.liteyuki.icu/"
 
 
 [tool.nonebot]