6 Commits
Author SHA1 Message Date
chun_qiu 488abb9730 i18n: translate web UI to Chinese
- Login/config/rules/status cards fully localized
- Event rule labels, buttons, placeholders and hints in Chinese
- Keep English only in code comments and API internals
2026-08-08 16:20:58 +08:00
chun_qiu f3a71e908e feat: user-customizable event voice rules (gift/blind box/enter/follow/like/guard/super chat)
- New event_rules.py: load/save event_rules.json, robust template renderer
  (missing vars -> empty, empty template -> muted), guard level mapping
- danmaku_handler.py: custom SEND_GIFT callback to extract blind_gift_name;
  implements all event callbacks feeding the TTS queue per user rules
- server.py: GET/POST /api/rules, load rules at startup, pass to client
- Web UI: 'Event Voice Rules' panel (per-event toggle + template + save)
- main.py CLI loads rules; build.py bundles event_rules.json
- skip_free_gift option to mute silver free gifts
- Rebuild portable zip (83.8 MB)
2026-08-08 16:17:28 +08:00
chun_qiu f4c99b1692 fix: serialize TTSBridge IO and TTSService lifecycle to prevent concurrency crashes
- TTSBridge.synthesize wraps stdin/stdout request-response in asyncio.Lock;
  concurrent calls previously caused 'readuntil() called while another
  coroutine is already waiting' and ENOENT (temp dir deleted while bridge
  still reading input.txt)
- TTSService.start/stop guarded by lifecycle lock to prevent race from
  rapid Start clicks creating duplicate bridge/worker/danmaku connections
- Verified: 20 concurrent synthesize calls all succeed with correct WAV
- Rebuild portable zip (83.8 MB)
2026-08-08 15:38:08 +08:00
chun_qiu f27bcfb055 fix: AquesTalk truncates speech on space - use 、 as word separator
Root cause: AquesTalk_Synthe silently drops all text after a space.
English segments previously joined with spaces (word gap), so anything
after the first space was silent. Replace all spaces with 、 (Japanese
comma pause) which AquesTalk supports natively.

- PUNCT_MAP maps space -> 、
- convertEnglishSegment joins words with 、
- synthesize collapses consecutive pauses, strips residual spaces

Verified: 'Actually这个proposal是非常creative的' WAV 19316 -> 79196 bytes
2026-08-08 15:26:55 +08:00
chun_qiu cb22e074af fix: OOV english words now transcribed via phonemize G2P + IPA
- Add phonemize (MIT) rule-based G2P as second fallback tier
- IPA phoneme -> katakana CV-syllable transcription table
- proposal -> プラポーザル, bilibili -> バイリーバイリー (no more letter-spelling)
- Letter spelling kept as last resort only
- Bridge logs SYNTH IN/OUT to stderr; tts.py drains stderr to Python logs
  so conversion results are visible in the server log
- Rebuild portable zip (83.8 MB)
2026-08-08 15:21:00 +08:00
chun_qiu 0c0e63161d feat: replace wanakana with english-to-kana 49K dictionary
- Vendor english-kana-matcher.js (english-to-kana, MIT, 49216 words)
- Rewrite bridge: dictionary lookup + letter-by-letter fallback for OOV words
- Convert English punctuation (,.!?) to Japanese pauses (、。)
- Fix Chinese fullwidth comma being stripped by filter_kana
- Preserve decimal points in numbers (3.5 -> スリーファイブ)
- main.py now applies filter_kana (matches web UI behavior)
- Remove wanakana dependency; rebuild portable zip (82.5 MB)
2026-08-08 15:06:34 +08:00
14 changed files with 917 additions and 143 deletions
+82 -2
View File
@@ -8,7 +8,7 @@ Bilibili 直播弹幕 + ゆっくり TTS 语音朗读器。
- 实时接收 Bilibili 直播弹幕
- 中文 → 拼音 → 片假名自动转换
- 英文按罗马音转片假名(wanakana
- 英文转片假名(english-to-kana 49K 词库 + 字母拼读兜底
- 可选扫码登录(获取未打码用户名)
- Web 控制面板(暗色主题)
- 支持 8 种ゆっくり音色 + 语速/音量调节
@@ -85,6 +85,41 @@ python main.py --room-id ROOM_ID [--voice VOICE] [--speed SPEED]
| `imd1` | imd1 |
| `jgr` | jgr |
## 事件语音规则
除弹幕外,礼物、盲盒、进房、上舰、醒目留言等事件也可以自定义语音。规则保存在 `event_rules.json`(首次运行自动生成),可在 Web UI 的 "Event Voice Rules" 面板编辑。
模板为空字符串 = 该事件静音。`enabled` 控制整类事件开关,`skip_free_gift` 跳过银瓜子免费礼物。
### 事件与变量
| 事件 | 变量 |
|------|------|
| `danmaku` 弹幕 | `{uname}` `{msg}` |
| `gift` 普通礼物 | `{viewer_name}` `{gift_name}` `{gift_num}` `{gift_price}` `{gift_coin_type}` |
| `blind_box` 盲盒礼物 | `{viewer_name}` `{gift_name}` `{gift_num}` `{blind_box_gift}` |
| `enter_room` 进房 | `{viewer_name}` `{user_uid}` |
| `follow` 关注 | `{viewer_name}` `{user_uid}` |
| `like` 点赞 | `{viewer_name}` `{user_uid}` |
| `interact_other` 其他互动 | `{viewer_name}` `{user_uid}` `{interact_type}`(分享/特别关注/互粉) |
| `buy_guard` 上舰 | `{ships_viewer_name}` `{ships_level}`(舰长/提督/总督) `{ships_num}` `{ships_price}` |
| `guard_toast` 上舰提示 | `{ships_viewer_name}` `{ships_level}` `{guard_days}` `{toast_msg}` |
| `super_chat` 醒目留言 | `{viewer_name}` `{super_chat_content}` `{super_chat_price}` `{super_chat_duration}` |
示例模板:
```json
{
"templates": {
"gift": "感谢由{viewer_name}投喂的{gift_name}",
"blind_box": "感谢由{viewer_name}投喂的{gift_name}爆出了{blind_box_gift}",
"enter_room": "欢迎{viewer_name}进入直播间",
"buy_guard": "恭喜{ships_viewer_name}成为尊贵的{ships_level}",
"super_chat": "{viewer_name}的醒目留言:{super_chat_content}"
}
}
```
## 技术架构
```
@@ -94,7 +129,7 @@ Bilibili 弹幕 (blivedm)
中文→片假名 (pypinyin + 映射表)
英文→片假名 (wanakana)
英文→片假名 (english-to-kana 49K 词库 + phonemize G2P 兜底)
AquesTalk TTS (aquestalk.js + v86 WASM 模拟)
@@ -121,3 +156,48 @@ python build.py
## 许可
MIT License
```
.
-==- +
::. %*%#*:
--.. #***:*+********-. .-+*#%%%%%%%##*+-: ..::-- -.
.*---%#=.=*+*+*+******+#@%%#%%%#%%%#%#%#%%%%#%#%%#%@%- .:-+###**+***+:+**:.-#+=
::+.-+++#+####****+**%#%%#%%#%%#%%#%#%%#%%#%#%%%%#%%#%#%%@@%#+**+****+**+**=-+#*::-+:
*-. #+.=+*+***%%%%%%#%%#%%#%#%#%#%#%%#%%#%#%#%#%#%%#%#%%%@@%%##*+***+**++*:=**#-.-+.
*###**::*%%%#%#%#%#%#%%#%%#%%#%%#%%%#%#%#%%#%#%#%%#%#%%#%#%@@@@@%#*+*+**+**=-::-*#. =
+-#=%##%#%#%#%#%%%#%#%#%#%#%#%%#%%#%#%%%#%#%%%%#%%#%%%#%%#%#%#%@@@@@#########=+*+*+=:+.
+---:%#%%%#%%#%%#%#%%%#%%#%%#%#%%#%#%%#%#%%%#%#%%#%%#%#%#%%#%#%%#%@%@@@######*++##--==
=-##%%#%#%#%%#%%#%#%#%#%%#%%%#%#%%#%#%#%#%#%#%#%#%#%#%%%#%%%%#%#%#%@%@@%##**+++%#==-
.%%%#%#%%%#%%#%#%%%#%%%%#%#%#%%#%#%%%#%%%#%%%#%%#%%#%#%#%@##%%#%%%%%%@%#+%##*+#: :-:+
:==#%#%%#%%%%#%%#%#%%#%@#%%%%%#%%#%%%%%#%#%#%#%#%%#%%%%%%%%@%%#%%##+-=-=-=-%#*++##**=:+
===--=--+%%%#%@%#%%#%%@@%%%#%@%%%%%#%@%#%#%%%#%@%%#%#%#%@%#%@**=--=--=+#%@@@%*+#%%=-%*
:%#%#@%%%%==-----+*%%@@%@#%%%@%%@@#%%#@@%%%#%@%%#@%%%#*+-----=##@%#%@@@%%@%@%@%%%*-+:%@:
%%#%@%%@@#%@@%%@%@%%@%@@%%%#%@@%@%%#%#@@@%%%%%@%@%@@%%%%@%@@@@%@@@%%%@@@%@@@@@%-+%##%@%*
*%#%@@%@@#@@%@%@@@@%%@@@@%%%%@@@@#==%@%@%@@%%#%%@@@@@%%%%%%%@%@@@%@%%#%@@@@%@%@@#######%@.
.%%%%@%@@%%@%@%%@%@%@@@%@%#@@%@%@#=-:*%@%@@%#%+%%%@%@%@%@@#%@@@%@@@@@%%%%%@@@@@%@####*+*+*%.
=%#@@@@@%%@@@%@@@@+@@%@#-#%+@@@@#-=:..*@@@%@=%*=+%@@@@@@%@%%%@@@%@@%@@%%#%@%@%@@@###*+*####%*-*
*%#@%@%@%@%@%@%@%-#%@%*+=##-@%@#==:....+@@@%-#*.:=@%@@=-%@@=*@%@@@%@@@@@%%%@@@@%@%##**+*--:++*-
##@@@%%#@@@@%@@*==*@====-%--+@*-=:... ..-%@*=.#%-=*#%%+=-+@*-=%@%@@@%@@%@@%%@%@@@%*+**#+-*%*+.
*%@%@- @%@%@@@+==#+ =####%-....=-.........+-:.# :######%=*+%.-+@@@%@@@@@@%@@@@@%@%*####+=%%.
=%%%. .@@@-#@=-=*:####*##%... ..... ... ......%####*##*%..*::.:@%@@*%%#%@@@%@%@@@#-:==+@@@@.
:%# -@--%%%--:- %******#........... .... ...*********+ +..+ *.=.-++-%@@@@@@@%@@%@%#%@@%@:
: :-#-==#:::-.=%#%%%%*.. ... .. .......... #######+ .:.:..=*=+#+#%==*%@%@%@@@@%@@@@@@@@:
#***%-:---+.-................... .. .......---=+--=:-:-**+###%=*=*@@@@@%@@@@%@%@%@@:
.#***%.:...:.......... .. .... ................:.:.:.:.-****###=-==%@%@@@%@%@@@@@@%@.
.#*+##...:...:............. ...... .. .................-*+*####=-=-@@@%@@@@@@%@%@@@%.
#***#............. .-... .........* ... ..............:**+###*=-=*%@@@%@@%@@@@@@@%%
.#+**%.................#@**+========-..................:****##*=-+%@%@@@%@@@%@%@%@@%
:=+=*##:......... ... ...*++========*..... .............:+-+*+#*=#@@@@@%@@@%@@@@@@*%*
#.=*--*...... .... .... .:#====-==-#.. ..... .........=-*=#*#%+%@%@%@@@@%@@@@%@%@-@=
-+ %@%-.. .................*+===*=....... .... .. ...=.+=-+#+=@%%@@%@@%@@@%@@@@*:%:
:%%@%@@-..... .. ... ... ......... .. .................@%+@@@@%#%@%@#%@@@%=%@%@-:#
:%%@@@%%#.. .......... ....... ........ ... .. ... . ==%#@%@%%%%%@@%-@@%@#:@@@% :*
:%%%@%.%%@#:... .. ......... .... .. .... .........+- -%%%@@@%#%@%@:-@#-@+ %@%= =.
:%%@@: %%-%@@*:..... ... ................... .. :+. =#@@%=#*%%@@- =@:.@: #@#
:%@@- %#.-@@@@#%*:....... .. .. .... .. ...:=+. #%@% -+-#@%= *- .#. *%.
+%@: #: @%@- *- .=*+=-.... ... ...-++=. .%@# . .%@= : = *.
*@: + %# =@- %:
#. .*
```
+6 -2
View File
@@ -135,14 +135,18 @@ def main() -> int:
safe_copy_tree(aq_src / "node_modules", aq_dst / "node_modules",
ignore_patterns=("bililive-touhou-tts",))
# ── 4. kuroshiro node_modules ──────────────────────────────────────
step("Copying kuroshiro node_modules")
# ── 4. node_modules (if any) ───────────────────────────────────────
if (ROOT / "node_modules").exists():
step("Copying node_modules")
safe_copy_tree(ROOT / "node_modules", BUILD_DIR / "node_modules")
# ── 5. Bridge + config ─────────────────────────────────────────────
step("Copying bridge files")
shutil.copy2(ROOT / "tts_bridge.js", BUILD_DIR / "tts_bridge.js")
shutil.copy2(ROOT / "english-kana-matcher.js", BUILD_DIR / "english-kana-matcher.js")
shutil.copy2(ROOT / "package.json", BUILD_DIR / "package.json")
if (ROOT / "event_rules.json").exists():
shutil.copy2(ROOT / "event_rules.json", BUILD_DIR / "event_rules.json")
print(" OK")
# ── 6. Launcher ────────────────────────────────────────────────────
+2 -2
View File
@@ -16,7 +16,7 @@ _KANA_SAFE_RE = re.compile(
r"\u3001\u3002\uFF01\uFF1F"
r"\u300C\u300D"
r"\u30FB\u3000"
r"a-zA-Z0-9 ]"
r"a-zA-Z0-9 ,.!?\uFF0C]"
)
_CHINESE_DIGITS = ["", "", "", "", "", "", "", "", "", ""]
@@ -85,7 +85,7 @@ def chinese_to_kana(text: str, convert_numbers: bool = True) -> str:
else:
result_parts.append(seg)
result = "".join(result_parts)
result = re.sub(r"(?<=\S) (?=\S)", "", result)
result = re.sub(r"(?<=[^\x00-\x7F]) (?=[^\x00-\x7F])", "", result)
return result
+122 -14
View File
@@ -1,7 +1,8 @@
"""Bilibili Live Danmaku Handler.
Connects to a Bilibili live room and puts incoming danmaku messages into an asyncio queue.
Supports cookie-based login with browser header emulation.
Connects to a Bilibili live room, receives danmaku/gift/guard/super-chat/
interact events, and puts user-customizable speech text into an asyncio queue.
Rules come from event_rules.json (see event_rules.py).
"""
import asyncio
@@ -9,24 +10,55 @@ import logging
from typing import Optional
import blivedm
from blivedm.models.web import DanmakuMessage
from blivedm.models.web import DanmakuMessage, GiftMessage, GuardBuyMessage, \
SuperChatMessage, InteractWordV2Message, UserToastV2Message
from bili_login import build_danmaku_session
from event_rules import (
DEFAULT_RULES, GUARD_LEVEL_NAMES, event_text, is_enabled, load_rules,
)
logger = logging.getLogger(__name__)
DANMAKU_FORMAT = "{uname}\u8bf4\u3001 {msg}"
INTERACT_TYPE_NAMES = {
3: "\u5206\u4eab", 4: "\u7279\u522b\u5173\u6ce8", 5: "\u4e92\u7c89",
}
class DanmakuHandler(blivedm.BaseHandler):
"""Handles incoming danmaku from a Bilibili live room."""
"""Handles incoming danmaku and live events from a Bilibili live room."""
def __init__(self, queue: asyncio.Queue, message_format: str = DANMAKU_FORMAT):
_CMD_CALLBACK_DICT = blivedm.BaseHandler._CMD_CALLBACK_DICT.copy()
def __init__(self, queue: asyncio.Queue, rules: dict | None = None,
message_format: str = DANMAKU_FORMAT):
super().__init__()
self._queue = queue
self._rules = rules or dict(DEFAULT_RULES)
self._message_format = message_format
self._count = 0
# ── custom SEND_GIFT callback to capture blind-gift info ──────────
def __gift_callback(self, client: blivedm.BLiveClient, command: dict):
message = GiftMessage.from_command(command["data"])
blind = command.get("data", {}).get("blind_gift") or {}
blind_name = blind.get("blind_gift_name", "") if isinstance(blind, dict) else ""
self._on_gift_raw(client, message, blind_name)
_CMD_CALLBACK_DICT["SEND_GIFT"] = __gift_callback
# ── helpers ────────────────────────────────────────────────────────
def _put(self, text: str) -> None:
if not text:
return
try:
self._queue.put_nowait(text)
except asyncio.QueueFull:
logger.warning("Danmaku queue full, dropping: %s", text)
# ── danmaku ────────────────────────────────────────────────────────
def _on_danmaku(self, client: blivedm.BLiveClient, message: DanmakuMessage) -> None:
if not message.msg.strip():
return
@@ -40,17 +72,91 @@ class DanmakuHandler(blivedm.BaseHandler):
logger.info("Danmaku #%d: %s", self._count, text)
else:
logger.debug("Danmaku #%d: %s", self._count, text)
try:
self._queue.put_nowait(text)
except asyncio.QueueFull:
logger.warning("Danmaku queue full, dropping: %s", text)
self._put(text)
def _on_gift(self, client: blivedm.BLiveClient, message) -> None:
logger.debug("Gift: %s x%d from %s", message.gift_name, message.num, message.uname)
# ── gift ───────────────────────────────────────────────────────────
def _on_gift_raw(self, client: blivedm.BLiveClient, message: GiftMessage,
blind_box_gift: str) -> None:
if self._rules.get("skip_free_gift") and message.coin_type == "silver":
logger.debug("Skip free gift: %s x%d from %s",
message.gift_name, message.num, message.uname)
return
def _on_interact_word_v2(self, client: blivedm.BLiveClient, message) -> None:
logger.debug("Interact: %s (type=%d)", message.uname, message.msg_type)
if blind_box_gift:
text = event_text(self._rules, "blind_box",
viewer_name=message.uname,
gift_name=message.gift_name,
gift_num=message.num,
blind_box_gift=blind_box_gift)
else:
text = event_text(self._rules, "gift",
viewer_name=message.uname,
gift_name=message.gift_name,
gift_num=message.num,
gift_price=message.price,
gift_coin_type=message.coin_type)
logger.info("Gift: %s x%d from %s (blind=%s)",
message.gift_name, message.num, message.uname, blind_box_gift or "-")
self._put(text)
def _on_gift(self, client: blivedm.BLiveClient, message: GiftMessage) -> None:
# fallback in case _CMD_CALLBACK_DICT override is bypassed
self._on_gift_raw(client, message, "")
# ── buy guard ──────────────────────────────────────────────────────
def _on_buy_guard(self, client: blivedm.BLiveClient, message: GuardBuyMessage) -> None:
text = event_text(self._rules, "buy_guard",
ships_viewer_name=message.username,
ships_level=GUARD_LEVEL_NAMES.get(message.guard_level, str(message.guard_level)),
ships_num=message.num,
ships_price=message.price)
logger.info("Buy guard: %s level=%d", message.username, message.guard_level)
self._put(text)
# ── guard toast ────────────────────────────────────────────────────
def _on_user_toast_v2(self, client: blivedm.BLiveClient, message: UserToastV2Message) -> None:
text = event_text(self._rules, "guard_toast",
ships_viewer_name=message.username,
ships_level=GUARD_LEVEL_NAMES.get(message.guard_level, str(message.guard_level)),
guard_days=message.num,
toast_msg=message.toast_msg)
logger.info("Guard toast: %s level=%d", message.username, message.guard_level)
self._put(text)
# ── super chat ─────────────────────────────────────────────────────
def _on_super_chat(self, client: blivedm.BLiveClient, message: SuperChatMessage) -> None:
text = event_text(self._rules, "super_chat",
viewer_name=message.uname,
super_chat_content=message.message,
super_chat_price=message.price,
super_chat_duration=message.time)
logger.info("Super chat: %s (%d yuan): %s", message.uname, message.price, message.message)
self._put(text)
# ── interact (enter room / follow / like / share) ──────────────────
def _on_interact_word_v2(self, client: blivedm.BLiveClient, message: InteractWordV2Message) -> None:
msg_type = message.msg_type
if msg_type == 1:
text = event_text(self._rules, "enter_room",
viewer_name=message.username,
user_uid=message.uid)
elif msg_type == 2:
text = event_text(self._rules, "follow",
viewer_name=message.username,
user_uid=message.uid)
elif msg_type == 6:
text = event_text(self._rules, "like",
viewer_name=message.username,
user_uid=message.uid)
else:
text = event_text(self._rules, "interact_other",
viewer_name=message.username,
user_uid=message.uid,
interact_type=INTERACT_TYPE_NAMES.get(msg_type, str(msg_type)))
logger.debug("Interact: %s (type=%d)", message.username, msg_type)
self._put(text)
# ── client stopped ─────────────────────────────────────────────────
def on_client_stopped(self, client, exception: Optional[Exception]) -> None:
if exception is not None:
logger.error("Client stopped with exception: %s", exception)
@@ -66,11 +172,13 @@ class DanmakuClient:
room_id: int,
queue: asyncio.Queue,
cookies: Optional[dict[str, str]] = None,
rules: dict | None = None,
message_format: str = DANMAKU_FORMAT,
):
self._room_id = room_id
self._queue = queue
self._cookies = cookies
self._rules = rules or dict(DEFAULT_RULES)
self._message_format = message_format
self._client: Optional[blivedm.BLiveClient] = None
self._session: Optional["aiohttp.ClientSession"] = None
@@ -79,7 +187,7 @@ class DanmakuClient:
import aiohttp
self._session = await build_danmaku_session(self._cookies)
self._client = blivedm.BLiveClient(self._room_id, session=self._session)
handler = DanmakuHandler(self._queue, self._message_format)
handler = DanmakuHandler(self._queue, self._rules, self._message_format)
self._client.set_handler(handler)
self._client.start()
logger.info("Connected to room %d", self._room_id)
File diff suppressed because one or more lines are too long
+27
View File
@@ -0,0 +1,27 @@
{
"enabled": {
"danmaku": true,
"gift": true,
"blind_box": true,
"enter_room": true,
"follow": false,
"like": false,
"interact_other": false,
"buy_guard": true,
"guard_toast": false,
"super_chat": true
},
"templates": {
"danmaku": "{uname}说、 {msg}",
"gift": "感谢由{viewer_name}投喂的{gift_name}",
"blind_box": "感谢由{viewer_name}投喂的{gift_name}爆出了{blind_box_gift}",
"enter_room": "欢迎{viewer_name}进入直播间",
"follow": "",
"like": "",
"interact_other": "",
"buy_guard": "恭喜{ships_viewer_name}成为尊贵的{ships_level}",
"guard_toast": "",
"super_chat": "{viewer_name}的醒目留言:{super_chat_content}"
},
"skip_free_gift": true
}
+125
View File
@@ -0,0 +1,125 @@
"""Event voice rules: load/save event_rules.json and render templates.
Each live event (danmaku, gift, blind box, enter room, follow, like,
interact, buy guard, guard toast, super chat) can be voiced using a
user-customizable template. Empty template = event is not spoken.
"""
import json
import logging
import pathlib
import re
import sys
logger = logging.getLogger(__name__)
DEFAULT_RULES = {
"enabled": {
"danmaku": True,
"gift": True,
"blind_box": True,
"enter_room": True,
"follow": False,
"like": False,
"interact_other": False,
"buy_guard": True,
"guard_toast": False,
"super_chat": True,
},
"templates": {
"danmaku": "{uname}\u8bf4\u3001 {msg}",
"gift": "\u611f\u8c22\u7531{viewer_name}\u6295\u5582\u7684{gift_name}",
"blind_box": "\u611f\u8c22\u7531{viewer_name}\u6295\u5582\u7684{gift_name}\u7206\u51fa\u4e86{blind_box_gift}",
"enter_room": "\u6b22\u8fce{viewer_name}\u8fdb\u5165\u76f4\u64ad\u95f4",
"follow": "",
"like": "",
"interact_other": "",
"buy_guard": "\u606d\u559c{ships_viewer_name}\u6210\u4e3a\u5c0a\u8d35\u7684{ships_level}",
"guard_toast": "",
"super_chat": "{viewer_name}\u7684\u9192\u76ee\u7559\u8a00\uff1a{super_chat_content}",
},
"skip_free_gift": True,
}
ALL_EVENTS = [
"danmaku", "gift", "blind_box", "enter_room", "follow", "like",
"interact_other", "buy_guard", "guard_toast", "super_chat",
]
_TEMPLATE_RE = re.compile(r"\{(\w+)\}")
def _base_dir() -> pathlib.Path:
if getattr(sys, "frozen", False):
return pathlib.Path(sys.executable).parent.parent.parent
return pathlib.Path(__file__).parent
def rules_path() -> pathlib.Path:
return _base_dir() / "event_rules.json"
def load_rules(path: str | None = None) -> dict:
"""Load rules from file; create default file if missing."""
p = pathlib.Path(path) if path else rules_path()
if not p.exists():
try:
p.write_text(json.dumps(DEFAULT_RULES, ensure_ascii=False, indent=2), encoding="utf-8")
logger.info("Created default event_rules.json at %s", p)
except OSError:
logger.warning("Cannot write default rules to %s", p)
return json.loads(json.dumps(DEFAULT_RULES))
try:
data = json.loads(p.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError) as e:
logger.error("Failed to load rules from %s: %s", p, e)
return json.loads(json.dumps(DEFAULT_RULES))
# merge with defaults to fill missing keys
merged = json.loads(json.dumps(DEFAULT_RULES))
for section in ("enabled", "templates"):
if isinstance(data.get(section), dict):
merged[section].update({k: v for k, v in data[section].items() if k in merged[section]})
if isinstance(data.get("skip_free_gift"), bool):
merged["skip_free_gift"] = data["skip_free_gift"]
return merged
def save_rules(rules: dict, path: str | None = None) -> None:
p = pathlib.Path(path) if path else rules_path()
p.write_text(json.dumps(rules, ensure_ascii=False, indent=2), encoding="utf-8")
logger.info("Saved event rules to %s", p)
def render_template(template: str, **vars_dict) -> str:
"""Render a template with given variables. Missing variables become empty.
Unknown variables are kept as-is (so users can see typos in the UI log)."""
if not template:
return ""
def _replace(m: re.Match) -> str:
name = m.group(1)
return str(vars_dict.get(name, "")) if name in vars_dict else m.group(0)
return _TEMPLATE_RE.sub(_replace, template)
def is_enabled(rules: dict, event: str) -> bool:
return bool(rules.get("enabled", {}).get(event, False))
def get_template(rules: dict, event: str) -> str:
return str(rules.get("templates", {}).get(event, ""))
def event_text(rules: dict, event: str, **vars_dict) -> str:
"""Build the speech text for an event, or '' if disabled/empty template."""
if not is_enabled(rules, event):
return ""
text = render_template(get_template(rules, event), **vars_dict)
return text.strip()
# ── guard level mapping ─────────────────────────────────────────────────
GUARD_LEVEL_NAMES = {1: "\u603b\u7763", 2: "\u63d0\u7763", 3: "\u8230\u957f"}
+5 -1
View File
@@ -12,7 +12,8 @@ import signal
import sys
from danmaku_handler import DanmakuClient
from chinese2kana import chinese_to_kana
from chinese2kana import chinese_to_kana, filter_kana
from event_rules import load_rules
from tts import TTSBridge
from audio_player import play_wav_async, get_output_devices
@@ -47,6 +48,7 @@ async def tts_worker(queue: asyncio.Queue, bridge: TTSBridge,
continue
try:
kana = chinese_to_kana(text, convert_numbers=convert_numbers)
kana = filter_kana(kana)
logger.info("Speaking: %s -> %s", text, kana)
wav_data = await bridge.synthesize(kana)
await play_wav_async(wav_data)
@@ -70,9 +72,11 @@ async def main_async(args: argparse.Namespace) -> int:
queue: asyncio.Queue = asyncio.Queue(maxsize=256)
shutdown_event = asyncio.Event()
bridge = TTSBridge(voice=args.voice, speed=args.speed)
rules = load_rules()
danmaku_client = DanmakuClient(
room_id=args.room_id, queue=queue,
cookies={"SESSDATA": args.sessdata} if args.sessdata else None,
rules=rules,
message_format=args.format,
)
+20 -7
View File
@@ -6,17 +6,30 @@
"": {
"name": "bililive-touhou-tts",
"dependencies": {
"wanakana": "^5.3.1"
"phonemize": "^1.2.0"
}
},
"node_modules/wanakana": {
"version": "5.3.1",
"resolved": "https://registry.npmmirror.com/wanakana/-/wanakana-5.3.1.tgz",
"integrity": "sha512-OSDqupzTlzl2LGyqTdhcXcl6ezMiFhcUwLBP8YKaBIbMYW1wAwDvupw2T9G9oVaKT9RmaSpyTXjxddFPUcFFIw==",
"node_modules/number-to-words": {
"version": "1.2.4",
"resolved": "https://registry.npmmirror.com/number-to-words/-/number-to-words-1.2.4.tgz",
"integrity": "sha512-/fYevVkXRcyBiZDg6yzZbm0RuaD6i0qRfn8yr+6D0KgBMOndFPxuW10qCHpzs50nN8qKuv78k8MuotZhcVX6Pw==",
"license": "MIT"
},
"node_modules/phonemize": {
"version": "1.2.0",
"resolved": "https://registry.npmmirror.com/phonemize/-/phonemize-1.2.0.tgz",
"integrity": "sha512-+zEpOPXrEaylYCIXMSDVhiAwFbVzIXRf+7vheuNxozg4hLKbQVDXCOpI0GAJw41xEgE9LcwMjLUN30aWrpwSkw==",
"license": "MIT",
"engines": {
"node": ">=12"
"dependencies": {
"number-to-words": "^1.2.4",
"pinyin-pro": "^3.26.0"
}
},
"node_modules/pinyin-pro": {
"version": "3.28.2",
"resolved": "https://registry.npmmirror.com/pinyin-pro/-/pinyin-pro-3.28.2.tgz",
"integrity": "sha512-jV38yxXHLfidirMC4hrXasLDozLCSq/4DfX88GnHcSEJ2+GpSedG6I9VOiEXJu6iQ5dbJC/RjmzyMuS5h/wH5A==",
"license": "MIT"
}
}
}
+1 -1
View File
@@ -2,6 +2,6 @@
"name": "bililive-touhou-tts",
"type": "module",
"dependencies": {
"wanakana": "^5.3.1"
"phonemize": "^1.2.0"
}
}
+45 -1
View File
@@ -18,6 +18,7 @@ from chinese2kana import chinese_to_kana, filter_kana
from tts import TTSBridge
from audio_player import play_wav_async, get_output_devices
from bili_login import BiliLoginSession
from event_rules import DEFAULT_RULES, load_rules, save_rules, rules_path
logger = logging.getLogger("bililive-tts-server")
@@ -27,6 +28,7 @@ def _get_static_dir() -> Path:
return Path(__file__).parent / "static"
STATIC_DIR = _get_static_dir()
event_rules = load_rules()
class TTSService:
@@ -37,6 +39,7 @@ class TTSService:
self._shutdown_event: asyncio.Event | None = None
self._worker_task: asyncio.Task | None = None
self._cookies: dict[str, str] | None = None
self._lifecycle_lock = asyncio.Lock()
self.running = False
self.start_time: float = 0
self.messages_processed = 0
@@ -48,6 +51,7 @@ class TTSService:
self._cookies = cookies
async def start(self, config: dict) -> None:
async with self._lifecycle_lock:
if self.running:
raise RuntimeError("Already running")
room_id = config["room_id"]
@@ -66,13 +70,15 @@ class TTSService:
await self._bridge.start()
self._danmaku_client = DanmakuClient(
room_id=room_id, queue=self._queue,
cookies=self._cookies, message_format=message_format,
cookies=self._cookies, rules=event_rules,
message_format=message_format,
)
await self._danmaku_client.start()
self.running = True
self._worker_task = asyncio.create_task(self._tts_worker(convert_numbers))
async def stop(self) -> None:
async with self._lifecycle_lock:
if not self.running:
return
self.running = False
@@ -237,6 +243,42 @@ async def api_logout(request: web.Request) -> web.Response:
return web.json_response({"status": "logged_out"})
async def api_rules_get(request: web.Request) -> web.Response:
return web.json_response(event_rules)
async def api_rules_post(request: web.Request) -> web.Response:
global event_rules
try:
new_rules = await request.json()
except Exception:
return web.json_response({"error": "Invalid JSON"}, status=400)
from event_rules import ALL_EVENTS, DEFAULT_RULES
merged = {
"enabled": dict(DEFAULT_RULES["enabled"]),
"templates": dict(DEFAULT_RULES["templates"]),
"skip_free_gift": True,
}
if isinstance(new_rules.get("enabled"), dict):
for k in ALL_EVENTS:
if isinstance(new_rules["enabled"].get(k), bool):
merged["enabled"][k] = new_rules["enabled"][k]
if isinstance(new_rules.get("templates"), dict):
for k in ALL_EVENTS:
if isinstance(new_rules["templates"].get(k), str):
merged["templates"][k] = new_rules["templates"][k]
if isinstance(new_rules.get("skip_free_gift"), bool):
merged["skip_free_gift"] = new_rules["skip_free_gift"]
try:
save_rules(merged)
except OSError as e:
return web.json_response({"error": f"Failed to save rules: {e}"}, status=500)
event_rules = merged
return web.json_response({"status": "saved", "rules": event_rules})
def create_app() -> web.Application:
app = web.Application()
app.router.add_get("/", index_handler)
@@ -248,6 +290,8 @@ def create_app() -> web.Application:
app.router.add_get("/api/qr/poll", api_qr_poll)
app.router.add_get("/api/login/status", api_login_status)
app.router.add_get("/api/logout", api_logout)
app.router.add_get("/api/rules", api_rules_get)
app.router.add_post("/api/rules", api_rules_post)
return app
+153 -42
View File
@@ -3,7 +3,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Bilibili Live → Yukkuri TTS</title>
<title>B站直播 → ゆっくり TTS</title>
<style>
:root {
--bg: #1a1a2e;
@@ -202,15 +202,36 @@
animation: spin 0.8s linear infinite; margin: 0 auto;
}
@keyframes spin { to { transform: rotate(360deg); } }
/* Event rules */
.rule-row {
display: flex; gap: 10px; align-items: center;
margin-bottom: 8px;
}
.rule-row .rule-name {
width: 110px; font-size: 0.82rem; color: var(--text-dim);
flex-shrink: 0;
}
.rule-row input[type="checkbox"] {
accent-color: var(--accent); width: 15px; height: 15px;
flex-shrink: 0;
}
.rule-row input[type="text"] {
flex: 1; padding: 6px 10px; border-radius: 6px;
border: 1px solid var(--border);
background: var(--input-bg); color: var(--text);
font-size: 0.85rem; outline: none;
}
.rule-row input[type="text"]:focus { border-color: var(--accent); }
</style>
</head>
<body>
<div class="container">
<h1>Bilibili Live &rarr; Yukkuri TTS</h1>
<h1>B站直播 &rarr; ゆっくり TTS</h1>
<!-- Login Card -->
<div class="card" id="login-card">
<h2>Login</h2>
<h2>登录</h2>
<div class="login-area">
<div class="qr-box" id="qr-box">
<div class="qr-placeholder" id="qr-placeholder">
@@ -225,18 +246,18 @@
<rect x="6" y="16" width="2" height="2"/>
<rect x="14" y="16" width="2" height="2"/>
</svg>
<span>Click "Login"<br/>to get QR code</span>
<span>点击"登录"<br/>获取二维码</span>
</div>
<img id="qr-img" src="" alt="QR Code" style="display:none">
<div id="qr-overlay" style="display:none"></div>
</div>
<div class="qr-info">
<div id="login-status" class="status-text">
Not logged in — anonymous danmaku may have censored usernames.
未登录 — 匿名弹幕的用户名可能被隐藏。
</div>
<div>
<button class="btn btn-login" id="btn-login" onclick="startLogin()">Login with QR</button>
<button class="btn btn-login" id="btn-logout" onclick="doLogout()" style="display:none">Logout</button>
<button class="btn btn-login" id="btn-login" onclick="startLogin()">扫码登录</button>
<button class="btn btn-login" id="btn-logout" onclick="doLogout()" style="display:none">退出登录</button>
</div>
</div>
</div>
@@ -244,39 +265,55 @@
<!-- Config Card -->
<div class="card">
<h2>Configuration</h2>
<h2>配置</h2>
<div class="form-row">
<div class="form-group">
<label for="room-id">Room ID</label>
<input type="number" id="room-id" placeholder="e.g. 12235923" value="">
<label for="room-id">房间号</label>
<input type="number" id="room-id" placeholder="例如 12235923" value="">
</div>
<div class="form-group" style="max-width:120px">
<label for="voice">Voice</label>
<label for="voice">音色</label>
<select id="voice">
<option value="f1">f1 (Reimu)</option>
<option value="f2">f2 (Marisa)</option>
<option value="m1">m1 (Male 1)</option>
<option value="m2">m2 (Male 2)</option>
<option value="r1">r1 (Robot)</option>
<option value="f1">f1 (灵梦)</option>
<option value="f2">f2 (魔理沙)</option>
<option value="m1">m1 (男声1)</option>
<option value="m2">m2 (男声2)</option>
<option value="r1">r1 (机器人)</option>
<option value="dvd">dvd</option>
<option value="imd1">imd1</option>
<option value="jgr">jgr</option>
</select>
</div>
<div class="form-group" style="max-width:140px">
<label for="speed">Speed (<span id="speed-val">100</span>)</label>
<label for="speed">语速 (<span id="speed-val">100</span>)</label>
<input type="range" id="speed" min="50" max="300" value="100"
oninput="document.getElementById('speed-val').textContent=this.value">
</div>
<div class="form-group" style="max-width:140px">
<label for="volume">Volume (<span id="volume-val">100</span>%)</label>
<label for="volume">音量 (<span id="volume-val">100</span>%)</label>
<input type="range" id="volume" min="0" max="200" value="100"
oninput="document.getElementById('volume-val').textContent=this.value">
</div>
</div>
<div class="btn-row">
<button class="btn btn-start" id="btn-start" onclick="startTTS()">Start</button>
<button class="btn btn-stop" id="btn-stop" onclick="stopTTS()" disabled>Stop</button>
<button class="btn btn-start" id="btn-start" onclick="startTTS()">开始</button>
<button class="btn btn-stop" id="btn-stop" onclick="stopTTS()" disabled>停止</button>
</div>
</div>
<!-- Event Voice Rules Card -->
<div class="card" id="rules-card">
<h2>事件语音规则</h2>
<div class="rule-list" id="rule-list"></div>
<div class="form-row" style="margin-top:10px; align-items:center">
<label class="chk-label"><input type="checkbox" id="rule-skip-free"> 跳过免费礼物</label>
<span style="flex:1"></span>
<button class="btn btn-login" onclick="saveRules()">保存规则</button>
</div>
<div class="rule-hint" style="margin-top:8px; font-size:0.78rem; color:var(--text-dim); line-height:1.6">
可用变量:{uname} {msg} | {viewer_name} {gift_name} {gift_num} {blind_box_gift} |
{ships_viewer_name} {ships_level} {ships_num} {guard_days} | {super_chat_content} {super_chat_price} |
{interact_type} {user_uid} — 模板留空 = 该事件静音。
</div>
</div>
@@ -284,14 +321,14 @@
<div class="card">
<div class="status-bar">
<span class="status-dot dot-off" id="status-dot"></span>
<span id="status-text">Stopped</span>
<span id="status-text">已停止</span>
</div>
<div class="stats">
<div>Messages: <span id="stat-msgs">0</span></div>
<div>Uptime: <span id="stat-uptime">00:00</span></div>
<div>消息数: <span id="stat-msgs">0</span></div>
<div>运行时长: <span id="stat-uptime">00:00</span></div>
</div>
<div id="log" style="margin-top:12px">
<span class="msg">Ready. Login with QR (optional), set Room ID, then click Start.</span>
<span class="msg">就绪。可选扫码登录,填写房间号后点击开始。</span>
</div>
</div>
</div>
@@ -311,13 +348,13 @@ function fmtUptime(sec) {
async function startLogin() {
document.getElementById('btn-login').disabled = true;
document.getElementById('login-status').innerHTML = '<div class="spinner"></div> Generating...';
document.getElementById('login-status').innerHTML = '<div class="spinner"></div> 正在生成...';
try {
const resp = await fetch('/api/qr/generate');
const data = await resp.json();
if (data.error) {
document.getElementById('login-status').textContent = 'Error: ' + data.error;
document.getElementById('login-status').textContent = '错误: ' + data.error;
document.getElementById('btn-login').disabled = false;
return;
}
@@ -328,13 +365,13 @@ async function startLogin() {
document.getElementById('qr-overlay').style.display = 'none';
document.getElementById('login-status').textContent = '请使用Bilibili客户端扫码';
document.getElementById('btn-login').disabled = false;
document.getElementById('btn-login').textContent = 'Refresh QR';
document.getElementById('btn-login').textContent = '刷新二维码';
loginPollKey = data.qrcode_key;
if (qrPollTimer) clearInterval(qrPollTimer);
qrPollTimer = setInterval(pollLoginStatus, 2000);
} catch (e) {
document.getElementById('login-status').textContent = 'Network error: ' + e.message;
document.getElementById('login-status').textContent = '网络错误: ' + e.message;
document.getElementById('btn-login').disabled = false;
}
}
@@ -354,11 +391,11 @@ async function pollLoginStatus() {
document.getElementById('qr-placeholder').style.display = 'none';
showOverlay(false, '', '');
document.getElementById('login-status').innerHTML =
'<div class="user-info"><span class="status-dot dot-on"></span> Login: ' + escapeHtml(data.username || 'Unknown') + '</div>';
'<div class="user-info"><span class="status-dot dot-on"></span> 已登录: ' + escapeHtml(data.username || '未知') + '</div>';
document.getElementById('btn-login').style.display = 'none';
document.getElementById('btn-logout').style.display = 'inline-block';
document.getElementById('btn-login').disabled = false;
document.getElementById('btn-login').textContent = 'Login with QR';
document.getElementById('btn-login').textContent = '扫码登录';
updateLoginStatusCard(true, data.username || '');
} else if (st === 86090) {
// Scanned, waiting
@@ -374,9 +411,9 @@ async function pollLoginStatus() {
loginPollKey = null;
showOverlay(true, 'expired', '二维码已过期');
document.getElementById('login-status').textContent = '二维码已过期,请点击刷新';
document.getElementById('btn-login').textContent = 'Refresh QR';
document.getElementById('btn-login').textContent = '刷新二维码';
} else {
document.getElementById('login-status').textContent = data.message || ('Status: ' + st);
document.getElementById('login-status').textContent = data.message || ('状态: ' + st);
}
} catch (e) {
// ignore network errors during polling
@@ -405,10 +442,10 @@ async function doLogout() {
document.getElementById('qr-img').src = '';
document.getElementById('qr-placeholder').style.display = 'flex';
document.getElementById('qr-overlay').style.display = 'none';
document.getElementById('login-status').textContent = 'Not logged in — anonymous danmaku may have censored usernames.';
document.getElementById('login-status').textContent = '未登录 — 匿名弹幕的用户名可能被隐藏。';
document.getElementById('btn-login').style.display = 'inline-block';
document.getElementById('btn-logout').style.display = 'none';
document.getElementById('btn-login').textContent = 'Login with QR';
document.getElementById('btn-login').textContent = '扫码登录';
}
function updateLoginStatusCard(loggedIn, username) {
@@ -419,7 +456,7 @@ function updateLoginStatusCard(loggedIn, username) {
async function startTTS() {
const roomId = document.getElementById('room-id').value;
if (!roomId) { alert('Please enter a Room ID'); return; }
if (!roomId) { alert('请输入房间号'); return; }
const config = {
room_id: parseInt(roomId),
@@ -435,13 +472,13 @@ async function startTTS() {
body: JSON.stringify(config)
});
const data = await resp.json();
if (data.error) { alert('Error: ' + data.error); return; }
if (data.error) { alert('错误: ' + data.error); return; }
running = true;
updateButtons();
startPolling();
} catch (e) {
alert('Failed to connect: ' + e.message);
alert('连接失败: ' + e.message);
}
}
@@ -454,7 +491,7 @@ async function stopTTS() {
updateButtons();
if (pollTimer) { clearInterval(pollTimer); pollTimer = null; }
document.getElementById('status-dot').className = 'status-dot dot-off';
document.getElementById('status-text').textContent = 'Stopped';
document.getElementById('status-text').textContent = '已停止';
}
function updateButtons() {
@@ -482,10 +519,10 @@ async function pollStatus() {
const txt = document.getElementById('status-text');
if (s.running) {
dot.className = 'status-dot dot-on';
txt.textContent = 'Running';
txt.textContent = '运行中';
} else {
dot.className = 'status-dot dot-off';
txt.textContent = 'Stopped';
txt.textContent = '已停止';
}
document.getElementById('stat-msgs').textContent = s.messages_processed;
@@ -495,7 +532,7 @@ async function pollStatus() {
const msgs = s.recent_messages || [];
log.innerHTML = msgs.length > 0
? msgs.map(m => '<span class="msg">' + escapeHtml(m) + '</span>').join('\n')
: '<span class="msg">Waiting for danmaku...</span>';
: '<span class="msg">等待弹幕...</span>';
log.scrollTop = log.scrollHeight;
} catch(e) {
// Server may have stopped
@@ -522,7 +559,7 @@ async function checkExistingLogin() {
const data = await resp.json();
if (data.logged_in) {
document.getElementById('login-status').innerHTML =
'<div class="user-info"><span class="status-dot dot-on"></span> Login: ' + escapeHtml(data.username) + '</div>';
'<div class="user-info"><span class="status-dot dot-on"></span> 已登录: ' + escapeHtml(data.username) + '</div>';
document.getElementById('btn-login').style.display = 'none';
document.getElementById('btn-logout').style.display = 'inline-block';
document.getElementById('qr-placeholder').style.display = 'none';
@@ -530,7 +567,81 @@ async function checkExistingLogin() {
} catch(e) {}
}
// ── Event voice rules ────────────────────────────────────────────────────
const RULE_LABELS = {
danmaku: '弹幕', gift: '礼物', blind_box: '盲盒礼物',
enter_room: '进入房间', follow: '关注', like: '点赞',
interact_other: '其他互动', buy_guard: '上舰',
guard_toast: '上舰提示', super_chat: '醒目留言',
};
async function loadRules() {
try {
const resp = await fetch('/api/rules');
const rules = await resp.json();
renderRules(rules);
} catch(e) {}
}
function renderRules(rules) {
const list = document.getElementById('rule-list');
list.innerHTML = '';
for (const key of Object.keys(RULE_LABELS)) {
const row = document.createElement('div');
row.className = 'rule-row';
const name = document.createElement('span');
name.className = 'rule-name';
name.textContent = RULE_LABELS[key];
const chk = document.createElement('input');
chk.type = 'checkbox';
chk.dataset.key = key;
chk.checked = !!(rules.enabled && rules.enabled[key]);
const input = document.createElement('input');
input.type = 'text';
input.dataset.key = key;
input.placeholder = '留空 = 静音';
input.value = (rules.templates && rules.templates[key]) || '';
row.appendChild(name);
row.appendChild(chk);
row.appendChild(input);
list.appendChild(row);
}
const skip = document.getElementById('rule-skip-free');
skip.checked = !!rules.skip_free_gift;
}
async function saveRules() {
const enabled = {};
const templates = {};
document.querySelectorAll('#rule-list input[type="checkbox"]').forEach(c => {
enabled[c.dataset.key] = c.checked;
});
document.querySelectorAll('#rule-list input[type="text"]').forEach(i => {
templates[i.dataset.key] = i.value;
});
const payload = {
enabled,
templates,
skip_free_gift: document.getElementById('rule-skip-free').checked,
};
try {
const resp = await fetch('/api/rules', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
});
const data = await resp.json();
if (data.error) { alert('保存失败: ' + data.error); return; }
renderRules(data.rules);
alert('规则已保存。重启TTS后生效。');
} catch(e) {
alert('保存失败: ' + e.message);
}
}
checkExistingLogin();
loadRules();
</script>
</body>
</html>
+28
View File
@@ -45,6 +45,8 @@ class TTSBridge:
self._process: asyncio.subprocess.Process | None = None
self._bridge_script = bridge_script or _resolve_path("tts_bridge.js")
self._node_exe = _get_node_exe()
self._stderr_task: asyncio.Task | None = None
self._io_lock = asyncio.Lock()
async def start(self) -> None:
if not pathlib.Path(self._bridge_script).exists():
@@ -61,6 +63,7 @@ class TTSBridge:
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
self._stderr_task = asyncio.create_task(self._drain_stderr())
line = await asyncio.wait_for(self._process.stdout.readline(), timeout=60)
line_str = line.decode("utf-8").strip()
@@ -73,10 +76,28 @@ class TTSBridge:
logger.info("Node.js TTS bridge ready.")
async def _drain_stderr(self) -> None:
"""Continuously read bridge stderr and forward to Python logs."""
if self._process is None or self._process.stderr is None:
return
try:
while True:
raw = await self._process.stderr.readline()
if not raw:
break
line = raw.decode("utf-8", errors="replace").rstrip()
if line:
logger.info("[bridge] %s", line)
except asyncio.CancelledError:
pass
except Exception:
logger.debug("Bridge stderr drain stopped", exc_info=True)
async def synthesize(self, kana_text: str) -> bytes:
if self._process is None or self._process.stdin is None:
raise RuntimeError("Bridge not started. Call start() first.")
async with self._io_lock:
tmpdir = pathlib.Path(tempfile.mkdtemp(prefix="tts_"))
try:
input_path = tmpdir / "input.txt"
@@ -114,6 +135,13 @@ class TTSBridge:
async def stop(self) -> None:
if self._process is not None:
logger.info("Stopping TTS bridge...")
if self._stderr_task is not None:
self._stderr_task.cancel()
try:
await self._stderr_task
except (asyncio.CancelledError, Exception):
pass
self._stderr_task = None
try:
if self._process.stdin is not None:
self._process.stdin.close()
+236 -7
View File
@@ -1,17 +1,40 @@
/** tts_bridge.js — Persistent Node.js bridge for aquestalk.js TTS synthesis.
Converts English/romaji to katakana via wanakana before synthesis.
English word → katakana pipeline (never skips):
1. english-to-kana dictionary (49K words, MIT, vendored english-kana-matcher.js)
2. phonemize (MIT) G2P → IPA → rule-based katakana transcription
3. letter-by-letter romanized spelling (last resort)
English punctuation (,.!?) is converted to Japanese pauses (、。).
*/
import { readFileSync, writeFileSync } from "fs";
import { createInterface } from "readline";
import { fileURLToPath, pathToFileURL } from "url";
import { dirname, join } from "path";
import wanakana from "wanakana";
import { createRequire } from "module";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
// ── vendored english-to-kana dictionary (auto-generated, MIT) ──────────
const matcherMod = pathToFileURL(join(__dirname, "english-kana-matcher.js")).href;
const { lookupKana } = await import(matcherMod);
// ── phonemize (G2P, MIT) via CJS entry (avoids JSON import issue in Node ESM) ──
const require = createRequire(import.meta.url);
let phonemize = null;
try {
phonemize = require("phonemize").phonemize;
console.error("[bridge] phonemize G2P loaded.");
} catch (err) {
console.error(`[bridge] phonemize unavailable (${err.message}), using letter fallback only.`);
}
// ── aquestalk.js ──────────────────────────────────────────────────────
const aquestalkMod = pathToFileURL(join(__dirname, "aquestalk.js", "dist", "index.js")).href;
const { load } = await import(aquestalkMod);
const args = process.argv.slice(2);
let voice = "f1";
let speed = 100;
@@ -24,9 +47,6 @@ for (let i = 0; i < args.length; i++) {
}
}
const aquestalkMod = pathToFileURL(join(__dirname, "aquestalk.js", "dist", "index.js")).href;
const { load } = await import(aquestalkMod);
console.error(`[bridge] Loading aquestalk.js with voice="${voice}", speed=${speed}...`);
let aq;
@@ -39,11 +59,220 @@ try {
process.exit(1);
}
// ── IPA → Katakana transcription (rule-based) ─────────────────────────
const KANA = {
k: { a: "カ", i: "キ", u: "ク", e: "ケ", o: "コ" },
g: { a: "ガ", i: "ギ", u: "グ", e: "ゲ", o: "ゴ" },
s: { a: "サ", i: "シ", u: "ス", e: "セ", o: "ソ" },
z: { a: "ザ", i: "ジ", u: "ズ", e: "ゼ", o: "ゾ" },
t: { a: "タ", i: "チ", u: "トゥ", e: "テ", o: "ト" },
d: { a: "ダ", i: "ジ", u: "ドゥ", e: "デ", o: "ド" },
n: { a: "ナ", i: "ニ", u: "ヌ", e: "ネ", o: "" },
h: { a: "ハ", i: "ヒ", u: "フ", e: "ヘ", o: "ホ" },
b: { a: "バ", i: "ビ", u: "ブ", e: "ベ", o: "ボ" },
p: { a: "パ", i: "ピ", u: "プ", e: "ペ", o: "ポ" },
m: { a: "マ", i: "ミ", u: "ム", e: "メ", o: "モ" },
r: { a: "ラ", i: "リ", u: "ル", e: "レ", o: "ロ" },
f: { a: "ファ", i: "フィ", u: "フ", e: "フェ", o: "フォ" },
v: { a: "バ", i: "ビ", u: "ブ", e: "ベ", o: "ボ" },
: { a: "チャ", i: "チ", u: "チュ", e: "チェ", o: "チョ" },
: { a: "ジャ", i: "ジ", u: "ジュ", e: "ジェ", o: "ジョ" },
ʃ: { a: "シャ", i: "シ", u: "シュ", e: "シェ", o: "ショ" },
ʒ: { a: "ジャ", i: "ジ", u: "ジュ", e: "ジェ", o: "ジョ" },
θ: { a: "サ", i: "シ", u: "ス", e: "セ", o: "ソ" },
ð: { a: "ザ", i: "ジ", u: "ズ", e: "ゼ", o: "ゾ" },
w: { a: "ワ", i: "ウィ", u: "ウ", e: "ウェ", o: "ウォ" },
j: { a: "ヤ", i: "イ", u: "ユ", e: "イェ", o: "ヨ" },
ŋ: { a: "ンガ", i: "ンギ", u: "ング", e: "ンゲ", o: "ンゴ" },
};
const CONS_ROW = {
"p": "p", "b": "b", "t": "t", "d": "d", "k": "k", "ɡ": "g", "g": "g",
"f": "f", "v": "v", "s": "s", "z": "z", "ʃ": "ʃ", "ʒ": "ʒ",
"h": "h", "tʃ": "tʃ", "dʒ": "dʒ", "θ": "θ", "ð": "ð",
"m": "m", "n": "n", "ŋ": "ŋ", "l": "r", "ɫ": "r", "ɹ": "r",
"r": "r", "j": "j", "w": "w", "ɾ": "r",
};
const VOWEL_KANA = { a: "ア", i: "イ", u: "ウ", e: "エ", o: "オ" };
const DIPH_KANA = { ai: "アイ", au: "アウ", oi: "オイ" };
function vClass(v) {
switch (v) {
case "ə": case "ɚ": case "ɝ": case "ɑ": case "ʌ": case "ɒ": case "æ": case "ɜ":
return { vowel: "a", long: false, diph: null };
case "ɪ": return { vowel: "i", long: false, diph: null };
case "i": return { vowel: "i", long: true, diph: null };
case "ʊ": return { vowel: "u", long: false, diph: null };
case "u": return { vowel: "u", long: true, diph: null };
case "ɛ": case "e": return { vowel: "e", long: false, diph: null };
case "eɪ": return { vowel: "e", long: true, diph: null };
case "ɔ": return { vowel: "o", long: false, diph: null };
case "o": case "oʊ": return { vowel: "o", long: true, diph: null };
case "aɪ": return { vowel: "a", long: false, diph: "ai" };
case "aʊ": return { vowel: "a", long: false, diph: "au" };
case "ɔɪ": return { vowel: "o", long: false, diph: "oi" };
default: return null;
}
}
const SONORANT = new Set(["m", "n", "ŋ", "l", "ɫ", "ɹ", "r", "j", "w", "ɾ"]);
function ipaToKana(ipaStr) {
const tokens = [];
let i = 0;
while (i < ipaStr.length) {
const two = ipaStr.slice(i, i + 2);
if (CONS_ROW[two] || vClass(two)) { tokens.push(two); i += 2; continue; }
const one = ipaStr[i];
if (CONS_ROW[one] || vClass(one)) { tokens.push(one); i += 1; continue; }
if (one === " " || one === "ː") { tokens.push(one); i += 1; continue; }
i += 1;
}
let out = "";
for (let i = 0; i < tokens.length; i++) {
const t = tokens[i];
if (t === " ") { out += " "; continue; }
if (t === "ː") { out += "ー"; continue; }
const vc = vClass(t);
if (vc) {
out += vc.diph ? DIPH_KANA[vc.diph] : (VOWEL_KANA[vc.vowel] + (vc.long ? "ー" : ""));
continue;
}
const row = CONS_ROW[t];
if (!row) continue;
const next = tokens[i + 1];
const nextVc = next !== undefined ? vClass(next) : null;
if (nextVc) {
if (nextVc.diph) {
out += KANA[row][nextVc.vowel] + DIPH_KANA[nextVc.diph].slice(1);
} else {
out += KANA[row][nextVc.vowel] + (nextVc.long ? "ー" : "");
}
i++;
} else if (next !== undefined) {
out += KANA[row]["u"];
} else {
out += (t === "n" || t === "ŋ") ? "ン" : KANA[row]["u"];
}
}
return out;
}
// ── Letter-by-letter fallback (last resort, AquesTalk1-safe) ──────────
const LETTER_KANA = {
a: "エー", b: "ビー", c: "シー", d: "ディー", e: "イー",
f: "エフ", g: "ジー", h: "エイチ", i: "アイ", j: "ジェー",
k: "ケー", l: "エル", m: "エム", n: "エヌ", o: "オー",
p: "ピー", q: "キュー", r: "アール", s: "エス", t: "ティー",
u: "ユー", v: "ブイ", w: "ダブリュー", x: "エックス", y: "ワイ", z: "ゼット",
};
const DIGIT_KANA = {
"0": "ゼロ", "1": "ワン", "2": "ツー", "3": "スリー", "4": "フォー",
"5": "ファイブ", "6": "シックス", "7": "セブン", "8": "エイト", "9": "ナイン",
};
function kanaForChar(ch) {
return LETTER_KANA[ch.toLowerCase()] || DIGIT_KANA[ch] || null;
}
function spellWord(word) {
let out = "";
for (const ch of word) {
const k = kanaForChar(ch);
out += k ? k : "";
}
return out;
}
// ── English punctuation → Japanese pause equivalents ──────────────────
const PUNCT_MAP = {
",": "、", ".": "。", "!": "", "?": "",
"": "、", "。": "。", "": "", "": "",
" ": "、", // AquesTalk truncates on space; use 、 pause instead
};
function wordToKana(word) {
const clean = word.replace(/'/g, "").toLowerCase();
if (!clean) return "";
// 1. dictionary
const hit = lookupKana(clean);
if (hit) return hit;
// 2. phonemize G2P → IPA → katakana
if (phonemize && /^[a-z]+$/.test(clean)) {
try {
const ipa = phonemize(clean, { stripStress: true });
if (ipa && /[^a-z]/.test(ipa)) {
const kana = ipaToKana(ipa);
if (kana) return kana;
}
} catch (err) {
// fall through to letter spelling
}
}
// 3. letter-by-letter spelling (never skip)
return spellWord(clean);
}
function convertEnglishSegment(text) {
// Protect decimal points between digits (3.5) from being treated as periods
const protectedText = text.replace(/(\d)\.(\d)/g, "$1\u30FB$2");
const words = protectedText.split(/([\s]+|[.,!?,。!?]+)/).filter((s) => s.length > 0);
const parts = [];
for (const token of words) {
if (/^\s+$/.test(token)) {
parts.push("、"); // word gap: AquesTalk truncates on space, use 、 instead
continue;
}
if (PUNCT_MAP[token] !== undefined) {
parts.push(PUNCT_MAP[token]);
continue;
}
parts.push(wordToKana(token));
}
return parts.join("").replace(/、+/g, "、");
}
function synthesize(kanaText) {
const trimmed = kanaText.trim();
if (!trimmed) throw new Error("EMPTY_TEXT");
const finalText = wanakana.toKatakana(trimmed, { convertLongVowelMark: true });
const wav = aq.run(finalText, speed);
let result = "";
let englishBuf = "";
for (const ch of trimmed) {
if ((ch >= "a" && ch <= "z") || (ch >= "A" && ch <= "Z") ||
(ch >= "0" && ch <= "9") || ch === "'" || ch === "," || ch === "." ||
ch === "!" || ch === "?") {
englishBuf += ch;
} else {
if (englishBuf) {
result += convertEnglishSegment(englishBuf);
englishBuf = "";
}
// Full-width punctuation that slipped through (e.g. ) → Japanese pause
result += PUNCT_MAP[ch] !== undefined ? PUNCT_MAP[ch] : ch;
}
}
if (englishBuf) {
result += convertEnglishSegment(englishBuf);
}
// Collapse consecutive pauses; strip any residual spaces (AquesTalk truncates on space)
result = result.replace(/ +/g, "、").replace(/、+/g, "、").replace(/^、|、$/g, "");
console.error(`[bridge] SYNTH IN: ${trimmed}`);
console.error(`[bridge] SYNTH OUT: ${result}`);
const wav = aq.run(result, speed);
return Buffer.from(wav);
}