AI语音克隆与角色扮演工具 - 从视频或照片生成真实人物声音进行对话

chat-with-anyone by noizai/skills

1,500 周安装量

402 GitHub Stars

GitHub

安装命令

npx skills add https://github.com/noizai/skills --skill chat-with-anyone

AI/机器学习开发音频处理

🇨🇳中文介绍

与任何人聊天

从在线视频中克隆真实人物的声音，或根据照片设计声音，然后通过 TTS 扮演该人物进行角色扮演。

重要提示：道德使用与版权

此技能合成模仿真实声音的语音。在继续之前，必须遵守以下规定：

绝不冒充他人进行欺骗、欺诈或骚扰。
仅使用公开可用的媒体（公开演讲、采访、新闻发布会）作为参考音频。
告知用户生成的音频是合成的，不应作为真实录音呈现。
拒绝针对未同意的私人个体的请求，或明显意图为欺骗、骚扰或诽谤的请求。

如果用户的意图看起来有害，请礼貌拒绝并解释原因。

先决条件

依赖项	类型	如何验证
`ffmpeg`	系统二进制文件	`ffmpeg -version`
`yt-dlp`	系统二进制文件

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

工作流 A：基于名称（从在线视频获取声音）

使用此清单跟踪进度：

- [ ] A1. 明确人物身份
- [ ] A2. 查找参考视频
- [ ] A3. 下载音频 + 字幕
- [ ] A4. 提取最佳参考片段
- [ ] A5. 生成语音

A1. 明确人物身份

如果身份不明确（例如"美国总统"、"蜘蛛侠演员"），请要求用户在继续之前指定确切的人物。

A2. 查找参考视频

使用网络搜索查找该人物说话清晰的 YouTube（或 Bilibili）视频。最佳候选：采访、演讲、新闻发布会。避免背景音乐过重的视频。

可尝试的搜索查询：

{CHARACTER_NAME} interview / {CHARACTER_NAME} 采访
{CHARACTER_NAME} speech / {CHARACTER_NAME} 演讲
{CHARACTER_NAME} press conference

A3. 下载音频和字幕

mkdir -p "tmp/chat_with_anyone/{CHARACTER_NAME}"
yt-dlp -x --audio-format mp3 \
  --write-subs --write-auto-subs --sub-langs "en,zh-Hans" \
  --convert-subs srt \
  -o "tmp/chat_with_anyone/{CHARACTER_NAME}/%(title)s.%(ext)s" \
  "{VIDEO_URL}"

下载后，列出输出目录以识别音频文件和 SRT 字幕文件：

ls tmp/chat_with_anyone/{CHARACTER_NAME}/

预期输出：一个 .mp3 音频文件和一个或多个 .srt 字幕文件。

如果没有出现字幕文件：尝试使用具有自动生成字幕的不同视频，或针对目标语言调整 --sub-langs。

A4. 提取最佳参考片段

使用自动提取脚本——它会解析 SRT，找到最密集的 3-12 秒语音窗口，并将其提取为 WAV：

python3 skills/chat-with-anyone/scripts/extract_ref_segment.py \
  --srt "tmp/chat_with_anyone/{CHARACTER_NAME}/{SRT_FILE}" \
  --audio "tmp/chat_with_anyone/{CHARACTER_NAME}/{AUDIO_FILE}" \
  -o "tmp/chat_with_anyone/{CHARACTER_NAME}/ref.wav"

脚本会打印选定的时间范围并保存参考 WAV 文件。在继续之前，请验证输出文件存在且非空。

如果脚本报告没有合适的片段：对于较短的剪辑，请尝试 --min-duration 2，或下载不同的视频。

A5. 生成语音并进行角色扮演

以角色身份撰写回复，然后合成语音：

python3 skills/tts/scripts/tts.py \
  -t "{RESPONSE_TEXT}" \
  --ref-audio "tmp/chat_with_anyone/{CHARACTER_NAME}/ref.wav" \
  -o "tmp/chat_with_anyone/{CHARACTER_NAME}/reply.wav"

将生成的音频文件连同文本一起呈现给用户。对于后续消息，请重用相同的 --ref-audio 路径。

工作流 B：基于图片（从照片设计声音）

使用此清单跟踪进度：

- [ ] B1. 分析图片
- [ ] B2. 设计声音
- [ ] B3. 预览（可选）
- [ ] B4. 生成语音

使用您的视觉能力检查图片：

如果人物是可识别的公众人物 --> 切换到工作流 A 以获取真实声音。
如果无法识别，则生成一个声音描述，涵盖：
- 性别（男 / 女）
- 大致年龄（例如"大约 30 岁"）
- 明显的气质（例如：开朗、权威、温和）
- 上下文线索（例如：西装 --> 专业语调；运动装 --> 充满活力）

将图片和描述传递给声音设计脚本：

python3 skills/chat-with-anyone/scripts/voice_design.py \
  --picture "{IMAGE_PATH}" \
  --voice-description "{VOICE_DESCRIPTION}" \
  -o "tmp/chat_with_anyone/voice_design"

检测到的声音特征（打印到标准输出）
输出目录中的预览音频文件
包含最佳语音 ID 的 voice_id.txt 文件

cat tmp/chat_with_anyone/voice_design/voice_id.txt

B3. 预览（可选）

呈现输出目录中的预览音频文件，以便用户可以听到声音。如果不满意，请使用调整后的 --voice-description 或 --guidance-scale 重新运行 B2。

B4. 生成语音并进行角色扮演

python3 skills/tts/scripts/tts.py \
  -t "{RESPONSE_TEXT}" \
  --voice-id "{VOICE_ID}" \
  -o "tmp/chat_with_anyone/voice_design/reply.wav"

对于后续消息，请继续使用相同的 --voice-id 以保持一致性。

示例：基于名称

用户 : 我想跟特朗普聊天，让他给我讲个睡前故事。

人物：唐纳德·特朗普。无需明确身份。
搜索 Donald Trump speech youtube，找到一个清晰的演讲视频。
下载：yt-dlp -x --audio-format mp3 --write-subs --write-auto-subs --sub-langs "en" --convert-subs srt -o "tmp/chat_with_anyone/trump/%(title)s.%(ext)s" "https://youtube.com/watch?v=..."
提取参考：python3 skills/chat-with-anyone/scripts/extract_ref_segment.py --srt "tmp/chat_with_anyone/trump/....srt" --audio "tmp/chat_with_anyone/trump/....mp3" -o "tmp/chat_with_anyone/trump/ref.wav"
以特朗普的风格生成 TTS：python3 skills/tts/scripts/tts.py -t "Let me tell you a tremendous bedtime story..." --ref-audio "tmp/chat_with_anyone/trump/ref.wav" -o "tmp/chat_with_anyone/trump/reply.wav"
将 reply.wav 和故事文本呈现给用户。

示例：基于图片

用户 : [上传 photo.jpg] 我想跟这张图片里的人聊天

视觉分析：无法识别的年轻女性，约 25 岁，休闲毛衣，温暖微笑。
设计声音：python3 skills/chat-with-anyone/scripts/voice_design.py --picture "photo.jpg" --voice-description "A young Chinese woman around 25, gentle and warm voice, friendly tone" -o "tmp/chat_with_anyone/voice_design"
从 tmp/chat_with_anyone/voice_design/voice_id.txt 读取语音 ID。
生成 TTS：python3 skills/tts/scripts/tts.py -t "你好呀！很高兴认识你！" --voice-id "{VOICE_ID}" -o "tmp/chat_with_anyone/voice_design/reply.wav"
呈现音频并使用相同的 --voice-id 继续角色扮演。

问题	解决方案
`yt-dlp` 下载失败或视频不可用	尝试不同的视频 URL；某些地区/视频受到限制。运行 `yt-dlp -U` 进行更新
没有 SRT 字幕文件	使用 `--sub-lang en,zh-Hans` 重新下载；如果仍然没有，请尝试具有自动字幕的不同视频
`extract_ref_segment.py` 找不到合适的窗口	对于较短的剪辑，使用 `--min-duration 2`，或尝试不同的视频
声音设计返回错误	检查 Noiz API 密钥；确保图片是清晰的人物照片
TTS 输出听起来不对	对于工作流 A，尝试不同的参考视频；对于工作流 B，调整 `--voice-description`

🇺🇸English

Chat with Anyone

Clone a real person's voice from online video, or design a voice from a photo, then roleplay as that person with TTS.

Important: Ethical Use & Copyright

This skill synthesizes speech that imitates real voices. Before proceeding, the agent must :

Never impersonate someone to deceive, defraud, or harass.
Only use publicly available media (public speeches, interviews, press conferences) as reference audio.
Inform the user that generated audio is synthetic and should not be presented as genuine recordings.
Decline requests that target private individuals who have not consented, or that are clearly intended for deception, harassment, or defamation.

If the user's intent appears harmful, refuse politely and explain why.

Prerequisites

Dependency	Type	How to verify
`ffmpeg`	System binary	`ffmpeg -version`
`yt-dlp`	System binary	`yt-dlp --version`
`tts` skill	Cursor skill	`ls skills/tts/scripts/tts.py`
`NOIZ_API_KEY`	Env var or file	`python3 skills/tts/scripts/tts.py config --show`

Before the first run , verify all dependencies are present:

ffmpeg -version && yt-dlp --version && ls skills/tts/scripts/tts.py

If yt-dlp is missing, install it:

uv pip install yt-dlp

If the Noiz API key is not configured:

python3 skills/tts/scripts/tts.py config --set-api-key YOUR_KEY

Mode Selection

User names a person (real or fictional) --> Workflow A
User provides an image , person is unrecognizable --> Workflow B
User provides an image , person is a recognizable public figure --> Workflow A (real voice is more authentic)
Multiple people in image --> Ask which person first

Workflow A: Name-based (voice from online video)

Track progress with this checklist:

- [ ] A1. Disambiguate character
- [ ] A2. Find reference video
- [ ] A3. Download audio + subtitles
- [ ] A4. Extract best reference segment
- [ ] A5. Generate speech

A1. Disambiguate Character

If ambiguous (e.g. "US President", "Spider-Man actor"), ask the user to specify the exact person before proceeding.

A2. Find a Reference Video

Use web search to find a YouTube (or Bilibili) video of the person speaking clearly. Best candidates: interviews, speeches, press conferences. Avoid videos with heavy background music.

Search queries to try:

{CHARACTER_NAME} interview / {CHARACTER_NAME} 采访
{CHARACTER_NAME} speech / {CHARACTER_NAME} 演讲
{CHARACTER_NAME} press conference

A3. Download Audio and Subtitles

mkdir -p "tmp/chat_with_anyone/{CHARACTER_NAME}"
yt-dlp -x --audio-format mp3 \
  --write-subs --write-auto-subs --sub-langs "en,zh-Hans" \
  --convert-subs srt \
  -o "tmp/chat_with_anyone/{CHARACTER_NAME}/%(title)s.%(ext)s" \
  "{VIDEO_URL}"

After download, list the output directory to identify the audio file and SRT subtitle file:

ls tmp/chat_with_anyone/{CHARACTER_NAME}/

Expected output: a .mp3 audio file and one or more .srt subtitle files.

If no subtitle files appear : try a different video that has auto-generated captions, or adjust --sub-langs for the target language.

A4. Extract Best Reference Segment

Use the automated extraction script — it parses the SRT, finds the densest 3-12 second speech window, and extracts it as a WAV:

python3 skills/chat-with-anyone/scripts/extract_ref_segment.py \
  --srt "tmp/chat_with_anyone/{CHARACTER_NAME}/{SRT_FILE}" \
  --audio "tmp/chat_with_anyone/{CHARACTER_NAME}/{AUDIO_FILE}" \
  -o "tmp/chat_with_anyone/{CHARACTER_NAME}/ref.wav"

The script prints the selected time range and saves the reference WAV. Verify the output exists and is non-empty before proceeding.

If the script reports no suitable segment : try --min-duration 2 for shorter clips, or download a different video.

A5. Generate Speech and Roleplay

Write a response in character, then synthesize it:

python3 skills/tts/scripts/tts.py \
  -t "{RESPONSE_TEXT}" \
  --ref-audio "tmp/chat_with_anyone/{CHARACTER_NAME}/ref.wav" \
  -o "tmp/chat_with_anyone/{CHARACTER_NAME}/reply.wav"

Present the generated audio file to the user along with the text. For subsequent messages, reuse the same --ref-audio path.

Workflow B: Image-based (voice from photo)

Track progress with this checklist:

- [ ] B1. Analyze image
- [ ] B2. Design voice
- [ ] B3. Preview (optional)
- [ ] B4. Generate speech

B1. Analyze the Image

Use your vision capability to examine the image:

If the person is a recognizable public figure --> switch to Workflow A for authentic voice.
If unrecognizable , produce a voice description covering:
- Gender (male / female)
- Approximate age (e.g. "around 30 years old")
- Apparent demeanor (e.g. cheerful, authoritative, gentle)
- Contextual cues (e.g. suit --> professional tone; athletic outfit --> energetic)

B2. Design the Voice

Pass both the image and the description to the voice-design script:

python3 skills/chat-with-anyone/scripts/voice_design.py \
  --picture "{IMAGE_PATH}" \
  --voice-description "{VOICE_DESCRIPTION}" \
  -o "tmp/chat_with_anyone/voice_design"

The script outputs:

Detected voice features (printed to stdout)
Preview audio files in the output directory
voice_id.txt containing the best voice ID

Read the voice ID:

cat tmp/chat_with_anyone/voice_design/voice_id.txt

B3. Preview (Optional)

Present the preview audio files from the output directory so the user can hear the voice. If unsatisfied, re-run B2 with adjusted --voice-description or --guidance-scale.

B4. Generate Speech and Roleplay

python3 skills/tts/scripts/tts.py \
  -t "{RESPONSE_TEXT}" \
  --voice-id "{VOICE_ID}" \
  -o "tmp/chat_with_anyone/voice_design/reply.wav"

For subsequent messages, keep using the same --voice-id for consistency.

Example: Name-based

User : 我想跟特朗普聊天，让他给我讲个睡前故事。

Agent steps :

Character: Donald Trump. No disambiguation needed.
Search Donald Trump speech youtube, find a clear speech video.
Download: yt-dlp -x --audio-format mp3 --write-subs --write-auto-subs --sub-langs "en" --convert-subs srt -o "tmp/chat_with_anyone/trump/%(title)s.%(ext)s" "https://youtube.com/watch?v=..."
Extract reference: python3 skills/chat-with-anyone/scripts/extract_ref_segment.py --srt "tmp/chat_with_anyone/trump/....srt" --audio "tmp/chat_with_anyone/trump/....mp3" -o "tmp/chat_with_anyone/trump/ref.wav"
Generate TTS in Trump's style: python3 skills/tts/scripts/tts.py -t "Let me tell you a tremendous bedtime story..." --ref-audio "tmp/chat_with_anyone/trump/ref.wav" -o "tmp/chat_with_anyone/trump/reply.wav"
Present reply.wav and the story text to the user.

Example: Image-based

User : [uploads photo.jpg] 我想跟这张图片里的人聊天

Agent steps :

Vision analysis: unrecognizable young woman, ~25, casual sweater, warm smile.
Design voice: python3 skills/chat-with-anyone/scripts/voice_design.py --picture "photo.jpg" --voice-description "A young Chinese woman around 25, gentle and warm voice, friendly tone" -o "tmp/chat_with_anyone/voice_design"
Read voice ID from tmp/chat_with_anyone/voice_design/voice_id.txt.
Generate TTS: python3 skills/tts/scripts/tts.py -t "你好呀！很高兴认识你！" --voice-id "{VOICE_ID}" -o "tmp/chat_with_anyone/voice_design/reply.wav"
Present audio and continue roleplay with same --voice-id.

Troubleshooting

Problem	Solution
`yt-dlp` download fails or video unavailable	Try a different video URL; some regions/videos are restricted. Run `yt-dlp -U` to update
No SRT subtitle files	Re-download with `--sub-lang en,zh-Hans`; if still none, try a different video with auto-captions
`extract_ref_segment.py` finds no suitable window	Use `--min-duration 2` for shorter clips, or try a different video
Voice design returns error	Check Noiz API key; ensure image is a clear photo of a person
TTS output sounds wrong	For Workflow A, try a different reference video; for Workflow B, adjust `--voice-description`

Weekly Installs

1.5K

Repository

noizai/skills

GitHub Stars

402

First Seen

Mar 3, 2026

Security Audits

Gen Agent Trust HubPass SocketPass SnykWarn

Installed on

opencode1.5K

gemini-cli1.5K

kimi-cli1.5K

cursor1.5K

cline1.5K

codex1.5K