docs: Update docs for the correctness (#13)
This commit is contained in:
1 parent
8dba55272c
commit
51a783f93a
60 files changed
+2456
-380
No files matched your search
@@ -0,0 +1,56 @@
|
||||
---
|
||||
title: Conversational AI Integration
|
||||
weight: 4
|
||||
---
|
||||
|
||||
**Goal:** Integrate Large Language Models (LLMs) while managing context and Discord's timeout constraints.
|
||||
|
||||
## 1. Context Management Strategy
|
||||
|
||||
Conversations require history to be coherent.
|
||||
|
||||
- **Scoping:** `Map<GuildID, Map<UserID, Message[]>>`
|
||||
- **Guild Key:** Prevents cross-server leaks.
|
||||
- **User Key:** Prevents cross-user confusion in public channels.
|
||||
- **Lifecycle:** Implement a rolling window (e.g., last 50 messages) to prevent context bloat.
|
||||
|
||||
## 2. Response Streaming
|
||||
|
||||
LLMs are slow; Discord is fast (3s timeout).
|
||||
|
||||
{{% steps %}}
|
||||
|
||||
### Step 1: Defer Interaction
|
||||
|
||||
Immediately call `interaction.deferReply()`. This buys you 15 minutes.
|
||||
|
||||
### Step 2: Invoke LLM
|
||||
|
||||
Call your AI service asynchronously.
|
||||
|
||||
### Step 3: Edit Response
|
||||
|
||||
When data arrives, use `interaction.editReply(content)`.
|
||||
{{% /steps %}}
|
||||
|
||||
## 3. Implementation Sketch
|
||||
|
||||
```typescript
|
||||
// commands/chat.ts
|
||||
export async function execute(interaction, controller) {
|
||||
// 1. Defer
|
||||
await interaction.deferReply();
|
||||
|
||||
// 2. Process
|
||||
const history = controller.getHistory(interaction.guildId, interaction.user.id);
|
||||
const response = await aiService.generate(interaction.options.getString('prompt'), history);
|
||||
|
||||
// 3. Respond
|
||||
await interaction.editReply(response);
|
||||
controller.updateHistory(interaction.guildId, interaction.user.id, response);
|
||||
}
|
||||
```
|
||||
|
||||
> [!WARNING]
|
||||
> **API Costs:**
|
||||
> AI APIs are often billed by token. Implement strict user-based rate limits in your Controller to prevent abuse.
|
||||
Reference in new issue
Block a user