Workers AI、Vectorize、AI Gateway

Workers AI:边缘 GPU 推理

Cloudflare 在大量 PoP 部署了 GPU(NVIDIA T4/L4/H100 混合),跑预装的开源模型——Llama 3/4、Mistral、Gemma、Whisper、Stable Diffusion、BGE 嵌入模型等。你按 token 付费,不用管部署。

# wrangler.toml
[ai]
binding = "AI"

app.post('/chat', async (c) => {
  const { question } = await c.req.json();
  const r = await c.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
    messages: [
      { role: 'system', content: 'You are a helpful Chinese assistant.' },
      { role: 'user', content: question },
    ],
  });
  return c.json({ answer: r.response });
});

流式响应

app.post('/stream', async (c) => {
  const stream = await c.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
    messages: [...],
    stream: true,
  });
  return new Response(stream, {
    headers: { 'content-type': 'text/event-stream' },
  });
});

模型类别

类别	代表模型	典型用途
Text Generation	llama-3.3-70b、mistral-small-3.1	聊天、生成
Embeddings	bge-base-en-v1.5、m3e	向量化
Rerank	bge-reranker	召回后精排
Classification	distilbert-sst-2	情感、标签
Speech-to-Text	whisper-large-v3-turbo	音频转录
Text-to-Image	flux-schnell、sdxl	图像生成
Vision	llava-hf、llama-3.2-vision	图片理解

Vectorize:边缘向量索引

wrangler vectorize create docs-index --dimensions=768 --metric=cosine

[[vectorize]]
binding = "VECTORS"
index_name = "docs-index"

嵌入 + 存入

async function embed(text: string, env: Env) {
  const r = await env.AI.run('@cf/baai/bge-base-en-v1.5', { text });
  return r.data[0];
}

app.post('/ingest', async (c) => {
  const { docId, chunks } = await c.req.json();
  const vectors = await Promise.all(chunks.map(async (chunk, i) => ({
    id: `${docId}:${i}`,
    values: await embed(chunk, c.env),
    metadata: { docId, text: chunk },
  })));
  await c.env.VECTORS.upsert(vectors);
  return c.json({ inserted: vectors.length });
});

检索

app.post('/search', async (c) => {
  const { query } = await c.req.json();
  const queryVec = await embed(query, c.env);
  const r = await c.env.VECTORS.query(queryVec, {
    topK: 5,
    returnMetadata: 'all',
    filter: { docId: { "$eq": 'manual-v2' } },
  });
  return c.json(r.matches);
});

端到端 RAG

app.post('/rag', async (c) => {
  const { question } = await c.req.json();

  // 1. Embed
  const qVec = await embed(question, c.env);

  // 2. 检索
  const r = await c.env.VECTORS.query(qVec, { topK: 5, returnMetadata: 'all' });
  const context = r.matches.map(m => m.metadata.text).join('\n---\n');

  // 3. 生成
  const ans = await c.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
    messages: [
      { role: 'system', content: `Use only the following context to answer:\n${context}` },
      { role: 'user', content: question },
    ],
  });

  return c.json({ answer: ans.response, sources: r.matches });
});

一个 /rag 端点,100 行以内把完整 RAG 跑起来——嵌入、检索、生成全在 Cloudflare,零外部依赖。

AI Gateway:给 LLM 加 observability

你在生产叫外部 LLM(OpenAI / Anthropic / Workers AI)时,AI Gateway 做 proxy:

// 把 OpenAI 请求改经 Gateway
const r = await fetch(`https://gateway.ai.cloudflare.com/v1/${ACCOUNT_ID}/my-gw/openai/chat/completions`, {
  method: 'POST',
  headers: { 'authorization': `Bearer ${OPENAI_KEY}` },
  body: JSON.stringify({ model: 'gpt-4o', messages: [...] }),
});

AutoRAG:一键托管 RAG

2025 Cloudflare 推出 AutoRAG——指向 R2 bucket,它自动做 chunk、embed、index、reindex,给你一个 aiSearch() 端点。适合"就想把文档灌进去问"的场景,少于 100 行代码。

const r = await c.env.AI.autorag('my-rag').aiSearch({
  query: '如何退款',
});
return c.json({ answer: r.response, sources: r.data });

Browser Rendering

在 Worker 里用 headless Chromium:截图、生成 PDF、抓动态渲染的页面:

import puppeteer from '@cloudflare/puppeteer';

app.get('/screenshot', async (c) => {
  const browser = await puppeteer.launch(c.env.BROWSER);
  const page = await browser.newPage();
  await page.goto(c.req.query('url'));
  const img = await page.screenshot();
  await browser.close();
  return new Response(img, { headers: { 'content-type': 'image/png' } });
});

成本参考

服务	计费	Free 额度
Workers AI(文本)	按 token,模型不同	每天 10K neurons
Workers AI(图像)	$0.00125/步	同上 pool
Vectorize 存储	$0.04/百万向量·月	5M 维度·月
Vectorize 查询	$0.01/百万查询	30M / 月
AI Gateway	免费	100K 日志/月

本章小结