feat: configurable report language (default zh-CN)
Gatekeep evals / eval-floor (push) Successful in 2s
Gatekeep evals / eval-floor (pull_request) Successful in 2s
gatekeep / security PR 风险 medium — 无未关闭安全阻断
Gatekeep / gatekeep (pull_request) Successful in 10m1s

Add providers.yaml `language` / Action input `language`; prompts and
published summary/inline honor it. Soften fail-closed messaging when
models 429; prefer cohere/nvidia in the first-3 parallel set for dogfood.
This commit is contained in:
layue13 committed 2026-10-27 08:29:48 +00:00
1 parent 93de34d0f0
commit ef13f8806a
10 files changed
+260 -71

No files matched your search

+11 -1
View File
@@ -17,6 +17,7 @@ const dir = args.dir;
const floor = args.floor || "low";
const agreementMin = Number(args["agreement-min"] || 2);
const scanHardFail = args["scan-hard-fail"] === "true";
const language = (args.language || process.env.GATEKEEP_LANGUAGE || "en").trim() || "en";
const outPath = args.out;
const tierRank = { low: 0, medium: 1, high: 2 };
@@ -177,8 +178,12 @@ const bySeverity = (a, b) => (sevRank[a.severity] ?? 9) - (sevRank[b.severity] ?
agreed.sort(bySeverity);
disputed.sort(bySeverity);
// Prefer successful model tiers; only if every call failed, fail-closed high.
const okModels = models.filter((m) => !m.error);
const tierSources = okModels.length ? okModels : models;
let risk_tier = floor;
for (const m of models) risk_tier = maxTier(risk_tier, m.risk_tier);
for (const m of tierSources) risk_tier = maxTier(risk_tier, m.risk_tier);
if (!okModels.length) risk_tier = maxTier(risk_tier, "high");
for (const f of agreed) {
if (f.kind === "security" || f.severity === "Critical") risk_tier = maxTier(risk_tier, "high");
else if (f.severity === "High" || f.tag === "Important") risk_tier = maxTier(risk_tier, "medium");
@@ -188,10 +193,15 @@ if (scanHardFail) risk_tier = maxTier(risk_tier, "high");
const openSecurityBlocking =
scanHardFail || agreed.some((f) => (f.kind === "security" || f.severity === "Critical") && f.blocking);
const modelErrors = models.filter((m) => m.error).length;
const result = {
risk_tier,
policy_floor: floor,
agreement_min: agreementMin,
language,
model_errors: modelErrors,
models_ok: okModels.length,
models: models.map((m) => ({
id: m.model,
risk_tier: m.risk_tier,
+172 -53
View File
@@ -2,6 +2,7 @@
/**
* Publish Gatekeep summary + agreed inline comments.
* Forge-agnostic (GitHub + Gitea) via fetch — no gh CLI required.
* Report language from merged.language / GATEKEEP_LANGUAGE (zh-CN, en, …).
*/
import { readFileSync, writeFileSync } from "node:fs";
@@ -24,8 +25,123 @@ const checkName = "gatekeep / security";
const apiBase = (process.env.GITHUB_API_URL || "https://api.github.com").replace(/\/$/, "");
const gitea = !apiBase.includes("api.github.com");
const language = (
merged.language ||
process.env.GATEKEEP_LANGUAGE ||
"en"
)
.trim()
.toLowerCase();
const zh = language.startsWith("zh");
const SEV_EMOJI = { Critical: "🔴", High: "🟠", Medium: "🟡", Low: "🟢" };
const T = zh
? {
title: "## Gatekeep 审查",
intro: "多模型并行审查结果如下。**先看严重级别**,Critical → Low 几秒扫完。",
risk: "PR 风险档",
floor: "策略底线",
agreement: "一致门槛",
agreedDisputed: "一致 / 分歧",
critHigh: "Critical / High(一致)",
modelsHdr: "### 模型(并行)",
modelCol: "模型",
tierCol: "其 PR 档",
findingsCol: "结论数",
agreedHdr: "### 一致结论",
agreedHelp:
"按严重级别排序。每条都写清 **严重级别**、**位置**、**为什么重要**。\n修复,或回复 `dismissed: <原因>` 后 resolve 讨论串。Gatekeep **没有**聊天 / @ 命令入口。",
agreedEmpty: "_模型之间没有达成一致的结论 — 可能很好,也可能该看分歧。_",
disputedHdr: "### 分歧(仅摘要,不上 inline)",
disputedHelp: "只有部分模型看到。有参考价值,本身不挡合并。",
disputedEmpty: "_没有值得单独列出的分歧。_",
scanHdr: "### 确定性安全扫描",
scanEmpty: "_无硬失败命中。_",
policyHdr: "### 策略底线信号",
policyNone: "- 无",
humansHdr: "### 仍需人工",
humansDefault: "- 品味、架构、计费/CI/鉴权、读证据",
footer:
"漏了真问题?开 **gatekeep-incident** issue — 这样才能变成 eval。\n_仅在模型一致时发 inline。PR 档失败关闭。结论以严重级别 + 位置 + 原因为先。_",
where: "位置",
why: "为什么重要",
whyFallback: "未写清影响 — 驳回前请先补一句。",
modelsLabel: "模型",
pre: " · 既有",
block: " · 阻断",
dispute: " · 分歧(未上 inline)",
preInline: "(既有)",
inlineFooter: (models) =>
`_由 ${models} 一致认定。请修复,或回复 \`dismissed: <原因>\` 后 resolve。_\n_无聊天命令 — Gatekeep 不响应 @。_`,
reviewBody: "Gatekeep 一致结论(严重级别 · 位置 · 原因)。请修复或驳回 — 无 @ 命令。",
fileLevel: (path) => `_文件级(无线号)于 \`${path}\`。_`,
noLoc: "(无位置)",
approveLow:
"看起来风险低 — Gatekeep + 安全绿灯后可不需人工(仅当 kill switch 打开才可自动合并)。",
approveMed: "需要 **1 名人工**过目。Bot 点赞不算人工。",
approveHigh: "需要 **2 名人工**批准。Bot 不能顶替。",
errNote: (n, ok) =>
`> **说明:** ${n} 个模型调用失败(常见于免费档限流 429)。风险档按**失败关闭**规则取高,**不等于**代码一定有问题。成功响应:${ok} 个。`,
statusBlock: "存在未关闭的安全 / Critical 结论 — 合并阻断",
statusOk: (tier) => `PR 风险 ${tier} — 无未关闭安全阻断`,
modelsAgree: "一致门槛 ≥",
}
: {
title: "## Gatekeep review",
intro:
"Hey — here's the multi-model pass. **Severity first** so you can skim Critical → Low in seconds.",
risk: "PR risk tier",
floor: "Policy floor",
agreement: "Agreement bar",
agreedDisputed: "Agreed / disputed",
critHigh: "Critical / High (agreed)",
modelsHdr: "### Models (parallel)",
modelCol: "Model",
tierCol: "Their PR tier",
findingsCol: "Findings",
agreedHdr: "### Agreed findings",
agreedHelp:
"Sorted by severity. Each one spells out **severity**, **where**, and **why it matters**.\nFix or dismiss with `dismissed: <reason>` (then resolve the thread). Gatekeep has **no** chat / @-command surface.",
agreedEmpty:
"_Nothing the models agreed on — nice, or spooky. Glance at disputed below._",
disputedHdr: "### Disputed (summary only — not posted inline)",
disputedHelp:
"One model saw it; others didn't. Useful signal, not a merge blocker by itself.",
disputedEmpty: "_No disagreements worth parking._",
scanHdr: "### Deterministic security scan",
scanEmpty: "_No hard-fail hits._",
policyHdr: "### Policy floor signals",
policyNone: "- none",
humansHdr: "### Still on humans",
humansDefault: "- taste, architecture, billing/CI/auth, reading the evidence",
footer:
"Missed something real? Open a **gatekeep-incident** issue — that's how it becomes an eval.\n_Inline comments only when models agree. PR tier is fail-closed. Findings lead with severity + location + why._",
where: "Where",
why: "Why it matters",
whyFallback: "Impact not spelled out — please clarify before dismissing.",
modelsLabel: "Models",
pre: " · pre-existing",
block: " · blocking",
dispute: " · disputed (not inlined)",
preInline: " (pre-existing)",
inlineFooter: (models) =>
`_Agreed by ${models}. Fix it, or reply \`dismissed: <reason>\` and resolve the thread._\n_No chat commands — Gatekeep doesn't take @-mentions._`,
reviewBody:
"Gatekeep agreed findings (severity · where · why). Fix or dismiss — no @-commands.",
fileLevel: (path) => `_File-level (no single line) on \`${path}\`._`,
noLoc: "(no location)",
approveLow:
"Looks low-risk — no human needed once Gatekeep + security are green (auto-merge only if the kill switch is on).",
approveMed: "Needs **1 human** look. Bot thumbs-up don't count as that look.",
approveHigh: "Needs **2 human** approvals. Bots never fill those seats.",
errNote: (n, ok) =>
`> **Note:** ${n} model call(s) failed (often free-tier 429). Risk tier is **fail-closed** high when calls fail — that does **not** mean the code is necessarily bad. Successful responses: ${ok}.`,
statusBlock: "Open security / Critical findings — merge blocked",
statusOk: (tier) => `PR risk ${tier} — no open security blockers`,
modelsAgree: "≥",
};
async function forgeApi(method, path, body) {
const p = path.startsWith("/") ? path : `/${path}`;
const url = `${apiBase}${p}`;
@@ -48,50 +164,46 @@ async function forgeApi(method, path, body) {
}
function approvals(tier) {
if (tier === "low")
return "Looks low-risk — no human needed once Gatekeep + security are green (auto-merge only if the kill switch is on).";
if (tier === "medium") return "Needs **1 human** look. Bot thumbs-up don't count as that look.";
return "Needs **2 human** approvals. Bots never fill those seats.";
if (tier === "low") return T.approveLow;
if (tier === "medium") return T.approveMed;
return T.approveHigh;
}
function loc(f) {
return f.location || (f.path ? `${f.path}${f.line ? `:${f.line}` : ""}` : "(no location)");
return f.location || (f.path ? `${f.path}${f.line ? `:${f.line}` : ""}` : T.noLoc);
}
function formatFindingBlock(f, { disputed = false } = {}) {
const sev = f.severity || "Medium";
const emoji = SEV_EMOJI[sev] || "⚪";
const pre = f.tag === "Pre-existing" ? " · pre-existing" : "";
const block = f.blocking ? " · blocking" : "";
const dispute = disputed ? " · disputed (not inlined)" : "";
const pre = f.tag === "Pre-existing" ? T.pre : "";
const block = f.blocking ? T.block : "";
const dispute = disputed ? T.dispute : "";
const tagNote = f.tag && f.tag !== "Important" ? ` · ${f.tag}` : "";
const why = f.why_it_matters || "Impact not spelled out — please clarify before dismissing.";
const why = f.why_it_matters || T.whyFallback;
const detail = (f.body || "").trim();
const detailBit = detail && detail !== why ? `\n\n${detail}` : "";
return (
`#### ${emoji} ${sev}${pre}${block}${dispute}${tagNote}: ${f.title}\n` +
`- **Where:** \`${loc(f)}\`\n` +
`- **Why it matters:** ${why}` +
`- **${T.where}:** \`${loc(f)}\`\n` +
`- **${T.why}:** ${why}` +
detailBit +
`\n- _Models:_ ${f.models.join(", ")}`
`\n- _${T.modelsLabel}:_ ${f.models.join(", ")}`
);
}
function formatInline(f) {
const sev = f.severity || "Medium";
const emoji = SEV_EMOJI[sev] || "⚪";
const pre = f.tag === "Pre-existing" ? " (pre-existing)" : "";
const why = f.why_it_matters || "Impact not spelled out — please clarify before dismissing.";
const pre = f.tag === "Pre-existing" ? T.preInline : "";
const why = f.why_it_matters || T.whyFallback;
const detail = (f.body || "").trim();
let out =
`### ${emoji} ${sev}${pre} — ${f.title}\n\n` +
`**Where:** \`${loc(f)}\`\n\n` +
`**Why it matters:** ${why}\n`;
`**${T.where}:** \`${loc(f)}\`\n\n` +
`**${T.why}:** ${why}\n`;
if (detail && detail !== why) out += `\n${detail}\n`;
out +=
`\n---\n` +
`_Agreed by ${f.models.join(", ")}. Fix it, or reply \`dismissed: <reason>\` and resolve the thread._\n` +
`_No chat commands — Gatekeep doesn't take @-mentions._`;
out += `\n---\n` + T.inlineFooter(f.models.join(", "));
return out;
}
@@ -100,12 +212,12 @@ const sortedDisputed = [...(merged.disputed_findings || [])];
const agreedList =
sortedAgreed.length === 0
? "_Nothing the models agreed on — nice, or spooky. Glance at disputed below._"
? T.agreedEmpty
: sortedAgreed.map((f) => formatFindingBlock(f)).join("\n\n");
const disputedList =
sortedDisputed.length === 0
? "_No disagreements worth parking._"
? T.disputedEmpty
: sortedDisputed.map((f) => formatFindingBlock(f, { disputed: true })).join("\n\n");
const modelRows = merged.models
@@ -118,49 +230,59 @@ const modelRows = merged.models
const critCount = sortedAgreed.filter((f) => f.severity === "Critical").length;
const highCount = sortedAgreed.filter((f) => f.severity === "High").length;
const errCount = Number(merged.model_errors || 0);
const okCount = Number(
merged.models_ok != null
? merged.models_ok
: (merged.models || []).filter((m) => !m.error).length,
);
const errBanner = errCount > 0 ? `\n${T.errNote(errCount, okCount)}\n` : "";
const agreementBar = zh
? `${T.modelsAgree}${merged.agreement_min} 个模型`
: `≥${merged.agreement_min} models`;
const body = `${marker}
## Gatekeep review
Hey — here's the multi-model pass. **Severity first** so you can skim Critical → Low in seconds.
${T.title}
${T.intro}
${errBanner}
| | |
| --- | --- |
| **PR risk tier** | \`${merged.risk_tier}\` |
| Policy floor | \`${merged.policy_floor}\` |
| Agreement bar | ≥${merged.agreement_min} models |
| Agreed / disputed | ${sortedAgreed.length} / ${sortedDisputed.length} |
| Critical / High (agreed) | ${critCount} / ${highCount} |
| **${T.risk}** | \`${merged.risk_tier}\` |
| ${T.floor} | \`${merged.policy_floor}\` |
| ${T.agreement} | ${agreementBar} |
| ${T.agreedDisputed} | ${sortedAgreed.length} / ${sortedDisputed.length} |
| ${T.critHigh} | ${critCount} / ${highCount} |
${approvals(merged.risk_tier)}
### Models (parallel)
| Model | Their PR tier | Findings |
${T.modelsHdr}
| ${T.modelCol} | ${T.tierCol} | ${T.findingsCol} |
| --- | --- | --- |
${modelRows}
### Agreed findings
Sorted by severity. Each one spells out **severity**, **where**, and **why it matters**.
Fix or dismiss with \`dismissed: <reason>\` (then resolve the thread). Gatekeep has **no** chat / @-command surface.
${T.agreedHdr}
${T.agreedHelp}
${agreedList}
### Disputed (summary only — not posted inline)
One model saw it; others didn't. Useful signal, not a merge blocker by itself.
${T.disputedHdr}
${T.disputedHelp}
${disputedList}
### Deterministic security scan
${(scan.hits || []).length ? (scan.hits || []).map((h) => `- ${h}`).join("\n") : "_No hard-fail hits._"}
${T.scanHdr}
${(scan.hits || []).length ? (scan.hits || []).map((h) => `- ${h}`).join("\n") : T.scanEmpty}
### Policy floor signals
${(floor.reasons || []).map((r) => `- ${r}`).join("\n") || "- none"}
${T.policyHdr}
${(floor.reasons || []).map((r) => `- ${r}`).join("\n") || T.policyNone}
### Still on humans
${(merged.human_focus || []).map((h) => `- ${h}`).join("\n") || "- taste, architecture, billing/CI/auth, reading the evidence"}
${T.humansHdr}
${(merged.human_focus || []).map((h) => `- ${h}`).join("\n") || T.humansDefault}
---
Missed something real? Open a **gatekeep-incident** issue — that's how it becomes an eval.
_Inline comments only when models agree. PR tier is fail-closed. Findings lead with severity + location + why._
${T.footer}
`;
async function main() {
@@ -182,7 +304,6 @@ async function main() {
.slice(0, 30)
.map((f) => ({
path: f.path,
// Gitea uses old_position/new_position in some versions; line works on GitHub
line: f.line,
body: formatInline(f),
}));
@@ -192,11 +313,11 @@ async function main() {
await forgeApi("POST", `/repos/${owner}/${name}/pulls/${pr.number}/reviews`, {
commit_id: pr.head.sha,
event: "COMMENT",
body: "Gatekeep agreed findings (severity · where · why). Fix or dismiss — no @-commands.",
body: T.reviewBody,
comments: inline,
});
} catch (e) {
console.error("Inline review failed; falling back to issue comments:", e.message);
console.error("inline review failed; falling back to issue comments:", e.message);
for (const f of inline) {
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, {
body: f.body + `\n\n_\`${f.path}:${f.line}\`_`,
@@ -212,7 +333,7 @@ async function main() {
f.path
) {
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, {
body: formatInline(f) + `\n\n_File-level (no single line) on \`${f.path}\`._`,
body: formatInline(f) + `\n\n${T.fileLevel(f.path)}`,
});
}
}
@@ -231,7 +352,6 @@ async function main() {
}
}
try {
// Gitea expects { labels: ["name"] }; GitHub same for issues API
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/labels`, {
labels: [want],
});
@@ -242,10 +362,9 @@ async function main() {
const conclusion = merged.open_security_blocking ? "failure" : "success";
const statusState = merged.open_security_blocking ? "failure" : "success";
const desc = merged.open_security_blocking
? "Open security / Critical findings — merge blocked"
: `PR risk ${merged.risk_tier} — no open security blockers`;
? T.statusBlock
: T.statusOk(merged.risk_tier);
// Prefer Checks API (GitHub); fall back to commit status (Gitea-friendly)
try {
await forgeApi("POST", `/repos/${owner}/${name}/check-runs`, {
name: checkName,
+7
View File
@@ -295,10 +295,16 @@ def main() -> None:
)
os.chmod(secrets_path, 0o600)
# Action input GATEKEEP_LANGUAGE overrides YAML; empty → YAML; missing → en
override_lang = (os.environ.get("GATEKEEP_LANGUAGE") or "").strip()
cfg_lang = str(cfg.get("language") or cfg.get("report_language") or "").strip()
language = override_lang or cfg_lang or "en"
plan = {
"agreement_min": int(cfg.get("agreement_min", 2)),
"temperature": cfg.get("temperature", 0),
"max_tokens": int(cfg.get("max_tokens", 4096)),
"language": language,
"models": plan_models,
"secrets_file": str(secrets_path),
"providers_used": sorted(provider_meta.keys()),
@@ -311,6 +317,7 @@ def main() -> None:
"models": [f"{m['provider']}:{m['model']}" for m in plan_models],
"keys": {m["provider"]: m["key_name"] for m in plan_models},
"agreement_min": plan["agreement_min"],
"language": plan["language"],
}
),
flush=True,
+23 -8
View File
@@ -39,6 +39,9 @@ python3 "$ACTION/scripts/resolve-providers.py" \
"${GATEKEEP_MODELS:-}"
AGREEMENT_MIN=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["agreement_min"])' "$PLAN")
# Prefer Action input / env override already applied in plan.language
export GATEKEEP_LANGUAGE
GATEKEEP_LANGUAGE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("language") or "en")' "$PLAN")
export GATEKEEP_MAX_TOKENS
GATEKEEP_MAX_TOKENS=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("max_tokens",4096))' "$PLAN")
SECRETS_FILE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["secrets_file"])' "$PLAN")
@@ -75,15 +78,25 @@ diff = (out / "diff.txt").read_text() if (out / "diff.txt").exists() else ""
discover = json.loads((out / "discover.json").read_text()) if (out / "discover.json").exists() else {}
plan = json.loads((out / "provider-plan.json").read_text())
system = """You are one reviewer in Gatekeep, a multi-model PR review system for the repo owner's own projects.
language = (plan.get("language") or os.environ.get("GATEKEEP_LANGUAGE") or "en").strip() or "en"
lang_note = (
f"LANGUAGE: Write ALL human-facing prose in {language} "
f"(summary, rationale, finding title / why_it_matters / body, human_focus). "
f"Keep JSON keys and enum tokens in English exactly as specified "
f"(risk_tier, severity Critical|High|Medium|Low, tag, kind)."
)
system = f"""You are one reviewer in Gatekeep, a multi-model PR review system for the repo owner's own projects.
Your job is useful review content that makes human triage cheap — not chat UX.
{lang_note}
Return ONLY valid JSON (no markdown fences) with:
{
{{
"risk_tier": "low|medium|high",
"summary": "short chatty overview",
"rationale": "why this PR risk tier",
"findings": [{
"findings": [{{
"severity": "Critical|High|Medium|Low",
"path": "file path",
"line": 1,
@@ -94,14 +107,14 @@ Return ONLY valid JSON (no markdown fences) with:
"why_it_matters": "concrete impact if ignored",
"body": "optional extra detail in a friendly tone",
"blocking": false
}],
}}],
"human_focus": ["string"]
}
}}
Finding rules (required on EVERY finding):
1. severity — Critical/High/Medium/Low so humans see critical vs minor instantly. (Distinct from PR risk_tier.)
2. exact location — path + line (use line_end for a range when needed).
3. why_it_matters — the impact/reason, not just \"please fix\".
3. why_it_matters — the impact/reason, not just "please fix".
Tone: clear and human; structured fields, not robotic checklist-speak.
tag is secondary (Important/Nit/Pre-existing); severity dominates.
Prefer real harm over style nits. Drop pure style. kind=security or severity=Critical ⇒ blocking true.
@@ -117,6 +130,7 @@ PR risk_tier rules:
user = {
"policy_floor": floor,
"policy_floor_reasons": [r.strip() for r in reasons.split("|") if r.strip()],
"language": language,
"discover": discover,
"reviewers": [f"{m['provider']}:{m['model']}" for m in plan.get("models", [])],
"pull_request": {
@@ -133,8 +147,8 @@ user = {
},
"diff": diff,
}
json.dump({"system": system, "user": json.dumps(user, indent=2)}, open(out / "prompt.json", "w"))
print(f"Prompt ready; floor={floor}; plan={len(plan.get('models',[]))} models", flush=True)
json.dump({"system": system, "user": json.dumps(user, indent=2), "language": language}, open(out / "prompt.json", "w"))
print(f"Prompt ready; floor={floor}; language={language}; plan={len(plan.get('models',[]))} models", flush=True)
PY
# Fire models in parallel (may be mixed providers)
@@ -167,6 +181,7 @@ node "$ACTION/scripts/merge-findings.js" \
--floor "$FLOOR" \
--agreement-min "$AGREEMENT_MIN" \
--scan-hard-fail "$SCAN_HARD" \
--language "$GATEKEEP_LANGUAGE" \
--out "$OUT/merged.json" | tee "$OUT/merge-meta.json"
set +e