Merge pull request 'feat: 可配置报告语言(默认 zh-CN)' (#3) from cursor/report-language-zh-165a into main
Gatekeep evals / eval-floor (push) Successful in 49s
Gatekeep evals / eval-floor (push) Successful in 49s
This commit was merged in pull request #3.
This commit is contained in:
commit
8993ebf841
12 files changed
+515
-159
No files matched your search
@@ -3,6 +3,12 @@
|
|||||||
#
|
#
|
||||||
# Secret: OPENROUTER_API_KEY (already configured on this repo)
|
# Secret: OPENROUTER_API_KEY (already configured on this repo)
|
||||||
# Action: in-repo composite at repository root (uses: ./)
|
# Action: in-repo composite at repository root (uses: ./)
|
||||||
|
#
|
||||||
|
# Important (uses: ./):
|
||||||
|
# - Checkout is the PR head, so Action fixes on main do NOT apply until the
|
||||||
|
# PR branch rebases/merges main. Stale heads can look "broken" after a fix.
|
||||||
|
# - Re-run: push an empty commit, or Actions → this workflow → Run workflow
|
||||||
|
# (workflow_dispatch; uses default branch / manual inputs as configured).
|
||||||
|
|
||||||
name: Gatekeep
|
name: Gatekeep
|
||||||
|
|
||||||
|
|||||||
@@ -22,30 +22,41 @@ AI-assisted PR review for **your own repos**: parallel multi-model review across
|
|||||||
|
|
||||||
`layue13/gatekeep` reviews **its own** PRs via [`.github/workflows/gatekeep-pr.yml`](.github/workflows/gatekeep-pr.yml) with `uses: ./` and `secrets.OPENROUTER_API_KEY`. Open a PR against `main` to see a review (Gitea Actions must have a runner — same pool as other `layue13` repos).
|
`layue13/gatekeep` reviews **its own** PRs via [`.github/workflows/gatekeep-pr.yml`](.github/workflows/gatekeep-pr.yml) with `uses: ./` and `secrets.OPENROUTER_API_KEY`. Open a PR against `main` to see a review (Gitea Actions must have a runner — same pool as other `layue13` repos).
|
||||||
|
|
||||||
|
Because dogfood uses `uses: ./`, **rebase/merge `main` into the PR branch** after Action fixes land, or the PR still runs the old Action. Re-run via new push or **Actions → Gatekeep → Run workflow**. Free-model 429s are retried; extras in `models:` after the first 3 are fallbacks; risk is force-high only if **zero** models succeed.
|
||||||
|
|
||||||
Allowlists / Gitea-on-github.com limitations: [`docs/remote-uses.md`](docs/remote-uses.md).
|
Allowlists / Gitea-on-github.com limitations: [`docs/remote-uses.md`](docs/remote-uses.md).
|
||||||
|
|
||||||
## Providers YAML
|
## Providers YAML
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
language: zh-CN # report language: zh-CN | en | … (prompts + published comments)
|
||||||
|
|
||||||
providers:
|
providers:
|
||||||
openrouter: # only include providers you configure
|
openrouter: # only include providers you configure
|
||||||
key: OPENROUTER_API_KEY # free-form secret/env name — pick any
|
key: OPENROUTER_API_KEY # free-form secret/env name — pick any
|
||||||
protocol: openai # openai | openai-completions | openai-responses | anthropic
|
protocol: openai # openai | openai-completions | openai-responses | anthropic
|
||||||
# base_url: optional (defaults exist for openrouter / anthropic / openai)
|
# base_url: optional (defaults exist for openrouter / anthropic / openai)
|
||||||
models:
|
models:
|
||||||
- google/gemma-4-31b-it:free
|
|
||||||
- google/gemma-4-26b-a4b-it:free
|
|
||||||
- cohere/north-mini-code:free
|
- cohere/north-mini-code:free
|
||||||
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
||||||
|
- google/gemma-4-31b-it:free
|
||||||
```
|
```
|
||||||
|
|
||||||
| Field | Meaning |
|
| Field | Meaning |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
|
| `language` | Report language for model prose + summary/inline comments (`zh-CN`, `en`, …). Alias: `report_language`. |
|
||||||
| `key` | **Free-form** name of the GitHub secret / env var holding the API key. **No** required `gatekeep_` prefix. |
|
| `key` | **Free-form** name of the GitHub secret / env var holding the API key. **No** required `gatekeep_` prefix. |
|
||||||
| `protocol` | `openai` \| `openai-completions` \| `openai-responses` \| `anthropic` |
|
| `protocol` | `openai` \| `openai-completions` \| `openai-responses` \| `anthropic` |
|
||||||
| `models` | Names to use (validated against the provider’s model list API at runtime) |
|
| `models` | Names to use (validated against the provider’s model list API at runtime) |
|
||||||
| `base_url` | Optional; known provider ids get defaults |
|
| `base_url` | Optional; known provider ids get defaults |
|
||||||
|
|
||||||
|
### Report language
|
||||||
|
|
||||||
|
- **This repo’s defaults:** `language: zh-CN` in `defaults/providers.yaml`.
|
||||||
|
- **Override without editing YAML:** Action input `language: en` (or `zh-CN`).
|
||||||
|
- Resolution order: Action input `language` → YAML `language` / `report_language` → `en`.
|
||||||
|
- JSON field names and enums stay English; human-facing prose follows the configured language.
|
||||||
|
|
||||||
Mix providers in one review by listing several entries. Secrets are read from the **workflow step `env`** using each entry’s `key` name.
|
Mix providers in one review by listing several entries. Secrets are read from the **workflow step `env`** using each entry’s `key` name.
|
||||||
|
|
||||||
Example suggestion only (still free-form): `key: gatekeep_openrouter` is fine if you named your secret that — not required.
|
Example suggestion only (still free-form): `key: gatekeep_openrouter` is fine if you named your secret that — not required.
|
||||||
|
|||||||
+17
-2
@@ -28,31 +28,46 @@ AI 辅助的 PR 审查 Action:多模型并行投票、按严重级别输出可
|
|||||||
|
|
||||||
`layue13/gatekeep` 用 [`.github/workflows/gatekeep-pr.yml`](.github/workflows/gatekeep-pr.yml) 对本仓库 PR 跑 Gatekeep:`uses: ./` + `secrets.OPENROUTER_API_KEY`。向 `main` 开 PR 即可触发(需有可用的 Gitea Actions runner)。
|
`layue13/gatekeep` 用 [`.github/workflows/gatekeep-pr.yml`](.github/workflows/gatekeep-pr.yml) 对本仓库 PR 跑 Gatekeep:`uses: ./` + `secrets.OPENROUTER_API_KEY`。向 `main` 开 PR 即可触发(需有可用的 Gitea Actions runner)。
|
||||||
|
|
||||||
|
**注意:** `uses: ./` 用的是 **PR 分支上的 Action**。main 上的修复要进 PR,需要 rebase/merge main,否则看起来像“修了还挂”。重跑:再推一次提交,或 Actions → Gatekeep → **Run workflow**。免费模型 429 会退避重试;`models` 列表第 4 个及以后作备用;**只有全部模型失败**才强制抬高风险档为 high。
|
||||||
|
|
||||||
github.com runner 若不允许非 github.com Action 源,见 `docs/remote-uses.md`。
|
github.com runner 若不允许非 github.com Action 源,见 `docs/remote-uses.md`。
|
||||||
|
|
||||||
## 多供应商(providers.yaml)
|
## 多供应商(providers.yaml)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
language: zh-CN # 报告语言:zh-CN | en | …(提示词 + 汇总/inline 评论)
|
||||||
|
|
||||||
providers:
|
providers:
|
||||||
openrouter:
|
openrouter:
|
||||||
key: OPENROUTER_API_KEY # 自由命名 — 任意 secret/环境变量名
|
key: OPENROUTER_API_KEY # 自由命名 — 任意 secret/环境变量名
|
||||||
protocol: openai # openai | openai-completions | openai-responses | anthropic
|
protocol: openai # openai | openai-completions | openai-responses | anthropic
|
||||||
models:
|
models:
|
||||||
- google/gemma-4-31b-it:free
|
|
||||||
- google/gemma-4-26b-a4b-it:free
|
|
||||||
- cohere/north-mini-code:free
|
- cohere/north-mini-code:free
|
||||||
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
||||||
|
- google/gemma-4-31b-it:free
|
||||||
```
|
```
|
||||||
|
|
||||||
|
- **`language`:报告语言**(本仓库默认 `zh-CN`)。也可用 Action 输入 `language:` 覆盖。别名 `report_language`。
|
||||||
- **`key`:自由命名**,**不要**要求必须 `gatekeep_` 前缀(你也可以叫 `gatekeep_openrouter`,只是举例)。
|
- **`key`:自由命名**,**不要**要求必须 `gatekeep_` 前缀(你也可以叫 `gatekeep_openrouter`,只是举例)。
|
||||||
- 每个供应商只需:`key` + `protocol`(`base_url` / `models` 按需;已知 id 有默认 base_url)。
|
- 每个供应商只需:`key` + `protocol`(`base_url` / `models` 按需;已知 id 有默认 base_url)。
|
||||||
- **不内置死模型清单**;启动时拉取供应商模型列表并校验名字。
|
- **不内置死模型清单**;启动时拉取供应商模型列表并校验名字。
|
||||||
- 可混用多个供应商;只写你真正配置了的条目。
|
- 可混用多个供应商;只写你真正配置了的条目。
|
||||||
|
|
||||||
|
### 如何设置语言
|
||||||
|
|
||||||
|
| 方式 | 示例 |
|
||||||
|
| --- | --- |
|
||||||
|
| `providers.yaml` | `language: zh-CN` 或 `language: en` |
|
||||||
|
| Action 输入(优先) | `with: { language: en }` |
|
||||||
|
| 缺省 | 无 YAML / 无输入时回落 `en`;本仓库 defaults 已写 `zh-CN` |
|
||||||
|
|
||||||
|
JSON 键名与枚举值(如 `Critical`)保持英文;摘要、结论正文、inline 评论跟人读的文案跟 `language`。
|
||||||
|
|
||||||
## 输入面
|
## 输入面
|
||||||
|
|
||||||
| 项 | 含义 |
|
| 项 | 含义 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
|
| `language` | 报告语言(`zh-CN` / `en` …);覆盖 YAML |
|
||||||
| `key` | Secret / 环境变量名(自由) |
|
| `key` | Secret / 环境变量名(自由) |
|
||||||
| `protocol` | `openai` / `openai-completions` / `openai-responses` / `anthropic` |
|
| `protocol` | `openai` / `openai-completions` / `openai-responses` / `anthropic` |
|
||||||
| `models` | 参与并行审查的模型名 |
|
| `models` | 参与并行审查的模型名 |
|
||||||
|
|||||||
+17
-2
@@ -21,6 +21,12 @@ inputs:
|
|||||||
(or bare names unique in config). Empty = providers.yaml lists.
|
(or bare names unique in config). Empty = providers.yaml lists.
|
||||||
required: false
|
required: false
|
||||||
default: ""
|
default: ""
|
||||||
|
language:
|
||||||
|
description: >
|
||||||
|
Report language for model prose + published summary/inline comments
|
||||||
|
(e.g. zh-CN, en). Empty = providers.yaml `language`, else en.
|
||||||
|
required: false
|
||||||
|
default: ""
|
||||||
skip-draft:
|
skip-draft:
|
||||||
description: Skip draft PRs
|
description: Skip draft PRs
|
||||||
required: false
|
required: false
|
||||||
@@ -52,10 +58,18 @@ runs:
|
|||||||
# Runner image / consumer workflow must provide Node >= 18.
|
# Runner image / consumer workflow must provide Node >= 18.
|
||||||
- name: Require Node.js
|
- name: Require Node.js
|
||||||
shell: bash
|
shell: bash
|
||||||
|
env:
|
||||||
|
GATEKEEP_LANGUAGE: ${{ inputs.language }}
|
||||||
run: |
|
run: |
|
||||||
if ! command -v node >/dev/null 2>&1; then
|
if ! command -v node >/dev/null 2>&1; then
|
||||||
echo "::error::Node.js >= 18 required for Gatekeep (merge-findings.js / publish.js)."
|
lang="${GATEKEEP_LANGUAGE:-}"
|
||||||
echo "Preinstall Node on the runner, or add a mirrored setup-node step before uses: ./."
|
if [[ "${lang,,}" == zh* ]]; then
|
||||||
|
echo "::error::Gatekeep 需要 Node.js ≥ 18(merge-findings.js / publish.js)。"
|
||||||
|
echo "请在 runner 预装 Node,或在 uses: ./ 之前加本 forge 已镜像的 setup-node;不要依赖未镜像的 actions/setup-node。"
|
||||||
|
else
|
||||||
|
echo "::error::Node.js >= 18 required for Gatekeep (merge-findings.js / publish.js)."
|
||||||
|
echo "Preinstall Node on the runner, or add a mirrored setup-node step before uses: ./."
|
||||||
|
fi
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
node --version
|
node --version
|
||||||
@@ -102,6 +116,7 @@ runs:
|
|||||||
GITHUB_TOKEN: ${{ inputs.github-token }}
|
GITHUB_TOKEN: ${{ inputs.github-token }}
|
||||||
CONFIG_DIR: ${{ inputs.config-dir }}
|
CONFIG_DIR: ${{ inputs.config-dir }}
|
||||||
GATEKEEP_MODELS: ${{ inputs.models }}
|
GATEKEEP_MODELS: ${{ inputs.models }}
|
||||||
|
GATEKEEP_LANGUAGE: ${{ inputs.language }}
|
||||||
SKIP_DRAFT: ${{ inputs.skip-draft }}
|
SKIP_DRAFT: ${{ inputs.skip-draft }}
|
||||||
GATEKEEP_ROOT: ${{ github.workspace }}
|
GATEKEEP_ROOT: ${{ github.workspace }}
|
||||||
GATEKEEP_ACTION: ${{ github.action_path }}
|
GATEKEEP_ACTION: ${{ github.action_path }}
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
# Example / owner consumer override — OpenRouter only.
|
# Example / owner consumer override — OpenRouter only.
|
||||||
# `key` is free-form; map it in the workflow step env from secrets.
|
# `key` is free-form; map it in the workflow step env from secrets.
|
||||||
|
|
||||||
|
language: zh-CN
|
||||||
|
|
||||||
agreement_min: 2
|
agreement_min: 2
|
||||||
temperature: 0
|
temperature: 0
|
||||||
max_tokens: 4096
|
max_tokens: 4096
|
||||||
@@ -10,7 +12,7 @@ providers:
|
|||||||
key: OPENROUTER_API_KEY
|
key: OPENROUTER_API_KEY
|
||||||
protocol: openai
|
protocol: openai
|
||||||
models:
|
models:
|
||||||
- google/gemma-4-31b-it:free
|
|
||||||
- google/gemma-4-26b-a4b-it:free
|
|
||||||
- cohere/north-mini-code:free
|
- cohere/north-mini-code:free
|
||||||
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
||||||
|
- google/gemma-4-31b-it:free
|
||||||
|
- google/gemma-4-26b-a4b-it:free
|
||||||
@@ -2,6 +2,10 @@
|
|||||||
# `key` is a FREE-FORM secret / env var name (no required gatekeep_ prefix).
|
# `key` is a FREE-FORM secret / env var name (no required gatekeep_ prefix).
|
||||||
# Put that name in the workflow step env, e.g. env: { OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} }
|
# Put that name in the workflow step env, e.g. env: { OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} }
|
||||||
|
|
||||||
|
# Report language for prompts + published summary/inline comments (BCP-47-ish: zh-CN, en, …).
|
||||||
|
# Override with Action input `language` when set.
|
||||||
|
language: zh-CN
|
||||||
|
|
||||||
agreement_min: 2
|
agreement_min: 2
|
||||||
temperature: 0
|
temperature: 0
|
||||||
max_tokens: 4096
|
max_tokens: 4096
|
||||||
@@ -10,8 +14,10 @@ providers:
|
|||||||
openrouter:
|
openrouter:
|
||||||
key: OPENROUTER_API_KEY
|
key: OPENROUTER_API_KEY
|
||||||
protocol: openai
|
protocol: openai
|
||||||
|
# First 3 are used in parallel. Prefer models that tend to answer on free tier;
|
||||||
|
# put rate-limit-prone free gemma after cohere/nvidia.
|
||||||
models:
|
models:
|
||||||
- google/gemma-4-31b-it:free
|
|
||||||
- google/gemma-4-26b-a4b-it:free
|
|
||||||
- cohere/north-mini-code:free
|
- cohere/north-mini-code:free
|
||||||
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
- nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
|
||||||
|
- google/gemma-4-31b-it:free
|
||||||
|
- google/gemma-4-26b-a4b-it:free
|
||||||
+7
-4
@@ -9,13 +9,16 @@ The Action repo itself runs Gatekeep on its own pull requests.
|
|||||||
| Secret | `OPENROUTER_API_KEY` → step `env` |
|
| Secret | `OPENROUTER_API_KEY` → step `env` |
|
||||||
| Trigger | `pull_request` (non-draft) + `workflow_dispatch` |
|
| Trigger | `pull_request` (non-draft) + `workflow_dispatch` |
|
||||||
| Proven | [PR #2](https://git.potatolab.uk/layue13/gatekeep/pulls/2) got `<!-- gatekeep-review -->`; run 15833 **success** |
|
| Proven | [PR #2](https://git.potatolab.uk/layue13/gatekeep/pulls/2) got `<!-- gatekeep-review -->`; run 15833 **success** |
|
||||||
|
| Report language | `defaults/providers.yaml` → `language: zh-CN` (override with Action `language:`) |
|
||||||
|
|
||||||
Gitea notes: no `actions/setup-node` mirror — Action requires runner Node ≥18. PyYAML installs via workspace venv (PEP 668). Free OpenRouter models may 429; review still posts (fail-closed tier).
|
Gitea notes: no `actions/setup-node` mirror — Action requires runner Node ≥18. PyYAML installs via workspace venv (PEP 668). Free OpenRouter models may 429: Action retries with backoff, tries `fallback_models` (list entries after the first 3), and only force-elevates risk to high when **zero** models succeed. Summary is Chinese when `language: zh-CN`.
|
||||||
|
|
||||||
## How to see a review
|
## How to see a review
|
||||||
|
|
||||||
1. Branch off `main`, make a tiny change, open a PR on https://git.potatolab.uk/layue13/gatekeep
|
1. Branch off **current** `main` (or merge/rebase main into the PR branch — `uses: ./` loads Action code from the PR head)
|
||||||
2. Wait for the **Gatekeep** workflow (same Actions runner pool as other `layue13` repos)
|
2. Open/push a PR on https://git.potatolab.uk/layue13/gatekeep
|
||||||
3. Look for the summary comment `<!-- gatekeep-review -->` on the PR
|
3. Wait for the **Gatekeep** workflow (same Actions runner pool as other `layue13` repos)
|
||||||
|
4. Look for the summary comment `<!-- gatekeep-review -->` (updated in place on re-run)
|
||||||
|
5. Manual re-run: Actions → Gatekeep → **Run workflow** (`workflow_dispatch`), or push another commit
|
||||||
|
|
||||||
Consumer sample workflows remain under top-level `workflows/` for copy-paste into *other* repos’ `.github/workflows/`.
|
Consumer sample workflows remain under top-level `workflows/` for copy-paste into *other* repos’ `.github/workflows/`.
|
||||||
Regular → Executable
+31
-21
@@ -2,6 +2,7 @@
|
|||||||
# Call one model via provider protocol.
|
# Call one model via provider protocol.
|
||||||
# Protocols: openai | openai-completions | openai-responses | anthropic
|
# Protocols: openai | openai-completions | openai-responses | anthropic
|
||||||
# (openai / openai-compatible → chat completions)
|
# (openai / openai-compatible → chat completions)
|
||||||
|
# Retries HTTP 429 with exponential backoff (GATEKEEP_HTTP_RETRIES, default 3).
|
||||||
# Args: <provider> <model> <protocol> <base_url> <prompt.json> <out.json> <secrets.json>
|
# Args: <provider> <model> <protocol> <base_url> <prompt.json> <out.json> <secrets.json>
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -21,6 +22,7 @@ if [[ -z "$KEY" ]]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
MAX_TOKENS="${GATEKEEP_MAX_TOKENS:-4096}"
|
MAX_TOKENS="${GATEKEEP_MAX_TOKENS:-4096}"
|
||||||
|
RETRIES="${GATEKEEP_HTTP_RETRIES:-3}"
|
||||||
SYSTEM=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["system"])' "$PROMPT_FILE")
|
SYSTEM=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["system"])' "$PROMPT_FILE")
|
||||||
USER=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["user"])' "$PROMPT_FILE")
|
USER=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["user"])' "$PROMPT_FILE")
|
||||||
|
|
||||||
@@ -54,13 +56,9 @@ print(json.dumps({
|
|||||||
}))
|
}))
|
||||||
PY
|
PY
|
||||||
)
|
)
|
||||||
HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \
|
URL="${BASE_URL%/}/chat/completions"
|
||||||
"${BASE_URL%/}/chat/completions" \
|
EXTRA_HDR=(-H "HTTP-Referer: https://github.com/${GITHUB_REPOSITORY:-gatekeep}" -H "X-Title: Gatekeep")
|
||||||
-H "Authorization: Bearer $KEY" \
|
AUTH=(-H "Authorization: Bearer $KEY")
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "HTTP-Referer: https://github.com/${GITHUB_REPOSITORY:-gatekeep}" \
|
|
||||||
-H "X-Title: Gatekeep" \
|
|
||||||
-d "$BODY")
|
|
||||||
EXTRACT=openai
|
EXTRACT=openai
|
||||||
elif [[ "$WIRE" == "openai-responses" ]]; then
|
elif [[ "$WIRE" == "openai-responses" ]]; then
|
||||||
BODY=$(python3 - "$MODEL" "$SYSTEM" "$USER" "$MAX_TOKENS" <<'PY'
|
BODY=$(python3 - "$MODEL" "$SYSTEM" "$USER" "$MAX_TOKENS" <<'PY'
|
||||||
@@ -77,11 +75,9 @@ print(json.dumps({
|
|||||||
}))
|
}))
|
||||||
PY
|
PY
|
||||||
)
|
)
|
||||||
HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \
|
URL="${BASE_URL%/}/responses"
|
||||||
"${BASE_URL%/}/responses" \
|
EXTRA_HDR=()
|
||||||
-H "Authorization: Bearer $KEY" \
|
AUTH=(-H "Authorization: Bearer $KEY")
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d "$BODY")
|
|
||||||
EXTRACT=openai-responses
|
EXTRACT=openai-responses
|
||||||
elif [[ "$WIRE" == "anthropic" ]]; then
|
elif [[ "$WIRE" == "anthropic" ]]; then
|
||||||
BODY=$(python3 - "$MODEL" "$SYSTEM" "$USER" "$MAX_TOKENS" <<'PY'
|
BODY=$(python3 - "$MODEL" "$SYSTEM" "$USER" "$MAX_TOKENS" <<'PY'
|
||||||
@@ -96,21 +92,36 @@ print(json.dumps({
|
|||||||
}))
|
}))
|
||||||
PY
|
PY
|
||||||
)
|
)
|
||||||
HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \
|
URL="${BASE_URL%/}/v1/messages"
|
||||||
"${BASE_URL%/}/v1/messages" \
|
EXTRA_HDR=(-H "anthropic-version: 2023-06-01")
|
||||||
-H "x-api-key: $KEY" \
|
AUTH=(-H "x-api-key: $KEY")
|
||||||
-H "anthropic-version: 2023-06-01" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d "$BODY")
|
|
||||||
EXTRACT=anthropic
|
EXTRACT=anthropic
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if [[ "$HTTP_CODE" != "200" ]]; then
|
attempt=0
|
||||||
|
HTTP_CODE="000"
|
||||||
|
while true; do
|
||||||
|
HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \
|
||||||
|
"$URL" \
|
||||||
|
"${AUTH[@]}" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
"${EXTRA_HDR[@]}" \
|
||||||
|
-d "$BODY" || echo "000")
|
||||||
|
if [[ "$HTTP_CODE" == "200" ]]; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
if [[ "$HTTP_CODE" == "429" && "$attempt" -lt "$RETRIES" ]]; then
|
||||||
|
sleep_s=$((2 ** attempt))
|
||||||
|
echo "$LABEL HTTP 429 — retry $((attempt + 1))/${RETRIES} after ${sleep_s}s" >&2
|
||||||
|
sleep "$sleep_s"
|
||||||
|
attempt=$((attempt + 1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
echo "$LABEL failed HTTP $HTTP_CODE" >&2
|
echo "$LABEL failed HTTP $HTTP_CODE" >&2
|
||||||
head -c 800 "$OUT_FILE.raw" >&2 || true
|
head -c 800 "$OUT_FILE.raw" >&2 || true
|
||||||
echo "{\"error\":\"http\",\"model\":\"$MODEL\",\"provider\":\"$PROVIDER\",\"http\":$HTTP_CODE}" > "$OUT_FILE"
|
echo "{\"error\":\"http\",\"model\":\"$MODEL\",\"provider\":\"$PROVIDER\",\"http\":$HTTP_CODE}" > "$OUT_FILE"
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
done
|
||||||
|
|
||||||
python3 - "$OUT_FILE.raw" "$OUT_FILE" "$MODEL" "$PROVIDER" "$EXTRACT" <<'PY'
|
python3 - "$OUT_FILE.raw" "$OUT_FILE" "$MODEL" "$PROVIDER" "$EXTRACT" <<'PY'
|
||||||
import json,sys
|
import json,sys
|
||||||
@@ -123,7 +134,6 @@ if kind == "anthropic":
|
|||||||
if c.get("type") == "text"
|
if c.get("type") == "text"
|
||||||
)
|
)
|
||||||
elif kind == "openai-responses":
|
elif kind == "openai-responses":
|
||||||
# Best-effort extract from Responses API
|
|
||||||
text = raw.get("output_text") or ""
|
text = raw.get("output_text") or ""
|
||||||
if not text:
|
if not text:
|
||||||
chunks = []
|
chunks = []
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ const dir = args.dir;
|
|||||||
const floor = args.floor || "low";
|
const floor = args.floor || "low";
|
||||||
const agreementMin = Number(args["agreement-min"] || 2);
|
const agreementMin = Number(args["agreement-min"] || 2);
|
||||||
const scanHardFail = args["scan-hard-fail"] === "true";
|
const scanHardFail = args["scan-hard-fail"] === "true";
|
||||||
|
const language = (args.language || process.env.GATEKEEP_LANGUAGE || "en").trim() || "en";
|
||||||
const outPath = args.out;
|
const outPath = args.out;
|
||||||
|
|
||||||
const tierRank = { low: 0, medium: 1, high: 2 };
|
const tierRank = { low: 0, medium: 1, high: 2 };
|
||||||
@@ -166,10 +167,22 @@ for (const m of models) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Prefer successful model tiers; only if every call failed, fail-closed high.
|
||||||
|
const okModels = models.filter((m) => !m.error);
|
||||||
|
const modelErrors = models.filter((m) => m.error).length;
|
||||||
|
const degraded = modelErrors > 0 || okModels.length < agreementMin;
|
||||||
|
|
||||||
|
// When some models 429/fail, lower the bar to surface usable findings from survivors
|
||||||
|
// (still require at least 1 successful model). Full fail-closed high only if zero ok.
|
||||||
|
let effectiveAgreement = agreementMin;
|
||||||
|
if (okModels.length > 0 && okModels.length < agreementMin) {
|
||||||
|
effectiveAgreement = Math.max(1, okModels.length);
|
||||||
|
}
|
||||||
|
|
||||||
const agreed = [];
|
const agreed = [];
|
||||||
const disputed = [];
|
const disputed = [];
|
||||||
for (const f of bucket.values()) {
|
for (const f of bucket.values()) {
|
||||||
if (f.models.length >= agreementMin) agreed.push(f);
|
if (f.models.length >= effectiveAgreement) agreed.push(f);
|
||||||
else disputed.push(f);
|
else disputed.push(f);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -177,8 +190,10 @@ const bySeverity = (a, b) => (sevRank[a.severity] ?? 9) - (sevRank[b.severity] ?
|
|||||||
agreed.sort(bySeverity);
|
agreed.sort(bySeverity);
|
||||||
disputed.sort(bySeverity);
|
disputed.sort(bySeverity);
|
||||||
|
|
||||||
|
const tierSources = okModels.length ? okModels : models;
|
||||||
let risk_tier = floor;
|
let risk_tier = floor;
|
||||||
for (const m of models) risk_tier = maxTier(risk_tier, m.risk_tier);
|
for (const m of tierSources) risk_tier = maxTier(risk_tier, m.risk_tier);
|
||||||
|
if (!okModels.length) risk_tier = maxTier(risk_tier, "high");
|
||||||
for (const f of agreed) {
|
for (const f of agreed) {
|
||||||
if (f.kind === "security" || f.severity === "Critical") risk_tier = maxTier(risk_tier, "high");
|
if (f.kind === "security" || f.severity === "Critical") risk_tier = maxTier(risk_tier, "high");
|
||||||
else if (f.severity === "High" || f.tag === "Important") risk_tier = maxTier(risk_tier, "medium");
|
else if (f.severity === "High" || f.tag === "Important") risk_tier = maxTier(risk_tier, "medium");
|
||||||
@@ -192,6 +207,11 @@ const result = {
|
|||||||
risk_tier,
|
risk_tier,
|
||||||
policy_floor: floor,
|
policy_floor: floor,
|
||||||
agreement_min: agreementMin,
|
agreement_min: agreementMin,
|
||||||
|
effective_agreement_min: effectiveAgreement,
|
||||||
|
language,
|
||||||
|
degraded,
|
||||||
|
model_errors: modelErrors,
|
||||||
|
models_ok: okModels.length,
|
||||||
models: models.map((m) => ({
|
models: models.map((m) => ({
|
||||||
id: m.model,
|
id: m.model,
|
||||||
risk_tier: m.risk_tier,
|
risk_tier: m.risk_tier,
|
||||||
|
|||||||
+187
-53
@@ -2,6 +2,7 @@
|
|||||||
/**
|
/**
|
||||||
* Publish Gatekeep summary + agreed inline comments.
|
* Publish Gatekeep summary + agreed inline comments.
|
||||||
* Forge-agnostic (GitHub + Gitea) via fetch — no gh CLI required.
|
* Forge-agnostic (GitHub + Gitea) via fetch — no gh CLI required.
|
||||||
|
* Report language from merged.language / GATEKEEP_LANGUAGE (zh-CN, en, …).
|
||||||
*/
|
*/
|
||||||
import { readFileSync, writeFileSync } from "node:fs";
|
import { readFileSync, writeFileSync } from "node:fs";
|
||||||
|
|
||||||
@@ -24,8 +25,133 @@ const checkName = "gatekeep / security";
|
|||||||
const apiBase = (process.env.GITHUB_API_URL || "https://api.github.com").replace(/\/$/, "");
|
const apiBase = (process.env.GITHUB_API_URL || "https://api.github.com").replace(/\/$/, "");
|
||||||
const gitea = !apiBase.includes("api.github.com");
|
const gitea = !apiBase.includes("api.github.com");
|
||||||
|
|
||||||
|
const language = (
|
||||||
|
merged.language ||
|
||||||
|
process.env.GATEKEEP_LANGUAGE ||
|
||||||
|
"en"
|
||||||
|
)
|
||||||
|
.trim()
|
||||||
|
.toLowerCase();
|
||||||
|
const zh = language.startsWith("zh");
|
||||||
|
|
||||||
const SEV_EMOJI = { Critical: "🔴", High: "🟠", Medium: "🟡", Low: "🟢" };
|
const SEV_EMOJI = { Critical: "🔴", High: "🟠", Medium: "🟡", Low: "🟢" };
|
||||||
|
|
||||||
|
const T = zh
|
||||||
|
? {
|
||||||
|
title: "## Gatekeep 审查",
|
||||||
|
intro: "多模型并行审查结果如下。**先看严重级别**,Critical → Low 几秒扫完。",
|
||||||
|
risk: "PR 风险档",
|
||||||
|
floor: "策略底线",
|
||||||
|
agreement: "一致门槛",
|
||||||
|
agreedDisputed: "一致 / 分歧",
|
||||||
|
critHigh: "Critical / High(一致)",
|
||||||
|
modelsHdr: "### 模型(并行)",
|
||||||
|
modelCol: "模型",
|
||||||
|
tierCol: "其 PR 档",
|
||||||
|
findingsCol: "结论数",
|
||||||
|
agreedHdr: "### 一致结论",
|
||||||
|
agreedHelp:
|
||||||
|
"按严重级别排序。每条都写清 **严重级别**、**位置**、**为什么重要**。\n修复,或回复 `dismissed: <原因>` 后 resolve 讨论串。Gatekeep **没有**聊天 / @ 命令入口。",
|
||||||
|
agreedEmpty: "_模型之间没有达成一致的结论 — 可能很好,也可能该看分歧。_",
|
||||||
|
disputedHdr: "### 分歧(仅摘要,不上 inline)",
|
||||||
|
disputedHelp: "只有部分模型看到。有参考价值,本身不挡合并。",
|
||||||
|
disputedEmpty: "_没有值得单独列出的分歧。_",
|
||||||
|
scanHdr: "### 确定性安全扫描",
|
||||||
|
scanEmpty: "_无硬失败命中。_",
|
||||||
|
policyHdr: "### 策略底线信号",
|
||||||
|
policyNone: "- 无",
|
||||||
|
humansHdr: "### 仍需人工",
|
||||||
|
humansDefault: "- 品味、架构、计费/CI/鉴权、读证据",
|
||||||
|
footer:
|
||||||
|
"漏了真问题?开 **gatekeep-incident** issue — 这样才能变成 eval。\n_仅在模型一致时发 inline。PR 档失败关闭。结论以严重级别 + 位置 + 原因为先。_",
|
||||||
|
where: "位置",
|
||||||
|
why: "为什么重要",
|
||||||
|
whyFallback: "未写清影响 — 驳回前请先补一句。",
|
||||||
|
modelsLabel: "模型",
|
||||||
|
pre: " · 既有",
|
||||||
|
block: " · 阻断",
|
||||||
|
dispute: " · 分歧(未上 inline)",
|
||||||
|
preInline: "(既有)",
|
||||||
|
inlineFooter: (models) =>
|
||||||
|
`_由 ${models} 一致认定。请修复,或回复 \`dismissed: <原因>\` 后 resolve。_\n_无聊天命令 — Gatekeep 不响应 @。_`,
|
||||||
|
reviewBody: "Gatekeep 一致结论(严重级别 · 位置 · 原因)。请修复或驳回 — 无 @ 命令。",
|
||||||
|
fileLevel: (path) => `_文件级(无线号)于 \`${path}\`。_`,
|
||||||
|
noLoc: "(无位置)",
|
||||||
|
approveLow:
|
||||||
|
"看起来风险低 — Gatekeep + 安全绿灯后可不需人工(仅当 kill switch 打开才可自动合并)。",
|
||||||
|
approveMed: "需要 **1 名人工**过目。Bot 点赞不算人工。",
|
||||||
|
approveHigh: "需要 **2 名人工**批准。Bot 不能顶替。",
|
||||||
|
errNote: (n, ok, degAgree) =>
|
||||||
|
`> **降级说明:** ${n} 个模型调用失败(常见免费档限流 429;已重试/尝试备用模型)。` +
|
||||||
|
(ok === 0
|
||||||
|
? `全部失败 → 风险档**失败关闭为 high**,**不等于**代码一定有问题。`
|
||||||
|
: `已用 ${ok} 个成功响应继续审查` +
|
||||||
|
(degAgree ? `(一致门槛临时降为 ${degAgree})` : "") +
|
||||||
|
`;失败关闭的 high **仅在零成功响应时**强制抬升。`),
|
||||||
|
statusBlock: "存在未关闭的安全 / Critical 结论 — 合并阻断",
|
||||||
|
statusOk: (tier) => `PR 风险 ${tier} — 无未关闭安全阻断`,
|
||||||
|
modelsAgree: "一致门槛 ≥",
|
||||||
|
}
|
||||||
|
: {
|
||||||
|
title: "## Gatekeep review",
|
||||||
|
intro:
|
||||||
|
"Hey — here's the multi-model pass. **Severity first** so you can skim Critical → Low in seconds.",
|
||||||
|
risk: "PR risk tier",
|
||||||
|
floor: "Policy floor",
|
||||||
|
agreement: "Agreement bar",
|
||||||
|
agreedDisputed: "Agreed / disputed",
|
||||||
|
critHigh: "Critical / High (agreed)",
|
||||||
|
modelsHdr: "### Models (parallel)",
|
||||||
|
modelCol: "Model",
|
||||||
|
tierCol: "Their PR tier",
|
||||||
|
findingsCol: "Findings",
|
||||||
|
agreedHdr: "### Agreed findings",
|
||||||
|
agreedHelp:
|
||||||
|
"Sorted by severity. Each one spells out **severity**, **where**, and **why it matters**.\nFix or dismiss with `dismissed: <reason>` (then resolve the thread). Gatekeep has **no** chat / @-command surface.",
|
||||||
|
agreedEmpty:
|
||||||
|
"_Nothing the models agreed on — nice, or spooky. Glance at disputed below._",
|
||||||
|
disputedHdr: "### Disputed (summary only — not posted inline)",
|
||||||
|
disputedHelp:
|
||||||
|
"One model saw it; others didn't. Useful signal, not a merge blocker by itself.",
|
||||||
|
disputedEmpty: "_No disagreements worth parking._",
|
||||||
|
scanHdr: "### Deterministic security scan",
|
||||||
|
scanEmpty: "_No hard-fail hits._",
|
||||||
|
policyHdr: "### Policy floor signals",
|
||||||
|
policyNone: "- none",
|
||||||
|
humansHdr: "### Still on humans",
|
||||||
|
humansDefault: "- taste, architecture, billing/CI/auth, reading the evidence",
|
||||||
|
footer:
|
||||||
|
"Missed something real? Open a **gatekeep-incident** issue — that's how it becomes an eval.\n_Inline comments only when models agree. PR tier is fail-closed. Findings lead with severity + location + why._",
|
||||||
|
where: "Where",
|
||||||
|
why: "Why it matters",
|
||||||
|
whyFallback: "Impact not spelled out — please clarify before dismissing.",
|
||||||
|
modelsLabel: "Models",
|
||||||
|
pre: " · pre-existing",
|
||||||
|
block: " · blocking",
|
||||||
|
dispute: " · disputed (not inlined)",
|
||||||
|
preInline: " (pre-existing)",
|
||||||
|
inlineFooter: (models) =>
|
||||||
|
`_Agreed by ${models}. Fix it, or reply \`dismissed: <reason>\` and resolve the thread._\n_No chat commands — Gatekeep doesn't take @-mentions._`,
|
||||||
|
reviewBody:
|
||||||
|
"Gatekeep agreed findings (severity · where · why). Fix or dismiss — no @-commands.",
|
||||||
|
fileLevel: (path) => `_File-level (no single line) on \`${path}\`._`,
|
||||||
|
noLoc: "(no location)",
|
||||||
|
approveLow:
|
||||||
|
"Looks low-risk — no human needed once Gatekeep + security are green (auto-merge only if the kill switch is on).",
|
||||||
|
approveMed: "Needs **1 human** look. Bot thumbs-up don't count as that look.",
|
||||||
|
approveHigh: "Needs **2 human** approvals. Bots never fill those seats.",
|
||||||
|
errNote: (n, ok, degAgree) =>
|
||||||
|
`> **Degraded:** ${n} model call(s) failed (often free-tier 429; retried / tried fallbacks). ` +
|
||||||
|
(ok === 0
|
||||||
|
? `Zero usable responses → risk tier **fail-closed high** (not proof the code is bad).`
|
||||||
|
: `Reviewing from ${ok} successful response(s)` +
|
||||||
|
(degAgree ? ` (agreement bar temporarily ${degAgree})` : "") +
|
||||||
|
`; fail-closed high is forced only when **zero** models succeed.`),
|
||||||
|
statusBlock: "Open security / Critical findings — merge blocked",
|
||||||
|
statusOk: (tier) => `PR risk ${tier} — no open security blockers`,
|
||||||
|
modelsAgree: "≥",
|
||||||
|
};
|
||||||
|
|
||||||
async function forgeApi(method, path, body) {
|
async function forgeApi(method, path, body) {
|
||||||
const p = path.startsWith("/") ? path : `/${path}`;
|
const p = path.startsWith("/") ? path : `/${path}`;
|
||||||
const url = `${apiBase}${p}`;
|
const url = `${apiBase}${p}`;
|
||||||
@@ -48,50 +174,46 @@ async function forgeApi(method, path, body) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function approvals(tier) {
|
function approvals(tier) {
|
||||||
if (tier === "low")
|
if (tier === "low") return T.approveLow;
|
||||||
return "Looks low-risk — no human needed once Gatekeep + security are green (auto-merge only if the kill switch is on).";
|
if (tier === "medium") return T.approveMed;
|
||||||
if (tier === "medium") return "Needs **1 human** look. Bot thumbs-up don't count as that look.";
|
return T.approveHigh;
|
||||||
return "Needs **2 human** approvals. Bots never fill those seats.";
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function loc(f) {
|
function loc(f) {
|
||||||
return f.location || (f.path ? `${f.path}${f.line ? `:${f.line}` : ""}` : "(no location)");
|
return f.location || (f.path ? `${f.path}${f.line ? `:${f.line}` : ""}` : T.noLoc);
|
||||||
}
|
}
|
||||||
|
|
||||||
function formatFindingBlock(f, { disputed = false } = {}) {
|
function formatFindingBlock(f, { disputed = false } = {}) {
|
||||||
const sev = f.severity || "Medium";
|
const sev = f.severity || "Medium";
|
||||||
const emoji = SEV_EMOJI[sev] || "⚪";
|
const emoji = SEV_EMOJI[sev] || "⚪";
|
||||||
const pre = f.tag === "Pre-existing" ? " · pre-existing" : "";
|
const pre = f.tag === "Pre-existing" ? T.pre : "";
|
||||||
const block = f.blocking ? " · blocking" : "";
|
const block = f.blocking ? T.block : "";
|
||||||
const dispute = disputed ? " · disputed (not inlined)" : "";
|
const dispute = disputed ? T.dispute : "";
|
||||||
const tagNote = f.tag && f.tag !== "Important" ? ` · ${f.tag}` : "";
|
const tagNote = f.tag && f.tag !== "Important" ? ` · ${f.tag}` : "";
|
||||||
const why = f.why_it_matters || "Impact not spelled out — please clarify before dismissing.";
|
const why = f.why_it_matters || T.whyFallback;
|
||||||
const detail = (f.body || "").trim();
|
const detail = (f.body || "").trim();
|
||||||
const detailBit = detail && detail !== why ? `\n\n${detail}` : "";
|
const detailBit = detail && detail !== why ? `\n\n${detail}` : "";
|
||||||
return (
|
return (
|
||||||
`#### ${emoji} ${sev}${pre}${block}${dispute}${tagNote}: ${f.title}\n` +
|
`#### ${emoji} ${sev}${pre}${block}${dispute}${tagNote}: ${f.title}\n` +
|
||||||
`- **Where:** \`${loc(f)}\`\n` +
|
`- **${T.where}:** \`${loc(f)}\`\n` +
|
||||||
`- **Why it matters:** ${why}` +
|
`- **${T.why}:** ${why}` +
|
||||||
detailBit +
|
detailBit +
|
||||||
`\n- _Models:_ ${f.models.join(", ")}`
|
`\n- _${T.modelsLabel}:_ ${f.models.join(", ")}`
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
function formatInline(f) {
|
function formatInline(f) {
|
||||||
const sev = f.severity || "Medium";
|
const sev = f.severity || "Medium";
|
||||||
const emoji = SEV_EMOJI[sev] || "⚪";
|
const emoji = SEV_EMOJI[sev] || "⚪";
|
||||||
const pre = f.tag === "Pre-existing" ? " (pre-existing)" : "";
|
const pre = f.tag === "Pre-existing" ? T.preInline : "";
|
||||||
const why = f.why_it_matters || "Impact not spelled out — please clarify before dismissing.";
|
const why = f.why_it_matters || T.whyFallback;
|
||||||
const detail = (f.body || "").trim();
|
const detail = (f.body || "").trim();
|
||||||
let out =
|
let out =
|
||||||
`### ${emoji} ${sev}${pre} — ${f.title}\n\n` +
|
`### ${emoji} ${sev}${pre} — ${f.title}\n\n` +
|
||||||
`**Where:** \`${loc(f)}\`\n\n` +
|
`**${T.where}:** \`${loc(f)}\`\n\n` +
|
||||||
`**Why it matters:** ${why}\n`;
|
`**${T.why}:** ${why}\n`;
|
||||||
if (detail && detail !== why) out += `\n${detail}\n`;
|
if (detail && detail !== why) out += `\n${detail}\n`;
|
||||||
out +=
|
out += `\n---\n` + T.inlineFooter(f.models.join(", "));
|
||||||
`\n---\n` +
|
|
||||||
`_Agreed by ${f.models.join(", ")}. Fix it, or reply \`dismissed: <reason>\` and resolve the thread._\n` +
|
|
||||||
`_No chat commands — Gatekeep doesn't take @-mentions._`;
|
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -100,12 +222,12 @@ const sortedDisputed = [...(merged.disputed_findings || [])];
|
|||||||
|
|
||||||
const agreedList =
|
const agreedList =
|
||||||
sortedAgreed.length === 0
|
sortedAgreed.length === 0
|
||||||
? "_Nothing the models agreed on — nice, or spooky. Glance at disputed below._"
|
? T.agreedEmpty
|
||||||
: sortedAgreed.map((f) => formatFindingBlock(f)).join("\n\n");
|
: sortedAgreed.map((f) => formatFindingBlock(f)).join("\n\n");
|
||||||
|
|
||||||
const disputedList =
|
const disputedList =
|
||||||
sortedDisputed.length === 0
|
sortedDisputed.length === 0
|
||||||
? "_No disagreements worth parking._"
|
? T.disputedEmpty
|
||||||
: sortedDisputed.map((f) => formatFindingBlock(f, { disputed: true })).join("\n\n");
|
: sortedDisputed.map((f) => formatFindingBlock(f, { disputed: true })).join("\n\n");
|
||||||
|
|
||||||
const modelRows = merged.models
|
const modelRows = merged.models
|
||||||
@@ -118,49 +240,64 @@ const modelRows = merged.models
|
|||||||
const critCount = sortedAgreed.filter((f) => f.severity === "Critical").length;
|
const critCount = sortedAgreed.filter((f) => f.severity === "Critical").length;
|
||||||
const highCount = sortedAgreed.filter((f) => f.severity === "High").length;
|
const highCount = sortedAgreed.filter((f) => f.severity === "High").length;
|
||||||
|
|
||||||
|
const errCount = Number(merged.model_errors || 0);
|
||||||
|
const okCount = Number(
|
||||||
|
merged.models_ok != null
|
||||||
|
? merged.models_ok
|
||||||
|
: (merged.models || []).filter((m) => !m.error).length,
|
||||||
|
);
|
||||||
|
const effAgree = merged.effective_agreement_min;
|
||||||
|
const degAgree =
|
||||||
|
merged.degraded && effAgree != null && effAgree < merged.agreement_min ? effAgree : null;
|
||||||
|
const errBanner =
|
||||||
|
errCount > 0 || merged.degraded ? `\n${T.errNote(errCount, okCount, degAgree)}\n` : "";
|
||||||
|
|
||||||
|
const agreementBar = zh
|
||||||
|
? `${T.modelsAgree}${merged.agreement_min} 个模型` +
|
||||||
|
(degAgree ? `(本轮降为 ${degAgree})` : "")
|
||||||
|
: `≥${merged.agreement_min} models` + (degAgree ? ` (this run: ${degAgree})` : "");
|
||||||
|
|
||||||
const body = `${marker}
|
const body = `${marker}
|
||||||
## Gatekeep review
|
${T.title}
|
||||||
|
|
||||||
Hey — here's the multi-model pass. **Severity first** so you can skim Critical → Low in seconds.
|
|
||||||
|
|
||||||
|
${T.intro}
|
||||||
|
${errBanner}
|
||||||
| | |
|
| | |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| **PR risk tier** | \`${merged.risk_tier}\` |
|
| **${T.risk}** | \`${merged.risk_tier}\` |
|
||||||
| Policy floor | \`${merged.policy_floor}\` |
|
| ${T.floor} | \`${merged.policy_floor}\` |
|
||||||
| Agreement bar | ≥${merged.agreement_min} models |
|
| ${T.agreement} | ${agreementBar} |
|
||||||
| Agreed / disputed | ${sortedAgreed.length} / ${sortedDisputed.length} |
|
| ${T.agreedDisputed} | ${sortedAgreed.length} / ${sortedDisputed.length} |
|
||||||
| Critical / High (agreed) | ${critCount} / ${highCount} |
|
| ${T.critHigh} | ${critCount} / ${highCount} |
|
||||||
|
|
||||||
${approvals(merged.risk_tier)}
|
${approvals(merged.risk_tier)}
|
||||||
|
|
||||||
### Models (parallel)
|
${T.modelsHdr}
|
||||||
| Model | Their PR tier | Findings |
|
| ${T.modelCol} | ${T.tierCol} | ${T.findingsCol} |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
${modelRows}
|
${modelRows}
|
||||||
|
|
||||||
### Agreed findings
|
${T.agreedHdr}
|
||||||
Sorted by severity. Each one spells out **severity**, **where**, and **why it matters**.
|
${T.agreedHelp}
|
||||||
Fix or dismiss with \`dismissed: <reason>\` (then resolve the thread). Gatekeep has **no** chat / @-command surface.
|
|
||||||
|
|
||||||
${agreedList}
|
${agreedList}
|
||||||
|
|
||||||
### Disputed (summary only — not posted inline)
|
${T.disputedHdr}
|
||||||
One model saw it; others didn't. Useful signal, not a merge blocker by itself.
|
${T.disputedHelp}
|
||||||
|
|
||||||
${disputedList}
|
${disputedList}
|
||||||
|
|
||||||
### Deterministic security scan
|
${T.scanHdr}
|
||||||
${(scan.hits || []).length ? (scan.hits || []).map((h) => `- ${h}`).join("\n") : "_No hard-fail hits._"}
|
${(scan.hits || []).length ? (scan.hits || []).map((h) => `- ${h}`).join("\n") : T.scanEmpty}
|
||||||
|
|
||||||
### Policy floor signals
|
${T.policyHdr}
|
||||||
${(floor.reasons || []).map((r) => `- ${r}`).join("\n") || "- none"}
|
${(floor.reasons || []).map((r) => `- ${r}`).join("\n") || T.policyNone}
|
||||||
|
|
||||||
### Still on humans
|
${T.humansHdr}
|
||||||
${(merged.human_focus || []).map((h) => `- ${h}`).join("\n") || "- taste, architecture, billing/CI/auth, reading the evidence"}
|
${(merged.human_focus || []).map((h) => `- ${h}`).join("\n") || T.humansDefault}
|
||||||
|
|
||||||
---
|
---
|
||||||
Missed something real? Open a **gatekeep-incident** issue — that's how it becomes an eval.
|
${T.footer}
|
||||||
_Inline comments only when models agree. PR tier is fail-closed. Findings lead with severity + location + why._
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
async function main() {
|
async function main() {
|
||||||
@@ -182,7 +319,6 @@ async function main() {
|
|||||||
.slice(0, 30)
|
.slice(0, 30)
|
||||||
.map((f) => ({
|
.map((f) => ({
|
||||||
path: f.path,
|
path: f.path,
|
||||||
// Gitea uses old_position/new_position in some versions; line works on GitHub
|
|
||||||
line: f.line,
|
line: f.line,
|
||||||
body: formatInline(f),
|
body: formatInline(f),
|
||||||
}));
|
}));
|
||||||
@@ -192,11 +328,11 @@ async function main() {
|
|||||||
await forgeApi("POST", `/repos/${owner}/${name}/pulls/${pr.number}/reviews`, {
|
await forgeApi("POST", `/repos/${owner}/${name}/pulls/${pr.number}/reviews`, {
|
||||||
commit_id: pr.head.sha,
|
commit_id: pr.head.sha,
|
||||||
event: "COMMENT",
|
event: "COMMENT",
|
||||||
body: "Gatekeep agreed findings (severity · where · why). Fix or dismiss — no @-commands.",
|
body: T.reviewBody,
|
||||||
comments: inline,
|
comments: inline,
|
||||||
});
|
});
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
console.error("Inline review failed; falling back to issue comments:", e.message);
|
console.error("inline review failed; falling back to issue comments:", e.message);
|
||||||
for (const f of inline) {
|
for (const f of inline) {
|
||||||
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, {
|
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, {
|
||||||
body: f.body + `\n\n_\`${f.path}:${f.line}\`_`,
|
body: f.body + `\n\n_\`${f.path}:${f.line}\`_`,
|
||||||
@@ -212,7 +348,7 @@ async function main() {
|
|||||||
f.path
|
f.path
|
||||||
) {
|
) {
|
||||||
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, {
|
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, {
|
||||||
body: formatInline(f) + `\n\n_File-level (no single line) on \`${f.path}\`._`,
|
body: formatInline(f) + `\n\n${T.fileLevel(f.path)}`,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -231,7 +367,6 @@ async function main() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
// Gitea expects { labels: ["name"] }; GitHub same for issues API
|
|
||||||
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/labels`, {
|
await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/labels`, {
|
||||||
labels: [want],
|
labels: [want],
|
||||||
});
|
});
|
||||||
@@ -242,10 +377,9 @@ async function main() {
|
|||||||
const conclusion = merged.open_security_blocking ? "failure" : "success";
|
const conclusion = merged.open_security_blocking ? "failure" : "success";
|
||||||
const statusState = merged.open_security_blocking ? "failure" : "success";
|
const statusState = merged.open_security_blocking ? "failure" : "success";
|
||||||
const desc = merged.open_security_blocking
|
const desc = merged.open_security_blocking
|
||||||
? "Open security / Critical findings — merge blocked"
|
? T.statusBlock
|
||||||
: `PR risk ${merged.risk_tier} — no open security blockers`;
|
: T.statusOk(merged.risk_tier);
|
||||||
|
|
||||||
// Prefer Checks API (GitHub); fall back to commit status (Gitea-friendly)
|
|
||||||
try {
|
try {
|
||||||
await forgeApi("POST", `/repos/${owner}/${name}/check-runs`, {
|
await forgeApi("POST", `/repos/${owner}/${name}/check-runs`, {
|
||||||
name: checkName,
|
name: checkName,
|
||||||
|
|||||||
+133
-60
@@ -46,8 +46,14 @@ KNOWN_DEFAULTS = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def die(msg: str, code: int = 1) -> None:
|
def _zh() -> bool:
|
||||||
print(f"gatekeep: {msg}", file=sys.stderr)
|
lang = (os.environ.get("GATEKEEP_LANGUAGE") or "").strip().lower()
|
||||||
|
return lang.startswith("zh")
|
||||||
|
|
||||||
|
|
||||||
|
def die(msg: str, code: int = 1, msg_zh: str | None = None) -> None:
|
||||||
|
text = msg_zh if (_zh() and msg_zh) else msg
|
||||||
|
print(f"gatekeep: {text}", file=sys.stderr)
|
||||||
sys.exit(code)
|
sys.exit(code)
|
||||||
|
|
||||||
|
|
||||||
@@ -57,7 +63,10 @@ def load_raw(path: Path) -> dict:
|
|||||||
try:
|
try:
|
||||||
import yaml # type: ignore
|
import yaml # type: ignore
|
||||||
except ImportError:
|
except ImportError:
|
||||||
die("PyYAML required to load providers.yaml (pip install pyyaml)")
|
die(
|
||||||
|
"PyYAML required to load providers.yaml (pip install pyyaml)",
|
||||||
|
msg_zh="加载 providers.yaml 需要 PyYAML(请 pip install pyyaml 或使用 Action 的 venv 步骤)",
|
||||||
|
)
|
||||||
raw = yaml.safe_load(text) or {}
|
raw = yaml.safe_load(text) or {}
|
||||||
else:
|
else:
|
||||||
raw = json.loads(text)
|
raw = json.loads(text)
|
||||||
@@ -109,16 +118,19 @@ def secret_value(key_name: str) -> str:
|
|||||||
return ""
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def http_json(url: str, headers: dict) -> dict | list:
|
def http_json(url: str, headers: dict, timeout: float = 25) -> dict | list:
|
||||||
req = urllib.request.Request(url, headers=headers, method="GET")
|
req = urllib.request.Request(url, headers=headers, method="GET")
|
||||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||||
return json.loads(resp.read().decode("utf-8"))
|
return json.loads(resp.read().decode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
def normalize_protocol(p: str) -> str:
|
def normalize_protocol(p: str) -> str:
|
||||||
p = (p or "").strip().lower()
|
p = (p or "").strip().lower()
|
||||||
if p not in PROTOCOL_WIRE:
|
if p not in PROTOCOL_WIRE:
|
||||||
die(f"unknown protocol {p!r} (want {' | '.join(PROTOCOLS)})")
|
die(
|
||||||
|
f"unknown protocol {p!r} (want {' | '.join(PROTOCOLS)})",
|
||||||
|
msg_zh=f"未知 protocol {p!r}(可选 {' | '.join(PROTOCOLS)})",
|
||||||
|
)
|
||||||
return p
|
return p
|
||||||
|
|
||||||
|
|
||||||
@@ -126,41 +138,72 @@ def wire_protocol(p: str) -> str:
|
|||||||
return PROTOCOL_WIRE[normalize_protocol(p)]
|
return PROTOCOL_WIRE[normalize_protocol(p)]
|
||||||
|
|
||||||
|
|
||||||
def discover_model_ids(base_url: str, protocol: str, api_key: str) -> set[str]:
|
def discover_model_ids(base_url: str, protocol: str, api_key: str) -> set[str] | None:
|
||||||
|
"""Return model id set, or None if discovery failed after retries (skip validation)."""
|
||||||
|
import time
|
||||||
|
|
||||||
base = base_url.rstrip("/")
|
base = base_url.rstrip("/")
|
||||||
wire = wire_protocol(protocol)
|
wire = wire_protocol(protocol)
|
||||||
ids: set[str] = set()
|
if wire not in ("openai-completions", "openai-responses", "anthropic"):
|
||||||
try:
|
die(f"unhandled protocol wire {wire!r}")
|
||||||
if wire in ("openai-completions", "openai-responses"):
|
|
||||||
data = http_json(
|
last_err: Exception | None = None
|
||||||
f"{base}/models",
|
attempts = int(os.environ.get("GATEKEEP_DISCOVER_RETRIES", "3"))
|
||||||
{"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
|
for attempt in range(max(1, attempts)):
|
||||||
|
try:
|
||||||
|
ids: set[str] = set()
|
||||||
|
if wire in ("openai-completions", "openai-responses"):
|
||||||
|
data = http_json(
|
||||||
|
f"{base}/models",
|
||||||
|
{
|
||||||
|
"Authorization": f"Bearer {api_key}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
for m in data.get("data") or []:
|
||||||
|
mid = m.get("id")
|
||||||
|
if mid:
|
||||||
|
ids.add(mid)
|
||||||
|
else:
|
||||||
|
data = http_json(
|
||||||
|
f"{base}/v1/models",
|
||||||
|
{
|
||||||
|
"x-api-key": api_key,
|
||||||
|
"anthropic-version": "2023-06-01",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
for m in data.get("data") or []:
|
||||||
|
mid = m.get("id")
|
||||||
|
if mid:
|
||||||
|
ids.add(mid)
|
||||||
|
return ids
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
body = e.read().decode("utf-8", errors="replace")[:400]
|
||||||
|
last_err = RuntimeError(f"HTTP {e.code} {body}")
|
||||||
|
# 401/403 are configuration errors — fail closed
|
||||||
|
if e.code in (401, 403):
|
||||||
|
die(
|
||||||
|
f"model discovery failed for {base}: {last_err}",
|
||||||
|
msg_zh=f"拉取模型列表失败({base}):{last_err} — 请检查 key",
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
last_err = e
|
||||||
|
if attempt + 1 < attempts:
|
||||||
|
sleep_s = 2**attempt
|
||||||
|
print(
|
||||||
|
f"gatekeep: model discovery retry {attempt + 1}/{attempts} "
|
||||||
|
f"after {sleep_s}s ({last_err})",
|
||||||
|
file=sys.stderr,
|
||||||
)
|
)
|
||||||
for m in data.get("data") or []:
|
time.sleep(sleep_s)
|
||||||
mid = m.get("id")
|
|
||||||
if mid:
|
print(
|
||||||
ids.add(mid)
|
f"gatekeep: model discovery failed for {base} after {attempts} tries "
|
||||||
elif wire == "anthropic":
|
f"({last_err}); skipping name validation and continuing",
|
||||||
data = http_json(
|
file=sys.stderr,
|
||||||
f"{base}/v1/models",
|
)
|
||||||
{
|
return None
|
||||||
"x-api-key": api_key,
|
|
||||||
"anthropic-version": "2023-06-01",
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
for m in data.get("data") or []:
|
|
||||||
mid = m.get("id")
|
|
||||||
if mid:
|
|
||||||
ids.add(mid)
|
|
||||||
else:
|
|
||||||
die(f"unhandled protocol wire {wire!r}")
|
|
||||||
except urllib.error.HTTPError as e:
|
|
||||||
body = e.read().decode("utf-8", errors="replace")[:400]
|
|
||||||
die(f"model discovery failed for {base}: HTTP {e.code} {body}")
|
|
||||||
except Exception as e:
|
|
||||||
die(f"model discovery failed for {base}: {e}")
|
|
||||||
return ids
|
|
||||||
|
|
||||||
|
|
||||||
def enrich_provider(pid: str, pconf: dict) -> dict:
|
def enrich_provider(pid: str, pconf: dict) -> dict:
|
||||||
@@ -189,6 +232,12 @@ def main() -> None:
|
|||||||
override_csv = sys.argv[3] if len(sys.argv) > 3 else ""
|
override_csv = sys.argv[3] if len(sys.argv) > 3 else ""
|
||||||
|
|
||||||
cfg = load_raw(cfg_path)
|
cfg = load_raw(cfg_path)
|
||||||
|
# Resolve language early so die() can speak Chinese when configured
|
||||||
|
override_lang = (os.environ.get("GATEKEEP_LANGUAGE") or "").strip()
|
||||||
|
cfg_lang = str(cfg.get("language") or cfg.get("report_language") or "").strip()
|
||||||
|
language = override_lang or cfg_lang or "en"
|
||||||
|
os.environ["GATEKEEP_LANGUAGE"] = language
|
||||||
|
|
||||||
providers_in = cfg["providers"]
|
providers_in = cfg["providers"]
|
||||||
providers: dict[str, dict] = {
|
providers: dict[str, dict] = {
|
||||||
pid: enrich_provider(pid, pconf) for pid, pconf in providers_in.items()
|
pid: enrich_provider(pid, pconf) for pid, pconf in providers_in.items()
|
||||||
@@ -240,54 +289,74 @@ def main() -> None:
|
|||||||
requested = uniq
|
requested = uniq
|
||||||
|
|
||||||
if len(requested) < 2:
|
if len(requested) < 2:
|
||||||
die(f"need 2–3 models for parallel review; got {len(requested)}")
|
die(
|
||||||
|
f"need 2–3 models for parallel review; got {len(requested)}",
|
||||||
|
msg_zh=f"并行审查至少需要 2–3 个模型,当前只有 {len(requested)} 个",
|
||||||
|
)
|
||||||
|
fallback_requested: list[dict] = []
|
||||||
if len(requested) > 3:
|
if len(requested) > 3:
|
||||||
print(f"gatekeep: truncating to first 3 of {len(requested)} models", file=sys.stderr)
|
print(
|
||||||
|
f"gatekeep: primary parallel set = first 3 of {len(requested)}; "
|
||||||
|
f"rest kept as fallbacks on 429/failure",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
fallback_requested = requested[3:]
|
||||||
requested = requested[:3]
|
requested = requested[:3]
|
||||||
|
|
||||||
provider_meta: dict[str, dict] = {}
|
provider_meta: dict[str, dict] = {}
|
||||||
for pid in {r["provider"] for r in requested}:
|
for pid in {r["provider"] for r in requested + fallback_requested}:
|
||||||
pconf = providers[pid]
|
pconf = providers[pid]
|
||||||
key_name = str(pconf["key"])
|
key_name = str(pconf["key"])
|
||||||
api_key = secret_value(key_name)
|
api_key = secret_value(key_name)
|
||||||
if not api_key:
|
if not api_key:
|
||||||
die(
|
die(
|
||||||
f"missing secret/env for provider {pid!r}: expected env var named {key_name!r} "
|
f"missing secret/env for provider {pid!r}: expected env var named {key_name!r} "
|
||||||
f"(from providers.*.key — free-form; set it on the workflow step env from secrets)."
|
f"(from providers.*.key — free-form; set it on the workflow step env from secrets).",
|
||||||
|
msg_zh=(
|
||||||
|
f"缺少供应商 {pid!r} 的密钥:需要环境变量 {key_name!r}"
|
||||||
|
f"(来自 providers.*.key;请在 workflow step 的 env 里从 secrets 注入)。"
|
||||||
|
),
|
||||||
)
|
)
|
||||||
available = discover_model_ids(pconf["base_url"], pconf["protocol"], api_key)
|
available = discover_model_ids(pconf["base_url"], pconf["protocol"], api_key)
|
||||||
if not available:
|
if available is not None and len(available) == 0:
|
||||||
die(f"provider {pid}: model list empty — check base_url/protocol/key")
|
die(
|
||||||
|
f"provider {pid}: model list empty — check base_url/protocol/key",
|
||||||
|
msg_zh=f"供应商 {pid} 模型列表为空 — 请检查 base_url / protocol / key",
|
||||||
|
)
|
||||||
provider_meta[pid] = {
|
provider_meta[pid] = {
|
||||||
"base_url": pconf["base_url"].rstrip("/"),
|
"base_url": pconf["base_url"].rstrip("/"),
|
||||||
"protocol": pconf["protocol"],
|
"protocol": pconf["protocol"],
|
||||||
"wire_protocol": wire_protocol(pconf["protocol"]),
|
"wire_protocol": wire_protocol(pconf["protocol"]),
|
||||||
"key_name": key_name,
|
"key_name": key_name,
|
||||||
"api_key": api_key,
|
"api_key": api_key,
|
||||||
"available_count": len(available),
|
"available_count": len(available) if available is not None else -1,
|
||||||
"_available": available,
|
"_available": available, # None ⇒ skip validation
|
||||||
}
|
}
|
||||||
|
|
||||||
plan_models = []
|
def to_plan_entry(r: dict) -> dict:
|
||||||
for r in requested:
|
|
||||||
pid, model = r["provider"], r["model"]
|
pid, model = r["provider"], r["model"]
|
||||||
avail = provider_meta[pid]["_available"]
|
avail = provider_meta[pid]["_available"]
|
||||||
if model not in avail:
|
if avail is not None and model not in avail:
|
||||||
sample = sorted(avail)[:8]
|
sample = sorted(avail)[:8]
|
||||||
die(
|
die(
|
||||||
f"model {model!r} not available on provider {pid} "
|
f"model {model!r} not available on provider {pid} "
|
||||||
f"({provider_meta[pid]['available_count']} listed). Examples: {sample}"
|
f"({provider_meta[pid]['available_count']} listed). Examples: {sample}",
|
||||||
|
msg_zh=(
|
||||||
|
f"模型 {model!r} 在供应商 {pid} 上不可用"
|
||||||
|
f"(列表共 {provider_meta[pid]['available_count']} 个)。示例:{sample}"
|
||||||
|
),
|
||||||
)
|
)
|
||||||
plan_models.append(
|
return {
|
||||||
{
|
"provider": pid,
|
||||||
"provider": pid,
|
"model": model,
|
||||||
"model": model,
|
"base_url": provider_meta[pid]["base_url"],
|
||||||
"base_url": provider_meta[pid]["base_url"],
|
"protocol": provider_meta[pid]["protocol"],
|
||||||
"protocol": provider_meta[pid]["protocol"],
|
"wire_protocol": provider_meta[pid]["wire_protocol"],
|
||||||
"wire_protocol": provider_meta[pid]["wire_protocol"],
|
"key_name": provider_meta[pid]["key_name"],
|
||||||
"key_name": provider_meta[pid]["key_name"],
|
}
|
||||||
}
|
|
||||||
)
|
plan_models = [to_plan_entry(r) for r in requested]
|
||||||
|
fallback_models = [to_plan_entry(r) for r in fallback_requested]
|
||||||
|
|
||||||
secrets_path = out_path.with_suffix(".secrets.json")
|
secrets_path = out_path.with_suffix(".secrets.json")
|
||||||
secrets_path.write_text(
|
secrets_path.write_text(
|
||||||
@@ -299,7 +368,9 @@ def main() -> None:
|
|||||||
"agreement_min": int(cfg.get("agreement_min", 2)),
|
"agreement_min": int(cfg.get("agreement_min", 2)),
|
||||||
"temperature": cfg.get("temperature", 0),
|
"temperature": cfg.get("temperature", 0),
|
||||||
"max_tokens": int(cfg.get("max_tokens", 4096)),
|
"max_tokens": int(cfg.get("max_tokens", 4096)),
|
||||||
|
"language": language,
|
||||||
"models": plan_models,
|
"models": plan_models,
|
||||||
|
"fallback_models": fallback_models,
|
||||||
"secrets_file": str(secrets_path),
|
"secrets_file": str(secrets_path),
|
||||||
"providers_used": sorted(provider_meta.keys()),
|
"providers_used": sorted(provider_meta.keys()),
|
||||||
}
|
}
|
||||||
@@ -309,8 +380,10 @@ def main() -> None:
|
|||||||
{
|
{
|
||||||
"providers": plan["providers_used"],
|
"providers": plan["providers_used"],
|
||||||
"models": [f"{m['provider']}:{m['model']}" for m in plan_models],
|
"models": [f"{m['provider']}:{m['model']}" for m in plan_models],
|
||||||
|
"fallbacks": [f"{m['provider']}:{m['model']}" for m in fallback_models],
|
||||||
"keys": {m["provider"]: m["key_name"] for m in plan_models},
|
"keys": {m["provider"]: m["key_name"] for m in plan_models},
|
||||||
"agreement_min": plan["agreement_min"],
|
"agreement_min": plan["agreement_min"],
|
||||||
|
"language": plan["language"],
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
flush=True,
|
flush=True,
|
||||||
|
|||||||
Regular → Executable
+70
-9
@@ -39,6 +39,9 @@ python3 "$ACTION/scripts/resolve-providers.py" \
|
|||||||
"${GATEKEEP_MODELS:-}"
|
"${GATEKEEP_MODELS:-}"
|
||||||
|
|
||||||
AGREEMENT_MIN=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["agreement_min"])' "$PLAN")
|
AGREEMENT_MIN=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["agreement_min"])' "$PLAN")
|
||||||
|
# Prefer Action input / env override already applied in plan.language
|
||||||
|
export GATEKEEP_LANGUAGE
|
||||||
|
GATEKEEP_LANGUAGE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("language") or "en")' "$PLAN")
|
||||||
export GATEKEEP_MAX_TOKENS
|
export GATEKEEP_MAX_TOKENS
|
||||||
GATEKEEP_MAX_TOKENS=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("max_tokens",4096))' "$PLAN")
|
GATEKEEP_MAX_TOKENS=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("max_tokens",4096))' "$PLAN")
|
||||||
SECRETS_FILE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["secrets_file"])' "$PLAN")
|
SECRETS_FILE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["secrets_file"])' "$PLAN")
|
||||||
@@ -75,15 +78,25 @@ diff = (out / "diff.txt").read_text() if (out / "diff.txt").exists() else ""
|
|||||||
discover = json.loads((out / "discover.json").read_text()) if (out / "discover.json").exists() else {}
|
discover = json.loads((out / "discover.json").read_text()) if (out / "discover.json").exists() else {}
|
||||||
plan = json.loads((out / "provider-plan.json").read_text())
|
plan = json.loads((out / "provider-plan.json").read_text())
|
||||||
|
|
||||||
system = """You are one reviewer in Gatekeep, a multi-model PR review system for the repo owner's own projects.
|
language = (plan.get("language") or os.environ.get("GATEKEEP_LANGUAGE") or "en").strip() or "en"
|
||||||
|
lang_note = (
|
||||||
|
f"LANGUAGE: Write ALL human-facing prose in {language} "
|
||||||
|
f"(summary, rationale, finding title / why_it_matters / body, human_focus). "
|
||||||
|
f"Keep JSON keys and enum tokens in English exactly as specified "
|
||||||
|
f"(risk_tier, severity Critical|High|Medium|Low, tag, kind)."
|
||||||
|
)
|
||||||
|
|
||||||
|
system = f"""You are one reviewer in Gatekeep, a multi-model PR review system for the repo owner's own projects.
|
||||||
Your job is useful review content that makes human triage cheap — not chat UX.
|
Your job is useful review content that makes human triage cheap — not chat UX.
|
||||||
|
|
||||||
|
{lang_note}
|
||||||
|
|
||||||
Return ONLY valid JSON (no markdown fences) with:
|
Return ONLY valid JSON (no markdown fences) with:
|
||||||
{
|
{{
|
||||||
"risk_tier": "low|medium|high",
|
"risk_tier": "low|medium|high",
|
||||||
"summary": "short chatty overview",
|
"summary": "short chatty overview",
|
||||||
"rationale": "why this PR risk tier",
|
"rationale": "why this PR risk tier",
|
||||||
"findings": [{
|
"findings": [{{
|
||||||
"severity": "Critical|High|Medium|Low",
|
"severity": "Critical|High|Medium|Low",
|
||||||
"path": "file path",
|
"path": "file path",
|
||||||
"line": 1,
|
"line": 1,
|
||||||
@@ -94,14 +107,14 @@ Return ONLY valid JSON (no markdown fences) with:
|
|||||||
"why_it_matters": "concrete impact if ignored",
|
"why_it_matters": "concrete impact if ignored",
|
||||||
"body": "optional extra detail in a friendly tone",
|
"body": "optional extra detail in a friendly tone",
|
||||||
"blocking": false
|
"blocking": false
|
||||||
}],
|
}}],
|
||||||
"human_focus": ["string"]
|
"human_focus": ["string"]
|
||||||
}
|
}}
|
||||||
|
|
||||||
Finding rules (required on EVERY finding):
|
Finding rules (required on EVERY finding):
|
||||||
1. severity — Critical/High/Medium/Low so humans see critical vs minor instantly. (Distinct from PR risk_tier.)
|
1. severity — Critical/High/Medium/Low so humans see critical vs minor instantly. (Distinct from PR risk_tier.)
|
||||||
2. exact location — path + line (use line_end for a range when needed).
|
2. exact location — path + line (use line_end for a range when needed).
|
||||||
3. why_it_matters — the impact/reason, not just \"please fix\".
|
3. why_it_matters — the impact/reason, not just "please fix".
|
||||||
Tone: clear and human; structured fields, not robotic checklist-speak.
|
Tone: clear and human; structured fields, not robotic checklist-speak.
|
||||||
tag is secondary (Important/Nit/Pre-existing); severity dominates.
|
tag is secondary (Important/Nit/Pre-existing); severity dominates.
|
||||||
Prefer real harm over style nits. Drop pure style. kind=security or severity=Critical ⇒ blocking true.
|
Prefer real harm over style nits. Drop pure style. kind=security or severity=Critical ⇒ blocking true.
|
||||||
@@ -117,6 +130,7 @@ PR risk_tier rules:
|
|||||||
user = {
|
user = {
|
||||||
"policy_floor": floor,
|
"policy_floor": floor,
|
||||||
"policy_floor_reasons": [r.strip() for r in reasons.split("|") if r.strip()],
|
"policy_floor_reasons": [r.strip() for r in reasons.split("|") if r.strip()],
|
||||||
|
"language": language,
|
||||||
"discover": discover,
|
"discover": discover,
|
||||||
"reviewers": [f"{m['provider']}:{m['model']}" for m in plan.get("models", [])],
|
"reviewers": [f"{m['provider']}:{m['model']}" for m in plan.get("models", [])],
|
||||||
"pull_request": {
|
"pull_request": {
|
||||||
@@ -133,11 +147,11 @@ user = {
|
|||||||
},
|
},
|
||||||
"diff": diff,
|
"diff": diff,
|
||||||
}
|
}
|
||||||
json.dump({"system": system, "user": json.dumps(user, indent=2)}, open(out / "prompt.json", "w"))
|
json.dump({"system": system, "user": json.dumps(user, indent=2), "language": language}, open(out / "prompt.json", "w"))
|
||||||
print(f"Prompt ready; floor={floor}; plan={len(plan.get('models',[]))} models", flush=True)
|
print(f"Prompt ready; floor={floor}; language={language}; plan={len(plan.get('models',[]))} models", flush=True)
|
||||||
PY
|
PY
|
||||||
|
|
||||||
# Fire models in parallel (may be mixed providers)
|
# Fire primary models in parallel (may be mixed providers)
|
||||||
pids=()
|
pids=()
|
||||||
i=0
|
i=0
|
||||||
while IFS= read -r line; do
|
while IFS= read -r line; do
|
||||||
@@ -159,6 +173,52 @@ for pid in "${pids[@]}"; do
|
|||||||
done
|
done
|
||||||
echo "Parallel model calls finished (wait_ec=$ec)" >&2
|
echo "Parallel model calls finished (wait_ec=$ec)" >&2
|
||||||
|
|
||||||
|
# On partial failure, try fallback_models from providers.yaml (beyond the first 3)
|
||||||
|
python3 - "$MODELS_DIR" "$PLAN" <<'PY' >"$OUT/fallback-needed.txt"
|
||||||
|
import json, pathlib, sys
|
||||||
|
models_dir = pathlib.Path(sys.argv[1])
|
||||||
|
plan = json.load(open(sys.argv[2]))
|
||||||
|
failed = ok = 0
|
||||||
|
for p in models_dir.glob("model-*.json"):
|
||||||
|
try:
|
||||||
|
j = json.loads(p.read_text())
|
||||||
|
except Exception:
|
||||||
|
failed += 1
|
||||||
|
continue
|
||||||
|
if j.get("error") or j.get("parse_error"):
|
||||||
|
failed += 1
|
||||||
|
else:
|
||||||
|
ok += 1
|
||||||
|
fallbacks = plan.get("fallback_models") or []
|
||||||
|
# Aim for ≥2 usable responses; if zero ok, try up to 3 reserves
|
||||||
|
if ok == 0:
|
||||||
|
need = min(len(fallbacks), 3)
|
||||||
|
elif ok < 2:
|
||||||
|
need = min(len(fallbacks), 2 - ok)
|
||||||
|
else:
|
||||||
|
need = 0
|
||||||
|
print(need)
|
||||||
|
print(f"gatekeep: primary ok={ok} failed={failed} fallbacks={len(fallbacks)} need={need}", file=sys.stderr)
|
||||||
|
PY
|
||||||
|
NEED_FB=$(head -1 "$OUT/fallback-needed.txt" | tr -d '[:space:]')
|
||||||
|
NEED_FB=${NEED_FB:-0}
|
||||||
|
if [[ "$NEED_FB" -gt 0 ]]; then
|
||||||
|
echo "gatekeep: trying $NEED_FB fallback model(s) after partial failure" >&2
|
||||||
|
fb_i=0
|
||||||
|
while IFS= read -r line; do
|
||||||
|
fb_i=$((fb_i + 1))
|
||||||
|
[[ "$fb_i" -gt "$NEED_FB" ]] && break
|
||||||
|
provider=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["provider"])')
|
||||||
|
model=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["model"])')
|
||||||
|
protocol=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["protocol"])')
|
||||||
|
base_url=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["base_url"])')
|
||||||
|
safe=$(echo "fb_${provider}_${model}" | tr '/:' '__')
|
||||||
|
bash "$ACTION/scripts/call-model.sh" \
|
||||||
|
"$provider" "$model" "$protocol" "$base_url" \
|
||||||
|
"$OUT/prompt.json" "$MODELS_DIR/model-fb$fb_i-$safe.json" "$SECRETS_FILE" || true
|
||||||
|
done < <(python3 -c 'import json,sys; [print(json.dumps(m)) for m in json.load(open(sys.argv[1])).get("fallback_models") or []]' "$PLAN")
|
||||||
|
fi
|
||||||
|
|
||||||
# Scrub secrets file from workspace artifact path if present
|
# Scrub secrets file from workspace artifact path if present
|
||||||
rm -f "$SECRETS_FILE"
|
rm -f "$SECRETS_FILE"
|
||||||
|
|
||||||
@@ -167,6 +227,7 @@ node "$ACTION/scripts/merge-findings.js" \
|
|||||||
--floor "$FLOOR" \
|
--floor "$FLOOR" \
|
||||||
--agreement-min "$AGREEMENT_MIN" \
|
--agreement-min "$AGREEMENT_MIN" \
|
||||||
--scan-hard-fail "$SCAN_HARD" \
|
--scan-hard-fail "$SCAN_HARD" \
|
||||||
|
--language "$GATEKEEP_LANGUAGE" \
|
||||||
--out "$OUT/merged.json" | tee "$OUT/merge-meta.json"
|
--out "$OUT/merged.json" | tee "$OUT/merge-meta.json"
|
||||||
|
|
||||||
set +e
|
set +e
|
||||||
|
|||||||
Reference in new issue
Block a user