ci+fix: add update-providers workflow + non-destructive fetch_models (#914)
Some checks failed
CI / pre-commit (push) Has been cancelled
CI / plano-tools-tests (push) Has been cancelled
CI / native-smoke-test (push) Has been cancelled
CI / docker-build (push) Has been cancelled
CI / validate-config (push) Has been cancelled
Publish docker image (latest) / build-arm64 (push) Has been cancelled
Publish docker image (latest) / build-amd64 (push) Has been cancelled
Build and Deploy Documentation / build (push) Has been cancelled
CI / security-scan (push) Has been cancelled
CI / test-prompt-gateway (push) Has been cancelled
CI / test-model-alias-routing (push) Has been cancelled
CI / test-responses-api-with-state (push) Has been cancelled
CI / e2e-plano-tests (3.10) (push) Has been cancelled
CI / e2e-plano-tests (3.11) (push) Has been cancelled
CI / e2e-plano-tests (3.12) (push) Has been cancelled
CI / e2e-plano-tests (3.13) (push) Has been cancelled
CI / e2e-plano-tests (3.14) (push) Has been cancelled
CI / e2e-demo-preference (push) Has been cancelled
CI / e2e-demo-currency (push) Has been cancelled
Publish docker image (latest) / create-manifest (push) Has been cancelled

* ci: add update-providers workflow

Adds .github/workflows/update-providers.yml so the provider_models.yaml
refresh can be triggered via workflow_dispatch (manual UI / gh CLI) or
repository_dispatch (from the PlanoHelper Slack bot).

The workflow:
  - Runs cargo run --bin fetch_models --features model-fetch with all
    provider API keys + AWS creds available as env from secrets.
  - Opens a PR via peter-evans/create-pull-request scoped to just
    crates/hermesllm/src/bin/provider_models.yaml.
  - On repository_dispatch, posts the PR link (or failure) back to Slack
    via the response_url in the dispatch payload.

Includes keys for the providers fetch_models reads today (OpenAI,
Anthropic, Mistral, DeepSeek, Grok, Moonshot, Dashscope/Qwen, Zhipu,
Xiaomi/Mimo, Google) plus forward-compat env for OpenRouter and Vercel
AI Gateway (added in #902).

The workflow has no push: or schedule: trigger, so landing this is inert
until something dispatches it. Required secrets are documented in
apps/planohelper/README.md (in a follow-up PR).

* fix(fetch_models): preserve existing providers when keys are missing

Previously fetch_models rebuilt provider_models.yaml from scratch on
every run, so running locally (or in CI) without e.g. ANTHROPIC_API_KEY,
GOOGLE_API_KEY, or AWS Bedrock credentials would silently drop those
providers' entries from the file. The user only meant to refresh what
they had keys for.

Now fetch_models loads the existing provider_models.yaml first and
treats each provider independently:

  - Successful fetch -> entry replaced with fresh data ("updated")
  - Missing API key  -> existing entry preserved ("skipped")
  - Failed fetch     -> existing entry preserved ("failed, kept existing")
  - Missing AWS creds -> Amazon entry preserved instead of running
    `aws bedrock list-foundation-models` and erroring out

If the file doesn't exist yet it starts fresh, same as before. If the
file exists but can't be parsed, the binary refuses to overwrite it and
exits with an error rather than silently nuking it.

Other changes that come along for the ride:

  - HashMap -> BTreeMap for the providers map. Output YAML now has a
    stable, alphabetical provider order across runs (eliminates
    HashMap-iteration churn in PR diffs). The first PR after this
    lands will reorder existing entries one time.
  - Per-provider summary at the end (updated / skipped / failed)
    so the workflow logs and Slack PR body make it obvious what
    actually changed vs. what was left alone.
  - File-level usage comment updated to match the new behavior and
    list the additional env vars (MISTRAL_API_KEY, MIMO_API_KEY).

No tests existed for this binary; manually verified with `env -i` (no
keys at all) that all 13 existing providers are preserved with their
original model counts.
This commit is contained in:
Musa 2026-05-05 14:19:52 -07:00 committed by GitHub
parent b71a555f19
commit 5a4487fc6e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 308 additions and 78 deletions

124
.github/workflows/update-providers.yml vendored Normal file
View file

@ -0,0 +1,124 @@
name: Update provider_models.yaml
on:
repository_dispatch:
types: [update-providers]
workflow_dispatch:
permissions:
contents: write
pull-requests: write
jobs:
update-providers:
runs-on: ubuntu-latest
env:
RESPONSE_URL: ${{ github.event.client_payload.response_url }}
SLACK_USER_ID: ${{ github.event.client_payload.user_id }}
SLACK_USER_NAME: ${{ github.event.client_payload.user_name }}
steps:
- name: Checkout main
uses: actions/checkout@v6
with:
ref: main
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Cache cargo build
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
crates/target
key: cargo-fetch-models-${{ hashFiles('crates/**/Cargo.lock', 'crates/**/Cargo.toml') }}
restore-keys: cargo-fetch-models-
- name: Run fetch_models
working-directory: crates/hermesllm
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
GROK_API_KEY: ${{ secrets.GROK_API_KEY }}
DASHSCOPE_API_KEY: ${{ secrets.DASHSCOPE_API_KEY }}
MOONSHOT_API_KEY: ${{ secrets.MOONSHOT_API_KEY }}
ZHIPU_API_KEY: ${{ secrets.ZHIPU_API_KEY }}
MIMO_API_KEY: ${{ secrets.MIMO_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
run: cargo run --bin fetch_models --features model-fetch
- name: Create pull request
id: cpr
uses: peter-evans/create-pull-request@v7
with:
branch: bot/update-providers-${{ github.run_id }}
base: main
commit-message: "chore: refresh provider_models.yaml"
title: "chore: refresh provider_models.yaml"
body: |
Automated refresh of `crates/hermesllm/src/bin/provider_models.yaml`
via `fetch_models`.
Requested by ${{ env.SLACK_USER_NAME && format('@{0}', env.SLACK_USER_NAME) || 'workflow_dispatch' }}${{ env.SLACK_USER_ID && format(' (Slack `{0}`)', env.SLACK_USER_ID) || '' }}.
Workflow run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
labels: automated, provider-models
add-paths: crates/hermesllm/src/bin/provider_models.yaml
- name: Notify Slack (success)
if: success() && env.RESPONSE_URL != ''
env:
PR_URL: ${{ steps.cpr.outputs.pull-request-url }}
PR_NUMBER: ${{ steps.cpr.outputs.pull-request-number }}
PR_OP: ${{ steps.cpr.outputs.pull-request-operation }}
run: |
if [ -z "$PR_URL" ]; then
TEXT=":information_source: No provider model changes detected \u2014 nothing to PR."
BLOCKS=$(jq -nc --arg text "$TEXT" '{response_type:"ephemeral", replace_original:true, text:$text, blocks:[{type:"section", text:{type:"mrkdwn", text:$text}}]}')
else
TEXT=":white_check_mark: provider_models.yaml PR ready: $PR_URL"
BLOCKS=$(jq -nc \
--arg pr "$PR_URL" \
--arg num "$PR_NUMBER" \
--arg op "$PR_OP" \
'{
response_type:"ephemeral",
replace_original:true,
text:(":white_check_mark: provider_models.yaml PR #" + $num + " " + $op + ": " + $pr),
blocks:[
{type:"section", text:{type:"mrkdwn", text:(":white_check_mark: *provider_models.yaml* PR <" + $pr + "|#" + $num + "> " + $op + ".")}},
{type:"actions", elements:[{type:"button", text:{type:"plain_text", text:"Open PR"}, url:$pr}]}
]
}')
fi
curl -sS -X POST -H 'Content-Type: application/json' -d "$BLOCKS" "$RESPONSE_URL"
- name: Notify Slack (failure)
if: failure() && env.RESPONSE_URL != ''
run: |
RUN_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
TEXT=":x: provider_models.yaml update failed. Logs: $RUN_URL"
jq -nc \
--arg text "$TEXT" \
--arg run "$RUN_URL" \
'{
response_type:"ephemeral",
replace_original:true,
text:$text,
blocks:[
{type:"section", text:{type:"mrkdwn", text:(":x: *provider_models.yaml update failed.*")}},
{type:"actions", elements:[{type:"button", text:{type:"plain_text", text:"View logs"}, url:$run}]}
]
}' | curl -sS -X POST -H 'Content-Type: application/json' -d @- "$RESPONSE_URL"