ReviewSentry

Overview

ReviewSentry is a GitHub Action that posts a structured AI code review on every pull request. It runs before a human reviewer looks at the code — catching the class of issues that are time-consuming to spot manually and fast to fix when caught early.

The review leads with a sensitive data scan: credentials, API keys, personal identifiers, private paths, and machine names are flagged before any other finding. This is the most important check, and it runs first, always.

ReviewSentry is provider-agnostic. It works with Anthropic, OpenAI, Google Gemini, any OpenAI-compatible endpoint (Groq, Azure, Ollama), and — at zero additional cost — GitHub Models using your existing GITHUB_TOKEN.

How it works

ReviewSentry is a composite GitHub Action. Add it to your repository and it triggers on every pull request — opened, updated, or reopened.

The action runs four steps in sequence:

1. Fetch configuration — reads .github/reviewsentry.yml from the PR branch if present. This allows per-repository criteria overrides without changing the workflow file.
2. Capture the diff — uses gh pr diff to fetch the pull request diff, truncating at the configured line limit to stay within provider token limits.
3. Run the AI review — sends the diff, PR title, and PR body to your configured AI provider with a structured system prompt and criteria checklist.
4. Post the comment — posts the AI review as a PR comment, prefixed with a visible advisory notice that the verdict is a recommendation, not an instruction.

Verdict extraction runs after posting. The action reads the last line of the review for the AI Recommendation marker (APPROVE, APPROVE WITH NOTES, or REQUEST CHANGES) and exposes it as an action output, optionally failing the workflow if configured to do so.

Features

Security-first review

Sensitive data disclosure is always criterion 1. Credentials, API keys, personal information, private paths, and computer names are reported before any other finding, with severity classification: Critical for credentials, High for personal identifiers, Moderate for machine names and repo paths.

Configurable criteria

The default checklist covers ten criteria: sensitive data, merge conflicts, correctness, cross-platform compatibility, bash quality, security, code quality, dependencies, documentation, and PR scope. Each criterion can be disabled per-repository via .github/reviewsentry.yml, and custom criteria can be added either via the config file or the review_criteria input.

Advisory verdict format

Every review ends with a clear verdict marker — ✅ APPROVE, 📝 APPROVE WITH NOTES, or ❌ REQUEST CHANGES — along with a reminder that this is an AI recommendation, not an instruction. The verdict is always the last line of the review, so readers engage with the full findings before seeing the outcome.

Configurable CI gate

Set fail_on: request_changes to exit non-zero when the verdict is REQUEST CHANGES, allowing you to block merges via required status checks. The default is never — informational only, no CI failure.

Draft PR control

Reviews fire on draft PRs by default. Set review_drafts: false to skip review until the PR is marked ready for review, saving AI tokens on work-in-progress branches.

Colour-coded findings

Each criterion section opens with ✅ (no issues) or ⚠️ (issues present). Individual findings are prefixed with 🔴 Critical, 🟠 High, or 🟡 Moderate/Low. Use show_passing_criteria: false to suppress clean sections and keep reviews concise on large PRs.

Recommended workflow

ReviewSentry works best as a first-pass reviewer that your team builds on — not as a gate that replaces human judgement.

1. Open your PR as a draft. ReviewSentry reviews it immediately. Any issues are visible before a human even looks at the code.
2. Address the findings. Fix genuine issues and commit — ReviewSentry re-reviews automatically. For false positives, leave a short comment explaining the intentional choice. This creates a record for the human reviewer.
3. Mark the PR ready for review. Push a final commit if you want a fresh review at this point (see KI-004 — the ready-for-review event does not currently auto-trigger).
4. Confirm all checks are green. Required status checks pass; AI review verdict is APPROVE or APPROVE WITH NOTES.
5. Request a peer review. Share the AI review comment as context. The reviewer gets a structured starting point rather than a blank diff.
6. Merge. The human reviewer merges when satisfied. The AI verdict is advisory — the final decision always rests with the maintainer.

Configuration

Workflow inputs

All inputs are set in your .github/workflows/ai-review.yml file.

- uses: Spyced-Concepts/ReviewSentry@<commit-sha>
  with:
    ai_api_key:            ${{ secrets.GITHUB_TOKEN }}   # or your provider key
    ai_model:              gpt-4o                         # any provider model
    ai_provider:           github-models                  # or anthropic, openai, gemini
    pr_number:             ${{ github.event.pull_request.number }}
    pr_title:              ${{ github.event.pull_request.title }}
    pr_body:               ${{ github.event.pull_request.body }}
    github_token:          ${{ secrets.GITHUB_TOKEN }}
    # Optional:
    fail_on:               never          # or request_changes
    review_drafts:         true           # set false to skip draft PRs
    show_passing_criteria: true           # set false for concise output
    diff_lines:            1500           # increase for large PRs
    review_criteria:       ""             # additional criteria, one per line
    custom_rules:          ""             # sensitive data patterns to add
    system_context:        ""             # project-specific context for the AI

Per-repository criteria config

Add .github/reviewsentry.yml to your repository to customise criteria without changing the workflow file.

# .github/reviewsentry.yml
# Disable criteria that don't apply to your project
cross_platform: false   # single-platform project
bash_quality: false     # no shell scripts

# Add custom criteria
custom:
  - "Flag any use of console.log in production code"
  - "Check that async functions have error handling"

# Disable a core criterion (requires explicit acknowledgement)
sensitive_data: false
acknowledge_disabled_core: true

Setting up

Step 1 — Choose your provider. GitHub Models is the zero-cost option using your existing GITHUB_TOKEN. Anthropic, OpenAI, and Gemini require an API key stored as a repository secret.

Step 2 — Add the workflow file. Create .github/workflows/ai-review.yml in your repository. See the README for complete example workflows per provider.

Step 3 — Pin to a SHA. Tags are mutable. Pin your uses: reference to a full 40-character commit SHA for cryptographic immutability. The SHA for each release is on the Releases page.

Step 4 — Open a PR. ReviewSentry fires on the first PR opened after the workflow is added. No further configuration is required.

Known issues

Four known limitations are documented in KNOWN_ISSUES.md:

• KI-001 — Large PR diffs can cause incomplete reviews (no verdict produced). Reduce PR scope or increase diff_lines.
• KI-002 — The self-review check stays red on feature branch PRs until the fix merges to the stable branch.
• KI-003 — The test suite pins gherkin-official to an older version constrained by pytest-bdd.
• KI-004 — The review does not re-run automatically when a draft PR is marked ready for review. Push a commit to trigger it.

Open source

ReviewSentry is MIT licensed and developed in the open at github.com/Spyced-Concepts/ReviewSentry. Contributions, bug reports, and feature requests are welcome via GitHub Issues.

The project uses BDD-driven development. Feature files in features/ define expected behaviour before code is written. A pytest-bdd test suite validates the implementation.

Built by Spyced Concepts Ltd. — a security-focused software company based in Cornwall, UK. Questions? Get in touch.