Review workflow

Modes and requests

The three modes

Each reviewer (each lineage, version and executor) has its own mode, set per repository on /connect and in the repository settings:

Wide tables: scroll horizontally to see more columns.

Mode What starts new work
Automatic Eligible pull-request events (opened, reopened, marked ready for review, pushed to) and explicit requests.
On request Explicit requests only. A push alone does not start this reviewer.
Off No new review or recheck work. History, deferrals and reading past findings remain available.

The saved mode controls each reviewer's future admissions. An ordinary job already accepted finishes under the mode it was admitted with. The one exception is a job started by an explicit reviewer selector (see Choosing a reviewer): it is checked against the live mode again while it runs and publishes, and stops if that one reviewer has been switched Off meanwhile.

What a newly connected repository starts with

  • Prerequisites: sign in with an admitted GitHub account. The Sigma Mnemoverse GitHub App installation must be within that account's approved scope, with current GitHub access to the repository.
  • Action: on /connect, tick the repository, choose the initial mode and save. Automatic is preselected for the instance owner; invited accounts start with On request. The page states the consequence before you save: "Automatic also queues existing open, non-draft pull requests. Before saving, move any PRs you want to exclude to Draft, or choose On request." and "On request connects new repositories without starting automatic reviews. Comment /sigma review on a PR when you want a review." It also reminds you that "Changing modes does not cancel reviews already queued." For invited beta accounts the same line starts "In the closed beta, reviews run on Sigma's model access at no charge to you, within a shared limit."; on the operator's own repositories it starts "With API keys, each review is paid."
  • Expected result: the installation's configured reviewers are bound to the repository with the chosen mode. With Automatic, every open non-draft pull request is queued at once.
  • Check: each queued pull request shows a neutral Review queued check per reviewer; the repository settings page lists the saved modes.
  • Recovery: set a reviewer to On request or Off in the repository settings to stop further admissions. Reviews already queued are not cancelled.

A repository you re-enable, or one that already has saved reviewer modes, keeps those modes; the initial-mode choice does not overwrite them. The reviewer version and executor follow the permitted roster; see the maintainer note under Executors.

If sign-in, installation or saving the repository does not finish, use the setup recovery table. Personal runner self-service has a separate availability limit.

Requesting a review explicitly

Explicit requests start enabled Automatic and On request reviewers on a pull request, including a draft:

  • a comment that is exactly /sigma review, optionally followed by a reviewer selector;
  • the App-mention form, @<the App's slug> review, with the same optional selector;
  • requesting a review from Sigma's own bot through GitHub's reviewer control. Requesting an unrelated reviewer does not start Sigma.

Each request is honoured once: a review request counts once per head, a comment command once per comment. Re-delivering or editing an old comment does not start another run, even on the same head. Requests start new model work: API stages are metered, subscription stages use the owner's allowance.

A request that names a reviewer Sigma cannot resolve posts a neutral Reviewer selection refused check: "the requested reviewer is unknown or not enabled for this repository; comment /sigma review to run this repository's reviewers". No model work happens. Checks published before public format v2 carry a longer sentence that also shows an example selector.

Who may call Sigma

A call requires a currently enabled repository within its connected account's scope. The instance owner may call directly for their own binding; other callers, including invited accounts, need GitHub-confirmed admin, maintain or write access to the repository. Without that access, or if Sigma cannot confirm it at that moment, the call is closed with the reason "pilot commands require write access", silently and without a retry; comment again once the cause is fixed. A confirmed permission is cached for one hour, so a permission change can take up to an hour to take effect.

Replying inside a finding's thread is a separate, narrower case: only the account that connected the repository can authorize it; see Replies and rechecks.

Drafts

An explicit request works on a draft pull request. Automatic events apply to open, non-draft pull requests only: a push to a draft does not start an Automatic reviewer, and a pull request that becomes a draft stops its running review (see below).

What a push does

A push does not restart a running review in place. If a run for the previous head is still queued or in progress when a newer head, a base change or a state change arrives, that run's check reads Review superseded with one of these reasons:

  • "The pull request was closed or merged; this review is no longer needed."
  • "The pull request became a draft, so Sigma stopped this review. Comment /sigma review to review the draft."
  • "A newer change superseded this run; it does not describe the current head."

A superseded run keeps its history and any cost it incurred; it is not a review of the new head. Automatic reviewers pick up the new head on their own; On request reviewers need a fresh /sigma review. What a completed review then does to earlier findings is described in What a push does to open findings.

If the maintainers pause Sigma after a job has fetched the code but before any model ran, the job returns to the queue with a fresh run identity, its check reads Review paused, and the abandoned run stays in history. No model work was done and nothing was charged.

For maintainers. A check that moves from "Review in progress" back to "Review paused" is this requeue, not a failure. The review starts again when the pause is lifted.

After Sigma was down

When Sigma comes back after downtime, it catches up on what it missed:

  • every open non-draft pull request is re-admitted against its current head, as if it had just been pushed, for Automatic reviewers;
  • missed /sigma review and App-mention comments are replayed, except comments that were edited before Sigma ever saw them, comments older than the repository's enablement (for every caller, the owner included) and, for callers other than the owner, comments older than the first catch-up scan;
  • owner replies in finding threads are re-read; other replies are not.

Nothing already consumed runs twice.