# Running tests from GitHub

Run tests and suites on every deployment, post results to the commit, and block pull requests when preview tests fail.

> **Plan:** available on the Free plan and above.

Connecting GitHub turns Testsnag from something you remember to run into something that runs itself at the moment risk is introduced. Instead of finding out at the next scheduled run that a deploy broke checkout, you find out as part of the deploy, and you can stop a pull request from merging until its preview passes.

![The GitHub section of a test’s notification settings, with a repository selector and Check Run toggles.](https://testsnag.com/images/docs/notifications/github-actions-1-test-binding.r1.jpg)

Marked on the screenshot:
- Pick the repository this test should watch.
- A successful deployment for that repo runs the test automatically.
- Post a Check Run back to the deployment commit when the test passes.
- Or when it fails. Leave both off to bind the repo without posting checks.

## Connecting

Install the Testsnag GitHub App from the Integrations page and grant it access to the repositories you want to use. This is done once per workspace. Repository access can be narrowed to specific repositories rather than an entire organisation, and revoked from GitHub at any time.

## What triggers a run

Testsnag runs on successful deployments. When your host or pipeline reports a GitHub deployment with the status "success", every test and suite bound to that repository runs. Pushes and pull requests on their own do not trigger anything: the deployment is what tells Testsnag there is something live to test. Hosts such as Vercel report deployments to GitHub automatically.

## Binding a test to a repository

In the GitHub card of a test’s Notifications & Integrations settings, choose the repository it should watch. Optionally set an environment filter, such as "production", so the test only runs on deployments to that environment. Leave the filter blank to run on every successful deployment.

When the run finishes, Testsnag can post a Check Run named "testsnag · <test name>" to the deployed commit. Use the two toggles to post it when the test passes, when it fails, or both. Leave both off to run the test on deployments without posting anything to GitHub.

## Deployment environments

When a deployment comes in, Testsnag compares its environment name with the [environments](https://testsnag.com/docs/building-blocks/environments.md) enabled for the test, ignoring case. A deployment to "staging" runs the test in your Staging environment, with its base URL and variables. If no enabled environment has that name, the test runs with no environment.

## Running a suite on deployments

Suites can watch a repository too. In the GitHub card of a suite’s Notifications & Integrations settings, pick the repository and an optional environment filter. Each successful deployment runs the whole suite as one suite run, in its usual order and with its stop-on-failure setting. If the suite runs in environments, the one whose name matches the deployment is used.

When the suite run finishes, Testsnag posts one Check Run named "testsnag · <suite name>" with a row per test, following the suite’s "Post a Check Run" toggles. The same toggles are also the default for member tests that run on their own from a deployment and do not set their own.

## Blocking pull requests on preview deployments

Web and API tests, and whole suites, can run against the preview site your host builds for each pull request, and stop the pull request from merging when they fail. Testsnag reads the preview address from the deployment GitHub reports (its environment URL), so this works with hosts that post GitHub deployments for previews, such as Vercel.

1. **Turn it on for each test or suite**: Pick the repository, set the environment filter to the exact name your host uses for previews (for example "Preview"), tick "Test PR previews and block merging on failure", and save. The environment filter is required for previews.
2. **Import the ruleset into GitHub**: After saving, download the ruleset from the same card. In the repository on GitHub, open Settings, Rules, Rulesets, choose New ruleset, then Import a ruleset, select the file and click Create. You only need to do this once per repository and environment.
3. **Open a pull request**: Every successful preview deployment runs all ticked tests and suites for that repository and environment against the preview address. One check named "testsnag (Preview)" appears on the pull request, shows the progress, and turns green only when every one of them passed.

## How the preview check works

- There is one check per environment, shared by every ticked test and suite in the repository. Its name never changes when you rename, add or remove tests, so the ruleset keeps working.
- The check fails if anything fails, and also if something could not run at all: the monthly run limit was reached, your plan does not include GitHub, the host did not send a preview address, or a run was cancelled or timed out. A pull request never merges untested.
- The check lists every test and suite with its result and a link to the run. Use Re-run on the check in GitHub to run everything again against the same preview. Re-run uses the tests and suites that are ticked at that moment.
- Runs use the preview address as their base URL and take variables from the environment with the same name, if you have one.
- A check that stops making progress is failed automatically after two hours, so a pull request is never stuck on "in progress" forever.

## Limitations

- Only web and API tests run on previews. Mobile and SEO tests cannot be ticked, and inside a ticked suite they are skipped and shown as skipped, without failing the check.
- API tests only switch requests that use a relative path, such as /api/orders, to the preview address. Requests with a full URL keep going to that URL. Web tests switch the start page and any page on the same site.
- Your host must report GitHub deployments with an environment URL. Hosts that only post commit statuses, such as Netlify, are not supported for previews yet. Use a webhook trigger from your pipeline instead.
- Once the check is required, a pull request cannot merge until it arrives. If the preview deployment fails, is skipped, or nothing is ticked any more, the pull request stays blocked. Remove the ruleset or fix the deployment to unblock it.
- Pull requests from forks and GitHub’s merge queue usually get no preview deployment, so a required preview check blocks them. Exclude those flows in the ruleset if you use them.
- The imported ruleset protects the default branch only. Edit it in GitHub to cover other branches.
- Importing a ruleset needs admin access to the repository. On private repositories, GitHub rulesets need a paid GitHub plan.
- If several projects in one repository deploy previews under the same environment name, their checks share a name and hide each other. Give each project its own environment name, for example "Preview - web" and "Preview - docs".
- A test that is ticked on its own and also belongs to a ticked suite runs twice on every preview.
- Tests on previews still post their own "testsnag · <test name>" check if their Check Run toggles are on. Turn them off if you only want the single preview check.
- Only require the preview check in GitHub. The per-test and per-suite checks are named after the test or suite, so renaming one would leave GitHub waiting for a check that never arrives.

> **Warning:** Every triggered run counts against your monthly allowance, and preview checks run on every push to every open pull request. Ten ticked tests on a busy repository can use hundreds of runs a week, so tick the tests that matter most and add more once you know the volume.

## The alternative: trigger URLs

If you are not on GitHub, your host does not report deployments, or you want the run to happen at a precise point in your own pipeline, use a [webhook trigger](https://testsnag.com/docs/running-tests/webhook-triggers.md) instead. It is a single POST from any CI system and works the same way.