Scarnox · Blog

Keeping CI green when your app has a captcha

Your sign-up form has a PopularCaptcha. Your end-to-end suite has to sign up. Now CI is red, and the tempting fix, turning the captcha off in the test build, means the flow you test is not the flow your users get.

There are several ways out of this. They differ in how much of the real flow they exercise, so pick the one that matches the question your test answers. Here are the options, roughly from cheapest to most realistic.

Scarnox solves PopularCaptcha today. Other captcha types are coming soon, so the solving option below is for PopularCaptcha only.

1. Test keys for unit and local tests

Many captcha providers publish test keys that always pass. Use them when you control the environment and only need the form to render and submit. Test keys do not exercise real verification, and they do not help once you test a deployed environment that uses real keys.

2. A bypass for your own staging

Some teams add a secret header that the backend accepts in place of a captcha token. It is fast and free, but it is a code path that exists only for tests, and one misconfiguration away from existing in production. If you use one, gate it behind an environment check, make the secret long and random, and test that it is rejected in production.

3. Mock the verification call

Stub the server-side verification request in integration tests. This is good for checking your backend logic, such as what happens when verification fails. It tells you nothing about whether the front end sends the token correctly, so pair it with one of the other options.

4. Solve it for real in staging and smoke tests

When you need to prove that the real flow works end to end, with real keys, the real widget and real verification, fetch a token from a solving API inside the test. This is the only option that catches a broken widget integration.

A Playwright helper looks like this:

// Playwright test helper
export async function solveCaptcha(page) {
  const sitekey = await page.getAttribute("[data-sitekey]", "data-sitekey");
  const r = await fetch("https://api.scarnox.com/api/tasks/create", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.SCARNOX_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      captcha_type: "PopularCaptchaTaskProxyless",
      site_key: sitekey,
      site_url: page.url(),
    }),
  });
  const { errorId, solution, errorCode, errorDescription } = await r.json();
  if (errorId !== 0) throw new Error(`${errorCode}: ${errorDescription}`);

  await page.evaluate((t) => {
    document.querySelectorAll('[name="h-captcha-response"],[name="g-recaptcha-response"]')
      .forEach((el) => { el.value = t; });
  }, solution.token);
}

Call it right before the form submit in the test, not at the start of the test. Tokens expire within about two minutes, and a slow step earlier in the test can use up that window.

Keep solves to a minimum

A suite that signs up on every run can make a lot of solves over a week. Keep the number down:

For retry rules inside the helper, reuse the policy from error handling and backoff.

Keep the key out of the repo

Store the key as a CI secret, named something like SCARNOX_KEY, and never put it in the test code or the fixtures. If it leaks, regenerate it from the dashboard. The old key stops working at once, so a leaked key cannot be used to run solves for long.

Use a separate key for CI. If one pipeline misbehaves, you can rotate that key without touching anything else.

Try it on your own staging

Create an account and start with free trial credit, no card needed. Wire the helper into one smoke test first, confirm it signs up cleanly, and only then add it to the rest of the suite. For what solving costs at volume, see current pricing.

What to assert after the submit

A solved captcha only proves the widget step worked. After the submit, assert the things that matter to the user: the next page loads, the success message appears, and the account or order exists in your database. A test that stops at "no error on the form" can pass while the signup silently fails on the backend.

Keep the assertions close to the business outcome. Check the URL, the visible message and one record the test created. Clean up that record at the end of the run so the next run starts from a known state.

Running the smoke test on a schedule

Run the real-solve smoke test on a schedule, for example once a day against staging, rather than on every pull request. Scheduled runs catch changes the site made overnight, such as a new widget version or a moved sitekey, without spending solves on every commit. Alert the team when the scheduled run fails, and keep the per-commit suite on the faster options.

Next steps