Tests

Running tests locally with the CLI tool

Install the Guardian CLI, sign in with guardian login, run a test plan against a server on your own machine or network, and upload the result to your test history.

The Guardian CLI (guardian) runs a test plan from your own computer or a CI job and, while you're signed in, uploads the result to your account. You can turn uploading off at any time. Use it when Guardian's cloud can't reach the server you want to test, such as a server behind your firewall, on a private network or in a sandbox. To test a public server from your account instead, see Run your first test.

Quick start

Download the CLI for your system from the Download page, verify it as shown in Install the CLI, put guardian on your PATH and check it runs:

guardian version

Sign in, then copy the code the terminal prints, paste it into the Approve CLI login page that opens in your browser, and select Approve:

guardian login

Save the example plan from Write a plan as plan.json, changing tool_name and arguments to a tool your server has.

Run it against a local server you own, and type y at the prompt:

guardian test --plan plan.json --target-url http://127.0.0.1:8000/mcp

The result is printed and uploaded to Test history. Add --no-upload to keep it on your machine.

Before you start

  • Only test servers you own or have permission to test. A test makes real tool calls. If a tool changes data, sends messages or makes payments, that happens.
  • You need a Guardian account with the Admin, Manager or Engineer role to approve a login or create a token. Viewers and Compliance users can't.
  • Results are self-reported. Guardian doesn't run the tests itself, so an uploaded run is labelled as self-reported and its results can't be verified. See CLI tokens for what that means.

Install the CLI

Open the Download page. It shows the latest version and the release's commit ID, and has these commands filled in for Linux, macOS and Windows. The files are public at https://get.guardiansamur.ai/cli/vX.Y.Z/. You need three: the archive for your system, checksums.txt and checksums.txt.sigstore.json. For example, on macOS (Linux is the same with its own archive name; on Windows use curl.exe in PowerShell):

V=X.Y.Z
base="https://get.guardiansamur.ai/cli/v$V"
for f in "guardian_${V}_darwin_all.tar.gz" checksums.txt checksums.txt.sigstore.json; do
  curl --proto '=https' --tlsv1.2 -fsSLO "$base/$f"
done

The version on the Download page comes from an unsigned file, so treat it as a hint. The check below is what proves the files are genuine.

Verify the download before you run it for the first time. This is required, not optional. Put the three files in an empty directory and check both the signature on checksums.txt and your archive's checksum. With cosign v3.1.3 (v2.6.0 also works; older versions don't), where <sha> is the commit ID shown on the Download page and V is the version without the v:

V=X.Y.Z
cosign verify-blob \
  --certificate-identity "https://github.com/Guardian-Samurai/guardian-client/.github/workflows/release.yml@refs/tags/v$V" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-github-workflow-trigger push \
  --certificate-github-workflow-repository Guardian-Samurai/guardian-client \
  --certificate-github-workflow-ref "refs/tags/v$V" \
  --certificate-github-workflow-sha <sha> \
  --bundle checksums.txt.sigstore.json \
  checksums.txt \
  && f="guardian_${V}_linux_amd64.tar.gz" \
  && grep -x "[0-9a-f]\{64\}  $f" checksums.txt > "$f.sha256" \
  && sha256sum -c "$f.sha256" \
  && echo "VERIFIED: safe to run"

On macOS use shasum -a 256 -c "$f.sha256" in place of sha256sum -c "$f.sha256", and set f to the archive you downloaded. The steps are chained with &&, so a failure stops the rest. Only the output VERIFIED: safe to run means the program is safe to run. If you see anything else, don't run it. The Download page and the CLI's README have the same steps.

The macOS and Windows programs aren't code-signed yet, so download them on the command line with curl, as above, rather than in a browser. Once the checks pass, unpack the archive, put the guardian program on your PATH and check it works:

guardian version

Sign in with guardian login

guardian login

The CLI prints a short code, such as BCDF-GHJK, and the address of the Approve CLI login page. It opens that page in your browser when it can. Then:

  1. Sign in to Guardian if asked.
  2. Type the code from your terminal into the page, check the details, and select Approve.
  3. The terminal confirms who you are signed in as.

Only type a code you got from guardian login yourself. Guardian never asks you for a code any other way. The page and the checks are described in CLI tokens.

Approving creates a token that lasts 90 days and can run tests and upload results. The CLI keeps it in your operating system's credential store (the keychain on macOS, Credential Manager on Windows, the Secret Service on Linux) and never prints it.

The computer name. Before it asks for approval, the CLI tells you that this computer's name will be saved with the new token. It sends only the part before the first dot, and only when it is made of letters, digits, hyphens and underscores. It appears in Guardian only after the token exists: next to the token in your CLI tokens list and on the runs it uploads, always labelled as reported by the CLI, because Guardian can't verify it. It never appears on the approval page. To leave it out, run:

guardian login --no-hostname

Other options:

  • --no-browser doesn't open a browser. Use it over SSH or on a machine with no screen: open the page yourself, on any device, and type the code there.
  • --with-token reads a token you created by hand from your CLI tokens page, from standard input, instead of using the browser. It can't be combined with --no-browser or --no-hostname.

If you have no usable credential store, such as in a container, the CLI says so and points you to GUARDIAN_TOKEN (see Use it in CI).

Check who you are

guardian whoami

This prints the Guardian address, where the token came from, your email, organization, tenant and role, the token's label, what it can do and when it expires. It never prints the token.

Write a plan

A plan is a JSON file that lists the calls to make. This one makes a single call to a tool named echo and expects it to succeed:

{
  "schema_version": 1,
  "plan_id": "my-first-plan",
  "target": { "url": "http://127.0.0.1:8000/mcp", "headers": {} },
  "cases": [
    {
      "case_id": "case-1",
      "rule_id": "mcp-test.functional.happy-path",
      "custom_case_id": null,
      "tool_name": "echo",
      "locator": null,
      "category": "functional",
      "action": { "type": "mcp.tools_call", "arguments": { "text": "hi" } },
      "repeat": 1,
      "timeout_ms": 5000,
      "expect": [ { "type": "call_succeeds" } ]
    }
  ]
}
  • Each case names a tool_name, the arguments to send, how many times to repeat the call, a timeout_ms, and what to expect: call_succeeds, or call_fails for a call the server should reject.
  • To upload a run, give each case one of Guardian's test rules in rule_id (this example's mcp-test.functional.happy-path is one), and set category to the rule's middle part (functional, negative or performance). A negative case must expect call_fails.
  • Mark a case "side_effects": true if its tool changes things. The CLI skips it unless you allow it (below).
  • The plan can't hold secrets. Put the target in --target-url and any headers on the command line.

The CLI checks the plan before it connects and exits with code 3 if it is malformed.

Run the plan

guardian test --plan plan.json --target-url http://127.0.0.1:8000/mcp

The CLI connects, lists the server's tools, and then shows a preview: the target, how many calls it will make, and which cases it will skip. It asks run N calls to HOST:PORT? [y/N]. Type y to go ahead; anything else stops the run before any call is made.

The result is printed to your terminal as one line of JSON. Add --out result.json to write it to a file instead. Messages about the run go to the error output, so they don't mix with the result.

Choose the target

  • Loopback is allowed by default. localhost, 127.0.0.1 and [::1] work without a flag.

  • Any other host needs --allow-target. Name the exact host or IP address, with no port, wildcard or https://:

    guardian test --plan plan.json --target-url https://mcp.internal.example/mcp --allow-target mcp.internal.example
    

    Private network addresses are fine once you allow them. A host you haven't allowed makes the CLI exit with code 4 before it connects.

  • --target-url replaces the plan's target.url. If you leave it out, the plan's address is used.

Skip the prompt with --yes

A run without --yes must be interactive, because you confirm it at the prompt. In a script, a CI job or any other place without a terminal, pass --yes; the preview then goes to the error output. --yes only skips the question: it never allows a target or a side-effect tool.

Send a secret header

If the server needs a header, such as an API key, read it from an environment variable. Set the variable without typing the secret into a command that your shell records in its history:

# bash or zsh: type the value; nothing is shown or saved to history
read -rs MCP_API_KEY && export MCP_API_KEY

# PowerShell 7.1 or later
$env:MCP_API_KEY = Read-Host -MaskInput

# Windows PowerShell 5.1
$env:MCP_API_KEY = [Runtime.InteropServices.Marshal]::PtrToStringBSTR([Runtime.InteropServices.Marshal]::SecureStringToBSTR((Read-Host -AsSecureString)))

In CI, use the job's secret instead. Then run:

guardian test --plan plan.json --target-url http://127.0.0.1:8000/mcp --header-env "Authorization=MCP_API_KEY"

The value of the variable named after the = becomes the header's value. Don't use --header "Name: value" for secrets: the value sits on the command line, where other users on the machine can see it in the process list and your shell history. guardian test never prints or uploads header values.

Allow tools with side effects

By default the CLI skips cases marked side_effects: true, and cases on tools it can't tell are read-only. To call a tool anyway, name it exactly:

guardian test --plan plan.json --target-url http://127.0.0.1:8000/mcp --allow-side-effects create_ticket

You can repeat the flag. Each name must match exactly one tool the server lists, or the run stops. Only allow a tool when you are sure what it does, because the CLI trusts the server's name for it.

Stop a long run

--deadline 90s (or 5m) limits how long the run lasts, from 1 second to 60 minutes; the default is 10 minutes. Press Ctrl-C to stop a run: no result is written and the exit code is 130.

Test a stdio server

A stdio MCP server is a program Guardian's cloud can't reach, so you run it on your own machine. Give the CLI the command to start it instead of a URL:

guardian test --plan plan.json --stdio "npx @modelcontextprotocol/server-everything"

The CLI starts the program, runs the plan against it and, while you're signed in, uploads the result like any other local run. The run shows a small stdio tag next to the server name in Test history.

What is uploaded. A masked server name made from the command, such as npx @modelcontextprotocol/server-everything, and hashes that identify the command so runs of the same one are grouped. The command line, environment variables and working folder are never uploaded. Use --server-name to choose the name that appears instead:

guardian test --plan plan.json --stdio "npx @modelcontextprotocol/server-everything" --server-name "Everything (local)"

Keep secrets off the command line. A secret in the command sits in your shell history and in the process list. If the server needs one, set it as an environment variable and pass it through by name with --pass-env:

guardian test --plan plan.json --stdio "my-server --stdio" --pass-env MY_API_KEY

How runs are grouped. Stdio entries are per person: your runs of a command are grouped together, and a teammate's runs of the same command form their own entry. The same command run from two different folders is grouped together. As with every run, members see their own runs, and Admins, Managers and Compliance see everyone's.

In pull-request CI jobs, use --no-upload, so a job that runs code from a pull request doesn't send results to your account.

Upload the run to Guardian

While you're signed in (or GUARDIAN_TOKEN is set), every guardian test run uploads its result. To stop for one run, add --no-upload; to stop for your user account on this computer, run guardian config set upload false (guardian config set upload true turns it back on). Add --upload when the run must fail if the upload fails, for example in CI. Give the server a name with --server-name:

guardian test --plan plan.json --target-url http://127.0.0.1:8000/mcp --server-name "Staging MCP"

The preview shows where the result will go. After the run, the CLI uploads it and prints the address of the run. What is uploaded is the result and a summary of the plan: the target's address without its path or query, the tools, the outcomes and the timings. Arguments, headers and file paths are never uploaded.

If you aren't signed in, or Guardian can't be reached, the CLI says the run wasn't uploaded and the exit code is the run's own.

The run appears in Test history with a Local badge. Opening it shows:

  • a line such as "Local run: tests executed from MY-LAPTOP on Oct 16, 2026, 10:00 AM", with the computer name that reported it unless you used --no-hostname (then it reads "Local run: tests executed on …");
  • the server by the name you gave, with a short tag from its address so two servers with the same name can be told apart.

Uploaded runs are counted apart from the runs Guardian does itself. Your dashboard's main tiles show Cloud runs only, with the self-reported runs in a separate group beneath them.

Exit codes

| Code | Meaning | |---|---| | 0 | At least one case passed, and none failed or errored. | | 1 | A case failed. | | 2 | A case errored, or no case ran (all were skipped). The result is still written. | | 3 | A problem with the command, config or plan; a run that isn't interactive without --yes, or that you didn't confirm; an --allow-side-effects name the server didn't list exactly once; or, with --upload, no token. | | 4 | The target isn't allowed. Nothing was called. | | 5 | With --upload only: Guardian refused or failed, before or after the run. It replaces 0, 1 and 2. Without --upload an upload problem never changes the exit code. | | 130 | You interrupted the run. |

Use it in CI

A CI job has no browser, so it can't use guardian login. Instead:

  1. Create a token on the CLI tokens page (select your name in the top-right corner, then Account settings, then Tokens). Choose Run tests and upload results.
  2. Save the token in your CI system's secret store, and expose it to the job as GUARDIAN_TOKEN. Never put it in the repository or a shell profile.
  3. Run guardian test with --yes. Add --upload if the job must fail when the result can't be uploaded, and --no-upload in jobs that shouldn't upload, such as pull-request checks. Give GUARDIAN_TOKEN only to the jobs that should upload. A config file baked into a CI image applies only to the user (and home directory) the job runs as, so use --no-upload or leave GUARDIAN_TOKEN out of jobs that shouldn't upload.
guardian test --plan plan.json --target-url http://127.0.0.1:8000/mcp --yes --upload --server-name "Staging MCP"

When GUARDIAN_TOKEN is set, the CLI uses it and ignores any stored token. It never saves it, and guardian login and guardian logout stop with an error while it is set. The token acts with your role, so see CLI tokens for how long it lasts and what revokes it.

Runs uploaded by guardian test carry the Local badge today, whether they ran on a laptop or in a CI job.

Sign out

guardian logout

This revokes the token Guardian issued to this machine and removes it from the credential store. If Guardian can't be reached, the token is still removed from this computer, but it stays valid until it expires or you revoke it on the CLI tokens page, and the CLI exits with code 5 to tell you so.