Repository navigation
[CLI] Make serpapi predictable for agents and scripts, and safe on terminals - #17
Merged
Merged
Conversation
The goreleaser job keeps its own contents: write to publish the release; the homebrew job pushes with the tap's deploy key and needs no GITHUB_TOKEN scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
release.yml took Go from go.mod's go line, 1.25.0, the minimum for go install users: v0.2.3's binaries carry 57 (darwin/arm64) and 58 (linux/amd64) Go standard library vulnerabilities by govulncheck -mode=binary. Release builds now use the latest stable Go; main built that way has none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A push or merge to main runs CI only; release.yml runs on v* tags, publishes 5 archives and checksums with GoReleaser, then pushes the formula to serpapi/homebrew-tap. AGENTS.md also covers the toolchain, the formula's test block, how to check a release and how to pull a bad one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gojq 0.12.19, pretty 1.2.2, x/term 0.45.0 and x/sys 0.47.0 are the newest releases that still build with go 1.25.0; x/term 0.46 and x/sys 0.48 require Go 1.26, which would force a toolchain upgrade on go install users. pflag goes to 1.0.10 with cobra. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Field and comment alignment in root.go and output.go, and two stray blank lines, so the gofmt gate in the next commit passes. No code changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test job runs gofmt and go test -race on two Go versions: 1.25, the minimum in go.mod for go install users, and the latest stable release, which builds the releases. The e2e job also uses the latest Go; it still skips fork pull requests, which get no SERPAPI_KEY. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A terminal acts on control characters in results: a page title holding ESC ] 52 can set the clipboard, ESC [ 2J clears the screen. On a terminal, JSON and --jq strings now go through ForTerminal, which writes C0 and C1 controls, DEL and bidi overrides as \u escapes. Piped output is unchanged and keeps the bytes. PrintJSON and PrintJQValue take the writer and whether it is a terminal; handleOutput passes stdout. PrintRaw and SameStream are for later commits (output=md, the large-result warning). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FprintError writes {"error":{"code","message","status"}} to any writer;
status is the HTTP status of the API's answer, 200 when a successful response
reported the error, and absent when there was none. It finds the message
through wrapped errors, keeps redacting api_key values, and escapes control
characters when stderr is a terminal. PrintWarning writes
{"warning":{"message"}} the same way, and InterruptedError is exit 130, code
interrupted. The API client sets the status in the next commit; warnings and
interrupts get their callers later.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A non-200 answer is an APIError with its status and SerpApi's message; an error inside a 200 body gets status 200. - Transport errors name the endpoint, not the request URL that carries the api_key. A timeout suggests twice the limit that ran out, at least 120 s, and says the search may still finish and be in the archive. - search, account and archive refuse a body that is not a JSON object, and locations one that is not an array, instead of printing it as a result. - SearchRaw and ArchiveRaw return output=md, html, csv and other formats as is, for a later commit; a JSON error body is still an error. - New takes WithBaseURL and WithTimeout options; NewWithTimeout stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Words without = are joined into one q, so serpapi search engine=google coffee shops searches for "coffee shops" instead of "shops"; mixing them with q=... is a usage error. Every parameter SerpApi reads a key from (api_key, apikey, key, serpapi_key, serp_api_key) is refused, so the key never lands in params or shell history. ApplyFields drops empty json_restrictor segments, which SerpApi answers with the whole response, and can append paths such as error that must survive the filter; Repeated lists parameters given twice. Both are used by the search command later in this stack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compile parses and compiles a filter, so a later commit can reject a typo as a usage error before a billed search runs. Run passes each result to a callback as gojq produces it and stops when the context is canceled; halt now stops output instead of being reported as an error. Apply is built on both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
os.Rename replaces the config file atomically on Windows too, so the old key is no longer deleted first and lost when the rename fails. A config.toml that isn't valid TOML, or can't be read, came out as "No API key found": account and archive now name the file and the line, and login rewrites the file. TOML parse errors quote part of the value, which may be the key, so only the line number is shown. The missing-key error says where a key can come from and where to get one. LoadAPIKey returns "" without an error when there is no file, so login and search keep treating a broken file as no key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release builds still set the version with -ldflags; go install builds now report the module version instead of dev, in --version and the User-Agent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Every command reports errors through run(): one JSON line on the command's own stderr, exit 2 for usage errors and 1 for API and network errors. serpapi without a command prints help to stderr and exits 2. - --timeout, --fields and --jq are checked before any request: --fields works only with search, a jq syntax error fails before a billed search, and --timeout takes seconds or a Go duration such as 2m, refusing NaN, negatives and values under a nanosecond. - The API key comes from --api-key, SERPAPI_KEY, SERPAPI_API_KEY (the name LangChain and the SerpApi MCP server use), then the config file, trimmed. A broken config file makes search warn and run without a key. - account and login refuse arguments without echoing them, since a stray argument is often the key; account masks the key by character, so an escaped or multi-byte key is never cut. - login takes --api-key, reads a hidden prompt on a terminal or the first line of stdin, keeps the saved key unless told to replace it, and warns when SERPAPI_KEY or SERPAPI_API_KEY overrides the saved key. - locations needs a query instead of fetching all 66 MB of locations. - Output goes to the command's stdout, so tests can run the CLI in-process, and --jq prints each result as gojq produces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmd_test.go runs commands in-process through run(), with a fake API from httptest, an isolated HOME and no key in the environment, and checks the exit code, stdout, every JSON line on stderr and the requests sent. edge_test.go is a table of 62 edge cases from stress testing, grouped by input: arguments, flags, API keys, the config file, server answers and output; fuzz_test.go fuzzes the key masking. pflag becomes a direct dependency for resetting flags between runs. Cases for features later in this stack are added with them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first Ctrl-C or SIGTERM cancels the command's context: an in-flight
request stops, --jq keeps what it printed, and the CLI reports
{"error":{"code":"interrupted",...}} and exits 130. A second signal kills
the process, and if the command is still stuck 3 s after the first, as when
stdout is a full pipe, the CLI exits 130 anyway. Before, a signal killed the
process with no message. --all-pages prints the pages fetched so far once
the paging rewrite later in this stack lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WriteObject writes a map as JSON indented by two spaces, byte for byte what json.Encoder with SetEscapeHTML(false) writes, but encodes array members one element at a time, so no encoded copy of the whole object is held in memory. It can report each member's encoded size. The next commit uses it to print --all-pages output; the tests fuzz it against encoding/json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- search with no parameters, --max-pages below 1, or --all-pages with
async=true is a usage error before any request.
- --fields keeps the error path, so an error inside the response still
shows, and a --fields that matches nothing returns {} with a warning.
- --all-pages carries json_restrictor, no_cache, zero_trace and output to
every page, since serpapi_pagination.next drops them. It stops at a page
--fields matches nothing on, at a repeated page and at --max-pages
(default 100, with a warning when more pages exist), and treats a 200
error such as "no results" after the first page as the end.
- When a later page fails or Ctrl-C stops the run, the pages fetched so far
are printed and the error names them.
- A pipe gets the merged pages written as they are encoded. 100 pages of
1 MB from a local fake API, median of 5 runs: peak RSS 722 -> 183 MB, and
669 -> 187 MB with --jq; wall time 0.63 -> 0.58 s. A terminal still gets
the buffered, colored form.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pipe, such as an agent reading stdout, now gets JSON indented by json.Indent: the bytes json.Encoder with SetIndent and WriteObject write, so a single search and merged --all-pages pages are formatted alike. tidwall/pretty, used for pipes until now, kept a short array on one line; the two agree on most responses. A body that isn't JSON comes back as is, and terminal output is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A flag guessed for a parameter, such as --num 10 or -e, gets "API parameters are key=value pairs, e.g. num=VALUE"; a search word starting with - is told to go inside q="..." or after --. - jq errors add the fix: ? for a value of another type, []? for a missing array, and the response's top-level keys for a path that isn't there. A filter that prints only null gets a warning listing those keys. - A piped result over 16 KB, which agent tools stop showing inline, comes with a warning that lists its 12 largest top-level keys by size and the free serpapi archive ID --jq to take parts of it, unless stderr is the same pipe (2>&1), where it would break a JSON parser. - A parameter given twice keeps its last value, with a warning; a 401 for a search sent without a key says how to give one. - output=md, html, raw_html, plain, csv and rank:DOMAIN print as is in search and archive. --jq or --all-pages with them, and unknown values such as output=xml, are usage errors before the search runs. - serpapi version prints what --version does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README covers search parameters and per-engine query keys, --fields and its cache key, output=md and the other formats, --all-pages limits and cost, async with archive polling, locations, login, version, key precedence, the config file, and what counts against the plan. "For AI Agents" states the stdout and stderr contract, the exit codes and the large result warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Offline tests run the compiled binary for usage errors, help and version, in an isolated HOME with no key in the environment. Tests that reach the real API, which can use searches, run only when SERPAPI_KEY is set; on main, the locations and invalid-key tests reached it without one. CI's e2e job now runs on fork pull requests too, where only the offline tests run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
login printed the account's email to the terminal as the API sent it: in a real PTY, an email holding ESC ] 52 (set the clipboard), ESC [ 2J and a bidi override reached the terminal in both "Logged in as" and "Already logged in as". The email now goes through ForTerminal, as search results do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AGENTS.md now tags the merge commit after its CI passed with no skipped live test, checks the tag points there, never moves a published tag, says how to publish written release notes, and pulls a bad version for go install users with retract. README's Quick Start links the API key page and the keyless demo, and the Go section says go install needs Go 1.25. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
serpapi with no command printed its 58-line usage to stderr before the JSON error, even when stderr wasn't a terminal: the one case of the 120 in the behavior matrix whose stderr wasn't JSON lines, which README promises scripts. The usage now shows only on a terminal, as login's messages do; off one, stderr is the error line alone, and it names --help. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6152d44 on main sentence-cased README's headings and dropped the tagline's blockquote, in sections this branch rewrote. The merge keeps this branch's text with both rules applied: 4 headings and the tagline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
ilyazub
added a commit
that referenced
this pull request
Oct 13, 2026
Since #17's signal handling, Ctrl-C at login's hidden key prompt waited the 3 s grace period, said "the output may be cut off", and left the terminal not echoing; the [y/N] prompt and a key piped through an open stdin waited too. In a pseudo-terminal, both prompts now exit 130 at once, the hidden one with echo restored, and nothing is saved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ilyazub
added a commit
that referenced
this pull request
Oct 13, 2026
v0.2.3 shipped on 2026-03-31; #7, #9, #15 and #17 merged after it and none reached users, because a release waited for someone to push a tag. Now, after both test legs and the live e2e pass on main, the tag job tags the commit with the next version and calls release.yml: a patch, or what the PR's release:minor or release:major label asks for. A release:skip label, or a merge where nothing that ships changed since the latest release, releases nothing. Outside pull requests a skipped live test fails CI, a published tag is never reused, a commit older than the latest release is never tagged, and runs on main go one at a time. A Release preview check tells each PR what merging releases. Dry run on this history: this branch with release:minor gives v0.3.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opus 5.5 Max wrote this:
Stop serpapi from echoing the API key or passing escape sequences from results to the terminal. Reject malformed commands before they reach the API, and keep fetched pages when Ctrl-C stops
--all-pages. Measured against a local fake SerpApi, not production. Resolves #10.On main, each of these can break an unattended run by a script or an AI agent.
output=mdalso fails with the error in #10, and releases build with go.mod's go1.25.0, which has known standard-library vulnerabilities.The fix separates errors from results and checks syntax and flag combinations before any request. Results go to stdout, escaped on a terminal and byte-exact in a pipe. Each error is one JSON line on stderr, and exit 2 means the command is wrong.
--all-pagesstreams its pages and keeps them on Ctrl-C. Releases build with the latest patched Go.Main against this PR: 120 commands on the fake SerpApi, plus terminal, memory and release-build probes.
--all-pagesoutput=md(#10)--jq, 0.78 → 0.54 s.--all-pagesstops at the first page--fieldsmatches nothing on. A misspelled--fieldspath over endless pages sent 100 requests on main and printed{}; here it sends 1. But results that start later are lost: on the fake SerpApi, page 5's 2 results became{}and a warning.output=md. Add --debug option #16's[debug]lines would fit the stderr contract as JSON lines when stderr isn't a terminal.To merge, accept the
--fieldstrade-off; the rest is measured above. Merging releases nothing: a maintainer tags v0.3.0 later, once CI passes with no skipped live test.What else scripts may notice
q=plus bare words; no parameters; the key askey,apikey,serpapi_keyorserp_api_key;--fieldsoutsidesearch; a--jqsyntax error,inputordebug;--max-pages 0;--all-pageswithasync=trueoroutput=md;--jqwithoutput=md; an unknownoutput=;locationswithout a query.q:engine=google coffee shopssendsq=coffee shops(main:q=shops).output=mdin any case,output=html,archive ID output=md,serpapi version,SERPAPI_API_KEY,login --api-key, jqhalt, and durations such as--timeout 2s(main: exit 1 or 2).--helplists exit codes 1, 2 and 130; README's "For AI agents" also lists 0. Error lines carry the HTTPstatuswhen there is one.--all-pagesaddsno_cache,zero_traceandoutputto every page; main sent only whatserpapi_pagination.nextheld. A 200 with an error on a later page is a warning, exit 0 (main: exit 1). When a later page fails, ornextisn't a URL, the pages so far print before the exit 1.api_error(main: exit 0). A paddedSERPAPI_KEYis trimmed.--fieldsaddserrortojson_restrictor;--fields ,,sends none (main sent,,and printed{}).serpapiwith no command prints its usage only on a terminal, on stderr (main: stdout); stderr always gets the JSON error.--jqthat fails on the response, such astonumberon a title, exits 2 after its search.Checks, reviews, release and rollback
Opus 5.5 Max checks on 982116a (macOS arm64, fake SerpApi on localhost, base main 6152d44):
Reviews: GPT-5.6 Sol Max (security and the plan); GPT-6 Astra Max and Fable 5.1 Max (earlier drafts of this description; their edits are applied).
After merge, a maintainer tags the merge commit as AGENTS.md says and checks the 5 archives, the
serpapi-cli 0.3.0commit in serpapi/homebrew-tap, andbrew test serpapi-cli. Kill criterion: a script-visible change not listed here. Rollback: revert the tap's formula commit, and release v0.3.1 withretract v0.3.0in go.mod. Review commit by commit; the bottom 3, up to 1bc0863 (release workflow and AGENTS.md), can go first as their own PR if a release is due sooner.