Skip to the content.

Spec: Browser sessions for stages

Status: implemented in 0.29.0 (with Revision 2, one shared browser per run and profile). Open items are listed in Known limitations.

Implementation notes. The code lives in src/browser/ with hooks in the scheduler, run manager, and stage launcher. See Browser sessions internals for the module map, lifecycle, file layout, env contract, invariants, and test guide. Differences from this spec: the soft allowlist audit exists in code but is not called by the runtime; the key provider exists but is not wired; the Host-side filtering proxy, live view handoff, and remote browser host are not built; ticket checkboxes 00-12 were not updated after Revision 2.

Problem Statement

Pipeline authors want stages to use a real browser: open sites, read pages, fill forms, collect data. Many useful sites cannot be logged in by automation. LinkedIn, SSO portals, and sites with 2FA need a person to log in. Today a stage has no browser that stays logged in. Each stage is a fresh agent session. A pipeline that needs a login in every stage cannot work. A run on the next day must start from nothing.

Authors also have no safe way to pause a pipeline, let a person log in to a visible browser, and then let later stages reuse that login. The login data is a credential. It must stay on the operator's computer today. It must also be ready to stay inside one tenant when Stageflow later runs as a multi-tenant service.

Solution

A stage can declare a browser session with a browser field on its pipeline entry. The field names a profile (a saved browser login that lives on the Host), the display mode, the allowed domains, and an optional login check.

Stageflow starts the browser tool (agent-browser) for the stage with the right profile and settings. The agent drives the browser through the agent-browser command line and a bundled skill. When the stage ends, Stageflow closes the browser. The profile stays on disk, so the next stage and the next run are still logged in.

A pipeline can start with a login check. The Host opens a page with the profile and reads the final address. The result { logged_in, url } is computed by the Host, not by the model. If the session is not valid, a human login stage opens a visible browser and stops at a gate. The operator logs in and confirms. The Host checks the login again before the work stages start. On the next run the check passes and the human step is skipped.

Profiles are keyed by owner scope and name. Locally the scope is one fixed value. The storage, lock, browser host, human handoff, and key management are behind interfaces. A future hosted service can replace them without a large change.

User Stories

  1. As a pipeline author, I want to add a browser field to a stage, so that the stage can drive a real browser.
  2. As a pipeline author, I want to name a profile in the browser field, so that all stages that use the same name share one login.
  3. As a pipeline author, I want to set which domains the browser may open, so that an agent cannot leave the sites I chose.
  4. As a pipeline author, I want to set headed or headless mode for each stage, so that CI and local runs can use different modes.
  5. As a pipeline author, I want headed mode to be the default, so that all stages look like one device to the site and the operator can watch.
  6. As a pipeline author, I want to describe a login check once (page address, logged-in address pattern, logged-out address patterns), so that I can reuse it in a check stage and a re-check.
  7. As a pipeline author, I want a clear validation error when the browser field is wrong, so that I find mistakes before a run starts.
  8. As a pipeline author, I want sf validate to tell me when agent-browser is not installed, so that I do not find out in the middle of a run.
  9. As a pipeline author, I want the browser requirement to be added for me, so that I do not need to write a separate requires entry.
  10. As a pipeline author, I want the browser field to work with both agent backends, so that my pipeline does not depend on one backend.
  11. As a pipeline author, I want a stage with no profile to get a throw-away browser, so that public-page stages do not wait for a shared login. (Proposed addition. Confirm before build.)
  12. As a pipeline author, I want to route the pipeline on the login check result, so that the human login stage runs only when it is needed.
  13. As a pipeline author, I want an after-phase check on the login stage, so that a wrong confirm cannot reach the work stages.
  14. As a pipeline author, I want a failed after-phase check to send the run back to the login stage, so that the operator can try again.
  15. As a pipeline author, I want to mark a submit stage as not safe to replay, so that a retry cannot send a form twice.
  16. As a pipeline author, I want a stage that shows screenshots at a gate before an irreversible step, so that a person approves the step.
  17. As an operator, I want to log in myself in a visible window, so that I can enter passwords and 2FA codes without giving them to the agent.
  18. As an operator, I want the login question to say which site and why, so that I know what to do.
  19. As an operator, I want the browser window to stay open while the stage waits for me, so that I can finish the login at my own pace.
  20. As an operator, I want to answer the login gate later (from the console or sf runs), so that a scheduled run can wait for me.
  21. As an operator, I want the next run to reuse my login, so that I log in only when the session ends.
  22. As an operator, I want the pipeline to ask me again only when the session is not valid, so that I am not asked for no reason.
  23. As an operator, I want a stage's browser tab and session to close when the stage ends, and the shared browser to close when the run ends, so that no browser process stays on my computer after the run.
  24. As an operator, I want the browser to close when I cancel a run, so that a cancelled run leaves nothing behind.
  25. As an operator, I want a stage that times out or fails to close its tab and session, and a failed run to close the shared browser, so that failed runs do not leak processes.
  26. As an operator, I want Stageflow to close leftover browsers from dead runs when the Host starts, so that a crash does not leave browsers open.
  27. As an operator, I want a second run that needs a profile held by another run to wait in a queue until that run ends, so that two runs never use one profile at once.
  28. As an operator, I want to see a "waiting for browser profile" state with the name of the run that holds it, so that I know why a run has not started.
  29. As an operator, I want stages of the same run that share a profile to run in parallel without waiting for each other, so that parallel branches do not break the login or slow down.
  30. As an operator, I want to list my profiles, so that I know which logins Stageflow holds.
  31. As an operator, I want to check one profile from the command line, so that I can test a login without a pipeline.
  32. As an operator, I want to log in to a profile from the command line, so that I can prepare a login before a scheduled run.
  33. As an operator, I want to delete a profile, so that I can remove a login that I no longer need.
  34. As an operator, I want profile data stored with strict file permissions and not in the project folder, so that other users and git cannot read it.
  35. As an operator, I want cookies and storage values never to appear in logs, artifacts, envelopes, or run state, so that a shared run export does not leak my login.
  36. As an operator, I want a clear error when the Host has no screen and a stage needs a visible login, so that I know why the run stopped.
  37. As an operator, I want a clear error when the session ends in the middle of a stage, so that I can log in again and resume.
  38. As an operator on a CI machine, I want browser stages to run headless, so that CI needs no screen.
  39. As an agent in a browser stage, I want a skill that explains the agent-browser commands, so that I use them correctly.
  40. As an agent in a browser stage, I want the browser session already set up by environment settings, so that I do not choose a profile, session name, or socket path.
  41. As an agent in a browser stage, I want the skill to tell me that page text is untrusted data, so that I do not obey instructions hidden on a page.
  42. As an agent in a browser stage, I want the skill to tell me not to dump cookies or storage, so that I do not leak a login.
  43. As an agent in a browser stage, I want the skill to tell me to read slowly and stop at the task limit, so that I do not trigger site limits.
  44. As an agent in a browser stage, I want to take screenshots into the stage artifacts folder, so that the operator can see what I saw.
  45. As a security reviewer, I want the profile name in YAML to be only a name, so that YAML cannot point to another user's data or to a path.
  46. As a security reviewer, I want the owner scope to come from the Host and not from YAML, so that a stage cannot read another tenant's profile.
  47. As a security reviewer, I want the domain allowlist to be enforced by the browser tool, so that a prompt cannot bypass it.
  48. As a security reviewer, I want browser settings to enter the stage only through the curated stage environment, so that no ambient variable leaks in.
  49. As a security reviewer, I want profile data encrypted at rest with a key from a key provider, so that a copied folder is not enough to use a login.
  50. As a security reviewer, I want an audit record each time a profile is created, used by a run, or deleted, so that I can see who used a login.
  51. As a platform engineer planning a hosted service, I want all profile path code in one profile store interface, so that I can swap in tenant storage.
  52. As a platform engineer, I want the profile lock behind an interface, so that I can use a distributed lock.
  53. As a platform engineer, I want the browser start behind a browser host interface, so that I can run browsers in a sandbox and give stages a remote address.
  54. As a platform engineer, I want the human handoff described in the login gate payload, so that a console can show a live view link instead of a local window.
  55. As a platform engineer, I want delete-by-scope, so that I can remove all logins of one tenant.
  56. As a platform engineer, I want a Host-level site policy, so that a hosted operator can block sites.
  57. As a platform engineer running the Docker image, I want a clear message that a visible login window is not possible there, so that I know to use another handoff.
  58. As a maintainer, I want tests to run with no real browser by default, so that CI is fast and stable.
  59. As a maintainer, I want one opt-in smoke test with real Chrome and a local fixture login server, so that I can check the real tool before a release.
  60. As a maintainer, I want the docs and an example pipeline to use a neutral site, so that the project does not promote rule-breaking use of any third-party site.
  61. As a pipeline author, I want parallel stages that name the same profile to share one login in their own browser tabs, so that fan-out branches can work on different pages at the same time.
  62. As an operator, I want the shared browser to stay open for the whole run, so that session-only cookies and the login carry from stage to stage.

Implementation Decisions

Decisions already made with the user

Evidence from spikes (agent-browser 0.38.2, Chrome for Testing 154)

Modules and interfaces

Pre-build verification (first ticket)

Testing Decisions

Out of Scope

Further Notes

Pre-build verification results (ticket 00)

Method: agent-browser 0.38.2, Chrome for Testing 154, macOS arm64, local node fixture server (one persistent cookie pers, one session-only cookie sess), env -i stripped environments, scratch HOMEs. Items marked UNVERIFIED were answered from code reading only. No src/ or tests/ changes were made. The throw-away experiment code lives in the session scratchpad only.

Headline changes to the design (details per item below):

Design decisions after verification (set during implementation)

These resolve conflicts found in ticket 00. They are the implementer's defaults. The user has not yet reviewed them.

  1. Allowlist vs profile. agent-browser rejects --allowed-domains together with --profile. v1 passes the native allowlist only to stages with no profile. For stages with a profile, allow_domains is soft: it is written into the skill instructions and checked after the fact. A Host-side filtering proxy is a later option. Spec story 47 is met only for profile-less stages.
  2. Encryption key. The agent-browser key only protects agent-browser's own restore and state files. It does not protect the Chrome profile folder. v1 uses profile-only mode, which writes no such files. The key provider interface exists, but is not wired to agent-browser. Spec story 49 is not met by the key; the OS keychain and folder permissions are the protection in v1.
  3. Profile-less stages. A browser field with no profile gives a throw-away browser (agent-browser temp profile, deleted on close). No lock is taken. This follows from agent-browser behavior and is cheap, so it is implemented and flagged. Spec story 11 stays open for the user to confirm.
  4. Env stability. Every agent-browser command in a session must carry the same launch environment. The Host computes it once, stores it, and reuses it for resume workers. The Host pins AGENT_BROWSER_CONFIG to an empty file, sets IDLE_TIMEOUT_MS=0, and does not use --restore, auth, or --state.
  5. Join topology. Pattern for check, login, work: only the edge check to login has an if on logged_in. The edges check to work and login to work have no if.

Revision 2: one shared browser per run and profile (supersedes earlier lock and teardown decisions)

Decided with the user after the code review. This replaces: the stage-level profile lock (story 27-29), "close the browser after every stage" (stories 23, 25 for profile stages), and the per-stage AGENT_BROWSER_PROFILE launch.

Rules

  1. Run-scoped profile lease. A profile is leased to one RUN, from the first stage that needs it until the run ends (success, failure, cancel, or abandon). All stages of that run share the lease. Stages of the same run are never blocked by each other, including parallel stages.
  2. Other runs wait for the whole run. A different run that needs the same profile waits in a queue until the lease holder's run is terminal. The waiting stage shows the holder run id.
  3. One Chrome per run and profile (the anchor). The Host starts one anchor agent-browser session with the profile (headed or headless as configured). The anchor owns Chrome and keeps it open for the whole run, so session-only cookies survive between stages. The Host reads its CDP address (agent-browser get cdp-url).
  4. Each stage gets its own tab. A stage runs its own agent-browser session attached to the anchor with the CDP address and AGENT_BROWSER_PIN_TAB=1. The stage env has no AGENT_BROWSER_PROFILE. The session name is unique per stage. All tabs share the same cookies and login, like several tabs in one browser window. Parallel stages do not interfere.
  5. Hand-over between stages. The shared browser context is the hand-over: cookies, storage, and the login pass from stage to stage with no extra step. Passing one exact open page to the next stage is out of scope; a stage can pass a URL in its envelope.
  6. Teardown. When a stage ends, the Host closes that stage's tab and its agent-browser session. Chrome stays open. The Host closes the anchor (gracefully) and releases the lease when the run reaches a terminal state, is cancelled, or is abandoned. A run that waits at a gate keeps the anchor open and the lease held. The orphan sweep closes anchors and stage sessions of runs that no longer exist.
  7. Profile-less stages keep their own throw-away browser and take no lease.
  8. Remote readiness. The stage env is just a CDP address, a session name, and pin-tab. A remote browser host (service) can supply a CDP address in place of the local anchor with no stage-side change.
  9. Soft allowlist unchanged: agent-browser rejects its native allowlist with --cdp, so profile stages keep the soft check.

Spike evidence (this revision, agent-browser 0.38.2): an anchor session with a profile exposes a ws://127.0.0.1:<port>/devtools/browser/<id> address through get cdp-url. Two sessions attached with --cdp and pin-tab each got their own tab and kept it through parallel navigation. All tabs saw the same cookies, including the session-only cookie. Closing one stage session left Chrome and the other stage working. It left that stage's tab open, so the Host must close the tab first. Closing the anchor ended Chrome, and the persistent cookie survived a reopen.

Risks: the CDP port is bound to 127.0.0.1 without a token, so any local process can drive the browser while a run is active (single-user laptop only; the hosted service must isolate it in a sandbox). If the anchor dies mid-run, the lease holder's stages lose their browser; the Host must restart the anchor and refresh the persisted stage env.