Document 0.1.0 changes, releasing, and the docs site

AGENTS.md

@@ -9,50 +9,56 @@ This repo contains `envsitter`: a TypeScript (ESM) library + CLI for safely insp
 - Prefer comparisons via deterministic fingerprints (HMAC) and boolean match results.
 - Prefer passing candidate secrets via stdin (to avoid shell history): `--candidate-stdin`.
 - Keep all generated peppers/secrets out of git (see `.gitignore`).
+- `envsitter-guard` (separate repo) exposes this package's exports to AI agents as tools, so every new export is agent-reachable.
 
 ## Project layout
 
-- `src/envsitter.ts`: main library surface (`EnvSitter`)
-- `src/cli.ts`: CLI entrypoint
-- `src/dotenv/parse.ts`: minimal dotenv parser
-- `src/pepper.ts`: pepper resolution + auto-create
-- `src/test/*.test.ts`: Node test suite (compiled to `dist/test/*.test.js`)
+- `src/index.ts`: public exports (the only surface npm users and `envsitter-guard` see)
+- `src/envsitter.ts`: `EnvSitter` read-only facade (keys, fingerprints, matchers, scan, redactor)
+- `src/cli.ts`: CLI entrypoint (`bin` -> `dist/cli.js`); one `if (cmd === ...)` block per command in `run()`
+- `src/file-ops.ts`: file-level mutations for the library API (read, edit, atomic write only when `write: true`)
+- `src/pepper.ts`: pepper resolution + auto-create; `src/fingerprint.ts`: HMAC-SHA-256
+- `src/sources/`: `DotenvFileSource`, `ExternalCommandSource` (both parse with `dotenv/parse.ts`)
+- `src/dotenv/document.ts`: the only dotenv grammar; round-trippable, keeps comments, blank lines, and line/column issues
+- `src/dotenv/parse.ts`: values-only view of `document.ts` (read path)
+- `src/dotenv/edit.ts`: pure string-in/string-out edits (copy, format, annotate, add/set/unset/delete); owns the edit types that `file-ops.ts` re-exports
+- `src/dotenv/io.ts`: `readTextFileOrEmpty`, `writeTextFileAtomic` (keeps file mode, writes through symlinks, new files `0600`); `src/dotenv/utils.ts`: value quoting
+- `src/paths.ts`: path policy (`isEnvFile`, `isExampleEnvFile`, `isSensitiveEnvFile`, `isEnvSitterPepperFile`)
+- `src/regex.ts`: `regexTest`; use it for every user-supplied regex (resets `lastIndex` for `g`/`y` flags)
+- `src/redact.ts`: `createRedactor` behind `EnvSitter#redactor()` and the `redact` command; decides which values look secret (key names, token shape, URL passwords, PEM)
+- `src/test/*.test.ts`: Node test suite (compiled to `dist/test/*.test.js`); `cli.test.ts` spawns `dist/cli.js`
+- `.github/workflows/`: `ci.yml` (push, PR, reusable), `release.yml` (tag `vX.Y.Z` -> CI -> npm trusted publishing -> GitHub Release), `pages.yml` (deploys `site/`); `.github/dependabot.yml` groups weekly npm and Actions updates
+- `.github/scripts/changelog-section.mjs`: prints a version's CHANGELOG section; fails if missing or still `(Unreleased)`
+- `site/`: static docs site for envsitter and envsitter-guard (no build step); `site/DESIGN.md` is its design contract and is not published
+
+### Gotchas
+
+- Grammar changes go in `dotenv/document.ts` only; `parseDotenv`, `validate`, and every edit follow automatically.
+- CLI file commands are thin wrappers over `file-ops.ts`; change behavior there, then keep the CLI's text (non-JSON) output in step.
+- `src/paths.ts` is the policy `envsitter-guard` enforces. Widening or narrowing `isSensitiveEnvFile` changes what agents are blocked from; add a case to `paths.test.ts` for every change.
+- `src/test/envsitter.test.ts` resolves the default pepper (`<cwd>/.envsitter/pepper`) unless `ENVSITTER_PEPPER` is set, so `npm test` creates that file on first run.
 
 ## Build / typecheck / test
 
-There is no linter configured (no ESLint/Prettier). Use `tsc` + tests as quality gates.
+There is no linter configured (no ESLint/Prettier). Use `tsc` + tests as quality gates. Husky pre-commit runs `npm test`.
 
-### Install
+```bash
+npm install
+npm run build       # tsc -p tsconfig.json -> dist/
+npm run typecheck   # tsc --noEmit
+npm test            # build, then scripts/run-tests.mjs runs every dist/test/*.test.js
+# tests execute from compiled JS, so build before running one file or one test
+npm run build && node --test dist/test/envsitter.test.js
+npm run build && node --test --test-name-pattern "outside-in" dist/test/envsitter.test.js
+```
 
-- `npm install`
+`--test-name-pattern` takes a substring/regex matching the `test('name', ...)` title.
 
-### Build
+### Releasing
 
-- `npm run build`
-  - Runs `tsc -p tsconfig.json` and outputs to `dist/`
-
-### Typecheck only
-
-- `npm run typecheck`
-
-### Run all tests
-
-- `npm test`
-  - Runs `npm run build` then `node scripts/run-tests.mjs`
-
-### Run a single test file
-
-Tests are written in TS but executed from compiled JS.
-
-- `npm run build`
-- `node --test dist/test/envsitter.test.js`
-
-### Run a single test by name/pattern
-
-- `npm run build`
-- `node --test --test-name-pattern "outside-in" dist/test/envsitter.test.js`
-
-(Use a substring/regex that matches the `test('name', ...)` title.)
+- Bump `package.json`, date the CHANGELOG heading, commit, push tag `vX.Y.Z`. The Release workflow does the rest; never `npm publish` by hand.
+- Push to `origin` (Gitea, `git.boxpositron.dev`); it mirrors to GitHub, where the workflows run. Jobs are guarded with `if: github.server_url == 'https://github.com'` so Gitea's runner skips them.
+- npm trusted publishing (OIDC) authenticates the publish; no npm token exists in the repo. Keep `release.yml` as the publishing workflow name, because the npm trusted-publisher config pins it.
 
 ### Bun
 

CHANGELOG.md

@@ -4,6 +4,42 @@ All notable changes to this project are documented in this file.
 
 This project follows [Semantic Versioning](https://semver.org/) and the format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
 
+## 0.1.0 (Unreleased)
+
+### Added
+
+- Path policy helpers: `isEnvFile`, `isSensitiveEnvFile`, and `isEnvSitterPepperFile`, alongside `isExampleEnvFile`.
+  - Env files are `.env`, `.env.<suffix>`, `<name>.env`, and `<name>.env.<suffix>`; source files such as `app.env.ts` are not.
+  - Sensitive env files are every env file except example/template files.
+  - `envsitter-guard` uses these helpers, so the library and the plugin agree on what is sensitive.
+- Type exports: `AnnotatePlan`, `CopyConflictPolicy`, `CopyPlanItem`, `DotenvIssue`, `FormatMode`, `FormatSort`.
+- Release automation: CI on every push and pull request (Node 22, 24, and 26), and tag-driven releases published with npm trusted publishing, so packages carry provenance attestations.
+- Documentation site at https://boxpositron.github.io/envsitter/ (source in `site/`, deployed by GitHub Pages).
+- `EnvSitter#redactor()` and the `envsitter redact` command replace secret-looking values in arbitrary text with `[envsitter: KEY redacted]` without returning the values. `envsitter-guard` uses it to scrub tool output.
+- When envsitter creates `.envsitter/`, it also writes `.envsitter/.gitignore` (`*`), so the pepper stays out of git even if the project's `.gitignore` does not list it.
+- CI also runs on macOS and Windows, and Dependabot proposes npm and GitHub Actions updates weekly (with a 7-day cooldown).
+
+### Changed
+
+- One dotenv grammar: `parseDotenv` (used by `EnvSitter` sources) now derives from the round-trip document parser used by `validate` and the file operations.
+  - Read-path parse errors now carry a `column` and use the same messages as `validate`.
+  - `export` followed by a tab is accepted on the read path.
+- CLI file commands (`validate`, `copy`, `format`/`reorder`, `annotate`, `add`, `set`, `unset`, `delete`) now run through the library file operations, so the CLI and the library cannot drift.
+  - `validate --json` output now includes `file`; `annotate --json` output now includes `key`.
+- Package version bumped to `0.1.0`.
+- `npm pack` and `npm publish` now build `dist/` first (`prepack`), so a publish can never ship stale or missing output.
+- Requires Node.js 22 or later (`engines`).
+- The npm package no longer ships the compiled test suite (`dist/test`).
+
+### Fixed
+
+- File writes no longer loosen permissions: a `0600` env file came back `0644` (world-readable) after any `--write` / `write: true` edit. Writes now keep the existing mode, and new files are created `0600`.
+- Writing to a symlinked env file now updates the link's target instead of replacing the symlink with a regular file.
+- An existing pepper file that is empty, too short, or unreadable is no longer silently replaced with a new pepper (which changed every fingerprint); envsitter now stops with an error naming the file. Two processes creating the pepper at the same time can no longer overwrite each other's.
+- `set`, `add`, and `copy` into an empty or new env file write `KEY=value` plus a trailing newline instead of a leading blank line and no trailing newline, and report the key on line 1.
+- Regexes with the `g` or `y` flag no longer skip every other key in `listKeys` filters, `scan` key filters, `partial_match_regex`, and `copy` include/exclude.
+- `match-by-key` rejects non-object JSON and non-string candidates with a clear error instead of a `TypeError`.
+
 ## 0.0.4 (2026-01-15)
 
 ### Added

README.md

@@ -1,7 +1,12 @@
 # envsitter
 
+[![CI](https://github.com/boxpositron/envsitter/actions/workflows/ci.yml/badge.svg)](https://github.com/boxpositron/envsitter/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/envsitter)](https://www.npmjs.com/package/envsitter)
+
 Safely inspect and match `.env` secrets **without ever printing values**.
 
+Docs: https://boxpositron.github.io/envsitter/
+
 `envsitter` is designed for LLM/agent workflows where you want to:
 
 - List keys present in an env source (`.env` file or external provider)
@@ -18,6 +23,7 @@ Related: https://github.com/boxpositron/envsitter-guard — an OpenCode plugin t
 - Deterministic matching uses **HMAC-SHA-256** with a local pepper.
   - This avoids publishing raw SHA-256 hashes that are easy to dictionary-guess.
 - Candidate secrets should be passed via stdin (`--candidate-stdin`) to avoid shell history.
+- File writes keep the target file's permission bits and go through symlinks to the real file; new env files are created with mode `0600`.
 
 Non-goals:
 
@@ -30,6 +36,8 @@ Non-goals:
 npm install envsitter
 ```
 
+Requires Node.js 22 or later.
+
 Or run the CLI without installing globally:
 
 ```bash
@@ -45,7 +53,7 @@ Resolution order:
 1. `process.env.ENVSITTER_PEPPER` (or `ENV_SITTER_PEPPER`)
 2. Pepper file at `.envsitter/pepper` (auto-created if missing)
 
-The pepper file is created with mode `0600` when possible, and `.envsitter/` is gitignored.
+The pepper file is created with mode `0600` when possible. When envsitter creates `.envsitter/`, it also writes `.envsitter/.gitignore` (`*`), so the pepper stays out of git even if your project's `.gitignore` does not list it. An existing pepper file is never replaced: if it is empty, malformed, or unreadable, envsitter stops with an error instead of generating a new pepper (which would change every fingerprint).
 
 ## CLI usage
 
@@ -58,6 +66,7 @@ The pepper file is created with mode `0600` when possible, and `.envsitter/` is
 | `match` | Match candidate value(s) against key(s) |
 | `match-by-key` | Bulk match candidates by key |
 | `scan` | Detect value shapes (JWT, URL, base64) |
+| `redact` | Replace secret values in text read from stdin |
 | `validate` | Check dotenv syntax |
 | `copy` | Copy keys between env files |
 | `format` / `reorder` | Sort and organize env files |
@@ -74,6 +83,7 @@ The pepper file is created with mode `0600` when possible, and `.envsitter/` is
 - `match --file <path> (--key <KEY> | --keys <K1,K2> | --all-keys) [--op <op>] [--candidate <value> | --candidate-stdin]`
 - `match-by-key --file <path> (--candidates-json <json> | --candidates-stdin)`
 - `scan --file <path> [--keys-regex <re>] [--detect jwt,url,base64]`
+- `redact --file <path> [--all] [--min-length <n>]`
 - `validate --file <path>`
 - `copy --from <path> --to <path> [--keys <K1,K2>] [--include-regex <re>] [--exclude-regex <re>] [--rename <A=B,C=D>] [--on-conflict error|skip|overwrite] [--write]`
 - `format --file <path> [--mode sections|global] [--sort alpha|none] [--write]`
@@ -201,6 +211,17 @@ Optionally restrict which keys to scan:
 envsitter scan --file .env --keys-regex "/(JWT|URL)/" --detect jwt,url
 ```
 
+### Redact secret values from text
+
+Replace secret values in any text (logs, command output, support bundles) before sharing it. `redact` reads stdin, writes the text to stdout with each value replaced by `[envsitter: KEY redacted]`, and reports counts per key on stderr (or in the output with `--json`):
+
+```bash
+envsitter redact --file .env < app.log > app.redacted.log
+envsitter redact --file .env --json < app.log
+```
+
+A value is redacted when it is at least 8 characters long, is not a plain number or boolean, and looks secret: its key name contains a word such as `TOKEN`, `SECRET`, `PASSWORD`, or `API_KEY`; it is shaped like a random token; it is a URL with a password (the bare password is redacted too); or it is a PEM block. Ordinary config such as `NODE_ENV=development` is left alone. `--all` redacts every value, and `--min-length <n>` changes the length floor. Encoded or transformed values (base64, split strings) are not detected.
+
 ### Validate dotenv syntax
 
 ```bash
@@ -294,10 +315,10 @@ JSON outputs:
   - with `--op`: `{ "op": string, "matches": Array<{ "key": string, "match": boolean }> }`
 - `match-by-key --json` -> `{ "matches": Array<{ "key": string, "match": boolean }> }`
 - `scan --json` -> `{ "findings": Array<{ "key": string, "detections": Array<"jwt"|"url"|"base64"> }> }`
-- `validate --json` -> `{ "ok": boolean, "issues": Array<{ "line": number, "column": number, "message": string }> }`
+- `validate --json` -> `{ "file": string, "ok": boolean, "issues": Array<{ "line": number, "column": number, "message": string }> }`
 - `copy --json` -> `{ "from": string, "to": string, "onConflict": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": Array<...> }`
 - `format --json` / `reorder --json` -> `{ "file": string, "mode": string, "sort": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...> }`
-- `annotate --json` -> `{ "file": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": { ... } }`
+- `annotate --json` -> `{ "file": string, "key": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": { ... } }`
 - `add --json` / `set --json` / `unset --json` -> `{ "file": string, "key": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": { "key": string, "action": "added"|"updated"|"unset"|"key_exists"|"not_found"|"no_change", "line"?: number } }`
 - `delete --json` -> `{ "file": string, "keys": string[], "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": Array<{ "key": string, "action": "deleted"|"not_found", "line"?: number }> }`
 
@@ -358,15 +379,28 @@ await deleteEnvFileKeys({ file: '.env', keys: ['DEPRECATED', 'UNUSED'], write: t
 ### Utility functions
 
 ```ts
-import { isExampleEnvFile } from 'envsitter';
+import { isEnvFile, isEnvSitterPepperFile, isExampleEnvFile, isSensitiveEnvFile } from 'envsitter';
+
+// Dotenv-style file names (any directory, POSIX or Windows separators)
+isEnvFile('.env.local');               // true
+isEnvFile('config.env.local');         // true
+isEnvFile('app.env.ts');               // false (source file)
 
 // Detect example/template env files
 isExampleEnvFile('.env.example');      // true
 isExampleEnvFile('api.env.sample');    // true
 isExampleEnvFile('.env');              // false
-isExampleEnvFile('api.env');           // false
+
+// Env files that may hold real values (every env file except examples/templates)
+isSensitiveEnvFile('.env.production'); // true
+isSensitiveEnvFile('.env.example');    // false
+
+// The default pepper location
+isEnvSitterPepperFile('.envsitter/pepper'); // true
 ```
 
+`envsitter-guard` uses `isSensitiveEnvFile` and `isEnvSitterPepperFile` to decide what agents may not read or edit.
+
 ### Match operators via the library
 
 ```ts
@@ -410,6 +444,18 @@ const es = EnvSitter.fromExternalCommand('my-secret-provider', ['export', '--for
 const keys = await es.listKeys();
 ```
 
+### Redact secret values
+
+```ts
+import { EnvSitter } from 'envsitter';
+
+const redactor = await EnvSitter.fromDotenvFile('.env').redactor();
+const { text, redactions } = redactor.redact(logOutput);
+// text has each secret value replaced by [envsitter: KEY redacted]; redactions is [{ key, count }]
+```
+
+`redactor.keys` lists the keys it covers, and the values never leave the redactor. Options: `{ all?: boolean; minLength?: number }`, with the same rules as the `redact` command.
+
 ## Development
 
 ```bash
@@ -432,6 +478,16 @@ npm run build
 node --test --test-name-pattern "outside-in" dist/test/envsitter.test.js
 ```
 
+## Releasing
+
+Releases are published by GitHub Actions with npm trusted publishing, so no npm token is stored anywhere and every version ships with a provenance attestation.
+
+1. Bump `version` in `package.json` and replace `(Unreleased)` in the matching `CHANGELOG.md` heading with the release date.
+2. Commit, then push a matching tag: `git tag v0.1.0 && git push origin main v0.1.0`.
+3. The Release workflow reruns CI, checks that the tag, `package.json`, and the CHANGELOG agree, publishes to npm (prereleases such as `v0.2.0-beta.1` go to the `next` dist-tag), and creates the GitHub Release from the CHANGELOG section.
+
+The docs site in `site/` deploys to GitHub Pages on every push to `main` that changes it.
+
 ## License
 
 MIT. See `LICENSE`.