From 2bd7359056a4bdc89ac2c8d482a76b3340d05e0d Mon Sep 17 00:00:00 2001 From: David Ibia Date: Tue, 20 Oct 2026 15:10:15 +0100 Subject: [PATCH] Document 0.1.0 changes, releasing, and the docs site --- AGENTS.md | 74 ++++++++++++++++++++++++++++------------------------ CHANGELOG.md | 36 +++++++++++++++++++++++++ README.md | 66 ++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 137 insertions(+), 39 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index bdc3666..591cc6a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,50 +9,56 @@ This repo contains `envsitter`: a TypeScript (ESM) library + CLI for safely insp - Prefer comparisons via deterministic fingerprints (HMAC) and boolean match results. - Prefer passing candidate secrets via stdin (to avoid shell history): `--candidate-stdin`. - Keep all generated peppers/secrets out of git (see `.gitignore`). +- `envsitter-guard` (separate repo) exposes this package's exports to AI agents as tools, so every new export is agent-reachable. ## Project layout -- `src/envsitter.ts`: main library surface (`EnvSitter`) -- `src/cli.ts`: CLI entrypoint -- `src/dotenv/parse.ts`: minimal dotenv parser -- `src/pepper.ts`: pepper resolution + auto-create -- `src/test/*.test.ts`: Node test suite (compiled to `dist/test/*.test.js`) +- `src/index.ts`: public exports (the only surface npm users and `envsitter-guard` see) +- `src/envsitter.ts`: `EnvSitter` read-only facade (keys, fingerprints, matchers, scan, redactor) +- `src/cli.ts`: CLI entrypoint (`bin` -> `dist/cli.js`); one `if (cmd === ...)` block per command in `run()` +- `src/file-ops.ts`: file-level mutations for the library API (read, edit, atomic write only when `write: true`) +- `src/pepper.ts`: pepper resolution + auto-create; `src/fingerprint.ts`: HMAC-SHA-256 +- `src/sources/`: `DotenvFileSource`, `ExternalCommandSource` (both parse with `dotenv/parse.ts`) +- `src/dotenv/document.ts`: the only dotenv grammar; round-trippable, keeps comments, blank lines, and line/column issues +- `src/dotenv/parse.ts`: values-only view of `document.ts` (read path) +- `src/dotenv/edit.ts`: pure string-in/string-out edits (copy, format, annotate, add/set/unset/delete); owns the edit types that `file-ops.ts` re-exports +- `src/dotenv/io.ts`: `readTextFileOrEmpty`, `writeTextFileAtomic` (keeps file mode, writes through symlinks, new files `0600`); `src/dotenv/utils.ts`: value quoting +- `src/paths.ts`: path policy (`isEnvFile`, `isExampleEnvFile`, `isSensitiveEnvFile`, `isEnvSitterPepperFile`) +- `src/regex.ts`: `regexTest`; use it for every user-supplied regex (resets `lastIndex` for `g`/`y` flags) +- `src/redact.ts`: `createRedactor` behind `EnvSitter#redactor()` and the `redact` command; decides which values look secret (key names, token shape, URL passwords, PEM) +- `src/test/*.test.ts`: Node test suite (compiled to `dist/test/*.test.js`); `cli.test.ts` spawns `dist/cli.js` +- `.github/workflows/`: `ci.yml` (push, PR, reusable), `release.yml` (tag `vX.Y.Z` -> CI -> npm trusted publishing -> GitHub Release), `pages.yml` (deploys `site/`); `.github/dependabot.yml` groups weekly npm and Actions updates +- `.github/scripts/changelog-section.mjs`: prints a version's CHANGELOG section; fails if missing or still `(Unreleased)` +- `site/`: static docs site for envsitter and envsitter-guard (no build step); `site/DESIGN.md` is its design contract and is not published + +### Gotchas + +- Grammar changes go in `dotenv/document.ts` only; `parseDotenv`, `validate`, and every edit follow automatically. +- CLI file commands are thin wrappers over `file-ops.ts`; change behavior there, then keep the CLI's text (non-JSON) output in step. +- `src/paths.ts` is the policy `envsitter-guard` enforces. Widening or narrowing `isSensitiveEnvFile` changes what agents are blocked from; add a case to `paths.test.ts` for every change. +- `src/test/envsitter.test.ts` resolves the default pepper (`/.envsitter/pepper`) unless `ENVSITTER_PEPPER` is set, so `npm test` creates that file on first run. ## Build / typecheck / test -There is no linter configured (no ESLint/Prettier). Use `tsc` + tests as quality gates. +There is no linter configured (no ESLint/Prettier). Use `tsc` + tests as quality gates. Husky pre-commit runs `npm test`. -### Install +```bash +npm install +npm run build # tsc -p tsconfig.json -> dist/ +npm run typecheck # tsc --noEmit +npm test # build, then scripts/run-tests.mjs runs every dist/test/*.test.js +# tests execute from compiled JS, so build before running one file or one test +npm run build && node --test dist/test/envsitter.test.js +npm run build && node --test --test-name-pattern "outside-in" dist/test/envsitter.test.js +``` -- `npm install` +`--test-name-pattern` takes a substring/regex matching the `test('name', ...)` title. -### Build +### Releasing -- `npm run build` - - Runs `tsc -p tsconfig.json` and outputs to `dist/` - -### Typecheck only - -- `npm run typecheck` - -### Run all tests - -- `npm test` - - Runs `npm run build` then `node scripts/run-tests.mjs` - -### Run a single test file - -Tests are written in TS but executed from compiled JS. - -- `npm run build` -- `node --test dist/test/envsitter.test.js` - -### Run a single test by name/pattern - -- `npm run build` -- `node --test --test-name-pattern "outside-in" dist/test/envsitter.test.js` - -(Use a substring/regex that matches the `test('name', ...)` title.) +- Bump `package.json`, date the CHANGELOG heading, commit, push tag `vX.Y.Z`. The Release workflow does the rest; never `npm publish` by hand. +- Push to `origin` (Gitea, `git.boxpositron.dev`); it mirrors to GitHub, where the workflows run. Jobs are guarded with `if: github.server_url == 'https://github.com'` so Gitea's runner skips them. +- npm trusted publishing (OIDC) authenticates the publish; no npm token exists in the repo. Keep `release.yml` as the publishing workflow name, because the npm trusted-publisher config pins it. ### Bun diff --git a/CHANGELOG.md b/CHANGELOG.md index da93638..b64ce96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,42 @@ All notable changes to this project are documented in this file. This project follows [Semantic Versioning](https://semver.org/) and the format is loosely based on [Keep a Changelog](https://keepachangelog.com/). +## 0.1.0 (Unreleased) + +### Added + +- Path policy helpers: `isEnvFile`, `isSensitiveEnvFile`, and `isEnvSitterPepperFile`, alongside `isExampleEnvFile`. + - Env files are `.env`, `.env.`, `.env`, and `.env.`; source files such as `app.env.ts` are not. + - Sensitive env files are every env file except example/template files. + - `envsitter-guard` uses these helpers, so the library and the plugin agree on what is sensitive. +- Type exports: `AnnotatePlan`, `CopyConflictPolicy`, `CopyPlanItem`, `DotenvIssue`, `FormatMode`, `FormatSort`. +- Release automation: CI on every push and pull request (Node 22, 24, and 26), and tag-driven releases published with npm trusted publishing, so packages carry provenance attestations. +- Documentation site at https://boxpositron.github.io/envsitter/ (source in `site/`, deployed by GitHub Pages). +- `EnvSitter#redactor()` and the `envsitter redact` command replace secret-looking values in arbitrary text with `[envsitter: KEY redacted]` without returning the values. `envsitter-guard` uses it to scrub tool output. +- When envsitter creates `.envsitter/`, it also writes `.envsitter/.gitignore` (`*`), so the pepper stays out of git even if the project's `.gitignore` does not list it. +- CI also runs on macOS and Windows, and Dependabot proposes npm and GitHub Actions updates weekly (with a 7-day cooldown). + +### Changed + +- One dotenv grammar: `parseDotenv` (used by `EnvSitter` sources) now derives from the round-trip document parser used by `validate` and the file operations. + - Read-path parse errors now carry a `column` and use the same messages as `validate`. + - `export` followed by a tab is accepted on the read path. +- CLI file commands (`validate`, `copy`, `format`/`reorder`, `annotate`, `add`, `set`, `unset`, `delete`) now run through the library file operations, so the CLI and the library cannot drift. + - `validate --json` output now includes `file`; `annotate --json` output now includes `key`. +- Package version bumped to `0.1.0`. +- `npm pack` and `npm publish` now build `dist/` first (`prepack`), so a publish can never ship stale or missing output. +- Requires Node.js 22 or later (`engines`). +- The npm package no longer ships the compiled test suite (`dist/test`). + +### Fixed + +- File writes no longer loosen permissions: a `0600` env file came back `0644` (world-readable) after any `--write` / `write: true` edit. Writes now keep the existing mode, and new files are created `0600`. +- Writing to a symlinked env file now updates the link's target instead of replacing the symlink with a regular file. +- An existing pepper file that is empty, too short, or unreadable is no longer silently replaced with a new pepper (which changed every fingerprint); envsitter now stops with an error naming the file. Two processes creating the pepper at the same time can no longer overwrite each other's. +- `set`, `add`, and `copy` into an empty or new env file write `KEY=value` plus a trailing newline instead of a leading blank line and no trailing newline, and report the key on line 1. +- Regexes with the `g` or `y` flag no longer skip every other key in `listKeys` filters, `scan` key filters, `partial_match_regex`, and `copy` include/exclude. +- `match-by-key` rejects non-object JSON and non-string candidates with a clear error instead of a `TypeError`. + ## 0.0.4 (2026-01-15) ### Added diff --git a/README.md b/README.md index d524124..524521a 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,12 @@ # envsitter +[![CI](https://github.com/boxpositron/envsitter/actions/workflows/ci.yml/badge.svg)](https://github.com/boxpositron/envsitter/actions/workflows/ci.yml) +[![npm](https://img.shields.io/npm/v/envsitter)](https://www.npmjs.com/package/envsitter) + Safely inspect and match `.env` secrets **without ever printing values**. +Docs: https://boxpositron.github.io/envsitter/ + `envsitter` is designed for LLM/agent workflows where you want to: - List keys present in an env source (`.env` file or external provider) @@ -18,6 +23,7 @@ Related: https://github.com/boxpositron/envsitter-guard — an OpenCode plugin t - Deterministic matching uses **HMAC-SHA-256** with a local pepper. - This avoids publishing raw SHA-256 hashes that are easy to dictionary-guess. - Candidate secrets should be passed via stdin (`--candidate-stdin`) to avoid shell history. +- File writes keep the target file's permission bits and go through symlinks to the real file; new env files are created with mode `0600`. Non-goals: @@ -30,6 +36,8 @@ Non-goals: npm install envsitter ``` +Requires Node.js 22 or later. + Or run the CLI without installing globally: ```bash @@ -45,7 +53,7 @@ Resolution order: 1. `process.env.ENVSITTER_PEPPER` (or `ENV_SITTER_PEPPER`) 2. Pepper file at `.envsitter/pepper` (auto-created if missing) -The pepper file is created with mode `0600` when possible, and `.envsitter/` is gitignored. +The pepper file is created with mode `0600` when possible. When envsitter creates `.envsitter/`, it also writes `.envsitter/.gitignore` (`*`), so the pepper stays out of git even if your project's `.gitignore` does not list it. An existing pepper file is never replaced: if it is empty, malformed, or unreadable, envsitter stops with an error instead of generating a new pepper (which would change every fingerprint). ## CLI usage @@ -58,6 +66,7 @@ The pepper file is created with mode `0600` when possible, and `.envsitter/` is | `match` | Match candidate value(s) against key(s) | | `match-by-key` | Bulk match candidates by key | | `scan` | Detect value shapes (JWT, URL, base64) | +| `redact` | Replace secret values in text read from stdin | | `validate` | Check dotenv syntax | | `copy` | Copy keys between env files | | `format` / `reorder` | Sort and organize env files | @@ -74,6 +83,7 @@ The pepper file is created with mode `0600` when possible, and `.envsitter/` is - `match --file (--key | --keys | --all-keys) [--op ] [--candidate | --candidate-stdin]` - `match-by-key --file (--candidates-json | --candidates-stdin)` - `scan --file [--keys-regex ] [--detect jwt,url,base64]` +- `redact --file [--all] [--min-length ]` - `validate --file ` - `copy --from --to [--keys ] [--include-regex ] [--exclude-regex ] [--rename ] [--on-conflict error|skip|overwrite] [--write]` - `format --file [--mode sections|global] [--sort alpha|none] [--write]` @@ -201,6 +211,17 @@ Optionally restrict which keys to scan: envsitter scan --file .env --keys-regex "/(JWT|URL)/" --detect jwt,url ``` +### Redact secret values from text + +Replace secret values in any text (logs, command output, support bundles) before sharing it. `redact` reads stdin, writes the text to stdout with each value replaced by `[envsitter: KEY redacted]`, and reports counts per key on stderr (or in the output with `--json`): + +```bash +envsitter redact --file .env < app.log > app.redacted.log +envsitter redact --file .env --json < app.log +``` + +A value is redacted when it is at least 8 characters long, is not a plain number or boolean, and looks secret: its key name contains a word such as `TOKEN`, `SECRET`, `PASSWORD`, or `API_KEY`; it is shaped like a random token; it is a URL with a password (the bare password is redacted too); or it is a PEM block. Ordinary config such as `NODE_ENV=development` is left alone. `--all` redacts every value, and `--min-length ` changes the length floor. Encoded or transformed values (base64, split strings) are not detected. + ### Validate dotenv syntax ```bash @@ -294,10 +315,10 @@ JSON outputs: - with `--op`: `{ "op": string, "matches": Array<{ "key": string, "match": boolean }> }` - `match-by-key --json` -> `{ "matches": Array<{ "key": string, "match": boolean }> }` - `scan --json` -> `{ "findings": Array<{ "key": string, "detections": Array<"jwt"|"url"|"base64"> }> }` -- `validate --json` -> `{ "ok": boolean, "issues": Array<{ "line": number, "column": number, "message": string }> }` +- `validate --json` -> `{ "file": string, "ok": boolean, "issues": Array<{ "line": number, "column": number, "message": string }> }` - `copy --json` -> `{ "from": string, "to": string, "onConflict": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": Array<...> }` - `format --json` / `reorder --json` -> `{ "file": string, "mode": string, "sort": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...> }` -- `annotate --json` -> `{ "file": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": { ... } }` +- `annotate --json` -> `{ "file": string, "key": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": { ... } }` - `add --json` / `set --json` / `unset --json` -> `{ "file": string, "key": string, "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": { "key": string, "action": "added"|"updated"|"unset"|"key_exists"|"not_found"|"no_change", "line"?: number } }` - `delete --json` -> `{ "file": string, "keys": string[], "willWrite": boolean, "wrote": boolean, "hasChanges": boolean, "issues": Array<...>, "plan": Array<{ "key": string, "action": "deleted"|"not_found", "line"?: number }> }` @@ -358,15 +379,28 @@ await deleteEnvFileKeys({ file: '.env', keys: ['DEPRECATED', 'UNUSED'], write: t ### Utility functions ```ts -import { isExampleEnvFile } from 'envsitter'; +import { isEnvFile, isEnvSitterPepperFile, isExampleEnvFile, isSensitiveEnvFile } from 'envsitter'; + +// Dotenv-style file names (any directory, POSIX or Windows separators) +isEnvFile('.env.local'); // true +isEnvFile('config.env.local'); // true +isEnvFile('app.env.ts'); // false (source file) // Detect example/template env files isExampleEnvFile('.env.example'); // true isExampleEnvFile('api.env.sample'); // true isExampleEnvFile('.env'); // false -isExampleEnvFile('api.env'); // false + +// Env files that may hold real values (every env file except examples/templates) +isSensitiveEnvFile('.env.production'); // true +isSensitiveEnvFile('.env.example'); // false + +// The default pepper location +isEnvSitterPepperFile('.envsitter/pepper'); // true ``` +`envsitter-guard` uses `isSensitiveEnvFile` and `isEnvSitterPepperFile` to decide what agents may not read or edit. + ### Match operators via the library ```ts @@ -410,6 +444,18 @@ const es = EnvSitter.fromExternalCommand('my-secret-provider', ['export', '--for const keys = await es.listKeys(); ``` +### Redact secret values + +```ts +import { EnvSitter } from 'envsitter'; + +const redactor = await EnvSitter.fromDotenvFile('.env').redactor(); +const { text, redactions } = redactor.redact(logOutput); +// text has each secret value replaced by [envsitter: KEY redacted]; redactions is [{ key, count }] +``` + +`redactor.keys` lists the keys it covers, and the values never leave the redactor. Options: `{ all?: boolean; minLength?: number }`, with the same rules as the `redact` command. + ## Development ```bash @@ -432,6 +478,16 @@ npm run build node --test --test-name-pattern "outside-in" dist/test/envsitter.test.js ``` +## Releasing + +Releases are published by GitHub Actions with npm trusted publishing, so no npm token is stored anywhere and every version ships with a provenance attestation. + +1. Bump `version` in `package.json` and replace `(Unreleased)` in the matching `CHANGELOG.md` heading with the release date. +2. Commit, then push a matching tag: `git tag v0.1.0 && git push origin main v0.1.0`. +3. The Release workflow reruns CI, checks that the tag, `package.json`, and the CHANGELOG agree, publishes to npm (prereleases such as `v0.2.0-beta.1` go to the `next` dist-tag), and creates the GitHub Release from the CHANGELOG section. + +The docs site in `site/` deploys to GitHub Pages on every push to `main` that changes it. + ## License MIT. See `LICENSE`.