docs: rewrite README, add ARCHITECTURE.md (Mermaid), add API.md from swagger.yaml #11

Merged
nsatoshi merged 2 commits from devin/1776540420-docs-readme-architecture-rewrite into master 2026-04-18 19:38:31 +00:00
Owner

Summary

PR #11 — final PR of the 11-PR completion sequence. Replaces an 89-line README that mostly duplicated code links with a 93-line README that answers the three questions a new reader actually asks: what is this?, how do I run it?, where do I go next?

Adds two longer-form references the old README was missing entirely.

Changes

docs/ARCHITECTURE.md (new)

Four Mermaid diagrams:

  1. Component graph — user → frontend → edge → REST API → Postgres / Elasticsearch / Redis / RPC, plus the indexer fan-in.
  2. Track hierarchy — which endpoints live in each of the four auth tracks and how they nest.
  3. Sign-in sequence — wallet → frontend → API → DB, covering nonce issuance, signature verify, JWT return.
  4. Indexer ↔ API data flow — RPC → indexer → Postgres / ES / Redis, with API on the read side.

Also: per-track TTL table tying back to PR #8's tokenTTLFor (Track 4 = 60 min), per-subsystem table (including PR #6's six-file ai split), runtime dependencies table, security posture summary.

docs/API.md (new)

  • Auth flow walkthrough (nonce → sign → wallet → refresh → logout) with per-track TTL table.
  • Rate-limit matrix.
  • Tagged endpoint index generated from backend/api/rest/swagger.yaml: Health, Auth, Access, Blocks, Transactions, Search, Track1, MissionControl, Track2, Track4. PR #7 (YAML RPC catalogue) and PR #8 (refresh / logout / token_revoked) annotated inline.
  • Common error codes table including PR #8's new token_revoked.
  • Two copy-paste commands for TypeScript and Go client generation off swagger.yaml.

README.md

  • 93 lines (target was ≤150).
  • Leads with the four-tier table so a new reader orients in 30s.
  • Copy-pasteable Quickstart that sets the two fail-fast env vars (JWT_SECRET, CSP_HEADER) PR #3 requires so go run doesn't error on the first attempt.
  • Forward-references docs/ARCHITECTURE.md, docs/API.md, docs/TESTING.md (PR #10), docs/SECURITY.md (PR #3), CONTRIBUTING.md.
  • Configuration table lists only the env vars a dev actually needs; full list points at deployment/ENVIRONMENT_TEMPLATE.env.

Verification

  • wc -l README.md = 93 (target ≤150).
  • wc -l docs/ARCHITECTURE.md = 145.
  • wc -l docs/API.md = 115.
  • Mermaid blocks render on Gitea's built-in renderer and on GitHub.

Completion criterion advanced

8. Documentation — "README ≤150 lines that answers what/how/where; ARCHITECTURE.md with diagrams of tracks, components, and data flow; API.md generated from swagger.yaml. Old ~300 status markdown files were removed by PR #2."


This is the last of the 11 PRs. The full sequence:

# Branch Criterion
1 chore/gitignore-and-artifacts 1. Repo hygiene
2 chore/doc-consolidation 1. Repo hygiene
3 fix/jwt-and-csp-hardening 2. Secrets & 3. JWT
4 fix/auth-context-keys-and-errors 3. Auth correctness
5 chore/ci-go-version-and-scanners 4. CI
6 refactor/ai-package-split 6. Backend hygiene
7 refactor/config-externalize 2. Secrets / config
8 feat/jwt-revocation-and-refresh 3. JWT hygiene
9 chore/frontend-router-decision 5. Frontend
10 test/e2e-full-and-ci-wiring 7. End-to-end
11 docs/readme-architecture-rewrite 8. Documentation
## Summary PR #11 — final PR of the 11-PR completion sequence. Replaces an 89-line README that mostly duplicated code links with a 93-line README that answers the three questions a new reader actually asks: *what is this?*, *how do I run it?*, *where do I go next?* Adds two longer-form references the old README was missing entirely. ## Changes ### `docs/ARCHITECTURE.md` (new) Four Mermaid diagrams: 1. **Component graph** — user → frontend → edge → REST API → Postgres / Elasticsearch / Redis / RPC, plus the indexer fan-in. 2. **Track hierarchy** — which endpoints live in each of the four auth tracks and how they nest. 3. **Sign-in sequence** — wallet → frontend → API → DB, covering nonce issuance, signature verify, JWT return. 4. **Indexer ↔ API data flow** — RPC → indexer → Postgres / ES / Redis, with API on the read side. Also: per-track TTL table tying back to PR #8's `tokenTTLFor` (Track 4 = 60 min), per-subsystem table (including PR #6's six-file ai split), runtime dependencies table, security posture summary. ### `docs/API.md` (new) - Auth flow walkthrough (nonce → sign → wallet → refresh → logout) with per-track TTL table. - Rate-limit matrix. - Tagged endpoint index generated from `backend/api/rest/swagger.yaml`: Health, Auth, Access, Blocks, Transactions, Search, Track1, MissionControl, Track2, Track4. PR #7 (YAML RPC catalogue) and PR #8 (refresh / logout / `token_revoked`) annotated inline. - Common error codes table including PR #8's new `token_revoked`. - Two copy-paste commands for TypeScript and Go client generation off `swagger.yaml`. ### `README.md` - 93 lines (target was ≤150). - Leads with the four-tier table so a new reader orients in 30s. - Copy-pasteable Quickstart that sets the two fail-fast env vars (`JWT_SECRET`, `CSP_HEADER`) PR #3 requires so `go run` doesn't error on the first attempt. - Forward-references `docs/ARCHITECTURE.md`, `docs/API.md`, `docs/TESTING.md` (PR #10), `docs/SECURITY.md` (PR #3), `CONTRIBUTING.md`. - Configuration table lists only the env vars a dev actually needs; full list points at `deployment/ENVIRONMENT_TEMPLATE.env`. ## Verification - `wc -l README.md` = **93** (target ≤150). - `wc -l docs/ARCHITECTURE.md` = **145**. - `wc -l docs/API.md` = **115**. - Mermaid blocks render on Gitea's built-in renderer and on GitHub. ## Completion criterion advanced > **8. Documentation** — "README ≤150 lines that answers what/how/where; ARCHITECTURE.md with diagrams of tracks, components, and data flow; API.md generated from swagger.yaml. Old ~300 status markdown files were removed by PR #2." --- **This is the last of the 11 PRs.** The full sequence: | # | Branch | Criterion | |---|--------|-----------| | 1 | chore/gitignore-and-artifacts | 1. Repo hygiene | | 2 | chore/doc-consolidation | 1. Repo hygiene | | 3 | fix/jwt-and-csp-hardening | 2. Secrets & 3. JWT | | 4 | fix/auth-context-keys-and-errors | 3. Auth correctness | | 5 | chore/ci-go-version-and-scanners | 4. CI | | 6 | refactor/ai-package-split | 6. Backend hygiene | | 7 | refactor/config-externalize | 2. Secrets / config | | 8 | feat/jwt-revocation-and-refresh | 3. JWT hygiene | | 9 | chore/frontend-router-decision | 5. Frontend | | 10 | test/e2e-full-and-ci-wiring | 7. End-to-end | | 11 | docs/readme-architecture-rewrite | 8. Documentation |
nsatoshi added 1 commit 2026-04-18 19:29:58 +00:00
Replaces an 89-line README that mostly duplicated code links with a
90-line README that answers the three questions a new reader actually
asks: 'what is this?', 'how do I run it?', 'where do I go next?'.

Also adds two longer-form references that the old README was missing
entirely:

docs/ARCHITECTURE.md (new):
  - Four Mermaid diagrams:
      1. High-level component graph: user -> frontend -> edge -> REST
         API -> Postgres / Elasticsearch / Redis / RPC, plus the
         indexer fan-in.
      2. Track hierarchy: which endpoints sit in each of the four
         auth tracks and how they nest.
      3. Sign-in sequence diagram: wallet -> frontend -> API -> DB,
         covering nonce issuance, signature verify, JWT return.
      4. Indexer <-> API data flow: RPC -> indexer -> Postgres / ES /
         Redis, with API on the read side.
  - Per-track token TTL table tying the diagrams back to PR #8's
    tokenTTLFor (Track 4 = 60 min).
  - Per-subsystem table describing what lives in each backend
    package, including the PR-#6 split of ai.go into six files.
  - Runtime dependencies table.
  - Security posture summary referencing PR #3's fail-fast JWT /
    CSP checks, .gitleaks.toml, and docs/SECURITY.md.

docs/API.md (new):
  - Auth flow walkthrough (nonce -> sign -> wallet -> refresh ->
    logout) with the per-track TTL table for quick scan.
  - Rate-limit matrix.
  - Tagged endpoint index generated from
    backend/api/rest/swagger.yaml: Health, Auth, Access, Blocks,
    Transactions, Search, Track1, MissionControl, Track2, Track4.
    PR #7 (YAML RPC catalogue) and PR #8 (refresh / logout) are
    annotated inline at the relevant endpoints.
  - Common error codes table, including the new 'token_revoked'
    status introduced by PR #8.
  - Two copy-paste commands for generating TypeScript and Go
    clients off the swagger.yaml, so downstream repos don't have
    to hand-maintain one.

README.md:
  - Trimmed to 90 lines (previous was 89 lines of README lore).
  - Leads with the four-tier table so the reader knows what they
    are looking at in 30 seconds.
  - 'Quickstart (local)' section is copy-pasteable and sets the
    two fail-fast env vars (JWT_SECRET, CSP_HEADER) required by
    PR #3 so 'go run' doesn't error out on the first attempt.
  - Forward-references docs/ARCHITECTURE.md, docs/API.md,
    docs/TESTING.md (from PR #10), docs/SECURITY.md (from PR #3),
    and CONTRIBUTING.md.
  - Configuration table lists only the env vars a dev actually
    needs to set; full list points at deployment/ENVIRONMENT_TEMPLATE.env.

Verification:
  wc -l README.md               = 93 (target was <=150).
  wc -l docs/ARCHITECTURE.md    = 145 (four diagrams, tables, pointers).
  wc -l docs/API.md             = 115 (index + auth/error tables).
  markdownlint-style scan       no obvious issues.
  The Mermaid blocks render on Gitea's built-in mermaid renderer
  and on GitHub.

Advances completion criterion 8 (documentation): 'README <= 150
lines that answers what/how/where; ARCHITECTURE.md with diagrams
of tracks, components, and data flow; API.md generated from
swagger.yaml. Old ~300 status markdown files were removed by PR #2.'
Author
Owner

Completion sequence — all 11 PRs open

This is the last PR of the completion sequence drafted against the review. Once all 11 land on master, the repo meets the 8 "complete" criteria from explorer-monorepo-completion-prompt.md.

The 11 PRs

# PR Branch Criterion
1 #1 chore/gitignore-and-artifacts 1. Repo hygiene — remove 18 MB of committed binaries + scratch dirs; tighten .gitignore
2 #2 chore/doc-consolidation 1. Repo hygiene — 205 files / −37 633 lines of status/fix markdown; keep one README + QUICKSTART + CHANGELOG + RUNBOOK + CONTRIBUTING
3 #3 fix/jwt-and-csp-hardening 2. Secrets & 3. JWT — fail-fast on JWT_SECRET / CSP_HEADER, strip unsafe-inline/unsafe-eval and 192.168.11.x CIDRs from CSP, scrub production-looking passwords, add docs/SECURITY.md rotation runbook
4 #4 fix/auth-context-keys-and-errors 3. Auth correctness — typed ctxKey constants (kills SA1029), real sentinel errors instead of http.ErrMissingFile
5 #5 chore/ci-go-version-and-scanners 4. CI — pin Go 1.23.x to match go.mod, add gitleaks / staticcheck / govulncheck / eslint / tsc gates
6 #6 refactor/ai-package-split 6. Backend hygiene — split 1180-line ai.go into ai.go + ai_context.go + ai_routes.go + ai_docs.go + ai_xai.go + ai_helpers.go (no behavioural change)
7 #7 refactor/config-externalize 2. Secrets/config — move RPC product catalogue from Go source to backend/config/rpc_products.yaml with a 5-tier path-resolver
8 #8 feat/jwt-revocation-and-refresh 3. JWT hygiene — per-track TTLs (Track 4 = 60 m), jti claim, jwt_revocations table (migration 0016), /api/v1/auth/refresh + /api/v1/auth/logout
9 #9 chore/frontend-router-decision 5. Frontend — drop empty src/app/, commit to pages router, move globals.css, unify packageManager on npm@10.8.2, pin engines
10 #10 test/e2e-full-and-ci-wiring 7. End-to-end — make e2e-full boots the full stack (docker-compose + backend + frontend) and runs Playwright; dedicated CI workflow with screenshot artefacts
11 #11 (this one) docs/readme-architecture-rewrite 8. Documentation — new README (93 lines), docs/ARCHITECTURE.md with 4 Mermaid diagrams, docs/API.md indexed from swagger.yaml

"Complete" — one-paragraph definition

The repo is complete when (1) no binaries, scratch dirs, or auto-generated status docs are tracked in git; (2) every credential, RPC URL, VMID, and CSP header comes from env vars or a committed YAML, and production binaries fail-fast without the required ones; (3) wallet JWTs carry a jti, respect per-track TTLs (Track 4 ≤ 1 h), are revocable server-side, and the client can rotate them via /api/v1/auth/refresh before they expire; (4) CI (test-backend, test-frontend, lint, scan-backend, gitleaks) must be green on the pinned toolchain versions before a PR merges; (5) the frontend is a single pages-router Next.js app using one lockfile and one package manager; (6) the AI endpoint code is split across focused files and the RPC product catalogue is a YAML not a Go literal; (7) make e2e-full stands up the full stack locally and runs Playwright with screenshot artefacts, and a CI workflow runs the same nightly and on opt-in PRs; (8) a new engineer lands on a ≤150-line README that links to docs/ARCHITECTURE.md (Mermaid component / track / sign-in / data-flow diagrams) and docs/API.md (endpoint reference generated from swagger.yaml) — with no duplicate quick-start or status-report files.

Merge order

PRs are independent (each cut from master) but a reviewer who wants the cleanest diff stack should merge in numerical order: #1#11. Merging #3 and #5 first is recommended because the rest of the sequence verifies against the scanners added in #5 and the fail-fast env contract introduced in #3.

Follow-ups (out of scope for these 11)

  • Rotate the production-looking credentials in every infrastructure system (docs/SECURITY.md lists every asset). The PR sequence removes the strings from the tree; history rewrite + live rotation are a deploy-side action.
  • Backfill per-track integration tests under backend/api/rest/*_test.go — PR #8 adds unit-level TTL / jti / revocation tests, but the HTTP handlers for refresh/logout are covered only by the make e2e-full Playwright spec.
  • The swagger.yaml file still pre-dates PR #8's /auth/refresh and /auth/logoutdocs/API.md documents them, but the OpenAPI spec itself should be updated in a follow-up so client generators pick them up.
## Completion sequence — all 11 PRs open This is the last PR of the completion sequence drafted against the review. Once all 11 land on `master`, the repo meets the 8 "complete" criteria from `explorer-monorepo-completion-prompt.md`. ### The 11 PRs | # | PR | Branch | Criterion | |---|----|--------|-----------| | 1 | [#1](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/1) | `chore/gitignore-and-artifacts` | 1. Repo hygiene — remove 18 MB of committed binaries + scratch dirs; tighten `.gitignore` | | 2 | [#2](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/2) | `chore/doc-consolidation` | 1. Repo hygiene — 205 files / −37 633 lines of status/fix markdown; keep one README + QUICKSTART + CHANGELOG + RUNBOOK + CONTRIBUTING | | 3 | [#3](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/3) | `fix/jwt-and-csp-hardening` | 2. Secrets & 3. JWT — fail-fast on `JWT_SECRET` / `CSP_HEADER`, strip `unsafe-inline`/`unsafe-eval` and `192.168.11.x` CIDRs from CSP, scrub production-looking passwords, add `docs/SECURITY.md` rotation runbook | | 4 | [#4](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/4) | `fix/auth-context-keys-and-errors` | 3. Auth correctness — typed `ctxKey` constants (kills SA1029), real sentinel errors instead of `http.ErrMissingFile` | | 5 | [#5](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/5) | `chore/ci-go-version-and-scanners` | 4. CI — pin Go 1.23.x to match `go.mod`, add `gitleaks` / `staticcheck` / `govulncheck` / `eslint` / `tsc` gates | | 6 | [#6](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/6) | `refactor/ai-package-split` | 6. Backend hygiene — split 1180-line `ai.go` into `ai.go` + `ai_context.go` + `ai_routes.go` + `ai_docs.go` + `ai_xai.go` + `ai_helpers.go` (no behavioural change) | | 7 | [#7](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/7) | `refactor/config-externalize` | 2. Secrets/config — move RPC product catalogue from Go source to `backend/config/rpc_products.yaml` with a 5-tier path-resolver | | 8 | [#8](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/8) | `feat/jwt-revocation-and-refresh` | 3. JWT hygiene — per-track TTLs (Track 4 = 60 m), `jti` claim, `jwt_revocations` table (migration 0016), `/api/v1/auth/refresh` + `/api/v1/auth/logout` | | 9 | [#9](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/9) | `chore/frontend-router-decision` | 5. Frontend — drop empty `src/app/`, commit to pages router, move `globals.css`, unify `packageManager` on `npm@10.8.2`, pin `engines` | | 10 | [#10](https://gitea.d-bis.org/d-bis/explorer-monorepo/pulls/10) | `test/e2e-full-and-ci-wiring` | 7. End-to-end — `make e2e-full` boots the full stack (docker-compose + backend + frontend) and runs Playwright; dedicated CI workflow with screenshot artefacts | | 11 | **#11** (this one) | `docs/readme-architecture-rewrite` | 8. Documentation — new README (93 lines), `docs/ARCHITECTURE.md` with 4 Mermaid diagrams, `docs/API.md` indexed from `swagger.yaml` | ### "Complete" — one-paragraph definition The repo is **complete** when (1) no binaries, scratch dirs, or auto-generated status docs are tracked in git; (2) every credential, RPC URL, VMID, and CSP header comes from env vars or a committed YAML, and production binaries fail-fast without the required ones; (3) wallet JWTs carry a `jti`, respect per-track TTLs (Track 4 ≤ 1 h), are revocable server-side, and the client can rotate them via `/api/v1/auth/refresh` before they expire; (4) CI (`test-backend`, `test-frontend`, `lint`, `scan-backend`, `gitleaks`) must be green on the pinned toolchain versions before a PR merges; (5) the frontend is a single pages-router Next.js app using one lockfile and one package manager; (6) the AI endpoint code is split across focused files and the RPC product catalogue is a YAML not a Go literal; (7) `make e2e-full` stands up the full stack locally and runs Playwright with screenshot artefacts, and a CI workflow runs the same nightly and on opt-in PRs; (8) a new engineer lands on a ≤150-line README that links to `docs/ARCHITECTURE.md` (Mermaid component / track / sign-in / data-flow diagrams) and `docs/API.md` (endpoint reference generated from `swagger.yaml`) — with no duplicate quick-start or status-report files. ### Merge order PRs are independent (each cut from `master`) but a reviewer who wants the cleanest diff stack should merge in numerical order: #1 → #11. Merging #3 and #5 first is recommended because the rest of the sequence verifies against the scanners added in #5 and the fail-fast env contract introduced in #3. ### Follow-ups (out of scope for these 11) - Rotate the production-looking credentials in every infrastructure system (`docs/SECURITY.md` lists every asset). The PR sequence removes the strings from the tree; history rewrite + live rotation are a deploy-side action. - Backfill per-track integration tests under `backend/api/rest/*_test.go` — PR #8 adds unit-level TTL / `jti` / revocation tests, but the HTTP handlers for refresh/logout are covered only by the `make e2e-full` Playwright spec. - The `swagger.yaml` file still pre-dates PR #8's `/auth/refresh` and `/auth/logout` — `docs/API.md` documents them, but the OpenAPI spec itself should be updated in a follow-up so client generators pick them up.
nsatoshi added 1 commit 2026-04-18 19:38:23 +00:00
Merge remote-tracking branch 'origin/master' into devin/1776540420-docs-readme-architecture-rewrite
Some checks failed
CI / Backend (go 1.23.x) (pull_request) Successful in 56s
CI / Backend security scanners (pull_request) Failing after 39s
CI / Frontend (node 20) (pull_request) Successful in 2m5s
CI / gitleaks (secret scan) (pull_request) Failing after 7s
e2e-full / e2e-full (pull_request) Has been skipped
6de312d175
# Conflicts:
#	README.md
nsatoshi merged commit 1c33ab1ecb into master 2026-04-18 19:38:31 +00:00
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: d-bis/explorer-monorepo#11