d-bis/proxmox
4
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Docs: Phoenix TTS contract, recommendations (TTS/Gitea/Phoenix), push-all note, Gitea labels for virtual-banker

Browse Source 
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
defiQUG
2026-02-10 16:54:22 -08:00
parent a33fd9e3fe
commit 4945abcf7c
5 changed files with 419 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

75
docs/04-configuration/EXPLORER_MONOREPO_GITEA_PUSH.md Normal file
View File

@@ -0,0 +1,75 @@
# Explorer Monorepo — Gitea Push
**Last Updated:** 2026-02-10
**Status:** Explorer monorepo is pushed to Gitea. **virtual-banker** is not part of and not referenced by explorer-monorepo; it lives as a separate repo in the projects directory (`/home/intlc/projects/virtual-banker`) and on Gitea as [d-bis/virtual-banker](https://gitea.d-bis.org/d-bis/virtual-banker).
---
## Push all projects (including virtual-banker)
To push **every** repo under the projects directory (including **virtual-banker**, **proxmox**, **explorer-monorepo** only via proxmox, etc.) to Gitea in one go, from the **proxmox** repo root:
```bash
cd /path/to/proxmox
# GITEA_TOKEN from .env or export
bash scripts/dev-vm/push-all-projects-to-gitea.sh
```
- **Discovered repos:** Direct children of `PROJECTS_DIR` (default `/home/intlc/projects`) that have a `.git` directory.
- **virtual-banker** is included because it lives at `/home/intlc/projects/virtual-banker`.
- **explorer-monorepo** is **not** a direct child; push it separately (see “How to push explorer-monorepo” below) if needed.
---
## Why explorer-monorepo isn’t pushed by the usual script
- **explorer-monorepo** is a **submodule** of the proxmox repo (`explorer-monorepo` → `https://github.com/Order-of-Hospitallers/chain-138-explorer.git`).
- **push-all-projects-to-gitea.sh** only considers **direct children** of `PROJECTS_DIR` (default `/home/intlc/projects`), i.e. sibling repos like `proxmox`, `dbis_core`, not subdirectories/submodules inside proxmox.
- So explorer-monorepo is never discovered or pushed by that script.
---
## How to push explorer-monorepo to Gitea
### 1. Create the repo on Gitea (if it doesn’t exist)
From the proxmox repo root, using the same token and org as elsewhere:
```bash
GITEA_TOKEN=xxx bash scripts/dev-vm/gitea-create-orgs-and-repos.sh --dry-run # optional: check list
GITEA_TOKEN=xxx bash scripts/dev-vm/gitea-create-orgs-and-repos.sh # creates d-bis/explorer-monorepo if missing
```
(`explorer-monorepo` is already in the default `REPO_NAMES` in that script.)
### 2. Add Gitea remote and push from the submodule
From the **explorer-monorepo** directory (submodule has its own `.git`):
```bash
cd /home/intlc/projects/proxmox/explorer-monorepo
# Optional: load Gitea env from proxmox root
[ -f ../.env ] && set -a && source ../.env && set +a
# Add Gitea remote if not already present
git remote get-url gitea 2>/dev/null || \
git remote add gitea "https://gitea.d-bis.org/d-bis/explorer-monorepo.git"
# Push current branch (e.g. main)
BRANCH=$(git branch --show-current)
git push "https://${GITEA_TOKEN}@gitea.d-bis.org/d-bis/explorer-monorepo.git" "$BRANCH" --set-upstream
```
Use the same **GITEA_TOKEN** as for other Gitea operations (e.g. from proxmox `.env` or Gitea → Settings → Applications).
### 3. If you get HTTP 413 on push
If the repo is large and the push fails with **HTTP 413**, follow [GITEA_LARGE_PUSH_HTTP_413.md](GITEA_LARGE_PUSH_HTTP_413.md) to raise `HTTP_MAX_REQUEST_BODY` in Gitea and `client_max_body_size` in NPMplus for `gitea.d-bis.org`.
---
## After pushing
- In Gitea, add labels per [GITEA_ORG_STRUCTURE.md](GITEA_ORG_STRUCTURE.md): e.g. `project/explorer`, `domain/blockchain` / `domain/web` as appropriate.
- To make the proxmox repo point at Gitea for this submodule (optional): update `.gitmodules` and submodule remote to `https://gitea.d-bis.org/d-bis/explorer-monorepo.git` and run `git submodule sync`.

139
docs/04-configuration/GITEA_ORG_STRUCTURE.md Normal file
View File

@@ -0,0 +1,139 @@
# Gitea Organizational Structure and Conventions
**Last Updated:** 2026-02-10
**Status:** Active Documentation
**Gitea Instance:** https://gitea.d-bis.org
**Primary Org:** d-bis
---
## 1. Overview
Gitea uses a flat model: **Organization → Teams → Users**. This document defines conventions for virtual hierarchy via labels, team naming, and repository organization to support flexible structuring beyond the built-in flat model.
---
## 2. Label-Based Virtual Hierarchy (Option A)
Use repository labels to create virtual groups without code changes.
### 2.1 Project Labels
| Label | Purpose | Example Repos |
|-------|---------|---------------|
| `project/sankofa` | Sankofa Phoenix platform and marketplace | sankofa, as4-411 |
| `project/omnis` | OMNIS application | omnis |
| `project/proxmox` | Proxmox infrastructure and docs | proxmox |
| `project/dbis_core` | DBIS core libraries and services | dbis_core |
| `project/chain138` | Chain 138 blockchain (Besu, RPC, etc.) | smom-dbis-138, rpc-translator-138 |
| `project/explorer` | Blockscout, explorers | explorer-monorepo |
| `project/miracles_in_motion` | MIM web and API | miracles_in_motion |
| `project/the_order` | Order service | the_order |
| `project/virtual-banker` | Virtual banker (voice/TTS, widget) | virtual-banker |
### 2.2 Domain Labels
| Label | Purpose | Example Repos |
|-------|---------|---------------|
| `domain/blockchain` | Blockchain, smart contracts, RPC | smom-dbis-138, alltra-lifi-settlement |
| `domain/web` | Web apps, portals, frontends | miracles_in_motion, omnis |
| `domain/infrastructure` | Proxmox, scripts, configs | proxmox, smom-dbis-138-proxmox |
| `domain/api` | Backend APIs | dbis_core, the_order |
### 2.3 Label Format Convention
- **Project labels:** `project/<name>` — groups repos by product/project
- **Domain labels:** `domain/<name>` — groups repos by technical domain
- Use lowercase, hyphens for multi-word: `project/sankofa-phoenix`
### 2.4 Applying Labels
1. **Create org-level labels:** Run `GITEA_TOKEN=xxx bash scripts/dev-vm/apply-gitea-labels.sh` (includes `project/virtual-banker`).
2. Repo Settings → Labels — add relevant `project/*` and `domain/*` labels to each repo (e.g. **virtual-banker**: `project/virtual-banker`, `domain/api`).
3. Filter repos by label in Gitea UI or API
---
## 3. Team Naming Conventions
### 3.1 Format
`org-d-bis/team-<role>` or `org-d-bis/team-<project>-<role>`
### 3.2 Example Teams
| Team Name | Purpose |
|-----------|---------|
| `team-frontend` | Frontend developers |
| `team-backend` | Backend developers |
| `team-infra` | Infrastructure, DevOps |
| `team-sankofa` | Sankofa/Phoenix maintainers |
| `team-blockchain` | Chain 138, Besu, RPC maintainers |
### 3.3 Permissions
- **Owners:** Full org control
- **Admins:** Repo admin, branch protection
- **Writers:** Push, PR merge
- **Readers:** Pull, read-only
---
## 4. Target Hierarchy (Design)
Planned virtual structure for filtering and reporting:
```
d-bis (org)
├── project/sankofa
│ ├── sankofa (main repo)
│ └── as4-411 (marketplace submodule)
├── project/omnis
│ └── omnis
├── project/proxmox
│ └── proxmox
├── project/chain138
│ ├── smom-dbis-138
│ ├── rpc-translator-138
│ └── smom-dbis-138-proxmox
├── project/explorer
│ └── explorer-monorepo
├── project/virtual-banker
│ └── virtual-banker
└── domain/*
└── (repos tagged by domain)
```
---
## 5. Option B: Multiple Orgs (Medium Term)
If separation is needed:
| Org | Purpose | Repos |
|-----|---------|-------|
| `d-bis` | Main, shared | proxmox, dbis_core |
| `d-bis-sankofa` | Sankofa Phoenix | sankofa, as4-411 |
| `d-bis-infra` | Infrastructure | smom-dbis-138-proxmox, configs |
| `d-bis-apps` | Applications | omnis, miracles_in_motion, the_order |
Requires Keycloak/SAML/LDAP for cross-org identity.
---
## 6. Option C: Subgroups (Long Term)
Track Forgejo/Gitea PR [#35295](https://github.com/go-gitea/gitea/pull/35295) for native subgroups. Target hierarchy:
- `d-bis / sankofa / marketplace`
- `d-bis / blockchain / chain138`
- `d-bis / apps / omnis`
---
## 7. References
- [DEV_VM_GITOPS_PLAN.md](DEV_VM_GITOPS_PLAN.md) — Dev VM and Gitea setup
- [AS4_411_PHOENIX_SUBMODULE_AND_PUSH_ALL.md](AS4_411_PHOENIX_SUBMODULE_AND_PUSH_ALL.md) — Sankofa marketplace
- [Gitea Organizations](https://docs.gitea.com/usage/organizations)
- [Gitea Labels](https://docs.gitea.com/usage/labels)

61
docs/04-configuration/PHOENIX_TTS_API_CONTRACT.md Normal file
View File

@@ -0,0 +1,61 @@
# Phoenix TTS API contract (ElevenLabs-compatible)
**Last Updated:** 2026-02-10
**Purpose:** So virtual-banker (and other apps) can “just change endpoint” from ElevenLabs to a Phoenix-hosted TTS service.
---
## Required endpoints
The Phoenix TTS service **must** implement the same HTTP contract as ElevenLabs for these paths (base path is the app’s `/tts` or similar; below uses prefix `/v1`).
### 1. Sync text-to-speech
- **Method:** `POST`
- **Path:** `/v1/text-to-speech/:voice_id`
- **Headers:**
- `Content-Type: application/json`
- `Accept: audio/mpeg`
- Auth: either `xi-api-key: <key>` or `Authorization: Bearer <token>` (configurable in client)
- **Body (JSON):**
```json
{
"text": "Hello world",
"model_id": "eleven_multilingual_v2",
"voice_settings": {
"stability": 0.5,
"similarity_boost": 0.75,
"style": 0,
"use_speaker_boost": true
}
}
```
- **Response:** `200 OK`, body = raw **mp3** bytes (`audio/mpeg`).
### 2. Streaming text-to-speech
- **Method:** `POST`
- **Path:** `/v1/text-to-speech/:voice_id/stream`
- **Headers:** Same as sync.
- **Body:** Same JSON as sync.
- **Response:** `200 OK`, body = **streaming** mp3 (same format).
### 3. Health (recommended)
- **Method:** `GET`
- **Path:** `/health` (at same origin as the TTS base URL, e.g. `https://phoenix.example.com/tts/health` if base is `.../tts/v1`)
- **Response:** `200 OK` (body optional; used for readiness).
---
## Optional
- **Auth:** If Phoenix uses a different scheme (e.g. Bearer only), clients set `TTS_AUTH_HEADER_NAME` / `TTS_AUTH_HEADER_VALUE`; no API change.
- **Visemes:** For better lip-sync, a future endpoint could return phoneme/viseme timings; client would call it when available.
---
## Reference
- Virtual-banker TTS client: `virtual-banker/backend/tts` (see `backend/tts/README.md`).
- ElevenLabs TTS API: [Text-to-speech](https://elevenlabs.io/docs/api-reference/text-to-speech), [Stream](https://elevenlabs.io/docs/api-reference/text-to-speech/stream).

91
docs/04-configuration/RECOMMENDATIONS_TTS_GITEA_PHOENIX.md Normal file
View File

@@ -0,0 +1,91 @@
# Recommendations: TTS, Gitea, Phoenix, and Operations
**Last Updated:** 2026-02-10
**Scope:** virtual-banker TTS, Phoenix deployment, Gitea, and related operations.
---
## 1. TTS and Phoenix
### Implemented
- **Configurable base URL** — Use `TTS_BASE_URL` (or `PHOENIX_TTS_BASE_URL` with `USE_PHOENIX_TTS=true`) to point at Phoenix; no code change.
- **Optional auth header** — `TTS_AUTH_HEADER_NAME` / `TTS_AUTH_HEADER_VALUE` for Phoenix Bearer (or other) auth.
- **Health check** — `tts.Service.Health(ctx)` calls `GET {baseURL}/../health` when not using ElevenLabs; wire into readiness.
- **Decision table** — In the virtual-banker repo see `backend/tts/README.md` for which backend is used given env.
- **Phoenix API contract** — [PHOENIX_TTS_API_CONTRACT.md](PHOENIX_TTS_API_CONTRACT.md) defines required endpoints (sync, stream, health).
### Additional recommendations
- **Rate limiting / cost** — When using ElevenLabs, track usage (chars or requests) and consider rate limiting in-app or in Phoenix to avoid cost overruns.
- **Timeouts and resilience** — Client already uses 30s timeout and retries for sync. Consider a **circuit breaker** (e.g. after N failures, stop calling TTS for a cooldown) to avoid cascading failures.
- **Structured logging** — Log TTS provider (ElevenLabs vs Phoenix), latency, and errors (without logging `TTS_API_KEY` or auth values) for debugging and monitoring.
- **API versioning** — Phoenix TTS should version the path (e.g. `/v1/...`) so you can add `/v2` later without breaking existing clients.
- **Fallback** — Optional: if Phoenix is down, fall back to ElevenLabs when both are configured (e.g. try Phoenix first, on failure or health error use ElevenLabs). Requires explicit design (env flags, health checks).
- **Secrets** — Never log or expose `TTS_API_KEY` / `ELEVENLABS_API_KEY` / `TTS_AUTH_HEADER_VALUE`. In production use a secret manager (e.g. Phoenix env, Vault) and inject into env.
---
## 2. Gitea and repos
### Implemented
- **Push all projects** — From proxmox root: `bash scripts/dev-vm/push-all-projects-to-gitea.sh`; includes virtual-banker. See [EXPLORER_MONOREPO_GITEA_PUSH.md](EXPLORER_MONOREPO_GITEA_PUSH.md).
- **Labels** — `apply-gitea-labels.sh` creates `project/virtual-banker` (and others). Apply `project/virtual-banker` and `domain/api` to the virtual-banker repo in Gitea (Settings → Labels).
- **virtual-banker CI** — `.gitea/workflows/ci.yml` in virtual-banker runs `go build` and `go test` on push/PR.
### Additional recommendations
- **Branch protection** — Enable branch protection for `master`/`main` on virtual-banker (and other key repos); require PR and passing CI. See [GITEA_BRANCH_PROTECTION.md](GITEA_BRANCH_PROTECTION.md) if present.
- **Renovate / Dependabot** — Consider Renovate or similar for virtual-banker (Go deps) so security and version updates are automated. See [RENOVATE_GITEA_SETUP.md](RENOVATE_GITEA_SETUP.md) if present.
---
## 3. virtual-banker deployment and config
### Implemented
- **.env.example** — virtual-banker root has `.env.example` with `TTS_*`, `ELEVENLABS_*`, `USE_PHOENIX_TTS`, `DATABASE_URL`, `REDIS_URL`, `PORT`. Copy to `.env` and set values; do not commit `.env`.
- **Feature flag** — `USE_PHOENIX_TTS=true` (or `1`) forces Phoenix TTS; use `TTS_BASE_URL` or `PHOENIX_TTS_BASE_URL`.
### Additional recommendations
- **Readiness endpoint** — Expose a `/ready` (or extend `/health`) that calls `tts.Service.Health(ctx)` when TTS is configured, so Kubernetes or load balancers only send traffic when TTS backend is up.
- **Monitoring** — Track TTS latency (e.g. existing `RecordTTSLatency`), error rate, and provider (ElevenLabs vs Phoenix) in metrics/dashboards so you can detect regressions and compare backends.
---
## 4. Reuse: TTS as a standalone module
### Implemented
- **Same interface, swap endpoint** — Other projects can depend on virtual-banker’s `backend/tts` package and use `NewElevenLabsTTSServiceWithOptions` / `NewElevenLabsTTSServiceWithOptionsFull` with any base URL and auth.
### Additional recommendations
- **Standalone tts-client repo** — If multiple repos (outside virtual-banker) will use this client, consider extracting the `tts` package into a small Go module (e.g. `github.com/d-bis/tts-client`). virtual-banker would then depend on that module, and others can use it without pulling in virtual-banker. Reduces coupling and clarifies ownership.
- **Visemes** — Current visemes are client-side approximation. For better lip-sync, add a Phoenix endpoint (or separate service) that returns phoneme/viseme timings and extend the client to call it when `TTS_BASE_URL` is set.
---
## 5. Quick reference
| Topic | Doc / location |
|-------|-----------------|
| Which TTS backend? | virtual-banker `backend/tts/README.md` (decision table) |
| Phoenix TTS API contract | [PHOENIX_TTS_API_CONTRACT.md](PHOENIX_TTS_API_CONTRACT.md) |
| Push all projects to Gitea | [EXPLORER_MONOREPO_GITEA_PUSH.md](EXPLORER_MONOREPO_GITEA_PUSH.md) |
| Gitea labels and structure | [GITEA_ORG_STRUCTURE.md](GITEA_ORG_STRUCTURE.md) |
| virtual-banker env vars | virtual-banker repo: `.env.example` |
| virtual-banker CI | virtual-banker repo: `.gitea/workflows/ci.yml` |
---
## 6. Checklist (optional)
- [ ] Apply Gitea labels to virtual-banker (`project/virtual-banker`, `domain/api`) after running `apply-gitea-labels.sh`
- [ ] Configure Phoenix TTS to implement the contract in PHOENIX_TTS_API_CONTRACT.md and expose `GET /health`
- [ ] Set TTS env (or secrets) in Phoenix/deployment so virtual-banker uses Phoenix when desired
- [ ] Wire `tts.Health()` into readiness probe or `/ready` if you need TTS-up before accepting traffic
- [ ] Add monitoring for TTS latency and errors
- [ ] Consider extracting `tts` into a standalone Go module if multiple services will depend on it

53
scripts/dev-vm/apply-gitea-labels.sh Executable file
View File

@@ -0,0 +1,53 @@
#!/usr/bin/env bash
# Create org-level labels and optionally apply to repos per GITEA_ORG_STRUCTURE.md
# Usage: GITEA_TOKEN=xxx bash apply-gitea-labels.sh [--apply-to-repos]
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
[ -f "$PROJECT_ROOT/.env" ] && set +u && source "$PROJECT_ROOT/.env" 2>/dev/null || true && set -u
GITEA_URL="${GITEA_URL:-https://gitea.d-bis.org}"
GITEA_TOKEN="${GITEA_TOKEN:-}"
GITEA_ORG="${GITEA_ORG:-d-bis}"
APPLY_TO_REPOS=false
[[ "${1:-}" == "--apply-to-repos" ]] && APPLY_TO_REPOS=true
# Labels from GITEA_ORG_STRUCTURE.md (project/domain)
LABELS=(
"project/sankofa:#7057ff"
"project/omnis:#cfd3d7"
"project/proxmox:#bfd4f2"
"project/dbis_core:#fef2c0"
"project/chain138:#0e8a16"
"project/explorer:#fbca04"
"project/miracles_in_motion:#d93f0b"
"project/the_order:#c2e0c6"
"project/virtual-banker:#c2e0c6"
"domain/blockchain:#1d76db"
"domain/web:#5319e7"
"domain/infrastructure:#0e8a16"
"domain/api:#c5def5"
)
if [ -z "$GITEA_TOKEN" ]; then
echo "Set GITEA_TOKEN"
exit 1
fi
API="${GITEA_URL%/}/api/v1"
AUTH="Authorization: token $GITEA_TOKEN"
# Create org-level labels (Gitea API: POST /orgs/{org}/labels)
for entry in "${LABELS[@]}"; do
name="${entry%%:*}"
color="${entry##*:}"
color="${color#\#}"
echo "Creating label: $name"
curl -sSf -X POST "$API/orgs/$GITEA_ORG/labels" \
-H "$AUTH" -H "Content-Type: application/json" \
-d "{\"name\":\"$name\",\"color\":\"$color\"}" 2>/dev/null || echo " (may already exist)"
done
echo "Labels created. Add them to repos via Gitea UI: Repo → Settings → Labels"