d-bis/the_order
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8649ad41242e8aa5e73c5a026a43ea7762b4db2e
the_order/docs/reports/REMAINING_TASKS.md
defiQUG 2633de4d33 feat(eresidency): Complete eResidency service implementation 
- Implement credential revocation endpoint with proper database integration
- Fix database row mapping (snake_case to camelCase) for eResidency applications
- Add missing imports (getRiskAssessmentEngine, VeriffKYCProvider, ComplyAdvantageSanctionsProvider)
- Fix environment variable type checking for Veriff and ComplyAdvantage providers
- Add required 'message' field to notification service calls
- Fix risk assessment type mismatches
- Update audit logging to use 'verified' action type (supported by schema)
- Resolve all TypeScript errors and unused variable warnings
- Add TypeScript ignore comments for placeholder implementations
- Temporarily disable security/detect-non-literal-regexp rule due to ESLint 9 compatibility
- Service now builds successfully with no linter errors

All core functionality implemented:
- Application submission and management
- KYC integration (Veriff placeholder)
- Sanctions screening (ComplyAdvantage placeholder)
- Risk assessment engine
- Credential issuance and revocation
- Reviewer console
- Status endpoints
- Auto-issuance service
2025-11-10 19:43:02 -08:00

701 lines
21 KiB
Markdown
Raw Blame History

# Remaining Tasks - The Order Monorepo
**Last Updated**: 2024-12-28
**Status**: Comprehensive review of all remaining work
---
## Table of Contents
1. [Critical Issues (Must Fix Immediately)](#critical-issues)
2. [High Priority Tasks](#high-priority-tasks)
3. [Medium Priority Tasks](#medium-priority-tasks)
4. [Low Priority / Nice to Have](#low-priority--nice-to-have)
5. [Implementation Details by Component](#implementation-details-by-component)
---
## Critical Issues (Must Fix Immediately)
### 1. Testing Infrastructure ❌
**Status**: No test files exist
**Impact**: Cannot verify functionality, regression risks, no CI confidence
**Effort**: 2-3 weeks
#### Tasks:
- [ ] Add unit tests for all packages (target: 80% coverage)
- [ ] `packages/auth` - OIDC, DID, eIDAS tests
- [ ] `packages/crypto` - KMS client tests
- [ ] `packages/storage` - Storage client and WORM tests
- [ ] `packages/schemas` - Schema validation tests
- [ ] `packages/workflows` - Workflow tests
- [ ] `packages/ui` - Component tests (if applicable)
- [ ] Add integration tests for all services
- [ ] `services/identity` - VC issuance/verification, signing
- [ ] `services/intake` - Document ingestion flow
- [ ] `services/finance` - Payment processing, ledger operations
- [ ] `services/dataroom` - Deal room operations, document access
- [ ] Add E2E tests for critical user flows
- [ ] `apps/portal-public` - Public portal flows
- [ ] `apps/portal-internal` - Internal admin flows
- [ ] Set up test coverage reporting in CI/CD
- [ ] Add test fixtures and mock factories to `packages/test-utils`
- [ ] Add database seeding utilities for tests
### 2. Incomplete Package Implementations ❌
**Status**: Multiple methods throw "Not implemented" errors
**Impact**: Application cannot function
**Effort**: 4-6 weeks
#### 2.1 Auth Package (`packages/auth`)
- [ ] **OIDC Provider** (`packages/auth/src/oidc.ts`)
- [ ] Implement `exchangeCodeForToken()` method
- [ ] **DID Resolver** (`packages/auth/src/did.ts`)
- [ ] Implement `resolve()` method
- [ ] Implement `verifySignature()` method
- [ ] **eIDAS Provider** (`packages/auth/src/eidas.ts`)
- [ ] Implement `requestSignature()` method
- [ ] Implement `verifySignature()` method
- [ ] Remove `@ts-expect-error` comment and properly type config
#### 2.2 Crypto Package (`packages/crypto`)
- [ ] **KMS Client** (`packages/crypto/src/kms.ts`)
- [ ] Implement `encrypt()` method
- [ ] Implement `decrypt()` method
- [ ] Implement `sign()` method
- [ ] Implement `verify()` method
- [ ] Remove `@ts-expect-error` comment and properly type config
- [ ] Add AWS KMS or GCP KMS implementation
#### 2.3 Storage Package (`packages/storage`)
- [ ] **Storage Client** (`packages/storage/src/storage.ts`)
- [ ] Implement `upload()` method (S3/GCS)
- [ ] Implement `download()` method
- [ ] Implement `delete()` method
- [ ] Implement `getPresignedUrl()` method
- [ ] Remove `@ts-expect-error` comment and properly type config
- [ ] **WORM Storage** (`packages/storage/src/worm.ts`)
- [ ] Implement `objectExists()` private method
#### 2.4 Workflows Package (`packages/workflows`)
- [ ] **Intake Workflow** (`packages/workflows/src/intake.ts`)
- [ ] Implement `intakeWorkflow()` function
- [ ] Integrate with Temporal or AWS Step Functions
- [ ] **Review Workflow** (`packages/workflows/src/review.ts`)
- [ ] Implement `reviewWorkflow()` function
- [ ] Integrate with Temporal or AWS Step Functions
### 3. Service Endpoint Implementations ❌
**Status**: All endpoints return placeholder messages
**Impact**: Services are non-functional
**Effort**: 3-4 weeks
#### 3.1 Identity Service (`services/identity`)
- [ ] Implement `/vc/issue` endpoint (verifiable credential issuance)
- [ ] Implement `/vc/verify` endpoint (verifiable credential verification)
- [ ] Implement `/sign` endpoint (document signing)
#### 3.2 Intake Service (`services/intake`)
- [ ] Implement `/ingest` endpoint
- [ ] Document upload handling
- [ ] OCR processing integration
- [ ] Document classification
- [ ] Routing logic
#### 3.3 Finance Service (`services/finance`)
- [ ] Implement `/ledger/entry` endpoint
- [ ] Ledger entry creation
- [ ] Transaction validation
- [ ] Database persistence
- [ ] Implement `/payments` endpoint
- [ ] Payment processing
- [ ] Payment gateway integration
- [ ] Transaction recording
#### 3.4 Dataroom Service (`services/dataroom`)
- [ ] Implement `POST /deals` endpoint (deal room creation)
- [ ] Implement `GET /deals/:dealId` endpoint (deal room retrieval)
- [ ] Implement `POST /deals/:dealId/documents` endpoint (document upload)
- [ ] Implement `GET /deals/:dealId/documents/:documentId/url` endpoint (presigned URL generation)
### 4. ESLint Configuration ❌
**Status**: Missing TypeScript ESLint plugins
**Impact**: Type safety issues undetected
**Effort**: 1 hour
- [ ] Install missing dependencies:
- [ ] `@typescript-eslint/eslint-plugin`
- [ ] `@typescript-eslint/parser`
- [ ] `eslint-plugin-security`
- [ ] `eslint-plugin-sonarjs`
- [ ] `eslint-config-prettier`
- [ ] Update `.eslintrc.js` with proper TypeScript configuration
- [ ] Add security-focused ESLint rules
- [ ] Configure ESLint-Prettier integration
### 5. Error Handling ❌
**Status**: No error handling middleware
**Impact**: Poor user experience, difficult debugging
**Effort**: 1 day
- [ ] Create `packages/shared` package (if doesn't exist)
- [ ] Implement error handling middleware
- [ ] Create `AppError` class
- [ ] Create error handler function
- [ ] Add structured error responses
- [ ] Add error handler to all services:
- [ ] `services/identity`
- [ ] `services/intake`
- [ ] `services/finance`
- [ ] `services/dataroom`
- [ ] Add error logging
- [ ] Add error recovery mechanisms
### 6. Input Validation ❌
**Status**: No request validation in endpoints
**Impact**: Security vulnerabilities, data corruption
**Effort**: 2-3 days
- [ ] Create Zod-to-JSON Schema converter utility
- [ ] Add Fastify schema validation to all endpoints
- [ ] Validate all request bodies using Zod schemas
- [ ] Validate all request parameters
- [ ] Validate all query parameters
- [ ] Return clear validation error messages
- [ ] Add validation to:
- [ ] `services/identity` endpoints
- [ ] `services/intake` endpoints
- [ ] `services/finance` endpoints
- [ ] `services/dataroom` endpoints
### 7. Security Middleware ❌
**Status**: No CORS, rate limiting, or security headers
**Impact**: Vulnerable to attacks
**Effort**: 1 day
- [ ] Install Fastify security plugins:
- [ ] `@fastify/helmet`
- [ ] `@fastify/rate-limit`
- [ ] `@fastify/cors`
- [ ] Create security middleware in `packages/shared`
- [ ] Configure CORS properly
- [ ] Configure rate limiting
- [ ] Configure security headers (helmet.js)
- [ ] Add to all services
- [ ] Remove hardcoded ports (use environment variables)
- [ ] Add request size limits
- [ ] Add HTTPS enforcement
---
## High Priority Tasks
### 8. Shared Package Creation
**Status**: Missing shared utilities package
**Effort**: 1-2 days
- [ ] Create `packages/shared` package structure
- [ ] Move error handling to shared package
- [ ] Move validation utilities to shared package
- [ ] Move security middleware to shared package
- [ ] Move logging utilities to shared package
- [ ] Add barrel exports
### 9. Environment Variable Validation
**Status**: No validation for environment variables
**Effort**: 2 hours
- [ ] Create `packages/shared/src/env.ts`
- [ ] Define Zod schema for all environment variables
- [ ] Validate environment variables on service startup
- [ ] Add to all services
- [ ] Document required environment variables
### 10. Database Integration
**Status**: No database client or migrations
**Effort**: 3-5 days
- [ ] Create `packages/database` package
- [ ] Add PostgreSQL client with connection pooling
- [ ] Set up database migrations (node-pg-migrate or kysely)
- [ ] Create migration scripts
- [ ] Add database connection to all services
- [ ] Create database schemas for:
- [ ] Identity service (users, credentials, signatures)
- [ ] Intake service (documents, classifications)
- [ ] Finance service (ledger entries, payments)
- [ ] Dataroom service (deals, documents, access control)
- [ ] Add migration validation in CI/CD
- [ ] Add database health checks
### 11. Structured Logging
**Status**: Fastify logger not structured
**Effort**: 1-2 days
- [ ] Install Pino logger
- [ ] Create logger configuration in `packages/shared`
- [ ] Configure structured JSON logging
- [ ] Add log levels configuration
- [ ] Add correlation IDs (request IDs)
- [ ] Add to all services
- [ ] Configure log rotation
- [ ] Add centralized logging setup
### 12. API Documentation
**Status**: No OpenAPI/Swagger documentation
**Effort**: 2-3 days
- [ ] Install Fastify Swagger plugins:
- [ ] `@fastify/swagger`
- [ ] `@fastify/swagger-ui`
- [ ] Configure Swagger for all services
- [ ] Document all endpoints with:
- [ ] Request/response schemas
- [ ] Description and tags
- [ ] Example requests/responses
- [ ] Set up Swagger UI routes
- [ ] Generate OpenAPI specs from Zod schemas
- [ ] Add to CI/CD for API documentation generation
### 13. Enhanced Health Checks
**Status**: Basic health checks only
**Effort**: 1 day
- [ ] Add comprehensive health check endpoints
- [ ] Check database connectivity
- [ ] Check storage connectivity
- [ ] Check KMS connectivity
- [ ] Check external service dependencies
- [ ] Return detailed health status
- [ ] Add readiness and liveness probes for Kubernetes
### 14. Monitoring & Observability
**Status**: No metrics, tracing, or alerting
**Effort**: 1 week
- [ ] Install OpenTelemetry SDK
- [ ] Configure distributed tracing
- [ ] Add Prometheus metrics client
- [ ] Add custom business metrics
- [ ] Expose metrics endpoints (`/metrics`)
- [ ] Add request tracing
- [ ] Configure Grafana dashboards
- [ ] Set up alerting rules
- [ ] Add performance monitoring
- [ ] Add error rate tracking
### 15. Authentication & Authorization Middleware
**Status**: No auth middleware
**Effort**: 2-3 days
- [ ] Create authentication middleware
- [ ] Implement JWT token verification
- [ ] Add OIDC token validation
- [ ] Add DID-based authentication
- [ ] Create authorization middleware
- [ ] Add role-based access control (RBAC)
- [ ] Add to protected endpoints
- [ ] Add API key authentication for service-to-service
---
## Medium Priority Tasks
### 16. Pre-commit Hooks
**Status**: Husky installed but not configured
**Effort**: 30 minutes
- [ ] Configure Husky pre-commit hook
- [ ] Install `lint-staged`
- [ ] Configure lint-staged for:
- [ ] TypeScript/JavaScript files (ESLint + Prettier)
- [ ] JSON/Markdown/YAML files (Prettier)
- [ ] Add commit message validation (optional)
### 17. CI/CD Enhancements
**Status**: Basic CI exists, needs enhancement
**Effort**: 2-3 days
- [ ] Review and enhance `.github/workflows/ci.yml`
- [ ] Add security scanning job:
- [ ] `pnpm audit`
- [ ] ESLint security rules
- [ ] Dependency vulnerability scanning
- [ ] Add test job with database service
- [ ] Add test coverage upload (Codecov)
- [ ] Add build artifact publishing
- [ ] Review and enhance `.github/workflows/release.yml`
- [ ] Add automated version bumping
- [ ] Add changelog generation
- [ ] Add Docker image building and publishing
- [ ] Add migration validation in CI
### 18. Code Documentation (JSDoc)
**Status**: Minimal JSDoc comments
**Effort**: 1 week
- [ ] Add JSDoc comments to all public APIs
- [ ] Document all classes and interfaces
- [ ] Document all function parameters
- [ ] Document return types
- [ ] Add usage examples
- [ ] Generate API documentation from JSDoc
### 19. TypeScript Improvements
**Status**: Some type suppressions present
**Effort**: 1-2 days
- [ ] Remove all `@ts-expect-error` comments
- [ ] Properly type all configurations
- [ ] Fix any type issues
- [ ] Ensure strict null checks everywhere
- [ ] Add proper type assertions where needed
### 20. Dependency Security
**Status**: No automated security scanning
**Effort**: 1 day
- [ ] Add `pnpm audit` to CI/CD
- [ ] Set up Dependabot or Renovate
- [ ] Configure automated dependency updates
- [ ] Add security update review process
- [ ] Document dependency update policy
### 21. Performance Optimizations
**Status**: No caching, connection pooling, or timeouts
**Effort**: 1 week
- [ ] Add Redis caching layer
- [ ] Implement caching middleware
- [ ] Add connection pooling for databases
- [ ] Add request timeouts
- [ ] Add circuit breakers for external services
- [ ] Implement request queuing
- [ ] Add response compression
- [ ] Optimize database queries
### 22. Service Communication
**Status**: No documented service-to-service patterns
**Effort**: 2-3 days
- [ ] Document service-to-service communication patterns
- [ ] Add service discovery mechanism
- [ ] Consider API gateway pattern
- [ ] Add service mesh (optional)
- [ ] Document inter-service authentication
### 23. Infrastructure as Code
**Status**: Terraform/K8s configs may be incomplete
**Effort**: 2-3 weeks
- [ ] Review and complete Terraform configurations
- [ ] Review and complete Kubernetes manifests
- [ ] Add Helm charts for all services
- [ ] Complete API gateway configurations
- [ ] Add infrastructure testing
- [ ] Document infrastructure setup
### 24. Brand Services Implementation
**Status**: Brand services exist but may be incomplete
**Effort**: TBD
- [ ] Review `services/omnis-brand` implementation
- [ ] Review `services/arromis-brand` implementation
- [ ] Complete any missing functionality
- [ ] Add tests for brand services
### 25. MCP Apps Implementation
**Status**: MCP apps exist but may be incomplete
**Effort**: TBD
- [ ] Review `apps/mcp-members` implementation
- [ ] Review `apps/mcp-legal` implementation
- [ ] Complete any missing functionality
- [ ] Add tests for MCP apps
---
## Low Priority / Nice to Have
### 26. Portal Apps Enhancement
**Status**: Portal apps exist but may need features
**Effort**: TBD
- [ ] Review `apps/portal-public` features
- [ ] Review `apps/portal-internal` features
- [ ] Add missing UI components
- [ ] Enhance user experience
- [ ] Add E2E tests
### 27. Documentation Enhancements
**Status**: Good documentation, could use more examples
**Effort**: 1 week
- [ ] Add more code examples to README files
- [ ] Add architecture diagrams
- [ ] Add sequence diagrams for workflows
- [ ] Add deployment guides
- [ ] Add troubleshooting guides
- [ ] Add developer onboarding guide
### 28. Load Testing
**Status**: No load testing setup
**Effort**: 1 week
- [ ] Set up load testing framework (k6, Artillery, etc.)
- [ ] Create load test scenarios
- [ ] Add load tests to CI/CD
- [ ] Document performance benchmarks
- [ ] Set up performance monitoring
### 29. Dependency Version Strategy
**Status**: No documented version locking strategy
**Effort**: 1 day
- [ ] Document dependency version policy
- [ ] Decide on exact vs. semver ranges
- [ ] Update package.json files accordingly
- [ ] Document update process
### 30. Git Practices
**Status**: Good commit guidelines, could enhance
**Effort**: 1 day
- [ ] Set up branch protection rules
- [ ] Require PR reviews
- [ ] Require CI checks to pass
- [ ] Require up-to-date branches
---
## Implementation Details by Component
### Packages
#### `packages/auth`
- [ ] Complete OIDC token exchange
- [ ] Complete DID resolution and verification
- [ ] Complete eIDAS signature operations
- [ ] Add comprehensive tests
- [ ] Add JSDoc documentation
#### `packages/crypto`
- [ ] Implement KMS client (AWS KMS or GCP KMS)
- [ ] Add encryption/decryption
- [ ] Add signing/verification
- [ ] Add comprehensive tests
- [ ] Add JSDoc documentation
#### `packages/storage`
- [ ] Implement S3/GCS storage client
- [ ] Implement WORM storage mode
- [ ] Add presigned URL generation
- [ ] Add comprehensive tests
- [ ] Add JSDoc documentation
#### `packages/workflows`
- [ ] Implement intake workflow (Temporal/Step Functions)
- [ ] Implement review workflow (Temporal/Step Functions)
- [ ] Add workflow orchestration
- [ ] Add comprehensive tests
- [ ] Add JSDoc documentation
#### `packages/schemas`
- [ ] Ensure all API schemas are defined
- [ ] Add schema validation tests
- [ ] Generate OpenAPI specs
- [ ] Document schema usage
#### `packages/shared` (NEW)
- [ ] Create package structure
- [ ] Add error handling
- [ ] Add validation utilities
- [ ] Add security middleware
- [ ] Add logging utilities
- [ ] Add environment validation
#### `packages/database` (NEW)
- [ ] Create package structure
- [ ] Add PostgreSQL client
- [ ] Add migration utilities
- [ ] Add connection pooling
- [ ] Add query builders
### Services
#### `services/identity`
- [ ] Implement VC issuance endpoint
- [ ] Implement VC verification endpoint
- [ ] Implement document signing endpoint
- [ ] Add error handling
- [ ] Add input validation
- [ ] Add security middleware
- [ ] Add database integration
- [ ] Add tests
- [ ] Add API documentation
#### `services/intake`
- [ ] Implement document ingestion endpoint
- [ ] Add OCR processing
- [ ] Add document classification
- [ ] Add routing logic
- [ ] Add error handling
- [ ] Add input validation
- [ ] Add security middleware
- [ ] Add database integration
- [ ] Add tests
- [ ] Add API documentation
#### `services/finance`
- [ ] Implement ledger entry endpoint
- [ ] Implement payment processing endpoint
- [ ] Add payment gateway integration
- [ ] Add error handling
- [ ] Add input validation
- [ ] Add security middleware
- [ ] Add database integration
- [ ] Add tests
- [ ] Add API documentation
#### `services/dataroom`
- [ ] Implement deal room creation
- [ ] Implement deal room retrieval
- [ ] Implement document upload
- [ ] Implement presigned URL generation
- [ ] Add access control
- [ ] Add error handling
- [ ] Add input validation
- [ ] Add security middleware
- [ ] Add database integration
- [ ] Add tests
- [ ] Add API documentation
### Apps
#### `apps/portal-public`
- [ ] Review and complete implementation
- [ ] Add E2E tests
- [ ] Add component tests
- [ ] Enhance UI/UX
#### `apps/portal-internal`
- [ ] Review and complete implementation
- [ ] Add E2E tests
- [ ] Add component tests
- [ ] Enhance UI/UX
#### `apps/mcp-members`
- [ ] Review and complete implementation
- [ ] Add tests
#### `apps/mcp-legal`
- [ ] Review and complete implementation
- [ ] Add tests
### Infrastructure
#### `infra/terraform`
- [ ] Review and complete configurations
- [ ] Add all required resources
- [ ] Add outputs
- [ ] Add documentation
#### `infra/k8s`
- [ ] Review and complete manifests
- [ ] Add Helm charts
- [ ] Add overlays for all environments
- [ ] Add documentation
#### `infra/gateways`
- [ ] Review and complete configurations
- [ ] Add API gateway setup
- [ ] Add WAF rules
- [ ] Add documentation
#### `infra/cicd`
- [ ] Review and complete CI/CD templates
- [ ] Add reusable workflows
- [ ] Add documentation
---
## Summary Statistics
### By Priority
- **Critical**: 7 major areas, ~50+ individual tasks
- **High Priority**: 8 major areas, ~40+ individual tasks
- **Medium Priority**: 10 major areas, ~30+ individual tasks
- **Low Priority**: 5 major areas, ~15+ individual tasks
### Estimated Effort
- **Critical Issues**: 8-12 weeks
- **High Priority**: 4-6 weeks
- **Medium Priority**: 6-8 weeks
- **Low Priority**: 3-4 weeks
- **Total Estimated Effort**: 21-30 weeks (5-7.5 months)
### Key Blockers
1. No tests (blocks CI/CD confidence)
2. Incomplete implementations (blocks functionality)
3. Missing security (blocks production deployment)
4. No error handling (blocks user experience)
5. No database integration (blocks data persistence)
---
## Recommended Implementation Order
### Phase 1: Foundation (Week 1-2)
1. Fix ESLint configuration
2. Create shared package
3. Add error handling middleware
4. Add input validation
5. Add security middleware
6. Add environment variable validation
7. Add basic tests for critical packages
### Phase 2: Core Functionality (Week 3-6)
1. Implement storage client
2. Implement KMS client
3. Add database integration
4. Implement service endpoints
5. Add structured logging
6. Add comprehensive tests
### Phase 3: Quality & Observability (Week 7-10)
1. Add comprehensive test coverage
2. Add monitoring and observability
3. Add API documentation
4. Implement workflows
5. Add E2E tests
### Phase 4: Production Ready (Week 11-14)
1. Performance optimization
2. Security hardening
3. Complete documentation
4. Load testing
5. Infrastructure completion
---
## Notes
- This list is comprehensive but may not be exhaustive
- Some tasks may be discovered during implementation
- Priorities may shift based on business requirements
- Estimated efforts are rough approximations
- Some tasks can be done in parallel
- Regular reviews should be conducted to update this list
---
## Next Steps
1. Review this list with the team
2. Prioritize based on business needs
3. Create GitHub issues for each task
4. Assign tasks to team members
5. Start with Phase 1 tasks
6. Update this document as tasks are completed
Reference in New Issue View Git Blame Copy Permalink