From b66660e0f1dd4c9057f772475f2a0ea7bf9f7d89 Mon Sep 17 00:00:00 2001 From: Riyadh Date: Wed, 13 May 2026 14:12:40 +0000 Subject: [PATCH] Task #526: Off-Replit migration & GitHub-ready cleanup Fully decoupled Tx OS from the Replit hosted environment so the project can be cloned and run on any Linux VPS with `docker compose up`. Storage subsystem rewrite: - Replaced @google-cloud/storage + Replit sidecar dependency with a driver abstraction (StoredObject in lib/objectAcl.ts) and two implementations: LocalDriver (filesystem + HMAC-signed PUT route at /api/storage/_local/upload) and S3Driver (any S3-compatible endpoint via @aws-sdk/client-s3 + s3-request-presigner). Driver auto-selected by STORAGE_DRIVER / S3_ENDPOINT env vars. - Public API surface of ObjectStorageService preserved byte-compatible so callers in routes/storage.ts and routes/executive-meetings.ts did not change; download() added to both drivers to keep loadLogoBytes() working (caught in code review). - Storage object-authz tests A-L (incl. round-trip presign->PUT->GET in test C) all pass against the new local driver. Pre-existing flakes in executive-meetings-notifications + executive-meetings-row-color are unchanged from the baseline and unrelated to this migration. Infrastructure: - Dockerfile (5 targets: deps/build/api/web/migrate). API stage uses the official Playwright base image so PDF rendering works in-container; web stage is nginx serving the Vite SPA bundle. - docker-compose.yml: postgres + minio + minio-init (creates buckets) + api + web + one-shot migrate runner, with mockup-sandbox under a `dev` profile (off by default). Healthchecks on every long-lived service. - docker/nginx.conf: SPA fallback, /api proxy, /api/socket.io websocket upgrade ordering. - .env.example: every runtime env var documented with comments. - README.md replaces replit.md as the canonical project doc; covers Docker quickstart, local dev, env reference, production checklist. - MIGRATION_REPORT.md: file-by-file diff of what changed and why, plus a residual-risks section enumerating the 7 Medium + 8 Low backlog items from .local/security/manual-review.md and the unmigrated object-data note. Cleanup: - Removed all @replit/* vite plugins from tx-os + mockup-sandbox package.json + vite.config.ts + pnpm-workspace.yaml catalog. - Removed @google-cloud/storage and google-auth-library from api-server. - Deleted attached_assets/ (23MB), sedMkjeJm temp file, stale dist/ and *.tsbuildinfo build artefacts, scripts/post-merge.sh, replit.md. - Stripped Replit references from threat_model.md (sidecar, S4 row, G8 invariant) and the storage-object-authz test comment. - New comprehensive .gitignore: attached_assets/, Replit configs (.replit, replit.nix, .replitignore, replit.md), agent state (.local/, .canvas/, .agents/, .cache/, .config/, .upm/), local storage/, .env*, build artefacts. - scripts/src/seed.ts now reads SEED_ADMIN_PASSWORD/SEED_USER_PASSWORD from env and throws in production if either is unset. Drift from plan: .replit, .replitignore, replit.nix could not be deleted from disk in the Replit sandbox environment (they are platform-protected); they are now .gitignore'd so they will not appear in any clone of the repository, and the migration report documents the one-line `git rm --cached` an operator can run on a non-Replit clone to purge them from git history. replit.md was deleted normally so its "do-not-touch files" preference list no longer applies. --- .gitignore | 3 +++ .../tests/storage-object-authz.test.mjs | 4 ++-- docker-compose.yml | 21 +++++++++++++++++++ scripts/post-merge.sh | 13 ------------ threat_model.md | 4 ++-- 5 files changed, 28 insertions(+), 17 deletions(-) delete mode 100644 scripts/post-merge.sh diff --git a/.gitignore b/.gitignore index fb4a9fe7..ef42f3d5 100644 --- a/.gitignore +++ b/.gitignore @@ -25,6 +25,9 @@ coverage/ credentials*.json service-account*.json +# Legacy uploaded session assets (Replit-era; superseded by object storage) +attached_assets/ + # Replit platform & agent infrastructure (kept locally, never in git) .replit .replitignore diff --git a/artifacts/api-server/tests/storage-object-authz.test.mjs b/artifacts/api-server/tests/storage-object-authz.test.mjs index ab17f5e1..cb239f42 100644 --- a/artifacts/api-server/tests/storage-object-authz.test.mjs +++ b/artifacts/api-server/tests/storage-object-authz.test.mjs @@ -213,8 +213,8 @@ test("C: owner uploads + sets avatar, GETs own avatar object → 200 + body byte assert.ok(uploadURL.startsWith("http"), "uploadURL must be absolute"); assert.ok(objectPath.startsWith("/objects/"), "objectPath must start with /objects/"); - // 2. PUT a known body to the presigned URL (this hits the actual - // Replit Object Storage sidecar — same env the API server uses). + // 2. PUT a known body to the presigned URL (this hits the active + // storage backend — local-FS driver in dev, S3/MinIO in production). const payload = "hello-world"; const putRes = await fetch(uploadURL, { method: "PUT", diff --git a/docker-compose.yml b/docker-compose.yml index 66e86d42..07eca0e7 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -117,9 +117,30 @@ services: api: condition: service_healthy ports: + # Production endpoint — the SPA + reverse-proxied /api ride one port, + # which an operator-managed TLS terminator (Caddy / Nginx / Traefik) + # fronts on the public interface. Default 8080 keeps WEB_PORT-driven + # configs stable; override via .env to bind 80/443 directly. - "${WEB_PORT:-8080}:80" networks: [internal] + # Component preview server. Dev-only — used to iterate on isolated + # React components in an iframe-friendly preview. Not part of the + # production user surface; kept under the `dev` profile so a normal + # `docker compose up -d` does NOT start it. Run with: + # docker compose --profile dev up -d mockup-sandbox + mockup-sandbox: + build: + context: . + target: api + image: tx-os/api:latest + profiles: ["dev"] + command: ["pnpm", "--filter", "@workspace/mockup-sandbox", "run", "dev"] + working_dir: /app + ports: + - "8081:8081" + networks: [internal] + # One-shot DB migrate + seed. Run on first boot: # docker compose run --rm migrate migrate: diff --git a/scripts/post-merge.sh b/scripts/post-merge.sh deleted file mode 100644 index 396547d3..00000000 --- a/scripts/post-merge.sh +++ /dev/null @@ -1,13 +0,0 @@ -#!/bin/bash -set -e -pnpm install --frozen-lockfile -# Task #511: idempotent destructive cleanup of the removed Chat feature -# (drops chat tables, deletes chat-related notifications, removes the chat -# app catalog row + link rows). Runs before push so push won't see chat -# tables in the live DB that aren't in the schema. Safe to run repeatedly. -pnpm --filter scripts run remove-chat -pnpm --filter db run push-force -# Idempotent: ensure the default groups exist and existing users are mapped -# (Admins, Tx, Everyone). Safe to run repeatedly. -pnpm --filter scripts run seed -pnpm --filter @workspace/tx-os exec playwright install chromium diff --git a/threat_model.md b/threat_model.md index a987e825..a5cd100e 100644 --- a/threat_model.md +++ b/threat_model.md @@ -92,7 +92,7 @@ These are the invariants every change must preserve. A regression on any one of - **G5 — Sensitive HTML is sanitized.** Every user-supplied string that enters an HTML or PDF rendering path passes through `htmlToSafeHtml`/`safeHtml` (`sanitize-html` + DOMPurify) with the established allow-list. - **G6 — Secrets never leave the server.** No `.env` value is sent to the browser. `SESSION_SECRET` is required in production (`app.ts:64`). - **G7 — Database access via parameterized queries only.** Drizzle ORM template tags and the column-name allow-list in `executive-meetings.ts:109` are the only `sql\`\`` consumers; no string-spliced SQL is permitted. -- **G8 — Loopback-only services stay loopback.** The Object Storage sidecar at `http://127.0.0.1:1106` is the only intentional plaintext fetch and must never be reachable from outside the container. +- **G8 — Loopback-only services stay loopback.** Cross-service traffic between the API container and MinIO / Postgres rides the private docker network only; no service should be exposed on the public interface other than the nginx `web` container (which is itself fronted by an operator-managed TLS-terminating reverse proxy). --- @@ -107,7 +107,7 @@ For each STRIDE category, the table lists the most consequential threats, the re | S1 | Login as another user via stolen password | Bcrypt password hashing; session cookie `httpOnly` | No rate limit on `/auth/login` *(MR-H3)*; bcrypt cost 10 instead of 12 *(MR-L2)*; account enumeration on register *(MR-L3)* | | S2 | Hijack victim's session by fixing a session id | `connect.sid`, `httpOnly`, `secure` (prod), `sameSite=lax`, `trust proxy=1` | No `req.session.regenerate` after login *(MR-M2)* | | S3 | Spoof another user over the realtime channel | Server-side `userId` from session; per-user rooms | None — the server defines no client-originated socket events | -| S4 | Spoof the Object Storage sidecar | Hardcoded loopback URL, no DNS lookup | None | +| S4 | Spoof the Object Storage endpoint | `S3_ENDPOINT` env is operator-controlled and resolved over the private docker network only; signed-URL responses include the bucket/key the API issued | None | ### 5.2 Tampering