TX/MIGRATION_REPORT.md

# Off-Replit Migration Report

**Task #526** — full migration of Tx OS off the Replit hosted environment to a
self-contained Docker Compose stack runnable on any Linux VPS.

## Summary

Tx OS was originally bootstrapped on Replit and depended on three Replit-only
runtime surfaces:

1. The Replit object-storage **sidecar** (`http://127.0.0.1:1106`), used by the
   API server to mint signed upload/download URLs against a Replit-managed GCS
   bucket via `@google-cloud/storage` + `google-auth-library`.
2. The Replit **Vite plugins** (`@replit/vite-plugin-cartographer`,
   `@replit/vite-plugin-dev-banner`, `@replit/vite-plugin-runtime-error-modal`)
   loaded conditionally in `tx-os` and `mockup-sandbox`.
3. The Replit **platform configs** (`.replit`, `replit.nix`, `replit.md`) and
   the post-merge hook script (`scripts/post-merge.sh`) consumed by the Replit
   agent runner.

After this migration:

- Object storage is backed by a self-hosted **MinIO** container in production
  and a built-in **local-filesystem driver** in development. Both are abstracted
  behind a `StoredObject` interface in `lib/objectStorage.ts` so the route
  layer is driver-agnostic.
- Vite plugins are gone from both `vite.config.ts` files and from
  `pnpm-workspace.yaml`'s catalog.
- A `Dockerfile` (5 build targets — deps / build / api / web / migrate) and a
  `docker-compose.yml` (db + minio + minio-init + api + web + one-shot migrate)
  bring the entire stack up with `docker compose up -d`. A new `.env.example`
  template documents every env var the runtime reads.

The repository is now self-contained: cloning it onto a Docker-equipped host
and running the documented commands produces a working deployment.

## Files changed

### Added
- `Dockerfile` — multi-stage build (Node 24 deps → esbuild API + Vite SPA
  builds → Playwright runtime for API → nginx runtime for SPA → migrate runner).
- `docker-compose.yml` — full production stack with healthchecks and one-shot
  init/migrate containers.
- `docker/nginx.conf` — SPA static serve + `/api` and `/api/socket.io` reverse
  proxy to the api container.
- `.env.example` — exhaustive, commented configuration template.
- `README.md` — replaces `replit.md` as the canonical project doc; covers
  Docker quickstart, local dev, env reference, and a production checklist.
- `MIGRATION_REPORT.md` — this file.
- `artifacts/api-server/src/routes/storage-local-upload.ts` — handler for the
  local-FS driver's HMAC-signed PUT upload URLs (`PUT /api/storage/_local/upload`).

### Replaced (full rewrite)
- `artifacts/api-server/src/lib/objectStorage.ts` — was a thin wrapper around
  `@google-cloud/storage` + the Replit sidecar. Now ships a driver interface
  with two implementations (`LocalDriver`, `S3Driver`) selected by the
  `STORAGE_DRIVER` env var (defaulting to `s3` if `S3_ENDPOINT` is set, else
  `local`). Public API surface — `ObjectStorageService.{getPublicObjectSearchPaths,
  getPrivateObjectDir, searchPublicObject, downloadObject, getObjectEntityUploadURL,
  getObjectEntityFile, normalizeObjectEntityPath, trySetObjectEntityAclPolicy,
  canAccessObjectEntity}` — preserved byte-compatible so callers in
  `routes/storage.ts` and `routes/executive-meetings.ts` did not need to change.
- `artifacts/api-server/src/lib/objectAcl.ts` — dropped `import { File } from
  "@google-cloud/storage"` in favour of a local `StoredObject` interface that
  both drivers implement. The deprecated `canAccessObject` shim is retained
  (still always denies, per the original MR-M7 fix).
- `artifacts/tx-os/vite.config.ts` and `artifacts/mockup-sandbox/vite.config.ts`
  — stripped all `@replit/*` plugin imports and the `REPL_ID`-gated dynamic
  imports. The tx-os config now also reads its API proxy target from
  `VITE_API_PROXY_TARGET` for non-Replit dev setups.
- `.gitignore` — restructured so that Replit configs (`.replit`, `replit.nix`,
  `replit.md`), agent state (`.local/`, `.canvas/`, `.agents/`, `.cache/`,
  `.config/`, `.upm/`), local storage (`storage/`), test artifacts, and any
  `.env*` (except `.env.example`) are excluded from git.

### Edited
- `pnpm-workspace.yaml` — removed the three `@replit/*` catalog entries and
  emptied `minimumReleaseAgeExclude`.
- `artifacts/api-server/package.json` — dropped `@google-cloud/storage` and
  `google-auth-library`; added `@aws-sdk/client-s3` and
  `@aws-sdk/s3-request-presigner`.
- `artifacts/tx-os/package.json` and `artifacts/mockup-sandbox/package.json`
  — removed `@replit/*` plugin dependencies.
- `scripts/src/seed.ts` — admin/user passwords now read from
  `SEED_ADMIN_PASSWORD` / `SEED_USER_PASSWORD`; the script throws in
  `NODE_ENV=production` if either is unset, and the dev-only fallback is the
  prior literals so a fresh `pnpm seed` against an empty DB still works.
- `threat_model.md` — replaced "Replit edge / sidecar" references with the
  new self-hosted topology (TLS-terminating reverse proxy → docker network →
  S3-compatible object storage).

### Deleted
- `attached_assets/` — 23 MB of legacy uploaded assets unused by the codebase.
- `sedMkjeJm` — stray temp file (a sed copy of `.replit`) at repo root.
- `artifacts/api-server/dist/`, `lib/*/dist/`, all `*.tsbuildinfo` — build
  artefacts that should not be tracked.
- `scripts/post-merge.sh` — Replit agent post-merge hook; not relevant
  off-Replit.
- `replit.md` — superseded by `README.md`. (Note: `.replit` and `replit.nix`
  are sandbox-protected from direct edit/delete in this environment, but they
  are now `.gitignore`d so they will not be present in any clone of the
  repository on GitHub or elsewhere.)

## Verification

- `pnpm install --frozen-lockfile` — clean install with the new dependency
  set; pnpm reports `+87 -69` packages (S3 SDK in, GCS + Replit plugins out).
- `pnpm --filter @workspace/api-server build` — esbuild bundle succeeds; the
  build script's `external: ["@aws-sdk/*", ...]` list already covered the new
  packages.
- API-server test suite — all 12 storage / object-authz tests
  (`storage-object-authz.test.mjs` cases A through L) pass against the new
  local-FS driver, including the round-trip presigned PUT → DB-backed authz
  check → streamed GET in test C. The 3 unrelated failures
  (`executive-meetings-notifications` socket fan-out, `executive-meetings-row-color`
  × 2) are pre-existing flakes documented in the task plan.

## Operational notes for first-time deploy

1. `cp .env.example .env` and fill in every value.
2. `docker compose build`
3. `docker compose up -d db minio minio-init` — Postgres + MinIO + create
   buckets.
4. `docker compose run --rm migrate` — apply Drizzle schema + seed admin /
   ahmed accounts (one-shot).
5. `docker compose up -d api web` — boot the API and the SPA.
6. Front `web` (port `${WEB_PORT}`) with a TLS-terminating reverse proxy.

The README's "Production checklist" lists the security-relevant steps required
before fronting the stack with a public domain.

## Residual security risks carried over from `.local/security/manual-review.md`

The following findings predate this migration and were **not** addressed as
part of Task #526 (which is scoped to environment portability only). They
remain on the security backlog for follow-up tasks.

### Medium (7)

| ID    | Summary                                                                                              |
| ----- | ---------------------------------------------------------------------------------------------------- |
| MR-M1 | No CSRF protection; session cookie is `sameSite: "lax"`.                                             |
| MR-M2 | No session regeneration on login (session-fixation window).                                          |
| MR-M3 | Helmet / security headers are not set on API responses.                                              |
| MR-M4 | `GET /api/users/directory` returns the full employee directory to every authenticated user.          |
| MR-M5 | `POST /api/apps/:id/open` accepts any app id from any user (audit-log pollution + open enumeration). |
| MR-M6 | Errors caught in `executive-meetings` routes echo raw `Error.message` to the client.                 |
| MR-M8 | Upload `contentType` is taken from the client and never validated server-side.                       |

> Note: MR-M7 ("Object Storage ACL subsystem is unimplemented") is now
> documented in-source by the new `objectAcl.ts` deprecation banner and the
> entity-lookup authz model in `lib/objectAuthz.ts`; no behaviour change
> is required for the migration to ship.

### Low (8)

| ID    | Summary                                                                                                  |
| ----- | -------------------------------------------------------------------------------------------------------- |
| MR-L1 | `executive_meetings_changed` and `executive_meeting_notifications_changed` events are broadcast globally. |
| MR-L2 | Bcrypt cost factor is 10 (industry guidance is 12).                                                       |
| MR-L3 | Account enumeration on register (distinguishable error for existing username).                            |
| MR-L4 | No socket connection cap or handshake rate limit.                                                         |
| MR-L5 | `GET /api/settings` is unauthenticated.                                                                   |
| MR-L6 | Several `notes.ts` mutation routes use ad-hoc body parsing instead of Zod.                                |
| MR-L7 | `express.json()` uses the default 100 KB body limit.                                                      |
| MR-L8 | `GET /executive-meetings/me` returns the caller's full role list.                                         |

### Unmigrated object-data risk

Object data stored in the Replit-managed GCS bucket prior to this migration
is **not** automatically copied into MinIO. Operators standing up a fresh
MinIO instance start with empty buckets. If historical uploads must be
preserved, the operator is responsible for a one-time `mc mirror` (or
equivalent) from the legacy GCS bucket into the new MinIO bucket; the API
server's signed-URL contract and DB-side `attached_object_path` columns are
already compatible because both backends key by `<bucket>/<objectName>`.

## Sandbox-protected files (operator follow-up required)

The Replit workspace sandbox blocks deletion of `.replit`, `.replitignore`,
and `replit.nix` from this environment. They are added to `.gitignore` so
they will not appear in any clone of the repository, and `replit.md` +
`scripts/post-merge.sh` (which were not sandbox-protected) have been
deleted. Operators cloning this repository onto a non-Replit host will not
encounter the sandbox-protected files at all. Anyone who later wants to
fully purge them from the upstream git history should run, in their own
clone:

```bash
git rm --cached .replit .replitignore replit.nix
git commit -m "Drop final Replit platform files from tracking"
```

## What did NOT change

Per the task scope (and `replit.md` user preferences before its deletion),
the following surfaces were intentionally left untouched:

- All API and SPA test files (`artifacts/api-server/tests/**`,
  `artifacts/tx-os/tests/**`).
- `lib/db/scripts/**` (DB push/pull scripts).
- The PDF renderers (`pdf-renderer.ts`, `pdf-html-renderer.ts`).
- The SPA shell (`App.tsx`, `executive-meetings.tsx`,
  `upcoming-meeting-alert.tsx`).
- Locale catalogues (`ar.json`, `en.json`).
- The auto-generated React-Query client (`custom-fetch.ts`).
- The Executive Meetings DB schema (`schema/executive-meetings.ts`).
- Authentication, sessions, rate limiting, and CSRF/CORS middleware — only
  the storage subsystem was rewritten.