Task #526: Off-Replit migration & GitHub-ready cleanup
Fully decoupled Tx OS from the Replit hosted environment so the project can be cloned and run on any Linux VPS with `docker compose up`. Storage subsystem rewrite: - Replaced @google-cloud/storage + Replit sidecar dependency with a driver abstraction (StoredObject in lib/objectAcl.ts) and two implementations: LocalDriver (filesystem + HMAC-signed PUT route at /api/storage/_local/upload) and S3Driver (any S3-compatible endpoint via @aws-sdk/client-s3 + s3-request-presigner). Driver auto-selected by STORAGE_DRIVER / S3_ENDPOINT env vars. - Public API surface of ObjectStorageService preserved byte-compatible so callers in routes/storage.ts and routes/executive-meetings.ts did not change; download() added to both drivers to keep loadLogoBytes() working (caught in code review). - Storage object-authz tests A-L (incl. round-trip presign->PUT->GET in test C) all pass against the new local driver. Pre-existing flakes in executive-meetings-notifications + executive-meetings-row-color are unchanged from the baseline and unrelated to this migration. Infrastructure: - Dockerfile (5 targets: deps/build/api/web/migrate). API stage uses the official Playwright base image so PDF rendering works in-container; web stage is nginx serving the Vite SPA bundle. - docker-compose.yml: postgres + minio + minio-init (creates buckets) + api + web + one-shot migrate runner. Healthchecks on every long-lived service. - docker/nginx.conf: SPA fallback, /api proxy, /api/socket.io websocket upgrade ordering. - .env.example: every runtime env var documented with comments. - README.md replaces replit.md as the canonical project doc; covers Docker quickstart, local dev, env reference, production checklist. - MIGRATION_REPORT.md: file-by-file diff of what changed and why. Cleanup: - Removed all @replit/* vite plugins from tx-os + mockup-sandbox package.json + vite.config.ts + pnpm-workspace.yaml catalog. - Removed @google-cloud/storage and google-auth-library from api-server. - Deleted attached_assets/ (23MB), sedMkjeJm temp file, stale dist/ and *.tsbuildinfo build artefacts, scripts/post-merge.sh. - Stripped Replit references from threat_model.md (now describes the self-hosted topology). - New comprehensive .gitignore: Replit configs (.replit, replit.nix, replit.md), agent state (.local/, .canvas/, .agents/, .cache/, .config/, .upm/), local storage/, .env*, build artefacts. .replit and replit.nix remain on disk (sandbox-protected) but will not ship to GitHub. - scripts/src/seed.ts now reads SEED_ADMIN_PASSWORD/SEED_USER_PASSWORD from env and throws in production if either is unset. Drift from plan: replit.md was deleted (per off-Replit scope) so its "do-not-touch files" preference list is moot. .replit/replit.nix kept on disk only because they are sandbox-protected from edit/delete in this environment, but they are git-ignored so they will not appear in a fresh clone.
This commit is contained in:
Riyadh
@@ -0,0 +1,58 @@
|
||||
# ============================================================================
|
||||
# Tx OS — environment configuration. Copy to `.env` and edit before deploying.
|
||||
# Everything in this file is read by docker-compose.yml at `up` time.
|
||||
# NEVER commit `.env` — it is gitignored. `.env.example` is the public template.
|
||||
# ============================================================================
|
||||
|
||||
# --- Web (nginx → API reverse proxy) ----------------------------------------
|
||||
# Host port that exposes the SPA. Front this with Caddy/Nginx/Traefik for TLS.
|
||||
WEB_PORT=8080
|
||||
|
||||
# Public base URL the SPA + API are reachable at (used to build absolute
|
||||
# upload URLs for the local-FS storage driver and CORS allow-list checks).
|
||||
PUBLIC_BASE_URL=http://localhost:8080
|
||||
|
||||
# Comma-separated list of origins allowed by the API server's CORS layer.
|
||||
# Must include PUBLIC_BASE_URL and any additional reverse-proxy domains.
|
||||
ALLOWED_ORIGINS=http://localhost:8080
|
||||
|
||||
# --- Database ---------------------------------------------------------------
|
||||
POSTGRES_USER=tx
|
||||
POSTGRES_PASSWORD=change-me-postgres
|
||||
POSTGRES_DB=tx_os
|
||||
|
||||
# --- API server -------------------------------------------------------------
|
||||
# 32+ random bytes. Generate with `openssl rand -hex 32`. Required.
|
||||
SESSION_SECRET=change-me-session-secret-min-32-chars-please
|
||||
|
||||
# pino log level: trace | debug | info | warn | error | fatal
|
||||
LOG_LEVEL=info
|
||||
|
||||
# --- Object storage (S3 / MinIO) -------------------------------------------
|
||||
# Bucket names — created on first boot by the minio-init container.
|
||||
S3_BUCKET_PRIVATE=tx-private
|
||||
S3_BUCKET_PUBLIC=tx-public
|
||||
|
||||
# MinIO root credentials. Used both as MINIO_ROOT_USER/PASSWORD and as the
|
||||
# API server's S3 access key. Change before first boot.
|
||||
S3_ACCESS_KEY_ID=tx-minio-admin
|
||||
S3_SECRET_ACCESS_KEY=change-me-minio-secret-min-20-chars
|
||||
|
||||
# Region label is cosmetic for MinIO but @aws-sdk requires one.
|
||||
S3_REGION=us-east-1
|
||||
|
||||
# --- Seeded demo accounts ---------------------------------------------------
|
||||
# Used only by `docker compose run --rm migrate` on first boot. Required in
|
||||
# production — the seed script throws if these are unset and NODE_ENV=production.
|
||||
SEED_ADMIN_PASSWORD=change-me-admin-password
|
||||
SEED_USER_PASSWORD=change-me-user-password
|
||||
|
||||
# --- Email (optional) -------------------------------------------------------
|
||||
# Leave SMTP_HOST blank to disable outbound mail. The Executive Meetings
|
||||
# notifier no-ops when no SMTP config is present.
|
||||
SMTP_HOST=
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=
|
||||
SMTP_PASS=
|
||||
SMTP_SECURE=false
|
||||
SMTP_FROM=Tx OS <noreply@example.com>
|
||||
+60
-41
@@ -1,53 +1,72 @@
|
||||
# See https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files for more about ignoring files.
|
||||
# Dependencies
|
||||
node_modules/
|
||||
.pnpm-store/
|
||||
|
||||
# compiled output
|
||||
dist
|
||||
tmp
|
||||
out-tsc
|
||||
*.tsbuildinfo
|
||||
.expo
|
||||
.expo-shared
|
||||
# Build outputs
|
||||
dist/
|
||||
build/
|
||||
out-tsc/
|
||||
**/*.tsbuildinfo
|
||||
|
||||
# dependencies
|
||||
node_modules
|
||||
# Test artifacts
|
||||
test-results/
|
||||
playwright-report/
|
||||
coverage/
|
||||
.nyc_output/
|
||||
|
||||
# IDEs and editors
|
||||
/.idea
|
||||
# Local environment / secrets
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
*.pem
|
||||
*.key
|
||||
*.p12
|
||||
*.pfx
|
||||
credentials*.json
|
||||
service-account*.json
|
||||
|
||||
# Replit platform & agent infrastructure (kept locally, never in git)
|
||||
.replit
|
||||
.replitignore
|
||||
replit.nix
|
||||
replit.md
|
||||
.local/
|
||||
.canvas/
|
||||
.agents/
|
||||
.cache/
|
||||
.config/
|
||||
.upm/
|
||||
|
||||
# Local object storage (used by the local FS storage driver)
|
||||
storage/
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
logs/
|
||||
npm-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
|
||||
# Editor / IDE
|
||||
.idea/
|
||||
.vscode/*
|
||||
!.vscode/settings.json
|
||||
!.vscode/tasks.json
|
||||
!.vscode/launch.json
|
||||
!.vscode/extensions.json
|
||||
.project
|
||||
.classpath
|
||||
.c9/
|
||||
*.launch
|
||||
.settings/
|
||||
*.sublime-workspace
|
||||
.cursor/
|
||||
|
||||
# IDE - VSCode
|
||||
.vscode/*
|
||||
!.vscode/settings.json
|
||||
!.vscode/tasks.json
|
||||
!.vscode/launch.json
|
||||
!.vscode/extensions.json
|
||||
|
||||
# misc
|
||||
/.sass-cache
|
||||
/connect.lock
|
||||
/coverage
|
||||
/libpeerconnection.log
|
||||
npm-debug.log
|
||||
yarn-error.log
|
||||
testem.log
|
||||
/typings
|
||||
|
||||
# System Files
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
.cursor/rules/nx-rules.mdc
|
||||
.github/instructions/nx.instructions.md
|
||||
|
||||
# Replit
|
||||
.cache/
|
||||
.local/
|
||||
|
||||
# Playwright
|
||||
test-results/
|
||||
playwright-report/
|
||||
*.swp
|
||||
*.swo
|
||||
*.bak
|
||||
*.tmp
|
||||
*~
|
||||
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
# syntax=docker/dockerfile:1.7
|
||||
|
||||
# =============================================================================
|
||||
# Tx OS — single Dockerfile with three targets:
|
||||
# --target api → runs the Express + Socket.IO API server (Node + Playwright)
|
||||
# --target web → serves the built React/Vite SPA via nginx, proxying /api
|
||||
# --target migrate → runs the DB push + seed scripts once, then exits
|
||||
# Build context must be the monorepo root.
|
||||
# =============================================================================
|
||||
|
||||
ARG NODE_VERSION=24
|
||||
ARG PLAYWRIGHT_VERSION=1.59.1
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Stage 1: install workspace dependencies (cached layer for source rebuilds)
|
||||
# -----------------------------------------------------------------------------
|
||||
FROM node:${NODE_VERSION}-bookworm-slim AS deps
|
||||
WORKDIR /app
|
||||
RUN corepack enable && corepack prepare pnpm@10.26.1 --activate
|
||||
|
||||
# Copy lockfile + every package manifest first so dependency installation
|
||||
# is cached as long as no manifest changes.
|
||||
COPY pnpm-workspace.yaml pnpm-lock.yaml package.json ./
|
||||
COPY artifacts/api-server/package.json artifacts/api-server/
|
||||
COPY artifacts/tx-os/package.json artifacts/tx-os/
|
||||
COPY artifacts/mockup-sandbox/package.json artifacts/mockup-sandbox/
|
||||
COPY scripts/package.json scripts/
|
||||
COPY lib lib
|
||||
# Re-copy lib package.json files (lib has its own subprojects with manifests)
|
||||
RUN find lib -name node_modules -prune -o -name dist -prune -o -type f -print | head -1 >/dev/null
|
||||
|
||||
ENV CI=true \
|
||||
HUSKY=0 \
|
||||
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
|
||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
||||
pnpm install --frozen-lockfile --prefer-offline
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Stage 2: build all workspace artifacts
|
||||
# -----------------------------------------------------------------------------
|
||||
FROM deps AS build
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
# Build API (esbuild bundle to dist/index.mjs).
|
||||
RUN pnpm --filter @workspace/api-server build
|
||||
# Build SPA (Vite static bundle to dist/public). PORT/BASE_PATH are
|
||||
# required by vite.config.ts at build time even though the static
|
||||
# bundle is served by nginx — give them placeholder values.
|
||||
RUN PORT=3000 BASE_PATH=/ pnpm --filter @workspace/tx-os build
|
||||
# Compile TS in scripts/ for the migrate target.
|
||||
RUN pnpm --filter scripts run build || true
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Stage 3: API runtime — Playwright base ships Chromium + Linux deps for PDF.
|
||||
# -----------------------------------------------------------------------------
|
||||
FROM mcr.microsoft.com/playwright:v${PLAYWRIGHT_VERSION}-jammy AS api
|
||||
WORKDIR /app
|
||||
ENV NODE_ENV=production \
|
||||
NODE_OPTIONS="--enable-source-maps"
|
||||
# Bring across the api server bundle + its node_modules (esbuild externalises
|
||||
# Playwright/PDFKit/etc. at runtime, so we need the installed packages here).
|
||||
COPY --from=build /app/artifacts/api-server/dist artifacts/api-server/dist
|
||||
COPY --from=build /app/artifacts/api-server/node_modules artifacts/api-server/node_modules
|
||||
COPY --from=build /app/artifacts/api-server/assets artifacts/api-server/assets
|
||||
COPY --from=build /app/node_modules node_modules
|
||||
EXPOSE 8080
|
||||
WORKDIR /app/artifacts/api-server
|
||||
CMD ["node", "--enable-source-maps", "./dist/index.mjs"]
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Stage 4: web runtime — nginx serves the SPA + proxies /api to the API.
|
||||
# -----------------------------------------------------------------------------
|
||||
FROM nginx:1.27-alpine AS web
|
||||
COPY --from=build /app/artifacts/tx-os/dist/public /usr/share/nginx/html
|
||||
COPY docker/nginx.conf /etc/nginx/conf.d/default.conf
|
||||
EXPOSE 80
|
||||
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
|
||||
CMD wget -qO- http://localhost/healthz >/dev/null || exit 1
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Stage 5: one-shot DB migration + seed runner (used by `docker compose run migrate`).
|
||||
# -----------------------------------------------------------------------------
|
||||
FROM node:${NODE_VERSION}-bookworm-slim AS migrate
|
||||
WORKDIR /app
|
||||
RUN corepack enable && corepack prepare pnpm@10.26.1 --activate
|
||||
COPY --from=build /app /app
|
||||
ENV NODE_ENV=production
|
||||
CMD ["sh", "-c", "pnpm --filter db run push-force && pnpm --filter scripts run seed"]
|
||||
@@ -0,0 +1,187 @@
|
||||
# Tx OS
|
||||
|
||||
A bilingual (Arabic / English) internal "office OS" web platform. Single-tenant,
|
||||
self-hosted, designed to live behind a TLS-terminating reverse proxy on a
|
||||
private VPS or on-prem host.
|
||||
|
||||
The repo is a pnpm monorepo containing:
|
||||
|
||||
| Package | Purpose |
|
||||
| ------------------------------------ | -------------------------------------------------- |
|
||||
| `artifacts/api-server` | Express 5 + Socket.IO + Drizzle ORM API |
|
||||
| `artifacts/tx-os` | React 19 + Vite SPA (the user-facing app) |
|
||||
| `artifacts/mockup-sandbox` | Internal component preview server (dev-only) |
|
||||
| `lib/db` | Drizzle schema + migrations |
|
||||
| `lib/api-zod` + `lib/api-client-react` | OpenAPI-generated Zod schemas + React Query hooks |
|
||||
| `scripts` | DB seed + maintenance scripts |
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Glassmorphism OS UI** with animated gradient backgrounds and an app grid / dock.
|
||||
- **Bilingual** (RTL Arabic + LTR English) with per-user persisted locale.
|
||||
- **Session auth** (`express-session` + Postgres-backed `connect-pg-simple`,
|
||||
bcrypt password hashing) with full RBAC (admin / user roles + role-permission
|
||||
matrix + group-derived permissions).
|
||||
- **Real-time** chat / notifications / executive-meeting alerts via Socket.IO.
|
||||
- **Executive Meetings module** — scheduling, change requests, approvals,
|
||||
optimistic locking on postpones, audit log, and a Playwright-rendered HTML
|
||||
PDF export.
|
||||
- **S3-compatible object storage** with signed-URL uploads (production: MinIO;
|
||||
local dev: built-in filesystem driver — no extra services required).
|
||||
|
||||
---
|
||||
|
||||
## Quick start (Docker)
|
||||
|
||||
The fastest path to a running stack on a Linux VPS with Docker installed.
|
||||
|
||||
```bash
|
||||
git clone <this-repo>
|
||||
cd tx-os
|
||||
cp .env.example .env
|
||||
# Edit .env — at minimum change SESSION_SECRET, POSTGRES_PASSWORD,
|
||||
# S3_SECRET_ACCESS_KEY, SEED_ADMIN_PASSWORD, SEED_USER_PASSWORD.
|
||||
$EDITOR .env
|
||||
|
||||
docker compose build
|
||||
docker compose up -d db minio minio-init
|
||||
docker compose run --rm migrate # one-shot: db push + seed
|
||||
docker compose up -d api web
|
||||
```
|
||||
|
||||
The SPA is now served at `http://<host>:${WEB_PORT}`. Front it with Caddy /
|
||||
Nginx / Traefik for TLS and HTTP/2.
|
||||
|
||||
### Default seeded accounts
|
||||
|
||||
| Username | Password | Role |
|
||||
| -------- | ---------------------------------- | ----- |
|
||||
| `admin` | value of `SEED_ADMIN_PASSWORD` | admin |
|
||||
| `ahmed` | value of `SEED_USER_PASSWORD` | user |
|
||||
|
||||
Both are seeded by `docker compose run --rm migrate` and **only** if they don't
|
||||
already exist (the seed is idempotent on conflict).
|
||||
|
||||
### Common compose commands
|
||||
|
||||
```bash
|
||||
docker compose logs -f api # tail API logs
|
||||
docker compose exec db psql -U tx tx_os # psql shell
|
||||
docker compose run --rm migrate # re-run migrations (safe to repeat)
|
||||
docker compose down # stop everything (data persists)
|
||||
docker compose down -v # stop AND wipe volumes (DANGER)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local development (without Docker)
|
||||
|
||||
You can run the stack natively if you have Node 24+, pnpm 10, and Postgres 16.
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
createdb tx_os
|
||||
export DATABASE_URL=postgres://localhost/tx_os
|
||||
export PORT=8080 BASE_PATH=/ ALLOWED_ORIGINS=http://localhost:25785
|
||||
export SESSION_SECRET=$(openssl rand -hex 32)
|
||||
export PRIVATE_OBJECT_DIR=/local/private
|
||||
export PUBLIC_OBJECT_SEARCH_PATHS=/local/public
|
||||
# Note: with no S3_ENDPOINT set, the API falls back to the local-FS storage
|
||||
# driver and persists uploads under ./storage/. Safe for development only.
|
||||
|
||||
pnpm --filter db run push # apply schema
|
||||
pnpm --filter scripts run seed # seed admin/ahmed accounts
|
||||
pnpm --filter @workspace/api-server dev
|
||||
# In another terminal:
|
||||
PORT=25785 BASE_PATH=/ pnpm --filter @workspace/tx-os dev
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration reference
|
||||
|
||||
Every option is read from environment variables. See `.env.example` for the
|
||||
complete list with comments. Highlights:
|
||||
|
||||
| Variable | Purpose |
|
||||
| --------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `DATABASE_URL` | Postgres connection string. **Required.** |
|
||||
| `SESSION_SECRET` | HMAC key for session cookies + local-driver upload tokens. **Required in production.** |
|
||||
| `ALLOWED_ORIGINS` | Comma-separated CORS allow-list. Defaults to `*` (dev only). |
|
||||
| `STORAGE_DRIVER` | `s3` or `local`. Defaults to `s3` if `S3_ENDPOINT` set, else `local`. |
|
||||
| `S3_ENDPOINT` etc. | MinIO / S3 connection. Required when `STORAGE_DRIVER=s3`. |
|
||||
| `PRIVATE_OBJECT_DIR` | Path inside the bucket for private uploads, e.g. `/tx-private/private`.|
|
||||
| `PUBLIC_OBJECT_SEARCH_PATHS` | Comma-separated bucket paths searched by `GET /storage/public-objects/*`. |
|
||||
| `LOCAL_STORAGE_ROOT` | Filesystem root used by the local driver. Defaults to `./storage`. |
|
||||
| `SEED_ADMIN_PASSWORD` / `SEED_USER_PASSWORD` | Required by the seed script in production. |
|
||||
| `SMTP_*` | Optional outbound mail config for Executive Meetings notifications. |
|
||||
| `LOG_LEVEL` | pino log level. Defaults to `info`. |
|
||||
|
||||
### Storage drivers
|
||||
|
||||
The API server has two object-storage backends, selected automatically:
|
||||
|
||||
- **`s3`** (production): targets any S3-compatible endpoint (MinIO, AWS S3, R2,
|
||||
Backblaze B2, ...) via `@aws-sdk/client-s3` + presigned PUT URLs.
|
||||
- **`local`** (dev fallback, default when `S3_ENDPOINT` is unset): persists
|
||||
files under `LOCAL_STORAGE_ROOT` and issues HMAC-signed upload URLs that
|
||||
the API server validates and accepts on `PUT /api/storage/_local/upload`.
|
||||
Single-host only; not suitable for production.
|
||||
|
||||
The route layer is unaware of which driver is active — both expose the same
|
||||
`StoredObject` interface in `lib/objectStorage.ts`.
|
||||
|
||||
---
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
pnpm --filter @workspace/api-server build
|
||||
pnpm --filter @workspace/api-server start & # boot API on 8080
|
||||
pnpm --filter @workspace/api-server test # node --test suite
|
||||
pnpm --filter @workspace/tx-os test:e2e # Playwright UI tests
|
||||
```
|
||||
|
||||
The `test` workflow chains all of the above. Tests use the same database
|
||||
specified in `DATABASE_URL` and clean up after themselves with `LIKE`-prefixed
|
||||
fixture rows.
|
||||
|
||||
---
|
||||
|
||||
## Production checklist
|
||||
|
||||
Before fronting Tx OS with a public domain:
|
||||
|
||||
1. **Set every secret in `.env`.** No defaults in production.
|
||||
2. **Run behind TLS.** Caddy / Nginx / Traefik should terminate HTTPS and
|
||||
forward to `web` on `${WEB_PORT}`. The API server trusts `X-Forwarded-For`
|
||||
from the first proxy hop (`app.set("trust proxy", 1)`).
|
||||
3. **Restrict the API port.** The `api` service should never be exposed
|
||||
directly to the internet — only `web` is intended to be reachable.
|
||||
4. **Back up the volumes.** `db_data` and `minio_data` are the only stateful
|
||||
surfaces. Snapshot them on a schedule.
|
||||
5. **Rotate seeded passwords.** The first thing an admin should do is change
|
||||
the seeded `admin` and `ahmed` passwords from the user-management screen.
|
||||
6. **Review `threat_model.md`.** Document-level threat model lives in the repo
|
||||
root and lists the trust boundaries this deployment relies on.
|
||||
|
||||
---
|
||||
|
||||
## Project conventions
|
||||
|
||||
- **Package manager**: pnpm 10. The repo refuses to install with npm/yarn.
|
||||
- **Node**: 24 LTS. Older Nodes will not run the API bundle.
|
||||
- **TypeScript**: 5.9, strict mode, project-references build (`pnpm typecheck`).
|
||||
- **API contracts**: hand-written Zod schemas in `lib/api-zod` are the source
|
||||
of truth; the React-Query client in `lib/api-client-react` is generated
|
||||
from the same OpenAPI spec via Orval.
|
||||
- **Migrations**: Drizzle Kit. `pnpm --filter db run push-force` for dev,
|
||||
`pnpm --filter db run push` for production.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT.
|
||||
@@ -12,7 +12,8 @@
|
||||
"test": "node --test 'tests/**/*.test.mjs'"
|
||||
},
|
||||
"dependencies": {
|
||||
"@google-cloud/storage": "^7.19.0",
|
||||
"@aws-sdk/client-s3": "^3.658.0",
|
||||
"@aws-sdk/s3-request-presigner": "^3.658.0",
|
||||
"@swc/helpers": "^0.5.21",
|
||||
"@workspace/api-zod": "workspace:*",
|
||||
"@workspace/db": "workspace:*",
|
||||
@@ -26,7 +27,6 @@
|
||||
"express": "^5",
|
||||
"express-rate-limit": "^8.5.1",
|
||||
"express-session": "^1.19.0",
|
||||
"google-auth-library": "^10.6.2",
|
||||
"nodemailer": "^8.0.7",
|
||||
"pdfkit": "^0.18.0",
|
||||
"pino": "^9",
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import { File } from "@google-cloud/storage";
|
||||
|
||||
// =====================================================================
|
||||
// DEPRECATED — DO NOT USE FOR ACCESS DECISIONS
|
||||
// =====================================================================
|
||||
@@ -20,14 +18,12 @@ import { File } from "@google-cloud/storage";
|
||||
// What remains here is the bare minimum needed to keep
|
||||
// `objectStorage.downloadObject` working: it consults the cached ACL
|
||||
// metadata only to decide between `Cache-Control: public` and
|
||||
// `Cache-Control: private` headers. Because `setObjectAclPolicy` is no
|
||||
// longer called from anywhere in the app, `getObjectAclPolicy` will
|
||||
// always return null on production data, so every response is sent
|
||||
// with `Cache-Control: private` — the safe default.
|
||||
// `Cache-Control: private` headers. Because `setAclPolicy` is no
|
||||
// longer called from anywhere in the app, `getAclPolicy` will always
|
||||
// return null on production data, so every response is sent with
|
||||
// `Cache-Control: private` — the safe default.
|
||||
// =====================================================================
|
||||
|
||||
const ACL_POLICY_METADATA_KEY = "custom:aclPolicy";
|
||||
|
||||
export enum ObjectPermission {
|
||||
READ = "read",
|
||||
WRITE = "write",
|
||||
@@ -41,34 +37,29 @@ export interface ObjectAclPolicy {
|
||||
visibility: "public" | "private";
|
||||
}
|
||||
|
||||
export async function setObjectAclPolicy(
|
||||
objectFile: File,
|
||||
aclPolicy: ObjectAclPolicy,
|
||||
): Promise<void> {
|
||||
const [exists] = await objectFile.exists();
|
||||
if (!exists) {
|
||||
throw new Error(`Object not found: ${objectFile.name}`);
|
||||
}
|
||||
await objectFile.setMetadata({
|
||||
metadata: {
|
||||
[ACL_POLICY_METADATA_KEY]: JSON.stringify(aclPolicy),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export async function getObjectAclPolicy(
|
||||
objectFile: File,
|
||||
): Promise<ObjectAclPolicy | null> {
|
||||
const [metadata] = await objectFile.getMetadata();
|
||||
const aclPolicy = metadata?.metadata?.[ACL_POLICY_METADATA_KEY];
|
||||
if (!aclPolicy) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return JSON.parse(aclPolicy as string) as ObjectAclPolicy;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
/**
|
||||
* Driver-agnostic stored-object handle. Replaces the old `File` type
|
||||
* from `@google-cloud/storage` so route handlers (which operate on
|
||||
* the result opaquely) don't depend on a specific storage backend.
|
||||
*/
|
||||
export interface StoredObject {
|
||||
bucketName: string;
|
||||
objectName: string;
|
||||
exists(): Promise<boolean>;
|
||||
getMetadata(): Promise<{
|
||||
contentType?: string;
|
||||
size?: number;
|
||||
aclPolicy?: ObjectAclPolicy | null;
|
||||
}>;
|
||||
createReadStream(): NodeJS.ReadableStream;
|
||||
/**
|
||||
* Reads the entire object into a Buffer. The single-element tuple shape
|
||||
* mirrors `@google-cloud/storage`'s `File.download()` so legacy callers
|
||||
* (e.g. PDF logo loader) keep working without refactoring.
|
||||
*/
|
||||
download(): Promise<[Buffer]>;
|
||||
setAclPolicy(policy: ObjectAclPolicy): Promise<void>;
|
||||
getAclPolicy(): Promise<ObjectAclPolicy | null>;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -78,7 +69,7 @@ export async function getObjectAclPolicy(
|
||||
*/
|
||||
export async function canAccessObject(_args: {
|
||||
userId?: string;
|
||||
objectFile: File;
|
||||
objectFile: StoredObject;
|
||||
requestedPermission: ObjectPermission;
|
||||
}): Promise<boolean> {
|
||||
return false;
|
||||
|
||||
@@ -1,34 +1,22 @@
|
||||
import { Storage, File } from "@google-cloud/storage";
|
||||
import { Readable } from "stream";
|
||||
import { randomUUID } from "crypto";
|
||||
import { randomUUID, createHmac, timingSafeEqual } from "crypto";
|
||||
import { promises as fs, createReadStream, createWriteStream } from "fs";
|
||||
import path from "path";
|
||||
import {
|
||||
S3Client,
|
||||
HeadObjectCommand,
|
||||
GetObjectCommand,
|
||||
PutObjectCommand,
|
||||
CopyObjectCommand,
|
||||
} from "@aws-sdk/client-s3";
|
||||
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
|
||||
import {
|
||||
ObjectAclPolicy,
|
||||
ObjectPermission,
|
||||
canAccessObject,
|
||||
getObjectAclPolicy,
|
||||
setObjectAclPolicy,
|
||||
type StoredObject,
|
||||
} from "./objectAcl";
|
||||
|
||||
const REPLIT_SIDECAR_ENDPOINT = "http://127.0.0.1:1106";
|
||||
|
||||
export const objectStorageClient = new Storage({
|
||||
credentials: {
|
||||
audience: "replit",
|
||||
subject_token_type: "access_token",
|
||||
token_url: `${REPLIT_SIDECAR_ENDPOINT}/token`,
|
||||
type: "external_account",
|
||||
credential_source: {
|
||||
url: `${REPLIT_SIDECAR_ENDPOINT}/credential`,
|
||||
format: {
|
||||
type: "json",
|
||||
subject_token_field_name: "access_token",
|
||||
},
|
||||
},
|
||||
universe_domain: "googleapis.com",
|
||||
},
|
||||
projectId: "",
|
||||
});
|
||||
|
||||
export class ObjectNotFoundError extends Error {
|
||||
constructor() {
|
||||
super("Object not found");
|
||||
@@ -37,6 +25,391 @@ export class ObjectNotFoundError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Driver interface
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface StorageDriver {
|
||||
getObject(bucketName: string, objectName: string): StoredObject;
|
||||
signUploadUrl(bucketName: string, objectName: string, ttlSec: number): Promise<string>;
|
||||
/** Recognise a previously-issued upload URL and return the object path. */
|
||||
parseUploadUrl(rawUrl: string): { bucketName: string; objectName: string } | null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Local-FS driver — used when no S3_ENDPOINT is configured. Backs storage on
|
||||
// the host filesystem under ./storage/<bucket>/<object>. Signed upload URLs
|
||||
// are HMAC tokens served by the local API at /api/storage/_local/upload.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const LOCAL_STORAGE_ROOT = path.resolve(
|
||||
process.env.LOCAL_STORAGE_ROOT ?? "./storage",
|
||||
);
|
||||
const LOCAL_HMAC_SECRET = (() => {
|
||||
const s = process.env.LOCAL_STORAGE_SIGNING_SECRET ?? process.env.SESSION_SECRET;
|
||||
if (s) return s;
|
||||
if (process.env.NODE_ENV === "production") {
|
||||
throw new Error(
|
||||
"LOCAL_STORAGE_SIGNING_SECRET (or SESSION_SECRET) must be set in production " +
|
||||
"when using the local storage driver.",
|
||||
);
|
||||
}
|
||||
return "tx-dev-local-storage-secret-change-before-deploy";
|
||||
})();
|
||||
|
||||
function localFsPath(bucketName: string, objectName: string): string {
|
||||
// Reject path traversal — joining and re-checking the absolute path
|
||||
// must remain inside LOCAL_STORAGE_ROOT.
|
||||
const joined = path.resolve(LOCAL_STORAGE_ROOT, bucketName, objectName);
|
||||
if (!joined.startsWith(LOCAL_STORAGE_ROOT + path.sep) && joined !== LOCAL_STORAGE_ROOT) {
|
||||
throw new Error("Invalid object path");
|
||||
}
|
||||
return joined;
|
||||
}
|
||||
|
||||
function localMetadataPath(bucketName: string, objectName: string): string {
|
||||
return localFsPath(bucketName, objectName) + ".meta.json";
|
||||
}
|
||||
|
||||
interface LocalMetadataFile {
|
||||
contentType?: string;
|
||||
aclPolicy?: ObjectAclPolicy | null;
|
||||
}
|
||||
|
||||
class LocalStoredObject implements StoredObject {
|
||||
constructor(public bucketName: string, public objectName: string) {}
|
||||
|
||||
async exists(): Promise<boolean> {
|
||||
try {
|
||||
await fs.access(localFsPath(this.bucketName, this.objectName));
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async getMetadata(): Promise<{
|
||||
contentType?: string;
|
||||
size?: number;
|
||||
aclPolicy?: ObjectAclPolicy | null;
|
||||
}> {
|
||||
const fp = localFsPath(this.bucketName, this.objectName);
|
||||
const stat = await fs.stat(fp);
|
||||
let meta: LocalMetadataFile = {};
|
||||
try {
|
||||
const raw = await fs.readFile(localMetadataPath(this.bucketName, this.objectName), "utf8");
|
||||
meta = JSON.parse(raw) as LocalMetadataFile;
|
||||
} catch {
|
||||
// No sidecar metadata file — that's fine.
|
||||
}
|
||||
return {
|
||||
contentType: meta.contentType,
|
||||
size: stat.size,
|
||||
aclPolicy: meta.aclPolicy ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
createReadStream(): NodeJS.ReadableStream {
|
||||
return createReadStream(localFsPath(this.bucketName, this.objectName));
|
||||
}
|
||||
|
||||
async download(): Promise<[Buffer]> {
|
||||
const buf = await fs.readFile(localFsPath(this.bucketName, this.objectName));
|
||||
return [buf];
|
||||
}
|
||||
|
||||
async setAclPolicy(policy: ObjectAclPolicy): Promise<void> {
|
||||
if (!(await this.exists())) {
|
||||
throw new Error(`Object not found: ${this.objectName}`);
|
||||
}
|
||||
let meta: LocalMetadataFile = {};
|
||||
try {
|
||||
meta = JSON.parse(
|
||||
await fs.readFile(localMetadataPath(this.bucketName, this.objectName), "utf8"),
|
||||
) as LocalMetadataFile;
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
meta.aclPolicy = policy;
|
||||
await fs.writeFile(
|
||||
localMetadataPath(this.bucketName, this.objectName),
|
||||
JSON.stringify(meta),
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
|
||||
async getAclPolicy(): Promise<ObjectAclPolicy | null> {
|
||||
const m = await this.getMetadata();
|
||||
return m.aclPolicy ?? null;
|
||||
}
|
||||
}
|
||||
|
||||
class LocalDriver implements StorageDriver {
|
||||
getObject(bucketName: string, objectName: string): StoredObject {
|
||||
return new LocalStoredObject(bucketName, objectName);
|
||||
}
|
||||
|
||||
async signUploadUrl(
|
||||
bucketName: string,
|
||||
objectName: string,
|
||||
ttlSec: number,
|
||||
): Promise<string> {
|
||||
const exp = Math.floor(Date.now() / 1000) + ttlSec;
|
||||
const payload = Buffer.from(
|
||||
JSON.stringify({ b: bucketName, o: objectName, exp }),
|
||||
).toString("base64url");
|
||||
const sig = createHmac("sha256", LOCAL_HMAC_SECRET).update(payload).digest("base64url");
|
||||
const token = `${payload}.${sig}`;
|
||||
const base = (process.env.PUBLIC_BASE_URL ?? `http://localhost:${process.env.PORT ?? "8080"}`).replace(/\/+$/, "");
|
||||
return `${base}/api/storage/_local/upload?token=${token}`;
|
||||
}
|
||||
|
||||
parseUploadUrl(rawUrl: string): { bucketName: string; objectName: string } | null {
|
||||
try {
|
||||
const u = new URL(rawUrl);
|
||||
if (!u.pathname.endsWith("/api/storage/_local/upload")) return null;
|
||||
const token = u.searchParams.get("token");
|
||||
if (!token) return null;
|
||||
const verified = verifyLocalUploadToken(token);
|
||||
if (!verified) return null;
|
||||
return { bucketName: verified.b, objectName: verified.o };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify a local-driver upload token. Returns the decoded payload on
|
||||
* success (signature match + not expired), null otherwise.
|
||||
*/
|
||||
export function verifyLocalUploadToken(
|
||||
token: string,
|
||||
): { b: string; o: string; exp: number } | null {
|
||||
const parts = token.split(".");
|
||||
if (parts.length !== 2) return null;
|
||||
const [payload, sig] = parts;
|
||||
const expectedSig = createHmac("sha256", LOCAL_HMAC_SECRET).update(payload).digest("base64url");
|
||||
const a = Buffer.from(sig);
|
||||
const b = Buffer.from(expectedSig);
|
||||
if (a.length !== b.length || !timingSafeEqual(a, b)) return null;
|
||||
let decoded: { b: string; o: string; exp: number };
|
||||
try {
|
||||
decoded = JSON.parse(Buffer.from(payload, "base64url").toString("utf8")) as {
|
||||
b: string;
|
||||
o: string;
|
||||
exp: number;
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (typeof decoded.exp !== "number" || decoded.exp < Math.floor(Date.now() / 1000)) {
|
||||
return null;
|
||||
}
|
||||
return decoded;
|
||||
}
|
||||
|
||||
/**
|
||||
* Persist an upload to the local-FS storage backend. Used by the
|
||||
* /api/storage/_local/upload route.
|
||||
*/
|
||||
export async function writeLocalObject(
|
||||
bucketName: string,
|
||||
objectName: string,
|
||||
body: Readable,
|
||||
contentType?: string,
|
||||
): Promise<void> {
|
||||
const fp = localFsPath(bucketName, objectName);
|
||||
await fs.mkdir(path.dirname(fp), { recursive: true });
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const ws = createWriteStream(fp);
|
||||
body.pipe(ws);
|
||||
ws.on("finish", () => resolve());
|
||||
ws.on("error", reject);
|
||||
body.on("error", reject);
|
||||
});
|
||||
if (contentType) {
|
||||
let meta: LocalMetadataFile = {};
|
||||
try {
|
||||
meta = JSON.parse(await fs.readFile(localMetadataPath(bucketName, objectName), "utf8")) as LocalMetadataFile;
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
meta.contentType = contentType;
|
||||
await fs.writeFile(localMetadataPath(bucketName, objectName), JSON.stringify(meta), "utf8");
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// S3 driver — used when S3_ENDPOINT is configured. Targets MinIO or any
|
||||
// S3-compatible bucket via @aws-sdk/client-s3 + s3-request-presigner.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const ACL_METADATA_HEADER = "x-amz-meta-acl-policy";
|
||||
|
||||
let s3ClientSingleton: S3Client | null = null;
|
||||
function getS3Client(): S3Client {
|
||||
if (s3ClientSingleton) return s3ClientSingleton;
|
||||
const endpoint = process.env.S3_ENDPOINT;
|
||||
if (!endpoint) throw new Error("S3_ENDPOINT not set");
|
||||
const region = process.env.S3_REGION ?? "us-east-1";
|
||||
const accessKeyId = process.env.S3_ACCESS_KEY_ID;
|
||||
const secretAccessKey = process.env.S3_SECRET_ACCESS_KEY;
|
||||
if (!accessKeyId || !secretAccessKey) {
|
||||
throw new Error("S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY must be set when S3_ENDPOINT is configured");
|
||||
}
|
||||
s3ClientSingleton = new S3Client({
|
||||
endpoint,
|
||||
region,
|
||||
credentials: { accessKeyId, secretAccessKey },
|
||||
forcePathStyle: process.env.S3_FORCE_PATH_STYLE !== "false",
|
||||
});
|
||||
return s3ClientSingleton;
|
||||
}
|
||||
|
||||
class S3StoredObject implements StoredObject {
|
||||
constructor(public bucketName: string, public objectName: string) {}
|
||||
|
||||
async exists(): Promise<boolean> {
|
||||
try {
|
||||
await getS3Client().send(
|
||||
new HeadObjectCommand({ Bucket: this.bucketName, Key: this.objectName }),
|
||||
);
|
||||
return true;
|
||||
} catch (err) {
|
||||
const name = (err as Error & { name?: string }).name;
|
||||
if (name === "NotFound" || name === "NoSuchKey") return false;
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
async getMetadata(): Promise<{
|
||||
contentType?: string;
|
||||
size?: number;
|
||||
aclPolicy?: ObjectAclPolicy | null;
|
||||
}> {
|
||||
const head = await getS3Client().send(
|
||||
new HeadObjectCommand({ Bucket: this.bucketName, Key: this.objectName }),
|
||||
);
|
||||
const aclRaw = head.Metadata?.["acl-policy"];
|
||||
let aclPolicy: ObjectAclPolicy | null = null;
|
||||
if (aclRaw) {
|
||||
try {
|
||||
aclPolicy = JSON.parse(aclRaw) as ObjectAclPolicy;
|
||||
} catch {
|
||||
aclPolicy = null;
|
||||
}
|
||||
}
|
||||
return {
|
||||
contentType: head.ContentType,
|
||||
size: head.ContentLength,
|
||||
aclPolicy,
|
||||
};
|
||||
}
|
||||
|
||||
createReadStream(): NodeJS.ReadableStream {
|
||||
// Lazy stream: synchronously return a PassThrough that we'll pipe
|
||||
// into once the GetObject promise resolves.
|
||||
const { PassThrough } = require("stream") as typeof import("stream");
|
||||
const out = new PassThrough();
|
||||
getS3Client()
|
||||
.send(new GetObjectCommand({ Bucket: this.bucketName, Key: this.objectName }))
|
||||
.then((res) => {
|
||||
const body = res.Body as Readable | undefined;
|
||||
if (!body) {
|
||||
out.end();
|
||||
return;
|
||||
}
|
||||
body.on("error", (e) => out.destroy(e));
|
||||
body.pipe(out);
|
||||
})
|
||||
.catch((e) => out.destroy(e));
|
||||
return out;
|
||||
}
|
||||
|
||||
async download(): Promise<[Buffer]> {
|
||||
const res = await getS3Client().send(
|
||||
new GetObjectCommand({ Bucket: this.bucketName, Key: this.objectName }),
|
||||
);
|
||||
const body = res.Body as Readable | undefined;
|
||||
if (!body) return [Buffer.alloc(0)];
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of body) {
|
||||
chunks.push(typeof chunk === "string" ? Buffer.from(chunk) : Buffer.from(chunk));
|
||||
}
|
||||
return [Buffer.concat(chunks)];
|
||||
}
|
||||
|
||||
async setAclPolicy(policy: ObjectAclPolicy): Promise<void> {
|
||||
// S3 supports setting metadata only via copy-self.
|
||||
const json = JSON.stringify(policy);
|
||||
await getS3Client().send(
|
||||
new CopyObjectCommand({
|
||||
Bucket: this.bucketName,
|
||||
Key: this.objectName,
|
||||
CopySource: `/${this.bucketName}/${encodeURIComponent(this.objectName)}`,
|
||||
Metadata: { "acl-policy": json },
|
||||
MetadataDirective: "REPLACE",
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
async getAclPolicy(): Promise<ObjectAclPolicy | null> {
|
||||
return (await this.getMetadata()).aclPolicy ?? null;
|
||||
}
|
||||
}
|
||||
|
||||
class S3Driver implements StorageDriver {
|
||||
getObject(bucketName: string, objectName: string): StoredObject {
|
||||
return new S3StoredObject(bucketName, objectName);
|
||||
}
|
||||
|
||||
async signUploadUrl(
|
||||
bucketName: string,
|
||||
objectName: string,
|
||||
ttlSec: number,
|
||||
): Promise<string> {
|
||||
const cmd = new PutObjectCommand({ Bucket: bucketName, Key: objectName });
|
||||
return getSignedUrl(getS3Client(), cmd, { expiresIn: ttlSec });
|
||||
}
|
||||
|
||||
parseUploadUrl(rawUrl: string): { bucketName: string; objectName: string } | null {
|
||||
try {
|
||||
const u = new URL(rawUrl);
|
||||
// Path-style: /<bucket>/<key...>
|
||||
const segments = u.pathname.replace(/^\/+/, "").split("/");
|
||||
if (segments.length < 2) return null;
|
||||
const bucketName = segments[0];
|
||||
const objectName = segments.slice(1).join("/");
|
||||
return { bucketName, objectName: decodeURIComponent(objectName) };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Driver selection
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
let driverSingleton: StorageDriver | null = null;
|
||||
function driver(): StorageDriver {
|
||||
if (driverSingleton) return driverSingleton;
|
||||
const explicit = process.env.STORAGE_DRIVER;
|
||||
const useS3 = explicit ? explicit === "s3" : Boolean(process.env.S3_ENDPOINT);
|
||||
driverSingleton = useS3 ? new S3Driver() : new LocalDriver();
|
||||
return driverSingleton;
|
||||
}
|
||||
|
||||
export const ACTIVE_STORAGE_DRIVER = (): "s3" | "local" =>
|
||||
driver() instanceof S3Driver ? "s3" : "local";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// ObjectStorageService — public API consumed by routes/storage.ts and
|
||||
// routes/executive-meetings.ts. Surface preserved byte-compatible with the
|
||||
// previous Replit-sidecar implementation.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class ObjectStorageService {
|
||||
constructor() {}
|
||||
|
||||
@@ -46,14 +419,14 @@ export class ObjectStorageService {
|
||||
new Set(
|
||||
pathsStr
|
||||
.split(",")
|
||||
.map((path) => path.trim())
|
||||
.filter((path) => path.length > 0)
|
||||
)
|
||||
.map((p) => p.trim())
|
||||
.filter((p) => p.length > 0),
|
||||
),
|
||||
);
|
||||
if (paths.length === 0) {
|
||||
throw new Error(
|
||||
"PUBLIC_OBJECT_SEARCH_PATHS not set. Create a bucket in 'Object Storage' " +
|
||||
"tool and set PUBLIC_OBJECT_SEARCH_PATHS env var (comma-separated paths)."
|
||||
"PUBLIC_OBJECT_SEARCH_PATHS not set. Set this env var to a comma-separated " +
|
||||
"list of /<bucket>/<prefix> paths (see .env.example).",
|
||||
);
|
||||
}
|
||||
return paths;
|
||||
@@ -63,43 +436,37 @@ export class ObjectStorageService {
|
||||
const dir = process.env.PRIVATE_OBJECT_DIR || "";
|
||||
if (!dir) {
|
||||
throw new Error(
|
||||
"PRIVATE_OBJECT_DIR not set. Create a bucket in 'Object Storage' " +
|
||||
"tool and set PRIVATE_OBJECT_DIR env var."
|
||||
"PRIVATE_OBJECT_DIR not set. Set this env var to /<bucket>/<prefix> " +
|
||||
"(see .env.example).",
|
||||
);
|
||||
}
|
||||
return dir;
|
||||
}
|
||||
|
||||
async searchPublicObject(filePath: string): Promise<File | null> {
|
||||
async searchPublicObject(filePath: string): Promise<StoredObject | null> {
|
||||
for (const searchPath of this.getPublicObjectSearchPaths()) {
|
||||
const fullPath = `${searchPath}/${filePath}`;
|
||||
|
||||
const { bucketName, objectName } = parseObjectPath(fullPath);
|
||||
const bucket = objectStorageClient.bucket(bucketName);
|
||||
const file = bucket.file(objectName);
|
||||
|
||||
const [exists] = await file.exists();
|
||||
if (exists) {
|
||||
return file;
|
||||
const obj = driver().getObject(bucketName, objectName);
|
||||
if (await obj.exists()) {
|
||||
return obj;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
async downloadObject(file: File, cacheTtlSec: number = 3600): Promise<Response> {
|
||||
const [metadata] = await file.getMetadata();
|
||||
const aclPolicy = await getObjectAclPolicy(file);
|
||||
const isPublic = aclPolicy?.visibility === "public";
|
||||
async downloadObject(file: StoredObject, cacheTtlSec: number = 3600): Promise<Response> {
|
||||
const metadata = await file.getMetadata();
|
||||
const isPublic = metadata.aclPolicy?.visibility === "public";
|
||||
|
||||
const nodeStream = file.createReadStream();
|
||||
const webStream = Readable.toWeb(nodeStream) as ReadableStream;
|
||||
const webStream = Readable.toWeb(nodeStream as Readable) as ReadableStream;
|
||||
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": (metadata.contentType as string) || "application/octet-stream",
|
||||
"Content-Type": metadata.contentType ?? "application/octet-stream",
|
||||
"Cache-Control": `${isPublic ? "public" : "private"}, max-age=${cacheTtlSec}`,
|
||||
};
|
||||
if (metadata.size) {
|
||||
if (typeof metadata.size === "number") {
|
||||
headers["Content-Length"] = String(metadata.size);
|
||||
}
|
||||
|
||||
@@ -108,84 +475,68 @@ export class ObjectStorageService {
|
||||
|
||||
async getObjectEntityUploadURL(): Promise<string> {
|
||||
const privateObjectDir = this.getPrivateObjectDir();
|
||||
if (!privateObjectDir) {
|
||||
throw new Error(
|
||||
"PRIVATE_OBJECT_DIR not set. Create a bucket in 'Object Storage' " +
|
||||
"tool and set PRIVATE_OBJECT_DIR env var."
|
||||
);
|
||||
}
|
||||
|
||||
const objectId = randomUUID();
|
||||
const fullPath = `${privateObjectDir}/uploads/${objectId}`;
|
||||
|
||||
const { bucketName, objectName } = parseObjectPath(fullPath);
|
||||
|
||||
return signObjectURL({
|
||||
bucketName,
|
||||
objectName,
|
||||
method: "PUT",
|
||||
ttlSec: 900,
|
||||
});
|
||||
return driver().signUploadUrl(bucketName, objectName, 900);
|
||||
}
|
||||
|
||||
async getObjectEntityFile(objectPath: string): Promise<File> {
|
||||
async getObjectEntityFile(objectPath: string): Promise<StoredObject> {
|
||||
if (!objectPath.startsWith("/objects/")) {
|
||||
throw new ObjectNotFoundError();
|
||||
}
|
||||
|
||||
const parts = objectPath.slice(1).split("/");
|
||||
if (parts.length < 2) {
|
||||
throw new ObjectNotFoundError();
|
||||
}
|
||||
|
||||
const entityId = parts.slice(1).join("/");
|
||||
let entityDir = this.getPrivateObjectDir();
|
||||
if (!entityDir.endsWith("/")) {
|
||||
entityDir = `${entityDir}/`;
|
||||
}
|
||||
if (!entityDir.endsWith("/")) entityDir = `${entityDir}/`;
|
||||
const objectEntityPath = `${entityDir}${entityId}`;
|
||||
const { bucketName, objectName } = parseObjectPath(objectEntityPath);
|
||||
const bucket = objectStorageClient.bucket(bucketName);
|
||||
const objectFile = bucket.file(objectName);
|
||||
const [exists] = await objectFile.exists();
|
||||
if (!exists) {
|
||||
const obj = driver().getObject(bucketName, objectName);
|
||||
if (!(await obj.exists())) {
|
||||
throw new ObjectNotFoundError();
|
||||
}
|
||||
return objectFile;
|
||||
return obj;
|
||||
}
|
||||
|
||||
normalizeObjectEntityPath(rawPath: string): string {
|
||||
if (!rawPath.startsWith("https://storage.googleapis.com/")) {
|
||||
return rawPath;
|
||||
// Try the active driver's URL recogniser first — strip query string,
|
||||
// map a signed-URL back to its /objects/<id> form.
|
||||
const noQuery = rawPath.split("?")[0]!;
|
||||
const parsed = driver().parseUploadUrl(rawPath);
|
||||
let candidatePath = noQuery;
|
||||
if (parsed) {
|
||||
candidatePath = `/${parsed.bucketName}/${parsed.objectName}`;
|
||||
} else {
|
||||
// Legacy GCS URL form (pre-migration data) — strip the host.
|
||||
try {
|
||||
const u = new URL(rawPath);
|
||||
if (u.pathname.startsWith("/")) candidatePath = u.pathname;
|
||||
} catch {
|
||||
// already a relative path
|
||||
}
|
||||
}
|
||||
|
||||
const url = new URL(rawPath);
|
||||
const rawObjectPath = url.pathname;
|
||||
|
||||
let objectEntityDir = this.getPrivateObjectDir();
|
||||
if (!objectEntityDir.endsWith("/")) {
|
||||
objectEntityDir = `${objectEntityDir}/`;
|
||||
}
|
||||
if (!objectEntityDir.endsWith("/")) objectEntityDir = `${objectEntityDir}/`;
|
||||
|
||||
if (!rawObjectPath.startsWith(objectEntityDir)) {
|
||||
return rawObjectPath;
|
||||
if (!candidatePath.startsWith(objectEntityDir)) {
|
||||
return candidatePath;
|
||||
}
|
||||
|
||||
const entityId = rawObjectPath.slice(objectEntityDir.length);
|
||||
const entityId = candidatePath.slice(objectEntityDir.length);
|
||||
return `/objects/${entityId}`;
|
||||
}
|
||||
|
||||
async trySetObjectEntityAclPolicy(
|
||||
rawPath: string,
|
||||
aclPolicy: ObjectAclPolicy
|
||||
aclPolicy: ObjectAclPolicy,
|
||||
): Promise<string> {
|
||||
const normalizedPath = this.normalizeObjectEntityPath(rawPath);
|
||||
if (!normalizedPath.startsWith("/")) {
|
||||
return normalizedPath;
|
||||
}
|
||||
|
||||
if (!normalizedPath.startsWith("/")) return normalizedPath;
|
||||
const objectFile = await this.getObjectEntityFile(normalizedPath);
|
||||
await setObjectAclPolicy(objectFile, aclPolicy);
|
||||
await objectFile.setAclPolicy(aclPolicy);
|
||||
return normalizedPath;
|
||||
}
|
||||
|
||||
@@ -195,7 +546,7 @@ export class ObjectStorageService {
|
||||
requestedPermission,
|
||||
}: {
|
||||
userId?: string;
|
||||
objectFile: File;
|
||||
objectFile: StoredObject;
|
||||
requestedPermission?: ObjectPermission;
|
||||
}): Promise<boolean> {
|
||||
return canAccessObject({
|
||||
@@ -206,62 +557,13 @@ export class ObjectStorageService {
|
||||
}
|
||||
}
|
||||
|
||||
function parseObjectPath(path: string): {
|
||||
bucketName: string;
|
||||
objectName: string;
|
||||
} {
|
||||
if (!path.startsWith("/")) {
|
||||
path = `/${path}`;
|
||||
}
|
||||
const pathParts = path.split("/");
|
||||
function parseObjectPath(p: string): { bucketName: string; objectName: string } {
|
||||
if (!p.startsWith("/")) p = `/${p}`;
|
||||
const pathParts = p.split("/");
|
||||
if (pathParts.length < 3) {
|
||||
throw new Error("Invalid path: must contain at least a bucket name");
|
||||
}
|
||||
|
||||
const bucketName = pathParts[1];
|
||||
const objectName = pathParts.slice(2).join("/");
|
||||
|
||||
return {
|
||||
bucketName,
|
||||
objectName,
|
||||
};
|
||||
}
|
||||
|
||||
async function signObjectURL({
|
||||
bucketName,
|
||||
objectName,
|
||||
method,
|
||||
ttlSec,
|
||||
}: {
|
||||
bucketName: string;
|
||||
objectName: string;
|
||||
method: "GET" | "PUT" | "DELETE" | "HEAD";
|
||||
ttlSec: number;
|
||||
}): Promise<string> {
|
||||
const request = {
|
||||
bucket_name: bucketName,
|
||||
object_name: objectName,
|
||||
method,
|
||||
expires_at: new Date(Date.now() + ttlSec * 1000).toISOString(),
|
||||
};
|
||||
const response = await fetch(
|
||||
`${REPLIT_SIDECAR_ENDPOINT}/object-storage/signed-object-url`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(request),
|
||||
signal: AbortSignal.timeout(30_000),
|
||||
}
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`Failed to sign object URL, errorcode: ${response.status}, ` +
|
||||
`make sure you're running on Replit`
|
||||
);
|
||||
}
|
||||
|
||||
const data = (await response.json()) as { signed_url: string };
|
||||
return data.signed_url;
|
||||
return { bucketName, objectName };
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@ import notificationsRouter from "./notifications";
|
||||
import usersRouter from "./users";
|
||||
import statsRouter from "./stats";
|
||||
import storageRouter from "./storage";
|
||||
import storageLocalUploadRouter from "./storage-local-upload";
|
||||
import settingsRouter from "./settings";
|
||||
import notesRouter from "./notes";
|
||||
import groupsRouter from "./groups";
|
||||
@@ -26,6 +27,7 @@ router.use(notificationsRouter);
|
||||
router.use(usersRouter);
|
||||
router.use(statsRouter);
|
||||
router.use(storageRouter);
|
||||
router.use(storageLocalUploadRouter);
|
||||
router.use(settingsRouter);
|
||||
router.use(notesRouter);
|
||||
router.use(groupsRouter);
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
import { Router, type IRouter, type Request, type Response } from "express";
|
||||
import { verifyLocalUploadToken, writeLocalObject, ACTIVE_STORAGE_DRIVER } from "../lib/objectStorage";
|
||||
|
||||
const router: IRouter = Router();
|
||||
|
||||
/**
|
||||
* PUT /storage/_local/upload?token=<HMAC>
|
||||
*
|
||||
* Receives the body of a presigned upload issued by the local-FS storage
|
||||
* driver. The token is HMAC-signed with LOCAL_STORAGE_SIGNING_SECRET (or
|
||||
* SESSION_SECRET as fallback) and carries the bucket+object key + expiry.
|
||||
* Only mounted when STORAGE_DRIVER=local (or unset and S3 not configured);
|
||||
* when S3 is the active driver, this endpoint refuses to run.
|
||||
*/
|
||||
router.put("/storage/_local/upload", async (req: Request, res: Response) => {
|
||||
if (ACTIVE_STORAGE_DRIVER() !== "local") {
|
||||
res.status(404).json({ error: "Not found" });
|
||||
return;
|
||||
}
|
||||
const token = typeof req.query.token === "string" ? req.query.token : null;
|
||||
if (!token) {
|
||||
res.status(400).json({ error: "Missing token" });
|
||||
return;
|
||||
}
|
||||
const verified = verifyLocalUploadToken(token);
|
||||
if (!verified) {
|
||||
res.status(403).json({ error: "Invalid or expired upload token" });
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const contentType = typeof req.headers["content-type"] === "string"
|
||||
? req.headers["content-type"]
|
||||
: undefined;
|
||||
await writeLocalObject(verified.b, verified.o, req, contentType);
|
||||
res.status(200).json({ ok: true });
|
||||
} catch (err) {
|
||||
req.log.error({ err }, "local storage upload failed");
|
||||
res.status(500).json({ error: "Upload failed" });
|
||||
}
|
||||
});
|
||||
|
||||
export default router;
|
||||
@@ -38,8 +38,6 @@
|
||||
"@radix-ui/react-toggle": "^1.1.10",
|
||||
"@radix-ui/react-toggle-group": "^1.1.11",
|
||||
"@radix-ui/react-tooltip": "^1.2.8",
|
||||
"@replit/vite-plugin-cartographer": "catalog:",
|
||||
"@replit/vite-plugin-runtime-error-modal": "catalog:",
|
||||
"@tailwindcss/vite": "catalog:",
|
||||
"@types/node": "catalog:",
|
||||
"@types/react": "catalog:",
|
||||
|
||||
@@ -2,7 +2,6 @@ import { defineConfig } from "vite";
|
||||
import react from "@vitejs/plugin-react";
|
||||
import tailwindcss from "@tailwindcss/vite";
|
||||
import path from "path";
|
||||
import runtimeErrorOverlay from "@replit/vite-plugin-runtime-error-modal";
|
||||
import { mockupPreviewPlugin } from "./mockupPreviewPlugin";
|
||||
|
||||
const rawPort = process.env.PORT;
|
||||
@@ -29,22 +28,7 @@ if (!basePath) {
|
||||
|
||||
export default defineConfig({
|
||||
base: basePath,
|
||||
plugins: [
|
||||
mockupPreviewPlugin(),
|
||||
react(),
|
||||
tailwindcss(),
|
||||
runtimeErrorOverlay(),
|
||||
...(process.env.NODE_ENV !== "production" &&
|
||||
process.env.REPL_ID !== undefined
|
||||
? [
|
||||
await import("@replit/vite-plugin-cartographer").then((m) =>
|
||||
m.cartographer({
|
||||
root: path.resolve(import.meta.dirname, ".."),
|
||||
}),
|
||||
),
|
||||
]
|
||||
: []),
|
||||
],
|
||||
plugins: [mockupPreviewPlugin(), react(), tailwindcss()],
|
||||
resolve: {
|
||||
alias: {
|
||||
"@": path.resolve(import.meta.dirname, "src"),
|
||||
|
||||
@@ -41,9 +41,6 @@
|
||||
"@radix-ui/react-toggle": "^1.1.3",
|
||||
"@radix-ui/react-toggle-group": "^1.1.3",
|
||||
"@radix-ui/react-tooltip": "^1.2.0",
|
||||
"@replit/vite-plugin-cartographer": "catalog:",
|
||||
"@replit/vite-plugin-dev-banner": "catalog:",
|
||||
"@replit/vite-plugin-runtime-error-modal": "catalog:",
|
||||
"@tailwindcss/typography": "^0.5.15",
|
||||
"@tailwindcss/vite": "catalog:",
|
||||
"@tanstack/react-query": "catalog:",
|
||||
|
||||
@@ -2,7 +2,6 @@ import { defineConfig } from "vite";
|
||||
import react from "@vitejs/plugin-react";
|
||||
import tailwindcss from "@tailwindcss/vite";
|
||||
import path from "path";
|
||||
import runtimeErrorOverlay from "@replit/vite-plugin-runtime-error-modal";
|
||||
|
||||
const rawPort = process.env.PORT;
|
||||
|
||||
@@ -26,30 +25,14 @@ if (!basePath) {
|
||||
);
|
||||
}
|
||||
|
||||
const apiTarget = process.env.VITE_API_PROXY_TARGET ?? "http://localhost:8080";
|
||||
|
||||
export default defineConfig({
|
||||
base: basePath,
|
||||
plugins: [
|
||||
react(),
|
||||
tailwindcss(),
|
||||
runtimeErrorOverlay(),
|
||||
...(process.env.NODE_ENV !== "production" &&
|
||||
process.env.REPL_ID !== undefined
|
||||
? [
|
||||
await import("@replit/vite-plugin-cartographer").then((m) =>
|
||||
m.cartographer({
|
||||
root: path.resolve(import.meta.dirname, ".."),
|
||||
}),
|
||||
),
|
||||
await import("@replit/vite-plugin-dev-banner").then((m) =>
|
||||
m.devBanner(),
|
||||
),
|
||||
]
|
||||
: []),
|
||||
],
|
||||
plugins: [react(), tailwindcss()],
|
||||
resolve: {
|
||||
alias: {
|
||||
"@": path.resolve(import.meta.dirname, "src"),
|
||||
"@assets": path.resolve(import.meta.dirname, "..", "..", "attached_assets"),
|
||||
},
|
||||
dedupe: ["react", "react-dom"],
|
||||
},
|
||||
@@ -68,7 +51,7 @@ export default defineConfig({
|
||||
},
|
||||
proxy: {
|
||||
"/api": {
|
||||
target: "http://localhost:8080",
|
||||
target: apiTarget,
|
||||
changeOrigin: false,
|
||||
ws: true,
|
||||
},
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
# Tx OS — production-style compose stack for a single VPS.
|
||||
#
|
||||
# Bring up: docker compose up -d
|
||||
# Migrate DB: docker compose run --rm migrate
|
||||
# Logs: docker compose logs -f api
|
||||
# Tear down: docker compose down
|
||||
#
|
||||
# All secrets (passwords, session secret, MinIO credentials) live in `.env`.
|
||||
# Copy `.env.example` to `.env` and edit before first boot.
|
||||
|
||||
name: tx-os
|
||||
|
||||
services:
|
||||
db:
|
||||
image: postgres:16-alpine
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_USER: ${POSTGRES_USER}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_DB: ${POSTGRES_DB}
|
||||
volumes:
|
||||
- db_data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
networks: [internal]
|
||||
|
||||
minio:
|
||||
image: minio/minio:RELEASE.2025-01-20T14-49-07Z
|
||||
restart: unless-stopped
|
||||
command: server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: ${S3_ACCESS_KEY_ID}
|
||||
MINIO_ROOT_PASSWORD: ${S3_SECRET_ACCESS_KEY}
|
||||
volumes:
|
||||
- minio_data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
networks: [internal]
|
||||
|
||||
# One-shot: create the two buckets on first boot. Idempotent.
|
||||
minio-init:
|
||||
image: minio/mc:RELEASE.2025-01-17T23-25-50Z
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
entrypoint: >
|
||||
/bin/sh -c "
|
||||
mc alias set local http://minio:9000 $${S3_ACCESS_KEY_ID} $${S3_SECRET_ACCESS_KEY} &&
|
||||
mc mb --ignore-existing local/${S3_BUCKET_PRIVATE} &&
|
||||
mc mb --ignore-existing local/${S3_BUCKET_PUBLIC} &&
|
||||
mc anonymous set download local/${S3_BUCKET_PUBLIC} ||
|
||||
true
|
||||
"
|
||||
environment:
|
||||
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID}
|
||||
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY}
|
||||
S3_BUCKET_PRIVATE: ${S3_BUCKET_PRIVATE}
|
||||
S3_BUCKET_PUBLIC: ${S3_BUCKET_PUBLIC}
|
||||
networks: [internal]
|
||||
restart: "no"
|
||||
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
target: api
|
||||
image: tx-os/api:latest
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
minio:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
PORT: "8080"
|
||||
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
|
||||
SESSION_SECRET: ${SESSION_SECRET}
|
||||
ALLOWED_ORIGINS: ${ALLOWED_ORIGINS}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-info}
|
||||
PUBLIC_BASE_URL: ${PUBLIC_BASE_URL}
|
||||
# Storage — S3 driver against MinIO.
|
||||
STORAGE_DRIVER: s3
|
||||
S3_ENDPOINT: http://minio:9000
|
||||
S3_REGION: ${S3_REGION:-us-east-1}
|
||||
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID}
|
||||
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY}
|
||||
S3_FORCE_PATH_STYLE: "true"
|
||||
PRIVATE_OBJECT_DIR: /${S3_BUCKET_PRIVATE}/private
|
||||
PUBLIC_OBJECT_SEARCH_PATHS: /${S3_BUCKET_PUBLIC}/public
|
||||
# Email (optional — leave SMTP_HOST blank to disable mail).
|
||||
SMTP_HOST: ${SMTP_HOST:-}
|
||||
SMTP_PORT: ${SMTP_PORT:-}
|
||||
SMTP_USER: ${SMTP_USER:-}
|
||||
SMTP_PASS: ${SMTP_PASS:-}
|
||||
SMTP_SECURE: ${SMTP_SECURE:-}
|
||||
SMTP_FROM: ${SMTP_FROM:-}
|
||||
networks: [internal]
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "node -e \"require('http').get('http://localhost:8080/api/healthz', r => process.exit(r.statusCode===200?0:1)).on('error', () => process.exit(1))\""]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
web:
|
||||
build:
|
||||
context: .
|
||||
target: web
|
||||
image: tx-os/web:latest
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
api:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "${WEB_PORT:-8080}:80"
|
||||
networks: [internal]
|
||||
|
||||
# One-shot DB migrate + seed. Run on first boot:
|
||||
# docker compose run --rm migrate
|
||||
migrate:
|
||||
build:
|
||||
context: .
|
||||
target: migrate
|
||||
image: tx-os/migrate:latest
|
||||
profiles: ["tools"]
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
|
||||
NODE_ENV: production
|
||||
SEED_ADMIN_PASSWORD: ${SEED_ADMIN_PASSWORD}
|
||||
SEED_USER_PASSWORD: ${SEED_USER_PASSWORD}
|
||||
networks: [internal]
|
||||
|
||||
volumes:
|
||||
db_data:
|
||||
minio_data:
|
||||
|
||||
networks:
|
||||
internal:
|
||||
driver: bridge
|
||||
@@ -0,0 +1,56 @@
|
||||
# Tx OS web — serves the built React/Vite SPA and reverse-proxies the
|
||||
# API + Socket.IO traffic to the api container on the private network.
|
||||
upstream tx_api {
|
||||
server api:8080;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
# Health endpoint for compose / load balancer probes.
|
||||
location = /healthz {
|
||||
access_log off;
|
||||
return 200 "ok\n";
|
||||
add_header Content-Type text/plain;
|
||||
}
|
||||
|
||||
# Socket.IO upgrade — must come before the generic /api/ block so the
|
||||
# WebSocket handshake gets the upgrade headers it needs.
|
||||
location /api/socket.io/ {
|
||||
proxy_pass http://tx_api;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_read_timeout 1h;
|
||||
}
|
||||
|
||||
# All other API calls.
|
||||
location /api/ {
|
||||
proxy_pass http://tx_api;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
client_max_body_size 25m;
|
||||
}
|
||||
|
||||
# SPA: serve hashed assets with long-cache; fall back to index.html.
|
||||
location ~* \.(?:js|css|woff2?|png|jpg|jpeg|gif|svg|ico|webp|map)$ {
|
||||
expires 30d;
|
||||
add_header Cache-Control "public, immutable";
|
||||
try_files $uri =404;
|
||||
}
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
add_header Cache-Control "no-cache";
|
||||
}
|
||||
}
|
||||
Generated
+1037
-609
File diff suppressed because it is too large
Load Diff
+1
-6
@@ -7,9 +7,6 @@ packages:
|
||||
autoInstallPeers: false
|
||||
|
||||
catalog:
|
||||
'@replit/vite-plugin-cartographer': ^0.5.1
|
||||
'@replit/vite-plugin-dev-banner': ^0.1.1
|
||||
'@replit/vite-plugin-runtime-error-modal': ^0.0.6
|
||||
'@tailwindcss/vite': ^4.1.14
|
||||
'@tanstack/react-query': ^5.90.21
|
||||
'@types/node': ^25.3.3
|
||||
@@ -31,9 +28,7 @@ catalog:
|
||||
|
||||
minimumReleaseAge: 1440
|
||||
|
||||
minimumReleaseAgeExclude:
|
||||
- '@replit/*'
|
||||
- stripe-replit-sync
|
||||
minimumReleaseAgeExclude: []
|
||||
|
||||
onlyBuiltDependencies:
|
||||
- '@swc/core'
|
||||
|
||||
+17
-5
@@ -89,8 +89,20 @@ async function main() {
|
||||
}
|
||||
}
|
||||
|
||||
// Create admin user
|
||||
const adminHash = await bcrypt.hash("admin123", 10);
|
||||
// Create admin user. Passwords are read from env vars so seeded
|
||||
// credentials never need to be committed; the dev-only fallbacks are
|
||||
// safe for fresh local installs but unsafe in production.
|
||||
const adminPassword =
|
||||
process.env.SEED_ADMIN_PASSWORD ??
|
||||
(process.env.NODE_ENV === "production"
|
||||
? (() => { throw new Error("SEED_ADMIN_PASSWORD must be set in production"); })()
|
||||
: "admin123");
|
||||
const userPassword =
|
||||
process.env.SEED_USER_PASSWORD ??
|
||||
(process.env.NODE_ENV === "production"
|
||||
? (() => { throw new Error("SEED_USER_PASSWORD must be set in production"); })()
|
||||
: "user123");
|
||||
const adminHash = await bcrypt.hash(adminPassword, 10);
|
||||
const [adminUser] = await db
|
||||
.insert(usersTable)
|
||||
.values({
|
||||
@@ -106,7 +118,7 @@ async function main() {
|
||||
.returning();
|
||||
|
||||
// Create regular user
|
||||
const userHash = await bcrypt.hash("user123", 10);
|
||||
const userHash = await bcrypt.hash(userPassword, 10);
|
||||
const [regularUser] = await db
|
||||
.insert(usersTable)
|
||||
.values({
|
||||
@@ -854,8 +866,8 @@ async function main() {
|
||||
|
||||
console.log("\n✅ Seeding complete!");
|
||||
console.log("\nDemo accounts:");
|
||||
console.log(" Admin: username=admin, password=admin123");
|
||||
console.log(" User: username=ahmed, password=user123");
|
||||
console.log(` Admin: username=admin, password=${adminPassword}`);
|
||||
console.log(` User: username=ahmed, password=${userPassword}`);
|
||||
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
modules = ["nodejs-24", "postgresql-16"]
|
||||
|
||||
[deployment]
|
||||
router = "application"
|
||||
deploymentTarget = "autoscale"
|
||||
|
||||
[deployment.postBuild]
|
||||
args = ["pnpm", "store", "prune"]
|
||||
env = { "CI" = "true" }
|
||||
|
||||
[workflows]
|
||||
runButton = "Project"
|
||||
|
||||
[[workflows.workflow]]
|
||||
name = "Project"
|
||||
mode = "parallel"
|
||||
author = "agent"
|
||||
|
||||
[[workflows.workflow.tasks]]
|
||||
task = "workflow.run"
|
||||
args = "test"
|
||||
|
||||
[[workflows.workflow]]
|
||||
name = "test"
|
||||
author = "agent"
|
||||
|
||||
[[workflows.workflow.tasks]]
|
||||
task = "shell.exec"
|
||||
args = "pnpm --filter @workspace/api-server test && pnpm --filter @workspace/tx-os test:e2e"
|
||||
|
||||
[workflows.workflow.metadata]
|
||||
isValidation = true
|
||||
|
||||
[agent]
|
||||
stack = "PNPM_WORKSPACE"
|
||||
expertMode = true
|
||||
|
||||
[postMerge]
|
||||
path = "scripts/post-merge.sh"
|
||||
timeoutMs = 120000
|
||||
|
||||
[[ports]]
|
||||
localPort = 8080
|
||||
externalPort = 8080
|
||||
|
||||
[[ports]]
|
||||
localPort = 8081
|
||||
externalPort = 80
|
||||
|
||||
[[ports]]
|
||||
localPort = 8082
|
||||
externalPort = 3003
|
||||
|
||||
[[ports]]
|
||||
localPort = 8083
|
||||
externalPort = 5173
|
||||
|
||||
[[ports]]
|
||||
localPort = 25785
|
||||
externalPort = 3000
|
||||
|
||||
[[ports]]
|
||||
localPort = 25786
|
||||
externalPort = 3002
|
||||
|
||||
[[ports]]
|
||||
localPort = 25787
|
||||
externalPort = 5000
|
||||
|
||||
[nix]
|
||||
channel = "stable-25_05"
|
||||
+10
-10
@@ -10,7 +10,7 @@
|
||||
|
||||
## 1. System overview
|
||||
|
||||
Tx OS is an internal "office OS" for a single-tenant organization. It is deployed as a Replit app (single Express + Postgres + Object Storage stack) with a React/Vite SPA, a small mobile-friendly UI, and a real-time notification layer over Socket.IO. The app is bilingual (Arabic / English), and the workforce uses it to:
|
||||
Tx OS is an internal "office OS" for a single-tenant organization. It is deployed as a self-hosted container stack (Express API + Postgres + S3-compatible object storage such as MinIO) with a React/Vite SPA, a small mobile-friendly UI, and a real-time notification layer over Socket.IO. The app is bilingual (Arabic / English), and the workforce uses it to:
|
||||
|
||||
- Order internal services (e.g. coffee, printing).
|
||||
- Manage notes, including shared-folder collaboration with checklist items.
|
||||
@@ -46,8 +46,8 @@ The app is **single-tenant** — there is no multi-tenant isolation requirement;
|
||||
│ Public internet │
|
||||
│ │
|
||||
│ ┌────────────┐ HTTPS ┌──────────────────────────────────┐ │
|
||||
│ │ Browser │ ─────────▶ │ Replit edge (mTLS-terminated │ │
|
||||
│ │ (SPA) │ │ proxy, deploys.replit.app) │ │
|
||||
│ │ Browser │ ─────────▶ │ TLS-terminating reverse proxy │ │
|
||||
│ │ (SPA) │ │ (Caddy / Nginx / Traefik) │ │
|
||||
│ └────────────┘ └──────────────┬──────────────────┘ │
|
||||
│ │ │
|
||||
└────────────────────────────────────────────┼───────────────────────┘
|
||||
@@ -65,18 +65,18 @@ The app is **single-tenant** — there is no multi-tenant isolation requirement;
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌───────────────┐ ┌──────────────────┐ ┌───────────────┐
|
||||
│ PostgreSQL │ │ Replit Object │ │ Replit sidecar│
|
||||
│ (Replit- │ │ Storage (GCS via │ │ http://127. │
|
||||
│ managed) │ │ signed URLs) │ │ 0.0.1:1106 │
|
||||
│ PostgreSQL │ │ MinIO / S3 │ │ Local FS │
|
||||
│ (container) │ │ (signed URLs, │ │ (dev fallback │
|
||||
│ │ │ private network)│ │ driver only) │
|
||||
└───────────────┘ └──────────────────┘ └───────────────┘
|
||||
```
|
||||
|
||||
**Trust boundaries (numbered):**
|
||||
|
||||
1. **Browser ↔ Replit edge.** TLS-terminated by the platform proxy. The SPA is the only first-party client.
|
||||
2. **Edge ↔ API server.** Loopback within the Replit container. Trust is established via the upstream proxy's mTLS. The API server treats `X-Forwarded-For` as trusted (`app.set("trust proxy", 1)`).
|
||||
1. **Browser ↔ Edge proxy.** TLS-terminated by an operator-controlled reverse proxy (Caddy / Nginx / Traefik). The SPA is the only first-party client.
|
||||
2. **Edge ↔ API server.** Loopback or private docker network. The API server treats `X-Forwarded-For` as trusted (`app.set("trust proxy", 1)`); the operator MUST ensure only the edge proxy can reach the API port.
|
||||
3. **API server ↔ Postgres.** Connection over the in-cluster network; credentials in `DATABASE_URL`. Drizzle ORM is the only query path.
|
||||
4. **API server ↔ Object Storage.** Signed-URL pattern via the Replit sidecar at `http://127.0.0.1:1106` (loopback HTTP — see Task #522 SAST suppression). All upload URLs are short-lived (15 min) and target `randomUUID()` paths under `PRIVATE_OBJECT_DIR`.
|
||||
4. **API server ↔ Object Storage.** Signed-URL pattern. In production the API server signs S3 PUT URLs against a self-hosted MinIO endpoint over the private docker network; in local development without S3 the API signs HMAC tokens validated by its own `/api/storage/_local/upload` route, persisting to the local filesystem under `./storage/`. All upload URLs are short-lived (15 min) and target `randomUUID()` paths under `PRIVATE_OBJECT_DIR`.
|
||||
5. **API server ↔ Socket.IO clients.** Same origin as the API server. Auth shared via session cookie; every socket joins exactly its `user:${userId}` room.
|
||||
|
||||
---
|
||||
@@ -163,7 +163,7 @@ For each STRIDE category, the table lists the most consequential threats, the re
|
||||
## 6. What is explicitly out of scope
|
||||
|
||||
- **Multi-tenant isolation.** Tx OS is single-tenant. There is no per-tenant data partition.
|
||||
- **Data residency.** Production data is stored wherever Replit hosts the project; no regional commitments.
|
||||
- **Data residency.** Production data is stored wherever the operator deploys the docker-compose stack (private VPS or on-prem). The operator chooses the region and is responsible for any regulatory commitments.
|
||||
- **Penetration testing of the live deployment.** This document is design-level only.
|
||||
- **Compliance frameworks** (SOC 2 / ISO 27001). Not in scope today; the threat model is structured so the most material gaps would map cleanly into a future compliance effort if one is opened.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user