Task #148: Make pnpm --filter @workspace/db run push work without manual SQL
## Original task
`pnpm --filter @workspace/db run push` was failing with a duplicate-key
error on `app_permissions` (and a missing FK on
`executive_meeting_notifications`) because legacy data in the dev DB
violates constraints the schema now declares. Devs had to drop into psql
to fix it, which made bootstrapping painful and left
`role_permission_audit` reliant on hand-applied SQL.
## Changes
- Added `lib/db/scripts/pre-push-cleanup.ts`, an idempotent cleanup that:
- Collapses duplicate `app_permissions` rows to one per
`(app_id, permission_id)` so the composite PK can be added.
- Deletes orphan `executive_meeting_notifications` rows so the new
`ON DELETE CASCADE` FK can be added.
- Skips both checks when the tables don't exist yet (fresh DB
no-op).
- Wired the script into `lib/db/package.json` so both `push` and
`push-force` run cleanup first (`pnpm run pre-push-cleanup && drizzle-kit push ...`).
- Added `tsx` to `lib/db` devDependencies (catalog version) so the
package can run the cleanup without leaning on another workspace.
- Updated `replit.md` Deployment / Migration Runbook to reflect that
cleanup is now automatic — no manual SQL required in any environment.
## Verification
- `pnpm --filter @workspace/db run push` now collapses 2 duplicate
groups + removes 1978 orphan notifications, then succeeds with
`[✓] Changes applied`.
- Re-running push is idempotent: second run reports no duplicates and
no orphans, then succeeds.
- `pnpm --filter @workspace/db run push-force` (used by
`scripts/post-merge.sh`) was also verified end-to-end.
- Confirmed `app_permissions` now has the composite PK,
`executive_meeting_notifications` has the cascade FK, and
`role_permission_audit` matches the schema.
## Notes
- A pre-existing unrelated typecheck error in
`artifacts/api-server/src/routes/executive-meetings.ts` (font
settings `scope` overload) was confirmed to exist on `main` before
any changes here and is out of scope for this task.
- Proposed follow-up #213 to move from `push`/`push-force` to
versioned `drizzle-kit migrate` so legacy-data backfills are checked
in instead of living in a generic pre-push script.
Replit-Task-Id: 2efc4e22-0c65-48a1-b573-319474698b96
This commit is contained in:
riyadhafraa
+5
-3
@@ -8,8 +8,9 @@
|
||||
"./schema": "./src/schema/index.ts"
|
||||
},
|
||||
"scripts": {
|
||||
"push": "drizzle-kit push --config ./drizzle.config.ts",
|
||||
"push-force": "drizzle-kit push --force --config ./drizzle.config.ts"
|
||||
"pre-push-cleanup": "tsx ./scripts/pre-push-cleanup.ts",
|
||||
"push": "pnpm run pre-push-cleanup && drizzle-kit push --config ./drizzle.config.ts",
|
||||
"push-force": "pnpm run pre-push-cleanup && drizzle-kit push --force --config ./drizzle.config.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"drizzle-orm": "catalog:",
|
||||
@@ -20,6 +21,7 @@
|
||||
"devDependencies": {
|
||||
"@types/node": "catalog:",
|
||||
"@types/pg": "^8.18.0",
|
||||
"drizzle-kit": "^0.31.9"
|
||||
"drizzle-kit": "^0.31.9",
|
||||
"tsx": "catalog:"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import pg from "pg";
|
||||
|
||||
const { Client } = pg;
|
||||
|
||||
if (!process.env.DATABASE_URL) {
|
||||
throw new Error(
|
||||
"DATABASE_URL must be set. Did you forget to provision a database?",
|
||||
);
|
||||
}
|
||||
|
||||
async function tableExists(client: pg.Client, table: string): Promise<boolean> {
|
||||
const result = await client.query(
|
||||
`SELECT 1 FROM information_schema.tables WHERE table_schema = 'public' AND table_name = $1`,
|
||||
[table],
|
||||
);
|
||||
return (result.rowCount ?? 0) > 0;
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const client = new Client({ connectionString: process.env.DATABASE_URL });
|
||||
await client.connect();
|
||||
try {
|
||||
await client.query("BEGIN");
|
||||
|
||||
if (await tableExists(client, "app_permissions")) {
|
||||
const dupes = await client.query(
|
||||
`SELECT
|
||||
count(*)::int AS dup_groups,
|
||||
coalesce(sum(c - 1), 0)::int AS extra_rows
|
||||
FROM (
|
||||
SELECT app_id, permission_id, count(*) AS c
|
||||
FROM app_permissions
|
||||
GROUP BY app_id, permission_id
|
||||
HAVING count(*) > 1
|
||||
) d`,
|
||||
);
|
||||
const dupGroups = (dupes.rows[0]?.dup_groups ?? 0) as number;
|
||||
const extraRows = (dupes.rows[0]?.extra_rows ?? 0) as number;
|
||||
if (dupGroups > 0) {
|
||||
await client.query(
|
||||
`CREATE TEMP TABLE app_permissions_dedup ON COMMIT DROP AS
|
||||
SELECT DISTINCT app_id, permission_id FROM app_permissions`,
|
||||
);
|
||||
await client.query(`DELETE FROM app_permissions`);
|
||||
await client.query(
|
||||
`INSERT INTO app_permissions (app_id, permission_id)
|
||||
SELECT app_id, permission_id FROM app_permissions_dedup`,
|
||||
);
|
||||
console.log(
|
||||
`[pre-push] Removed ${extraRows} duplicate app_permissions row(s) across ${dupGroups} (app_id, permission_id) key(s).`,
|
||||
);
|
||||
} else {
|
||||
console.log("[pre-push] app_permissions has no duplicates.");
|
||||
}
|
||||
} else {
|
||||
console.log("[pre-push] app_permissions table not present yet — skipping dedup.");
|
||||
}
|
||||
|
||||
if (
|
||||
(await tableExists(client, "executive_meeting_notifications")) &&
|
||||
(await tableExists(client, "executive_meetings"))
|
||||
) {
|
||||
const orphan = await client.query(
|
||||
`DELETE FROM executive_meeting_notifications n
|
||||
WHERE n.meeting_id IS NOT NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM executive_meetings m WHERE m.id = n.meeting_id
|
||||
)`,
|
||||
);
|
||||
const removed = orphan.rowCount ?? 0;
|
||||
if (removed > 0) {
|
||||
console.log(
|
||||
`[pre-push] Removed ${removed} orphan executive_meeting_notifications row(s).`,
|
||||
);
|
||||
} else {
|
||||
console.log("[pre-push] No orphan executive_meeting_notifications rows.");
|
||||
}
|
||||
} else {
|
||||
console.log(
|
||||
"[pre-push] executive_meeting_notifications/executive_meetings not both present — skipping orphan cleanup.",
|
||||
);
|
||||
}
|
||||
|
||||
await client.query("COMMIT");
|
||||
} catch (err) {
|
||||
await client.query("ROLLBACK").catch(() => {});
|
||||
throw err;
|
||||
} finally {
|
||||
await client.end();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error("[pre-push] cleanup failed:", err);
|
||||
process.exit(1);
|
||||
});
|
||||
Generated
+3
@@ -669,6 +669,9 @@ importers:
|
||||
drizzle-kit:
|
||||
specifier: ^0.31.9
|
||||
version: 0.31.9
|
||||
tsx:
|
||||
specifier: 'catalog:'
|
||||
version: 4.21.0
|
||||
|
||||
lib/object-storage-web:
|
||||
devDependencies:
|
||||
|
||||
@@ -73,31 +73,11 @@
|
||||
|
||||
- **Rich-text columns are PostgreSQL `text` (no length cap).** `executive_meetings.title_ar`, `executive_meetings.title_en`, and `executive_meeting_attendees.name` were widened from `varchar(500)` / `varchar(255)` to `text` to hold sanitized Tiptap HTML. The schema declarations live in `lib/db/src/schema/executive-meetings.ts`.
|
||||
- **`pnpm --filter @workspace/db run push-force` must run in every environment** (dev, staging, production) after deploying schema changes. Dev is covered automatically by `scripts/post-merge.sh`. Staging and production must run the same command on each deploy so their `title_ar` / `title_en` / `name` columns match the code; otherwise long rich-text saves will be rejected by the old varchar limits.
|
||||
- **One-time pre-push cleanup (run BEFORE `pnpm --filter @workspace/db run push-force` in any environment that has not been pushed since these constraints were added).** Two pieces of legacy data block the push because the old DB never enforced the constraints the schema now declares:
|
||||
1. Duplicate rows in `app_permissions` (the schema declares a composite primary key on `(app_id, permission_id)`).
|
||||
2. Orphan rows in `executive_meeting_notifications` whose `meeting_id` no longer exists (the schema declares `ON DELETE CASCADE`, which was never enforced because the FK was missing).
|
||||
- **Pre-push cleanup is automatic.** Both `pnpm --filter @workspace/db run push` and `push-force` now run `lib/db/scripts/pre-push-cleanup.ts` first. That script is idempotent and:
|
||||
1. Collapses duplicate rows in `app_permissions` to one per `(app_id, permission_id)` so the composite primary key declared by the schema can be created on legacy DBs.
|
||||
2. Deletes orphan `executive_meeting_notifications` rows whose `meeting_id` no longer exists, so the new `ON DELETE CASCADE` foreign key the schema declares can be added on legacy DBs.
|
||||
|
||||
Run this idempotent SQL once per environment before the push:
|
||||
|
||||
```sql
|
||||
BEGIN;
|
||||
|
||||
-- Collapse duplicate app_permissions rows to one per (app_id, permission_id)
|
||||
CREATE TEMP TABLE app_permissions_dedup AS
|
||||
SELECT DISTINCT app_id, permission_id FROM app_permissions;
|
||||
DELETE FROM app_permissions;
|
||||
INSERT INTO app_permissions (app_id, permission_id)
|
||||
SELECT app_id, permission_id FROM app_permissions_dedup;
|
||||
|
||||
-- Drop notifications whose meeting was already deleted
|
||||
DELETE FROM executive_meeting_notifications n
|
||||
WHERE n.meeting_id IS NOT NULL
|
||||
AND NOT EXISTS (SELECT 1 FROM executive_meetings m WHERE m.id = n.meeting_id);
|
||||
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
Then run `pnpm --filter @workspace/db run push-force`. The push has been verified end-to-end against the dev database and is idempotent on subsequent runs.
|
||||
Both checks are skipped automatically on a fresh DB (the table-existence guard makes them no-ops). No manual SQL is needed in any environment — `pnpm --filter @workspace/db run push` runs cleanly against both fresh and existing dev DBs, and `scripts/post-merge.sh` continues to use `push-force` so the same cleanup runs after every task merge. All schema tables (notably `role_permission_audit` and `permission_audit`) are created via the normal push path.
|
||||
|
||||
## Task #207 — Custom subheadings inside attendee cells (April 2026)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user