feat: notification categories#3958
Conversation
|
🍹 The Update (preview) for dailydotdev/api/prod (at b1c542e) was successful. ✨ Neo Code ReviewThis is a standard new-image rollout adding notification category filtering, with two moderate concerns: the non-concurrent index build inside the migration transaction (should be pre-built concurrently in prod) and a Kubernetes Secret replacement that removes OneSignal keys, creating a brief replacement window. 🟡 Moderate Risk.This deployment ships the notification category filtering feature: a new 🟡 Warning — Index creation under migration transaction: The 🟡 Warning — Secret replacement ( 🔵 Info — Historical rows will have Resource Changes Name Type Operation
~ vpc-native-clean-gifted-plus-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-current-streak-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-tag-materialized-views-cron kubernetes:batch/v1:CronJob update
~ vpc-native-daily-digest-cron kubernetes:batch/v1:CronJob update
~ vpc-native-rotate-weekly-quests-cron kubernetes:batch/v1:CronJob update
~ vpc-native-calculate-top-readers-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-source-public-threshold-cron kubernetes:batch/v1:CronJob update
+- vpc-native-k8s-secret kubernetes:core/v1:Secret create-replacement
~ vpc-native-squad-posts-analytics-refresh-cron kubernetes:batch/v1:CronJob update
~ vpc-native-materialize-yearly-best-post-archives-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-images-cron kubernetes:batch/v1:CronJob update
~ vpc-native-hourly-notification-cron kubernetes:batch/v1:CronJob update
~ vpc-native-validate-active-users-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-opportunities-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-analytics-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-expire-super-agent-trial-cron kubernetes:batch/v1:CronJob update
~ vpc-native-post-lifecycle-state-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-post-analytics-history-day-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-achievement-rarity-cron kubernetes:batch/v1:CronJob update
+ vpc-native-api-db-migration-5e12561c kubernetes:batch/v1:Job create
~ vpc-native-update-highlighted-views-cron kubernetes:batch/v1:CronJob update
~ vpc-native-sync-subscription-with-cio-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-trending-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-users-cron kubernetes:batch/v1:CronJob update
~ vpc-native-worker-job-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-generate-search-invites-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-user-companies-cron kubernetes:batch/v1:CronJob update
~ vpc-native-temporal-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-clean-stale-user-transactions-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-updated-sync-cron kubernetes:batch/v1:CronJob update
~ vpc-native-check-analytics-report-cron kubernetes:batch/v1:CronJob update
~ vpc-native-deployment kubernetes:apps/v1:Deployment update
- vpc-native-api-clickhouse-migration-3571d872 kubernetes:batch/v1:Job delete
~ vpc-native-personalized-digest-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-update-tags-str-cron kubernetes:batch/v1:CronJob update
~ vpc-native-post-analytics-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-personalized-digest-cron kubernetes:batch/v1:CronJob update
~ vpc-native-materialize-monthly-best-post-archives-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-analytics-history-clickhouse-cron kubernetes:batch/v1:CronJob update
- vpc-native-api-db-migration-3571d872 kubernetes:batch/v1:Job delete
~ vpc-native-bg-deployment kubernetes:apps/v1:Deployment update
... and 15 other changes |
| // by type using a (userId, type, date) index without scanning a heavy user's | ||
| // rows and discarding non-matching types post-join. | ||
| @Column({ type: 'text', nullable: true }) | ||
| type?: NotificationType; |
| -- user_notification is one of the largest tables, so this is intentionally a | ||
| -- bounded loop rather than a single UPDATE. | ||
|
|
||
| DO $$ |
There was a problem hiding this comment.
I'm not sure on the consequences, but i'd do the limit per user and not 10k globally
|
@rebelchris another approach, instead of materializing the category, maybe we should just store the notification types to category mapping and then in sql we will just look for specific notification types instead of category? this will make it easy to change categories |
|
|
||
| public async up(queryRunner: QueryRunner): Promise<void> { | ||
| await queryRunner.query(/* sql */ ` | ||
| ALTER TABLE "user_notification" ADD COLUMN IF NOT EXISTS "type" text; |
There was a problem hiding this comment.
why do we need type here, would join not work?
| const notificationCategoryToTypes: Partial< | ||
| Record<NotificationFilterCategory, NotificationType[]> | ||
| > = { | ||
| [NotificationFilterCategory.Upvotes]: [ | ||
| NotificationType.ArticleUpvoteMilestone, | ||
| NotificationType.CommentUpvoteMilestone, | ||
| ], |
There was a problem hiding this comment.
We don't have this kind of mapping somewhere else, eg. per CIO topics for example?
| // (userId, type, date) index can drive the query for heavy users. | ||
| if (categoryFilter && 'include' in categoryFilter) { | ||
| builder.queryBuilder.andWhere(`un."type" = ANY(:types)`, { | ||
| types: categoryFilter.include, |
There was a problem hiding this comment.
we need to check categoryFilter.include is non empty, if it is it will break SQL syntax i think, same below
Adding categories to notifications so we can filter them out frontend wise