Notification Priority?
Does Novu have any ability to make some notifications more important than others? For example I'm using Novu to send Reset Password emails, and I also want to use it to send some marketing messages. Issues that we've had in the past mixing platforms that send both critical messages and marketing messages, always come back to a problem where there was a massive amount of marketing messages queued up, and the critical ones are waiting in line behind that.
So lets say I had a topic that kicked off an email to 100k users. This is a marketing message and is "low" priority to send.
Is there anyway for me to make sure that the "Reset Password" emails go to the front of the queue when they are triggered?
30 Replies
@isaiahdahl
Currently, novu does not support notification priority.
We have concept of critical workflows. Critical workflows are those workflows of which preferences can not be changed by subscriber.
https://docs.novu.co/concepts/preferences#exclude-workflows-from-preferences-critical-workflow
@Pawan Jain yes I’m aware and make use of that. But will events for critical workflows in the FIFO queue be processed in line with everything else?
@isaiahdahl, you just advanced to level 2!
Other than ignoring subscriber status is there any other difference to how critical workflows are processed?
@isaiahdahl not really, they are processed the same. To make sure I understand the full use case. Can you explain what you do, and where would this comes hand, as the full use case. I know of 2 (could make the claim of 3) kind of priorities. So I'd love to deep dive into the use case and a bit on how would this affect your product.
Thanks 😄
@Tomer Barnea Sure thing!
Simply put I want to use Novu for system notifications that are critical ie. "Password Reset" or "You've Been Invited to Join "Person's" Team". But then I also want to use it to power some marketing drip/bucket campaigns. I would have like a "new-music-for-you" workflow that is triggered from our API with personalization context to generate an email with personalized songs (context: we're a Music Publishing company) . That "new-music-for-you" notification could trigger to 100k+ people. And when that happens, I wouldn't want the next "password-reset" notification to be stuck in line behind all of those "new-music-for-you" emails.
Does that make sense?
Ok, so if I understand this correctly you are looking to solve priority for the queue management, and this is to ensure that no matter what (based on priority) some notifications will be processed. Is that correct? Are you self-hosting?
It's less about whether or not the notification will eventually be processed or not. I'm assuming they would all process, It's about the timely manner for critical notifications.
Most importantly, I'm looking for critical notifications (ones people can't unsubscribe from) to not wait in queue behind less important marketing notifications. If I trigger 100k 'new-music' workflows when I trigger the 'password-reset' workflow for a customer, the queue picks that 'password-reset' workflow up earlier then if it was just FIFO.
That 'password-reset' workflow should end up in a customers inbox within ~45 seconds. But if i happen to have just queued up a big marketing drip, it could be like 20min before they got that email.
Yes self-hosting.
That 'password-reset' workflow should end up in a customers inbox within ~45 seconds. But if i happen to have just queued up a big marketing drip, it could be like 20min before they got that email.
Yes self-hosting.
Ok, got it, thanks!
So, there are two options here: one is a capability, and the other is the deployment. Right now, we don't have that capability; I'll add that feedback to our priority item to make sure we address this when we build this. As for a current solution, you can add more workers to make the queue processing faster. I am not sure if that's an option, but in the cloud version we have auto spawn for more workers, so that might be a solution.
Ok so are you indicating that you already have some kind of priority management in the roadmap and you'll add these comments?
Is this the type of thing you'd accept a partial implementation PR on? Like if it simply was just assigning a default priority of 2 to everything and then 1 if the workflow is critical=true?
We would defintly consider PR for this. What version are you using?
And yes, this is in the backlog. And I just added our conversation in 🙂
0.23
Got it. Yes, we would consider that, keep in mind we are moving to V2 now (code-backed workflows) and this is the main focus.
Yeah I've seen that. That's mostly workflow creating/editing changes though right? or or is there alot of re-write in the whole system that's breaking?
It is a major rewrite of the system, but most of the APIs should be the same. When it comes to workflow creation as well as execution there are more changes. Write now we are adding new GUI-based capabilities for V2.
tbh, major might be an overstretch, but many parts are updated, and more updates are coming soon. Most of the system is the same.
It looks like the WorkflowQueueService and QueueBase Service already pass in job options to bullmq, so it's just a matter of adding priority as an option to the event trigger usecases.
for example https://github.dev/novuhq/novu/blob/08bd4266fab88417a974b06b3a502fd96d7354ed/apps/api/src/app/events/usecases/parse-event-request/parse-event-request.usecase.ts#L140-L141
if at here there was a way to know whether or not the workflow was critical or not, which seems liek it's readily available from line 47, then it should be a pretty simple check to just set the
options: { priority : 1 | 2} sorta thing.I smell a PR cooking here😂 🧑🍳
Could it be just this easy?
https://github.com/novuhq/novu/pull/6489
GitHub
feat(api): allow workflow trigger to specify a priority that's pass...
What changed? Why was the change needed?
Allows workflow trigger to specify a priority with a value that's then passed onto BullMQ and determines the priority processing of the items on the...
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
@isaiahdahl I just had a quick look at the PR, I'll pass it over to the team to take a deeper look. But this looks nice and clean. Noice!
Nice!
Hey @isaiahdahl I should have an answer tomorrow how can we push it 🙂
Sweet, let me know if I need to rebase or whip it up from a different branch or something.
@isaiahdahl, you just advanced to level 3!
Hey @isaiahdahl we would like to add this, just one thing we need a flag to control this. The team should leave a comment on the PR soon, let's get this merged 😄
🙌
@Tomer Barnea seems like the PRs for this feature have been stuck in a review after all changes were fixed for quite some time. Would love to see them merged!
https://github.com/novuhq/novu/pull/6489
https://github.com/novuhq/novu/pull/6528
GitHub
feat(api): Allow Events/Workflows to be processed/triggered with pr...
What changed? Why was the change needed?
Allows workflow trigger to specify a priority with a value that's then passed onto BullMQ and determines the priority processing of the items on the...
Hey @isaiahdahl let me follow up on that tomorrow, the team right now working on big updates, but let me keep you posted.
Hi @isaiahdahl , thanks for following up. I have approved the PR whilst noting a caveat with regards to the Community Edition OpenAPI spec invalidity.
There are a couple of outstanding comments to address a few linting issues. If you can commit those changes, we will merge these today.