Kysely•14mo ago

Transaction response time

Hey everyone, in my code I have a createUser(), createProfile(), createBusiness(), and createAddress() for this example. Currently, I have creaeUser() execute on it's own and then the other 3 free functions execute as one transaction. Which gives me the benefit of having a fast response time for when a user creates an account. (the loading screen is quick) but I don't know if that benefit outweighs the risk of having user records in the database without a profile, business, and/or address records - *if for some reason the transaction fails but the createUser() executed fine. I really like the fast signup user experience provided by just awaiting createUser() without having to wait for a whole transaction to complete. Is there any way to handle this scenario other than optimizing the functions within the transaction as much as possible?

21 Replies

Igal•14mo ago

Hey 👋 You could create user immediately and offload the rest of the data + user id to a queue with a retry mechanism and dead letter queue. thus the other entities will be created eventually and asynchronously.

NazCodeland•14mo ago

async createUser(data) {
  const userData = coerceInputData(data, 'emailVerified');

  get(isCurrentSignUpABusinessStore) ? (userData.role = 'business') : (userData.role = 'client');

  // create 'profile','address' and 'business' if it's a   business signup
  const result = db.transaction().execute(async (trx) => {
  // create user
  const query = db.insertInto('User').values(userData);
  // return user
  const result = supportsReturning ? await query.returningAll().executeTakeFirstOrThrow() : await query.executeTakeFirstOrThrow().then(async () => {
    return await db.selectFrom('User').selectAll().where('email', '=', `${userData.email}`).executeTakeFirstOrThrow();
});

  const userId = result.id;
  const profile = await createProfile(
{ 
  ...get(currentSigningUpBusinessStore).profile,
  userId: userId
},
  trx
);

let business;
if (get(isCurrentSignUpABusinessStore)) {
  business = await createBusiness(
    {
      ...get(currentSigningUpBusinessStore).business,          userId: userId
    },
    trx
  );
}

const address = await createAddress(                  {
        ...get(currentSigningUpBusinessStore).address,
        ...(get(isCurrentSignUpABusinessStore) ? {   businessId: business?.id 
      } : { profileId: profile.id })
    },
    trx
  );
  return result;
});

return coerceReturnData(await result, 'emailVerified');
},

async createUser(data) {
  const userData = coerceInputData(data, 'emailVerified');

  get(isCurrentSignUpABusinessStore) ? (userData.role = 'business') : (userData.role = 'client');

  // create 'profile','address' and 'business' if it's a   business signup
  const result = db.transaction().execute(async (trx) => {
  // create user
  const query = db.insertInto('User').values(userData);
  // return user
  const result = supportsReturning ? await query.returningAll().executeTakeFirstOrThrow() : await query.executeTakeFirstOrThrow().then(async () => {
    return await db.selectFrom('User').selectAll().where('email', '=', `${userData.email}`).executeTakeFirstOrThrow();
});

  const userId = result.id;
  const profile = await createProfile(
{ 
  ...get(currentSigningUpBusinessStore).profile,
  userId: userId
},
  trx
);

let business;
if (get(isCurrentSignUpABusinessStore)) {
  business = await createBusiness(
    {
      ...get(currentSigningUpBusinessStore).business,          userId: userId
    },
    trx
  );
}

const address = await createAddress(                  {
        ...get(currentSigningUpBusinessStore).address,
        ...(get(isCurrentSignUpABusinessStore) ? {   businessId: business?.id 
      } : { profileId: profile.id })
    },
    trx
  );
  return result;
});

return coerceReturnData(await result, 'emailVerified');
},

Hey! Just for context, I have createUser(), createProfile(), createBusiness() and createAddress() so, now I need to learn about how to make a queue with a retry mechanism and dead letter queue, but before I do that I have a question about "... thus the other entities will be created eventually and asynchronously." If the response is returned immediately after createUser() has been created, so the front-end UI proceeds as it's awaiting the return of createUser() and then say later, in the transaction, something goes wrong, how will that work. Does the user get signed out? Does the record created by createUser() get deleted?

Igal•14mo ago

and then say later, in the transaction, something goes wrong, how will that work.

you either poll for the eventually consistent data OR its errors, or use web sockets (preferably a managed service for that) that inform the FE things succeeded/failed.

Does the user get signed out? Does the record created by createUser() get deleted?

depends on product requirements and UX you're going for.

NazCodeland•14mo ago

What do you mean by "Poll"? as in choose?

Igal•14mo ago

https://en.wikipedia.org/wiki/Polling_(computer_science)

Polling (computer science)

Polling, or interrogation, refers to actively sampling the status of an external device by a client program as a synchronous activity. Polling is most often used in terms of input/output (I/O), and is also referred to as polled I/O or software-driven I/O. A good example of hardware implementation is a watchdog timer.

Igal•14mo ago

polling is when you keep requesting for the data or its status in intervals from the FE

NazCodeland•14mo ago

I see, I heard of it but I thought ppl were saying "pulling" lol hmm, in the above code if I moved

...
 const userData = coerceInputData(data, 'emailVerified');

  get(isCurrentSignUpABusinessStore) ? (userData.role = 'business') : (userData.role = 'client');

  // create 'profile','address' and 'business' if it's a   business signup
  const result = db.transaction().execute(async (trx) => {
  // create user
  const query = db.insertInto('User').values(userData);
  // return user
  const result = supportsReturning ? await query.returningAll().executeTakeFirstOrThrow() : await query.executeTakeFirstOrThrow().then(async () => {
    return await db.selectFrom('User').selectAll().where('email', '=', `${userData.email}`).executeTakeFirstOrThrow();
});
...

...
 const userData = coerceInputData(data, 'emailVerified');

  get(isCurrentSignUpABusinessStore) ? (userData.role = 'business') : (userData.role = 'client');

  // create 'profile','address' and 'business' if it's a   business signup
  const result = db.transaction().execute(async (trx) => {
  // create user
  const query = db.insertInto('User').values(userData);
  // return user
  const result = supportsReturning ? await query.returningAll().executeTakeFirstOrThrow() : await query.executeTakeFirstOrThrow().then(async () => {
    return await db.selectFrom('User').selectAll().where('email', '=', `${userData.email}`).executeTakeFirstOrThrow();
});
...

out of the transaction, that would leave createProfile, createBusiness and createAddress in the one transaction. I need to keep track of the status of this transaction I can keep track of it by polling it or use web sockets to do it Just wanted to make sure I'm following but if that is correct it's on me to implement error handling to notify myself and maybe the user along with keeping a record of which user (userId) the transaction failed for

Igal•14mo ago

you're too focused on the low-level details

handler(req, res) {
  const Aresult = await A(req.body);

  await enqueue({ body: req.body, Aresult });

  res.status(201).json(Aresult);
}

handler(req, res) {
  const Aresult = await A(req.body);

  await enqueue({ body: req.body, Aresult });

  res.status(201).json(Aresult);
}

enqueue would just enqueue the message in a queue (SQS, RabbitMQ, etc.). another place in your codebase will consume the queue and do the transaction.

NazCodeland•14mo ago

Okay, I follow so far, I am assuming these SQS, RabbitMW platforms /API offer error handling etc or somewhere inbetween I need to add that incase something in the queue goes wrong

Igal•14mo ago

yeah messages can be "dead-lettered" when they fail too much (a configurable threshold) or they expire (TTL) or the queue is full

NazCodeland•14mo ago

okay, I am assuming messages are the individual queued items in the queue awaiting exeuction

Igal•14mo ago

yeah the structure of a message body is totally your decision

NazCodeland•14mo ago

I haven't looked into SQS, RabbitMW etc but you are saying that I can just pass it my transaction as a queue and somewhere else in the code I will be using SQS/RabbitMQ to process those queues very nice if I am following you correctly so far, that brings me to my other question I asked you in the other channel I guess I'll close that channel and continue here, unless you think it's a good idea to continue there?

Igal•14mo ago

let's continue here

NazCodeland•14mo ago

You said "... monitored (+ alerts), you might have queues in between with retry mechanism + dead letter queue. dead letter queues could be consumed too for error handling ..." and "The system could be event-sourced, so you always have the event persisted and could replay it from the event stream. " or just returning a response the moment as error occurs is also fine depending on the use case we already talked about queues with retry mechanisms, you said "messages" can be dead lettered - I am assuming this is how you determine when to stop retrying ( a threshold that's configurable) "The system could be event-sourced, so you always have the event persisted and could replay it from the event stream. " - that's new does event-sourced mean that any function call, api call or get/mutations are recorded?

Gaming

Programming

Transaction response time