Best practices when designing a workflow engine using timers?

Question

I'm working in a OOP language called Fantom, similar to Java, to create a workflow engine. My code sends emails to users, and it awaits for users to click a button within the email. In my current working code, I send the email, then I suspend the thread with something like Thread.sleep(1000), and check every few seconds to see if the user has responded. I send another email depending on their response, and have several of these emails and timers. I'm expecting users to have up to two weeks to respond to each of the email. While my solution works, I'm wondering what are some best practices for an application like this on an architectural level? What are some resources to help me design this better? Thanks

Answer 1

You can design your control flow the other way around to avoid multiple timers.

1. When you send an email, store any context you need for a response. E.g. write this context to a database or file.
2. Regularly check for responses. E.g. you might have a single thread that checks incoming emails. This may involve a timer, or may simply await push-notifications (e.g. when using push-IMAP).
3. Once you receive a response, load the necessary information (from the database) and send the next email.

Why might this be a better solution?

• We can handle many more pending responses because we don't need one thread per pending response. This saves you memory and CPU time, at the cost of a tiny amount of database storage.
• As a consequence this also deals much better with the case if a response never comes.
• If your application crashes it can be restarted and will continue with the saved state.

As a general observation, it is often better to have events trigger reactions, instead of polling or waiting for events. Vaguely related concepts include the Tell, don't Ask Principle, promises, reactive user interfaces (e.g. JavaFX), asynchronous web servers like NodeJS, and the idea of continuations: we can store the context of a thread and continue it later. And of course the broad class of event-driven architectures in general, in particular ideas such as event sourcing architectures.