Use Database Transaction Logs to Implement Observer Pattern

Having an observer in the application is not good

“To understand is to perceive patterns” — Isaiah Berlin

Design patterns help you solve common problems and keep your code maintainable, and extensible.

\
The observer pattern is one such pattern.

\
A set of observers keep watching a change in an object. On any change in the object, the observers are notified of the change. This pattern is used when a change in an object should trigger changes in the observing objects.

\
In the case of web applications, all state is maintained in the database entries and tables. There are many scenarios when you need to post-process an entry or trigger a job on data change in a table. The observer pattern is used for this purpose.

How does the Observer Pattern work?

Observer pattern

\
Essentially what happens is observers subscribe to changes in the subject. When there is a change in the subject it notifies the observers. Observers subscribe to these events to take actions based on specific changes in the subject.

\
Once the observers receive this notification, they take the action they are supposed to take.

Ways to implement Observer Pattern in Web apps

There are many ways this pattern can be implemented. Here are the most common ways I have personally witnessed —

Utilizing Web Framework’s Observers

Many web frameworks have this pattern built-in or there are libraries that bring in its functionality. You can simply add observers in model code that get triggered on record creation, update, or deletion.

\
A typical observer in web frameworks looks like this. Example shows how Ruby on Rails provides the feature to implement observers.

\
Drawbacks — As the project grows it becomes hard to maintain the observers. You tend to create multiple services and applications. What if a service needs to trigger a job in another service, inbuilt observer patterns can complicate the code. And keeping track of observers in all projects can be a nightmare.

Periodic jobs

This is the most common approach I have seen programmers take, primarily because of time constraints. A simple background job is run to scan through the database tables and based on the changes the observer code is run either synchronously or asynchronously.

\
Drawbacks — This approach is an expensive one. Database scans take time and since the jobs are run periodically the changes done by these jobs are delayed. Moreover, these jobs become a single point of failure.

Using Transaction Logs to implement Observer Pattern

Observer pattern using transaction logs of DB

Transaction logs are generated by the database master. These logs help the slave to catch up with the changes. In this design, DB transaction logs are read to find changes in various tables, and observers are triggered based on the type of events. Queues can be used to solve the producer-consumer problem.

Why use this design?

There are many advantages to this design, here are a few of them —

  • It’s easy to maintain observers in one place.
  • This design helps you to maintain the observers irrespective of the frameworks you use for your apps.
  • Avoids spaghetti in your applications and services.
  • Observers can be scaled independently.
  • Almost instantaneous triggering of observers.

Final Words

We had a spaghetti of code that was triggering observers in different web services. Implementing this scheme took us some time but it was worth it. We use this pipeline to denormalize objects, maintain audit logs, push data to analytics, etc.

Leave a Reply

Your email address will not be published. Required fields are marked *

Instagram

Why do People Say: "Developers are Lazy"?
The saying “work smart, not hard” is applicable for programmers.
.
https://hackernoon.com/why-do-people-say-developers-are-lazy

.
Author: Aga Wozniak
.
.
.
.
.
#blog #100Daysofcode #javascript #vuejs #datascientist #peoplewhocode #learntocode #coding #developerlife #frontenddeveloper #backenddeveloper #fullstackdeveloper #developer #webdeveloper #thedevlife #phpdeveloper #computerscience #programmer #programmingisfun #codingdays
...

Mitigating the DDOS Threats Facing Banks and Fintechs
As much as digitization and cyber simplified banking, the Fintech sector has left digital payment activity exposed to malicious and suspicious activity.
.
https://hackernoon.com/mitigating-the-ddos-threats-facing-banks-and-fintechs

.
Author: Josh Horowitz
.
.
.
.
.
#blog #100Daysofcode #javascript #vuejs #datascientist #peoplewhocode #learntocode #coding #developerlife #frontenddeveloper #backenddeveloper #fullstackdeveloper #developer #webdeveloper #thedevlife #phpdeveloper #computerscience #programmer #programmingisfun #codingdays
...

24 Best JavaScript Blogs and Websites
In this overview, we have compiled a list of popular sites, as well as JS blogs that are worth reading and keeping in your bookmarks.
.
https://hackernoon.com/24-best-javascript-blogs-and-websites

.
Author: natashatsybliyenko
.
.
.
.
.
#blog #100Daysofcode #javascript #vuejs #datascientist #peoplewhocode #learntocode #coding #developerlife #frontenddeveloper #backenddeveloper #fullstackdeveloper #developer #webdeveloper #thedevlife #phpdeveloper #computerscience #programmer #programmingisfun #codingdays
...

The Projects Working to Lower Ethereum Gas Fees
As more investors try their hand at DeFi, gas fees are shooting over the roof, making engaging with decentralized apps uneconomical for most users.
.
https://hackernoon.com/ethereum-gas-fees-are-there-any-projects-working-to-optimize-eth-gas-fees

.
Author: CryptoVirally SLR
.
.
.
.
.
#blog #100Daysofcode #javascript #vuejs #datascientist #peoplewhocode #learntocode #coding #developerlife #frontenddeveloper #backenddeveloper #fullstackdeveloper #developer #webdeveloper #thedevlife #phpdeveloper #computerscience #programmer #programmingisfun #codingdays
...

On the Edge of a New Year: IT Predictions for 2022
The single biggest cause of network errors are people.
.
https://hackernoon.com/an-interview-with-uplogix-ceo-lisa-frankovitch

.
Author: Mignonette Garnier
.
.
.
.
.
#blog #100Daysofcode #javascript #vuejs #datascientist #peoplewhocode #learntocode #coding #developerlife #frontenddeveloper #backenddeveloper #fullstackdeveloper #developer #webdeveloper #thedevlife #phpdeveloper #computerscience #programmer #programmingisfun #codingdays
...

How to Modernize IBM i Applications
If you’re like most IBM i users, you know how much value your IBM i data and applications bring to your business. Your end-users, however, may not. In today’s world of rich user experience, fast-paced application development, and constantly evolving customer expectations, IBM i applications are unde…
.
https://hackernoon.com/how-to-modernize-ibm-i-applications

.
Author: Lansa
.
.
.
.
.
#blog #100Daysofcode #javascript #vuejs #datascientist #peoplewhocode #learntocode #coding #developerlife #frontenddeveloper #backenddeveloper #fullstackdeveloper #developer #webdeveloper #thedevlife #phpdeveloper #computerscience #programmer #programmingisfun #codingdays
...