At any time you modify data in a database no matter the type: SQL, No-SQL, even File-System I/O then a transaction is something to know about. One could liken a transaction to a checkout process at the store, yet I am speaking about bulk transactions.

Speedier and More Efficient

The fastest systems to work with are your immediate memory, like using Redis, Aerospike or Memcache. File systems (I/O) would often be the slowest. It's important to consider that when you work with I/O you are locking a file when you modify it. This may happen extremely fast but it's worth noting, especially if you were to write a data set inside a text file. Databases can have a table, or row-level locking while it is in use. This is only to prevent a collision. With that said, all things generally happen extremely fast and systems often have a mechanism in place that does a behind the scenes queue.

Use Case

When we come back to bulk transactions imagine a bank updating 20,000 records in real-time a second. This would be quite insane, or the quantity of orders Amazon receives in a minute. Of course, there are enough resources in the giants but a smarter way to handle it would be the following scenario.

  • Use a temporary in-memory database such as Redis.
  • Store records until they reach a quantity of 200, or a timer or one minute, and possibly even a size limit.
  • Process all data from Redis into the permanent database and flush the in-memory data.
  • This bulk transaction will be one query rather than 200, it will happen to be far more efficient.

Summary

If your updating records as fast as possible, you also need efficiency. That is why transactions exist in most database software. Do you need to use them? Of course not, you can use them and probably only will if you have a very popular system doing something.