Recently, the tech scene was rattled by the news that Google Cloud somehow lost the entire account of an Australian financial institution. Due to a misconfiguration, all servers, databases, and backups were gone overnight.

It's any founder's worst nightmare. One day, you open your laptop and find your entire platform gone. The company was saved by a diligent system engineer who had the foresight to back up its data to another cloud provider. They were able to rebuild their platform using the data they could salvage from this external file server.

Cloud providers like Google have an excellent track record for storing your data, and this case is extremely uncommon. But data loss, unfortunately, isn't.

It opens the uncomfortable debate around backup strategies and data loss. What is the right amount of backups?

That really depends on the acceptable amount of data loss. While the knee-jerk reaction is to claim that data loss is never acceptable, the real answer is a lot more pragmatic. Financial institutions can't lose a single transaction, but most SaaS platforms can tolerate some data loss. It’s a high-impact, low-frequency risk that needs to be managed. While it would hurt to lose 24 hours of data, in most cases, it doesn’t spell the end of the company. 

Your backup strategy should match this tolerance for data loss.

At a minimum, there should be a daily backup mechanism; luckily, most companies have that covered.

Managed databases often come with robust daily or hourly backups and point-in-time restoration. If your startup currently doesn't support that, stop reading and set that up today!

But even if you have such a managed instance, all it takes is one human error to delete an entire database, including its backups. This isn't as uncommon as it sounds. 

So, the next level of security will be to store snapshots of the database on a file system. If you want to take it up a notch, consider scheduling a daily data dump and uploading those to a file server. Since you're already putting in the effort, you might as well host this in another cloud.

But the most important part of the backup strategy is often overlooked. It doesn't matter how many layers of backup you have if you don't know how to restore your data. At a minimum, you should try to restore your data from a backup once every year. It's fascinating how many companies have best-in-class backups that have never been battle tested. 

Document the process

Disaster recovery is hopefully a rare event. There is no need to spend a lot of time automating these processes, but it is absolutely critical to document the manual steps. The last thing you want to do is figure out how to restore a database when your service is down! When disaster strikes, you want to have a step-by-step guide on how to recover.

Schedule the restore

That's why it's vital to practice the restoration process once every 6 to 12 months. This allows the team to train those skills so that there is no panic when the need arises. Set up a recurring meeting every few months to restore the database. By doing this regularly, data recovery becomes a habit.

Adapt the strategy

SaaS products evolve fast, and it's very likely that the backup strategy needs to evolve, too. When a system's complexity rises, the team might need to set up a more elaborate backup strategy. Adding a second database will make data consistency harder. Adding a messaging queue creates yet another layer of complexity. Whenever the team practices the restore, they should adapt the process and its documentation to evolve with the product.

Data loss is a scary subject, but most startups can afford a little bit. So, in most cases, daily backups are enough. Unless your system gets more complex or your tolerance for data loss is extremely low. In those cases more robust data recovery strategies become necessary.

For most SaaS platforms, a managed database with snapshots stored on a filesystem is enough to secure their data.