This is awesome. Want to just mention my experience trying to replicate sqlite here.
I host a multiplayer game on fly. The way I've designed it is, each game server has it's own sqlite database. And each fly server can host multiple game servers, to keep a high utilization. I currently use Litestream to replicate each database to s3 for disaster recovery. I am planning to move from S3 to sftp to save on the high post/put costs that s3 incurs (the actual storage costs are negligible).
I thought what I am doing would be more common place. But it seems that running single machine instances that can recover after a crash is not common after all (or atleast the tooling does not focus on that). Most use cases seem to be serving high availability or scalability.
In the unnecessary (IMO) desire to make everything highly available, I think simpler solutions have been over looked. I can't help but feel that if you need LiteFS, it is possible that you should be looking at a server oriented database like Postgres or Mysql. In that respect, I feel Litestream is underrated and deserves more attention. It serves a use case which is perhaps more in-line with an in-process DB :)
PS. this thread has some really interesting tools though (Marmot, mycelite). Great to see so many options.
Litestream & LiteFS author here. I think single-node, easily-recoverable systems are great and it fits a lot of people's use cases. VPS providers are pretty reliable too so even using a regular hourly backup can be good enough for some people. We even have docs for a cron-based backup [1] on the Litestream docs site.
You're right that HA is one benefit of LiteFS. But I think another important difference is reducing geographic latency. It's possible to spin up read replicas & failovers for Postgres or MySQL and then run application servers for each one of those but it's a huge pain. Or you can pay a serverless database provider but that's expensive. One of the goals with LiteFS is to simply be able to add application nodes in different regions and automagically have faster read latency to people near those regions.
I host a multiplayer game on fly. The way I've designed it is, each game server has it's own sqlite database. And each fly server can host multiple game servers, to keep a high utilization. I currently use Litestream to replicate each database to s3 for disaster recovery. I am planning to move from S3 to sftp to save on the high post/put costs that s3 incurs (the actual storage costs are negligible).
I thought what I am doing would be more common place. But it seems that running single machine instances that can recover after a crash is not common after all (or atleast the tooling does not focus on that). Most use cases seem to be serving high availability or scalability.
In the unnecessary (IMO) desire to make everything highly available, I think simpler solutions have been over looked. I can't help but feel that if you need LiteFS, it is possible that you should be looking at a server oriented database like Postgres or Mysql. In that respect, I feel Litestream is underrated and deserves more attention. It serves a use case which is perhaps more in-line with an in-process DB :)
PS. this thread has some really interesting tools though (Marmot, mycelite). Great to see so many options.