-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
LND 0.14 ships with support for Postgres as a database backend. Unfortunately the full potential of Postgres can't be reached because internally LND still uses key/value storage everywhere.
This means that currently with a Postgres backend, the data that is stored is mostly unreadable. It uses custom serialization/deserialization in several flavours. Any tool that wants to operate on this data needs to implement the exact same serialization methods.
More importantly, maintaining custom serialization code is time-consuming and error-prone. This becomes apparent in an extreme way for database migrations. For each migration, a frozen copy of the serialization code needs to be added to the code base, to ensure that the migration won't ever break.
Furthermore, a minimalistic kv store like bbolt doesn't offer functionality like indexes (other than on the key) and constraints. Functionality that could be put to good use to guard database integrity in a simple way and make lnd more reliable and faster overall.
These are all advantages that could be unlocked by a step-by-step migration from kv to structured sql. The problem with that migration is that there is still bbolt. If bbolt support needs to stay, a large amount of code needs to be added to have two implementations for every store in the system. This means more potential for bugs and increased maintenance costs.
One way to address this problem is to replace bbolt by SQLite. SQLite is probably one of the top five most deployed software modules in the world, so reliability shouldn't be an issue.
The initial replacement will stick to the kv model for data. But once fully on the sql platform, it becomes possible to migrate select sub-systems that need it the most to structured sql tables. This migration can happen at the same time for both SQLite and Postgres, so that there won't be duplicate implementations.
Ideas for a light abstraction to iron out the differences in SQL dialect between Postgres and SQLite can be taken from c-lightning.
A first test with LND on SQLite has been carried out successfully.