Cannot use SQLite as the SQL registry #5565
Description
Expected Behavior
feature_store.yaml with SQLite as sql registry works.
project: my_project
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
# registry: data/registry.db # <--- works with file registry but not with sqlite as sql registry.
registry:
registry_type: sql
path: sqlite:///data/registry.db
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
offline_store:
type: file
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 3
# By default, no_auth for authentication and authorization, other possible values kubernetes and oidc. Refer the documentation for more details.
auth:
type: no_auth
Current Behavior
feature_store = FeatureStore(repo_path=".")
...
OperationalError: (sqlite3.OperationalError) database is locked
[SQL: INSERT INTO feast_metadata (project_id, metadata_key, metadata_value, last_updated_timestamp) VALUES (?, ?, ?, ?)]
[parameters: ('my_project', 'last_updated_timestamp', '1754965241', 1754965241)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
Steps to reproduce
Create a project/repository structure.
my_project/feature_repo
├── data
│ └── driver_stats.parquet
├── example_repo.py
└── feature_store.yaml
Run.
feature_store = FeatureStore(repo_path=".")
example_repo.py:
# This is an example feature definition file
from datetime import timedelta
from feast import (
Entity,
FeatureService,
FeatureView,
Field,
FileSource,
Project,
)
from feast.feature_logging import LoggingConfig
from feast.infra.offline_stores.file_source import FileLoggingDestination
from feast.types import Float32, Int64
project = Project(name="my_project", description="A project for driver statistics")
driver_hourly_stats = FileSource(
name="driver_hourly_stats_source",
path="data/driver_stats.parquet",
timestamp_field="event_timestamp",
created_timestamp_column="created",
)
# You can think of an entity as a primary key used to fetch features.
driver = Entity(name="driver", join_keys=["driver_id"])
driver_hourly_stats_view = FeatureView(
name="driver_hourly_stats",
entities=[driver],
ttl=timedelta(days=1),
# The list of features defined below act as a schema to both define features
# for both materialization of features into a store, and are used as references
# during retrieval for building a training dataset or serving features
schema=[
Field(name="conv_rate", dtype=Float32),
Field(name="acc_rate", dtype=Float32),
Field(name="avg_daily_trips", dtype=Int64, description="Average daily trips"),
],
# Tell FEAST to materialise into online store.
online=True,
source=driver_hourly_stats, # <--- Link to the raw data storage technology
tags={"team": "driver_performance"},
)
# This groups features into a model version
driver_activity_v1 = FeatureService(
name="driver_activity_v1",
features=[
# driver_hourly_stats_view[["conv_rate", "acc_rate"]], # Sub-selects a feature from a feature view
driver_hourly_stats_view,
],
logging_config=LoggingConfig(
destination=FileLoggingDestination(path="data")
),
)
Specifications
- Version:
import feast
feast.__version__
'0.51.0'
- Platform: MacOS M2 (15.5 (24F74))
- Subsystem: Python 3.11
Activity
oonisim commentedon Aug 12, 2025
Full stack trace.
oonisim commentedon Aug 12, 2025
SDK without feature_store.yaml cause the same.
ntkathole commentedon Aug 12, 2025
@oonisim This looks like concurrency issue at database level. Can you try to configure registry with
oonisim commentedon Aug 18, 2025
@ntkathole thanks for the reply. Tried the suggestion but no luck.
oonisim commentedon Sep 1, 2025
The documentation says Sqlite works as the SQL Registry DB. But it is not working with FEAST 0.51.
Please verify if SQLite does work or update the document if it does not work.
https://docs.feast.dev/reference/registries/sql
oonisim commentedon Sep 1, 2025
ntkathole commentedon Sep 1, 2025
@oonisim Hey, I checked again and it looks like the problem with transaction handling within feast itself. The
_set_last_updated_metadatamethod was being called from within the_apply_objectmethod's transaction context, but_set_last_updated_metadatawas trying to start its own transaction withself.write_engine.begin(). This created a nested transaction scenario that SQLite doesn't handle well.I am working on the fix.
oonisim commentedon Sep 2, 2025
Thanks @ntkathole, confirmed the fix. It worked (replacing site-packages/feast/infra/registry/sql.py)
feat(feast): switch registry from file to SQLite SQL backend