Repository navigation
Update database schema automatically #3343
Description
** Please make sure you read the contribution guide and file the issues in the right place. **
Contribution guide.
Is your feature request related to a problem? Please describe.
We have setup our google-adk application with a database using DatabaseSessionService in a cloud deployment with a PostgreSQL database.
There were two adk version updates (1.14.0 and 1.17.0) that enforced schema changes.
We as the users are obliged to manually update the schema and have to catch updates to existing tables ourselves.
This is also stated in 1.14.0 Release Notes:
NOTE: This requires DB migration, run ALTER TABLE events ADD COLUMN custom_metadata JSON; to migrate existing database tables.With 1.17.0, the required changes were not even mentioned and I had to manually find out what changes to the DB schema are necessary.
The existing migration mechanisms - to my understanding - does only create new tables.
It does not update existing tables by either editind or deleting columns.
Describe the solution you'd like
The db schema should be automatically updated by the google-adk.
That must be done on startup, but only if a DB is configured for sessions.
Describe alternatives you've considered
%
Additional context
Our solution looks like this:
if session_db_config is None:
logger.info("using no database")
session_db_url = None
else:
try:
ensure_schema_created(session_db_config)
except Exception as e:
logger.error(f"Error creating schema: {e}, ignoring and continuing...", exc_info=e)
try:
migrate_if_needed(session_db_config)
except Exception as e:
logger.error(f"Error migrating tables: {e}, ignoring and continuing...", exc_info=e)
session_db_url = session_db_config.get_url(schema=SCHEMA_NAME)
app = get_fast_api_app(
agents_dir=str(agents_dir),
web=True,
session_service_uri=session_db_url,
)
def migrate_if_needed(db_config: DBConfig):
engine = create_engine(db_config.get_url(schema=SCHEMA_NAME))
with engine.begin() as connection:
# required for 1.14.0 migration
# https://github.com/google/adk-python/releases/tag/v1.14.0
connection.execute(text("ALTER TABLE sessions.events ADD COLUMN IF NOT EXISTS custom_metadata JSONB"))
# required for 1.17.0 migration
# https://github.com/google/adk-python/commit/6ab1498aa0bad783e17e97a2ceede97b4bd3016f
connection.execute(text("ALTER TABLE sessions.events ADD COLUMN IF NOT EXISTS usage_metadata JSONB"))
connection.execute(text("ALTER TABLE sessions.events ADD COLUMN IF NOT EXISTS citation_metadata JSONB"))
logger.info("Altered events table to add custom_metadata column")
engine.dispose()
Activity
gundeep-dialpad commentedon Oct 31, 2025
@flansen can you please share that sql you had to run to migrate to 1.17.0? or where can I find it. Thanks
flansen commentedon Oct 31, 2025
@gundeep-dialpad this should do it, I will update my initial post as well :):
dandolf81 commentedon Nov 3, 2025
Our service account functional users are not granted the ability to execute DDL statements. There has got to be a better way to manage or publish these requisite schema changes within the release documentation.
ryanovas commentedon Nov 23, 2025
This is absolutely necessary, especially since changes to the schema are happening on minor version bumps. Just small incremental updates will surprise me when I deploy something to staging and no one can connect anymore because once again a column is missing that's expected and I need to go hunt for a migration script in some doc somewhere or an issue with the SQL commands necessary.
klateefa commentedon Nov 24, 2025
@flansen Thank you for reporting this. I understand the request is currently under review. Please keep me posted if anything else is needed. Thanks for your patience.
achilleasatha commentedon Jan 13, 2026
As a platform engineer running plenty of ADK based apps in multiple envs please do address this before you push any more breaking changes.
It looks like future versions of adk will deprecate the old schema - this SHOULD NOT happen without a way to properly handle DB migrations or should be part of a major release.
Minor / patches should not break core functionality.
[warning ] The database is using the legacy v0 schema, which uses Pickle to serialize event actions. The v0 schema will not be supported going forward and will be deprecated in a few rollouts. Please migrate to the v1 schema which uses JSON serialization for event data. You can useadk migrate sessioncommand to migrate your database. [google_adk.google.adk.sessions.migration._schema_check_utils]flansen commentedon Jan 13, 2026
What @achilleasatha points out, this would be really unfortunate.
It would be great to have schema versioning as part of the adk.
We deploy the adk in many cloud environments as an agent backend microservice.
Running adk cli commands is not really preferred in this scenario.
wuliang229 commentedon Jan 21, 2026
@achilleasatha @flansen @dandolf81 @ryanovas thanks for the feedback about automatic schema update. In the 1.22.1 release, ADK provided a script in adk/sessions/migration/migrate_from_sqlalchemy_pickle.py, and a generic migration runner adk/sessions/migration/migration_runner.py that you can use to run any migration needed before running the agents. In addition, there's a new table in the database to store the version of the schema, so only necessary migration steps are performed.
In the future, ADK will use these structures for any future schema migrations, with clear documentation and guide in the release notes.
Please take a look at these and let me know if they can be used in your projects
achilleasatha commentedon Jan 21, 2026
@wuliang229 thanks for the response.
The reason the script is not sufficient is because it's not tenable to ssh into every service that runs adk across all envs and manually run it.
I don't even think we are an extreme case with 5 envs and 10s of adk agents per env.
Doing the migration alone would be a day's worth of work every time you bump your schema.
This is a well solved problem.
Would you accept contributions on this?
Edit: to provide an example from LiteLLM's implementation of Prisma migrations https://docs.litellm.ai/docs/proxy/prod#9-use-prisma-migrate-deploy but do note that they are introducing a beta feature to run those with a helm pre-sync hook https://docs.litellm.ai/docs/proxy/prod#7-use-helm-presync-hook-for-database-migrations-beta which is definitely the better choice to avoid race conditions upon deployment and multiple pods trying to run the migration simultaneously (there is a lock to protect but it can cause issues on rollouts).
6 remaining items
achilleasatha commentedon Feb 3, 2026
Not yet unfortunately :( It's been a very busy period for us. I have allocated capacity in our current sprint for this so I am hoping to get around to:
My desired approach will be to bring in alembic to manage migrations I don't know if this will have support from the team / community. I really feel like this kind of initiative should have a dedicated working group.
wuliang229 commentedon Feb 5, 2026
@achilleasatha You can start with a draft RFC and some pseudocode. We can work on it and integrate it into our solution if it seems reasonable.
achilleasatha commentedon Feb 5, 2026
Raised draft RFC relating to issue #4387
Feedback most welcome - @flansen @dandolf81 @ryanovas
feat: adopt Alembic for database schema migrations
feat: adopt Alembic for database schema migrations
achilleasatha commentedon Feb 7, 2026
Raised draft PR #4408
ftnext commentedon Feb 11, 2026
I agree ADK needs a first-class migration story for DB schema changes.
In the meantime, I created a small helper repo that tracks schema diffs across ADK versions:
https://github.com/ftnext/adk-python-db-schema-history
It provides per-version PostgreSQL/MySQL schema snapshots, and is intended to be used with
sqldef(psqldef/mysqldef) to identify/apply upgrade-timeALTER TABLEchanges before rollout.DeanChensj commentedon Jul 21, 2026
Thank you for raising this issue.
The problem of having to manually run
ALTER TABLEstatements for new metadata columns has been resolved in the latest ADK versions (specifically starting with the V1 schema).In the V1 schema, the
eventstable now uses a single JSON column (event_data) to serialize event details. This means future additions to theEventmodel will be dynamically stored in this JSON column without requiring any database schema changes (like adding new columns).If you are upgrading from an older version (e.g. V0 pickle-based schema), you can migrate your database using the ADK CLI:
Note that ADK intentionally does not support automatic in-place migrations on startup for safety reasons (to prevent production database corruption if a migration fails). However, the
DatabaseSessionServiceis designed to remain backward-compatible with older schemas for several releases, allowing you to plan your migration.Closing this issue as the underlying schema rigidity has been resolved with the V1 JSON schema.
achilleasatha commentedon Jul 21, 2026
The direction the team is taking with this is rather unfortunate :(
If what you're saying here is true can I expect my RFC and PRs that have been dangling for months to be closed as well?
What happens later down the line when that dumping ground of a json column is an unmaintainable monstrosity? We dump it and create a new one for every major bump of ADK?
I just don't understand why we are refusing to take a methodical and systematic approach to solving this. :(
ftnext commentedon Jul 21, 2026
Hi @DeanChensj , I agree with @achilleasatha and don’t consider this issue resolved. Using
event_datareduces some schema changes, but it does not provide a systematic migration strategy for future relational changes, upgrades, rollbacks, or CI/CD deployments.Since RFC #4387 and PR #4408 are still open, could this issue be reopened, or could the maintainers clarify where the broader migration requirements will be tracked?
DeanChensj commentedon Jul 22, 2026
Reopened the issue