The last_update_time provided in the session object '2025-07-01 18:59:42' is earlier than the update_time in the storage_session '2025-07-01 18:59:56'. Please check if it is a stale session

Describe the bug

{"error": "The last_update_time provided in the session object '2025-07-01 18:59:42' is earlier than the update_time in the storage_session '2025-07-01 18:59:56'. Please check if it is a stale session."}

main.py

session_mysql_url = os.getenv('SESSION_MYSQL_URL')

app = get_fast_api_app(
    agents_dir="./sql_agent",
    session_service_uri=session_mysql_url,
    allow_origins=["*"],
    web=False,
    trace_to_cloud=False,
)

if __name__ == "__main__":
    import uvicorn

    config = uvicorn.Config(
        "main:app",
        log_level="info",
        port=8000,
        reload=True,
        # Disable Uvicorn's default logging config
        log_config=None
    )
    server = uvicorn.Server(config)
    server.run()

Desktop (please complete the following information):

OS:
Python version(python -V): 3.12
ADK version(pip show google-adk): 1.4.1
uvicorn: 0.34.3

Model Information:

qwen-max model

Same problem..Happens when I use MySQL DB for session service uri

Thanks for reporting this bug! Could you please provide the following information? which should be helpful for me to understand the problem

Does your mysql db has a different timezone from your environment where you run the agent?
Does the same issue happens when using different databases, e.g. sqlite?

Does your mysql db has a different timezone from your environment where you run the agent?

SELECT @@global.time_zone;
output：+08:00

SELECT @@session.time_zone;
outpu: +08:00

SELECT @@system_time_zone;
outpu: CST

I used Alibaba Cloud's RDS database service, which is based on MySQL.

SELECT VERSION();
outpu: 5.7.44-log

Thanks for the additional information! Is it possible for you to share a minimal runnable example to reproduce it?

I tried with the following simple example and I cannot reproduce the error, so I suspect the error you saw was caused by some race conditions, for example, your application might be processing multiple events for the same session concurrently, leading to a race condition where an older session object is used to update the state.

If it is not easy to share a minimal runnable example, could you please let me know how I might modify my following test code to reproduce the error?

folder tree

├── hello_world
│   ├── agent.py
│   └── __init__.py
├── main.py
└── test.py

main.py

from google.adk.cli.fast_api import get_fast_api_app
import uvicorn

# session_service_url = "sqlite:///./sessions.db"
session_service_url = "mysql+pymysql://root:[email protected]:3306/sessions"

app = get_fast_api_app(
    agents_dir=".",
    session_service_uri=session_service_url,
    allow_origins=["*"],
    web=True,
    trace_to_cloud=False,
)

if __name__ == "__main__":
  config = uvicorn.Config(
      "main:app",
      log_level="info",
      port=8080,
      reload=True,
      # Disable Uvicorn's default logging config
      log_config=None,
  )
  server = uvicorn.Server(config)
  server.run()

test.py

from typing import Any
import requests

APP_NAME = "hello_world"
USER_ID = "user"

BASE_URL = "http://localhost:8080"

headers = {}


def get_request(
    url: str, params: dict[str, Any] | None = None
) -> dict[str, Any]:
  if params is None:
    params = {}
  response = requests.get(
      BASE_URL + url, headers=headers, params=params, timeout=60
  )
  response.raise_for_status()
  return response.json()


def post_request(url: str, payload: Any) -> dict[str, Any]:
  response = requests.post(
      BASE_URL + url, headers=headers, json=payload, timeout=60
  )
  response.raise_for_status()
  return response.json()


if __name__ == "__main__":
  print("list-app:", get_request("/list-apps"))

  # request = requests.post(
  #     "http://localhost:8080/apps/test-app/users/test-user/sessions",
  #     json={
  #         "app_name": "test-app",
  #         "user_id": "test-user",
  #     },
  # )
  # print("Created a new session")

  print(
      "list-sessions:",
      get_request(f"/apps/{APP_NAME}/users/{USER_ID}/sessions"),
  )

  print(
      "run:",
      post_request(
          "/run",
          {
              "appName": "hello_world",
              "userId": "user",
              "sessionId": "94c095a0-5379-421d-ae93-3673200d9ee5",
              "newMessage": {
                  "parts": [{"text": "Roll a 10 sided dice"}],
                  "role": "user",
              },
              "streaming": False,
          },
      ),
  )

agent.py

import random

from google.adk import Agent
from google.adk.tools.tool_context import ToolContext
from google.genai import types


def roll_die(sides: int, tool_context: ToolContext) -> int:
  """Roll a die and return the rolled result.

  Args:
    sides: The integer number of sides the die has.

  Returns:
    An integer of the result of rolling the die.
  """
  result = random.randint(1, sides)
  if not 'rolls' in tool_context.state:
    tool_context.state['rolls'] = []

  tool_context.state['rolls'] = tool_context.state['rolls'] + [result]
  return result


async def check_prime(nums: list[int]) -> str:
  """Check if a given list of numbers are prime.

  Args:
    nums: The list of numbers to check.

  Returns:
    A str indicating which number is prime.
  """
  primes = set()
  for number in nums:
    number = int(number)
    if number <= 1:
      continue
    is_prime = True
    for i in range(2, int(number**0.5) + 1):
      if number % i == 0:
        is_prime = False
        break
    if is_prime:
      primes.add(number)
  return (
      'No prime numbers found.'
      if not primes
      else f"{', '.join(str(num) for num in primes)} are prime numbers."
  )


root_agent = Agent(
    model='gemini-2.0-flash',
    name='hello_world_agent',
    description=(
        'hello world agent that can roll a dice of 8 sides and check prime'
        ' numbers.'
    ),
    instruction="""
      You roll dice and answer questions about the outcome of the dice rolls.
      You can roll dice of different sizes.
      You can use multiple tools in parallel by calling functions in parallel(in one request and in one round).
      It is ok to discuss previous dice roles, and comment on the dice rolls.
      When you are asked to roll a die, you must call the roll_die tool with the number of sides. Be sure to pass in an integer. Do not pass in a string.
      You should never roll a die on your own.
      When checking prime numbers, call the check_prime tool with a list of integers. Be sure to pass in a list of integers. You should never pass in a string.
      You should not check prime numbers before calling the tool.
      When you are asked to roll a die and check prime numbers, you should always make the following two function calls:
      1. You should first call the roll_die tool to get a roll. Wait for the function response before calling the check_prime tool.
      2. After you get the function response from roll_die tool, you should call the check_prime tool with the roll_die result.
        2.1 If user asks you to check primes based on previous rolls, make sure you include the previous rolls in the list.
      3. When you respond, you must include the roll_die result from step 1.
      You should always perform the previous 3 steps when asking for a roll and checking prime numbers.
      You should not rely on the previous history on prime results.
    """,
    tools=[
        roll_die,
        check_prime,
    ],
    generate_content_config=types.GenerateContentConfig(
        safety_settings=[
            types.SafetySetting(  # avoid false alarm about rolling dice.
                category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
        ]
    ),
)

I tried with the following simple example and I cannot reproduce the error, so I suspect the error you saw was caused by some race conditions, for example, your application might be processing multiple events for the same session concurrently, leading to a race condition where an older session object is used to update the state.

If it is not easy to share a minimal runnable example, could you please let me know how I might modify my following test code to reproduce the error?

folder tree

├── hello_world
│   ├── agent.py
│   └── __init__.py
├── main.py
└── test.py

main.py

from google.adk.cli.fast_api import get_fast_api_app
import uvicorn

# session_service_url = "sqlite:///./sessions.db"
session_service_url = "mysql+pymysql://root:[email protected]:3306/sessions"

app = get_fast_api_app(
    agents_dir=".",
    session_service_uri=session_service_url,
    allow_origins=["*"],
    web=True,
    trace_to_cloud=False,
)

if __name__ == "__main__":
  config = uvicorn.Config(
      "main:app",
      log_level="info",
      port=8080,
      reload=True,
      # Disable Uvicorn's default logging config
      log_config=None,
  )
  server = uvicorn.Server(config)
  server.run()

test.py

from typing import Any
import requests

APP_NAME = "hello_world"
USER_ID = "user"

BASE_URL = "http://localhost:8080"

headers = {}


def get_request(
    url: str, params: dict[str, Any] | None = None
) -> dict[str, Any]:
  if params is None:
    params = {}
  response = requests.get(
      BASE_URL + url, headers=headers, params=params, timeout=60
  )
  response.raise_for_status()
  return response.json()


def post_request(url: str, payload: Any) -> dict[str, Any]:
  response = requests.post(
      BASE_URL + url, headers=headers, json=payload, timeout=60
  )
  response.raise_for_status()
  return response.json()


if __name__ == "__main__":
  print("list-app:", get_request("/list-apps"))

  # request = requests.post(
  #     "http://localhost:8080/apps/test-app/users/test-user/sessions",
  #     json={
  #         "app_name": "test-app",
  #         "user_id": "test-user",
  #     },
  # )
  # print("Created a new session")

  print(
      "list-sessions:",
      get_request(f"/apps/{APP_NAME}/users/{USER_ID}/sessions"),
  )

  print(
      "run:",
      post_request(
          "/run",
          {
              "appName": "hello_world",
              "userId": "user",
              "sessionId": "94c095a0-5379-421d-ae93-3673200d9ee5",
              "newMessage": {
                  "parts": [{"text": "Roll a 10 sided dice"}],
                  "role": "user",
              },
              "streaming": False,
          },
      ),
  )

agent.py

import random

from google.adk import Agent
from google.adk.tools.tool_context import ToolContext
from google.genai import types


def roll_die(sides: int, tool_context: ToolContext) -> int:
  """Roll a die and return the rolled result.

  Args:
    sides: The integer number of sides the die has.

  Returns:
    An integer of the result of rolling the die.
  """
  result = random.randint(1, sides)
  if not 'rolls' in tool_context.state:
    tool_context.state['rolls'] = []

  tool_context.state['rolls'] = tool_context.state['rolls'] + [result]
  return result


async def check_prime(nums: list[int]) -> str:
  """Check if a given list of numbers are prime.

  Args:
    nums: The list of numbers to check.

  Returns:
    A str indicating which number is prime.
  """
  primes = set()
  for number in nums:
    number = int(number)
    if number <= 1:
      continue
    is_prime = True
    for i in range(2, int(number**0.5) + 1):
      if number % i == 0:
        is_prime = False
        break
    if is_prime:
      primes.add(number)
  return (
      'No prime numbers found.'
      if not primes
      else f"{', '.join(str(num) for num in primes)} are prime numbers."
  )


root_agent = Agent(
    model='gemini-2.0-flash',
    name='hello_world_agent',
    description=(
        'hello world agent that can roll a dice of 8 sides and check prime'
        ' numbers.'
    ),
    instruction="""
      You roll dice and answer questions about the outcome of the dice rolls.
      You can roll dice of different sizes.
      You can use multiple tools in parallel by calling functions in parallel(in one request and in one round).
      It is ok to discuss previous dice roles, and comment on the dice rolls.
      When you are asked to roll a die, you must call the roll_die tool with the number of sides. Be sure to pass in an integer. Do not pass in a string.
      You should never roll a die on your own.
      When checking prime numbers, call the check_prime tool with a list of integers. Be sure to pass in a list of integers. You should never pass in a string.
      You should not check prime numbers before calling the tool.
      When you are asked to roll a die and check prime numbers, you should always make the following two function calls:
      1. You should first call the roll_die tool to get a roll. Wait for the function response before calling the check_prime tool.
      2. After you get the function response from roll_die tool, you should call the check_prime tool with the roll_die result.
        2.1 If user asks you to check primes based on previous rolls, make sure you include the previous rolls in the list.
      3. When you respond, you must include the roll_die result from step 1.
      You should always perform the previous 3 steps when asking for a roll and checking prime numbers.
      You should not rely on the previous history on prime results.
    """,
    tools=[
        roll_die,
        check_prime,
    ],
    generate_content_config=types.GenerateContentConfig(
        safety_settings=[
            types.SafetySetting(  # avoid false alarm about rolling dice.
                category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
        ]
    ),
)

main.py

app = get_fast_api_app(
    agents_dir="./sql_agent",
    session_service_uri=session_mysql_url,
    allow_origins=["*"],
    web=False,
    trace_to_cloud=False,
)

I used this shell script to start project

uvicorn main:app --workers 4 --host 0.0.0.0

the workers param setting is 4, I guess cased by this

@code2tan please share the contents under sql_agent folder for us to reproduce and debug.

@seanzhou1023

@seanzhou1023 Thanks for looking into this issue! I just reproduced the error with my above code. Here are the steps:

Started the server by running uvicorn main:app --workers 4 --host 0.0.0.0
Call /run api twice at roughly the same time on the same session id

3. Saw the error `ValueError: The last_update_time provided in the session object '2025-07-08 16:55:47'` is earlier than the update_time in the storage_session '2025-07-08 16:57:11'. Please check if it is a stale session.`

Actually, even if we use --workers 1, this error can still happen, because the requests are handled asynchronously anyway, as lone as the first request is awaiting some IO, the server will work on the second request.

@code2tan Could you please check if you have similar simultaneous requests on the same session id when you interact with the server? Is it intentional?

@xuanyang15

@xuanyang15 any update the issue still exists for me, if i call simultaneous /un api

@akshatshaw

@akshatshaw Thanks for letting us know you have the same problem! We were waiting for @code2tan to confirm the root cause. Could you please confirm if the issue is caused by calling the api multiple times on the session id simultaneously? If so, could you please help us understand the use case for this simultaneous calls on the same session id?

@dennisgera

We also encountered this stale session issue when we try to mutate session state (which calls append_event) in the middle of /run_sse (ADK version 1.19).

We adopted @dennisgera 's solution and successfully mitigated the stale session issue 🎉 .

Maybe @dennisgera 's implementation should be built-in in ADK.

Hi, I am closing this issue now as I am not able to reproduce this on the current repo. Please reopen this issue if you notice this still

@GWeale

@GWeale Maybe I’m missing it, but did any of the PRs for this issue get merged?
I can tell it's still happening on 1.20

@GWeale

@GWeale
I'm using 1.18 - Same Issue is still happening - Using CloudSQL PostGres as session DB

I think 1063fa5 was submitted as a fix for this issue. Could you please have a try with the latest version?

fix: Strip timezone for PostgreSQL timestamps in DatabaseSessionService

I am using 1.21.0 along with cloud sql DB. I am also getting this issue.

fix: Strip timezone for PostgreSQL timestamps in DatabaseSessionService

Resolved by 1063fa5

fix: Reload the stale session in database_session_service when the up…

fix: Strip timezone for PostgreSQL timestamps in DatabaseSessionService

Navigation Menu

Repository navigation

The last_update_time provided in the session object '2025-07-01 18:59:42' is earlier than the update_time in the storage_session '2025-07-01 18:59:56'. Please check if it is a stale session #1733

26 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

The last_update_time provided in the session object '2025-07-01 18:59:42' is earlier than the update_time in the storage_session '2025-07-01 18:59:56'. Please check if it is a stale session #1733

Description

Activity

veevortexiq commented on Jul 2, 2025

xuanyang15 commented on Jul 2, 2025

CodyTonn commented on Jul 3, 2025

xuanyang15 commented on Jul 4, 2025

xuanyang15 commented on Jul 7, 2025

CodyTonn commented on Jul 8, 2025

seanzhou1023 commented on Jul 8, 2025

xuanyang15 commented on Jul 8, 2025

akshatshaw commented on Jul 28, 2025

xuanyang15 commented on Jul 28, 2025

26 remaining items

johnson-liang-appier commented on Feb 6, 2026

GWeale commented on Feb 9, 2026

TomShlaim commented on Feb 9, 2026

veevortexiq commented on Feb 9, 2026

xuanyang15 commented on Feb 9, 2026

chanchalchitwan commented on Feb 10, 2026

sasha-gitg commented on Mar 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions