Preparestmt use LRU Map instead default map by Yaccc · Pull Request #7435 · go-gorm/gorm

Yaccc · 2025-04-24T08:02:15Z

GROM default map may be cause memory leak

What did this pull request do?

PrepareStmt support use lrumap instead default Map
PreparedStmtDB stmt change to interface to defined maps

type StmtStore interface {
	Get(key string) (*Stmt, bool)
	Set(key string, value *Stmt)
	Delete(key string)
	AllMap() map[string]*Stmt
}

func NewPreparedStmtDB(connPool ConnPool, prepareStmtLruConfig *PrepareStmtLruConfig) *PreparedStmtDB {
	return &PreparedStmtDB{
		ConnPool: connPool,
		Stmts: func() StmtStore {
			var stmts StmtStore
			if prepareStmtLruConfig != nil && prepareStmtLruConfig.Open {
				lru := &LruStmtStore{}
				lru.NewLru(prepareStmtLruConfig.Size, prepareStmtLruConfig.TTL)
				stmts = lru
			} else {
				stmts = &DefaultStmtStore{}
			}
			return stmts
		}(),
		Mux: &sync.RWMutex{},
	}
}

original maps use defaultStmtStore,lur maps use LruStmtStore
PrepareStmtLruConfig init lru size and expired time
size must lt zero ande default ttl is

const noEvictionTTL = time.Hour * 24 * 365 * 10

const DEFAULT_MAX_SIZE = (1 << 63) - 1
const DEFAULT_TTL = time.Hour * 24
	// PrepareStmt cache support LRU expired
	PrepareStmtMaxSize int
	PrepareStmtTTL     time.Duration

User Case Description

TestPreparedStmtLruFromTransaction check lru active

conn, ok := tx.ConnPool.(*gorm.PreparedStmtDB)
	lens := len(conn.Stmts.AllMap())
	if lens == 0 {
		t.Fatalf("lru should not be empty")
	}
	time.Sleep(time.Second * 40)
	AssertEqual(t, ok, true)
	AssertEqual(t, len(conn.Stmts.AllMap()), 0)

jinzhu · 2025-04-24T08:19:10Z

gorm.go

 	callbacks  *callbacks
 	cacheStore *sync.Map
 }
+type PrepareStmtLruConfig struct {


This is no longer needed?

already deleted

jinzhu · 2025-04-24T08:22:53Z

gorm.go

 			preparedStmt = v.(*PreparedStmtDB)
 		} else {
-			preparedStmt = NewPreparedStmtDB(db.ConnPool)
+			preparedStmt = NewPreparedStmtDB(db.ConnPool, db.Config.PrepareStmtMaxSize, db.Config.PrepareStmtTTL)


Do we need to allow session-level configuration? For example, configure it here: https://github.com/go-gorm/gorm/blob/master/gorm.go#L107

if config.PrepareStmt { preparedStmt := NewPreparedStmtDB(db.ConnPool, config.PrepareStmtMaxSize, config.PrepareStmtTTL) db.cacheStore.Store(preparedStmtDBKey, preparedStmt) db.ConnPool = preparedStmt }

session default use static config

jinzhu · 2025-04-24T08:28:40Z

prepare_stmt.go

 }

-func NewPreparedStmtDB(connPool ConnPool) *PreparedStmtDB {
+const DEFAULT_MAX_SIZE = (1 << 63) - 1


Could we relocate the majority of the following code to an internal package, such as internal/stmt_store, to prevent the exposure of additional structs or methods?

cause circular dependency

…o feature/sup_lru_prepareStmt

iTanken · 2025-04-27T01:22:24Z

Build failed in 32bit environment

a827495#r155931475

GOARCH=386 go test -timeout 20m -v ./...
  shell: /usr/bin/bash -e {0}
# gorm.io/gorm/internal/stmt_store
Error: 
../../../go/pkg/mod/gorm.io/[email protected]/internal/stmt_store/stmt_store.go:99:10: 
cannot use defaultMaxSize (untyped int constant 9223372036854775807) as int value in assignment (overflows)

Refs: go-gorm#7435

…#7439) Refs: #7435

andig · 2025-05-03T08:34:35Z

We've just hit this as well in #7442. A .1 release would be highly appreciated.

* 支持lru淘汰preparestmt cache * 支持lru淘汰preparestmt cache * 支持lru淘汰preparestmt cache * 只使用lru * 只使用lru * 只使用lru * 只使用lru * 只使用lru * 只使用lru * 只使用lru * 只使用lru * 只使用lru * change const export * Add stmt_store * refact prepare stmt store * Rename lru store * change const export * ADD UT * format code and add session level prepare stmt config * code format according to golinter ci * ADD UT --------- Co-authored-by: xiezhaodong <[email protected]> Co-authored-by: Jinzhu <[email protected]>

…go-gorm#7439) Refs: go-gorm#7435

xiezhaodong added 10 commits April 14, 2025 13:07

支持lru淘汰preparestmt cache

5225c20

支持lru淘汰preparestmt cache

3ae5fde

支持lru淘汰preparestmt cache

618e8c1

只使用lru

3f922a7

只使用lru

00f5a04

只使用lru

e233e7c

只使用lru

2a08d0e

只使用lru

b0dcec6

只使用lru

3ccf0b0

只使用lru

cfd5f25

jinzhu reviewed Apr 24, 2025

View reviewed changes

只使用lru

4be2fdb

jinzhu reviewed Apr 24, 2025

View reviewed changes

只使用lru

3758fb9

Yaccc changed the title ~~Preparestmt use lur instead default map~~ Preparestmt use LRU Map instead default map Apr 24, 2025

xiezhaodong and others added 10 commits April 24, 2025 16:47

change const export

dfa1b81

Add stmt_store

886a406

refact prepare stmt store

14dc8ed

Rename lru store

2ec55b5

change const export

9dd6263

ADD UT

f4e7282

format code and add session level prepare stmt config

29a3f7e

code format according to golinter ci

c588e74

ADD UT

bb11909

Merge remote-tracking branch 'origin/feature/sup_lru_prepareStmt' int…

4571df5

…o feature/sup_lru_prepareStmt

jinzhu merged commit a827495 into master Apr 25, 2025
23 of 41 checks passed

jinzhu deleted the feature/sup_lru_prepareStmt branch April 25, 2025 08:22

iTanken added a commit to iTanken/gorm that referenced this pull request Apr 27, 2025

fix: int type variable defaultMaxSize overflows in 32-bit environment

bfc66cc

Refs: go-gorm#7435

iTanken mentioned this pull request Apr 27, 2025

fix: int type variable defaultMaxSize overflows in 32-bit environment #7439

Merged

3 tasks

jinzhu pushed a commit that referenced this pull request Apr 27, 2025

fix: int type variable defaultMaxSize overflows in 32-bit environment (…

8c4e8e2

…#7439) Refs: #7435

wchargin mentioned this pull request May 23, 2025

1.2.7 breaks migrations when updating 1.2.5 #6888

Closed

ivila mentioned this pull request Nov 7, 2025

Prepare stmt caching #7623

Open

phroggyy pushed a commit to incident-io/gorm that referenced this pull request Jan 23, 2026

fix: int type variable defaultMaxSize overflows in 32-bit environment (…

a1aa85c

…go-gorm#7439) Refs: go-gorm#7435

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Preparestmt use LRU Map instead default map#7435

Preparestmt use LRU Map instead default map#7435
jinzhu merged 22 commits intomasterfrom
feature/sup_lru_prepareStmt

Yaccc commented Apr 24, 2025 •

edited

Loading

Uh oh!

jinzhu Apr 24, 2025

Uh oh!

Yaccc Apr 24, 2025 •

edited

Loading

Uh oh!

jinzhu Apr 24, 2025

Uh oh!

Yaccc Apr 24, 2025

Uh oh!

jinzhu Apr 24, 2025

Uh oh!

Yaccc Apr 24, 2025

Uh oh!

Uh oh!

iTanken commented Apr 27, 2025 •

edited

Loading

Uh oh!

andig commented May 3, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Yaccc commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GROM default map may be cause memory leak

What did this pull request do?

User Case Description

Uh oh!

jinzhu Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Yaccc Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jinzhu Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Yaccc Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

jinzhu Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Yaccc Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

iTanken commented Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andig commented May 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Yaccc commented Apr 24, 2025 •

edited

Loading

Yaccc Apr 24, 2025 •

edited

Loading

iTanken commented Apr 27, 2025 •

edited

Loading

andig commented May 3, 2025 •

edited

Loading