Skip to content

(WIP) Implement Generics API#7424

Merged
jinzhu merged 26 commits intomasterfrom
generics
May 25, 2025
Merged

(WIP) Implement Generics API#7424
jinzhu merged 26 commits intomasterfrom
generics

Conversation

@jinzhu
Copy link
Copy Markdown
Member

@jinzhu jinzhu commented Apr 17, 2025

feat(gorm): add generic CRUD & query interfaces

This commit introduces a type-safe, generic API layer on top of GORM that
allows you to perform Create, Read, Update, Delete and raw SQL operations

Helper constructor:

  func G[T any](db *DB, opts ...clause.Expression) Interface[T]

Basic usage:

// Create a user
u := User{Name: "Alice"}
if err := gorm.G[User](DB).Create(ctx, &u); err != nil {
  // handle error
}

// Query by name and count
cnt, err := gorm.G[User](DB).
  Where("name = ?", "Alice").
  Count(ctx, "*")

// Raw + Find
users, err := gorm.G[User](DB).
  Raw("SELECT * FROM users WHERE created_at > ?", since).
  Find(ctx)

// Update one field
rows, err := gorm.G[User](DB).
  Where("id = ?", u.ID).
  Update(ctx, "email", "[email protected]")

// Delete
deleted, err := gorm.G[User](DB).
  Where("id = ?", u.ID).
  Delete(ctx)

@jinzhu jinzhu requested review from a631807682 and Copilot May 21, 2025 14:58

This comment was marked as outdated.

Copy link
Copy Markdown
Member

@a631807682 a631807682 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be necessary to add unit tests for nested join/preload.
For generics API, how should we reuse it? (Similar to the original Session)

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 22, 2025

It may be necessary to add unit tests for nested join/preload. For generics API, how should we reuse it? (Similar to the original Session)

There are two ways to reuse db:

  1. Just share the db var with condition, as shown in this test https://github.com/go-gorm/gorm/blob/generics/tests/generics_test.go#L758-L771 , which no longer causes any DB pollution issues.

  2. Using Scopes to reuse code — though I'm still thinking about whether there's a better way to implement this.

Do you have any suggestions?

@jinzhu jinzhu requested a review from Copilot May 22, 2025 14:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a type-safe, generic API layer on top of GORM to enable generic CRUD and query operations. Key changes include:

  • Removal of DB.Debug() calls in tests to simplify logging.
  • Upgrades of dependency versions in tests/go.mod.
  • Significant refactoring in core files (e.g. statement.go, callbacks/*) to support generic interfaces and result handling.

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/joins_test.go Removed Debug() calls from Preload chains in join tests.
tests/go.mod Upgraded GORM and related dependency versions.
tests/connpool_test.go Assigned Logger to db immediately after connection open.
statement.go Refactored GetInstance logic and BuildCondition to properly use table context and result propagation.
clause/joins.go Added helper functions for join target creation and aliasing.
callbacks/* Updated CRUD and query callbacks to set and propagate Result fields.
chainable_api.go Added documentation for Unscoped usage.

Comment on lines +122 to 125
db.Logger = DB.Logger
if err != nil {
t.Fatalf("Should open db success, but got %v", err)
}
Copy link

Copilot AI May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assigning the Logger immediately after opening the DB may lead to a nil pointer if an error occurs. Consider checking the error before setting db.Logger.

Suggested change
db.Logger = DB.Logger
if err != nil {
t.Fatalf("Should open db success, but got %v", err)
}
if err != nil {
t.Fatalf("Should open db success, but got %v", err)
}
db.Logger = DB.Logger

Copilot uses AI. Check for mistakes.
case *DB:
subdb := v.Session(&Session{Logger: logger.Discard, DryRun: true}).getInstance()
if v.Statement.SQL.Len() > 0 {
case interface{ getInstance() *DB }:
Copy link

Copilot AI May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider defining and using a named interface for types that provide the getInstance() method to enhance readability and maintainability.

Suggested change
case interface{ getInstance() *DB }:
case InstanceProvider:

Copilot uses AI. Check for mistakes.
@a631807682
Copy link
Copy Markdown
Member

There are two ways to reuse db:

  1. Just share the db var with condition, as shown in this test generics/tests/generics_test.go#L758-L771 , which no longer causes any DB pollution issues.
  2. Using Scopes to reuse code — though I'm still thinking about whether there's a better way to implement this.

Do you have any suggestions?

II think the generics API is simple enough without polluting the db.

Copy link
Copy Markdown
Member

@a631807682 a631807682 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@NeariX67
Copy link
Copy Markdown

NeariX67 commented May 23, 2025

I am currently adopting to the generics branch and noticed that there is no .ToSQL() method. Would be nice to see that added as well, to basicly stay method compatible with the old api.
Edit: .Save(), .Pluck() and .FirstOrCreate() are also missing

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 23, 2025

I am currently adopting to the generics branch and noticed that there is no .ToSQL() method. Would be nice to see that added as well, to basicly stay method compatible with the old api. Edit: .Save() and .FirstOrCreate() are also missing

@NeariX67 you can write it like this https://github.com/go-gorm/gorm/blob/generics/tests/generics_test.go#L842-L847

@NeariX67
Copy link
Copy Markdown

@jinzhu Are you going to implement FirstOrCreate, Pluck, Create (for multiple objects, without batches) and Save for this generics API?

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 25, 2025

@jinzhu Are you going to implement FirstOrCreate, Pluck, Create (for multiple objects, without batches) and Save for this generics API?

We are not planning to implement those methods. FirstOrCreate and Save are convenience APIs, but they can be misleading — for example, they don’t handle concurrency issues well. It's often better to write more explicit and meaningful code instead.

Also, for Pluck, you can use Select with Scan; and for batch creation, you can use CreateInBatches based on the length of your data to control the batch size manually.

With the generics API, our goal is to take a more restrained approach to API design to help avoid potential misuse.

@jinzhu jinzhu merged commit c44405a into master May 25, 2025
36 of 41 checks passed
@jinzhu jinzhu deleted the generics branch May 25, 2025 08:15
@devhaozi devhaozi mentioned this pull request May 26, 2025
1 task
@Allendar
Copy link
Copy Markdown

Allendar commented May 27, 2025

@jinzhu A small bug that prevents the correct execution of Updates;

Updates(ctx context.Context, t T) (rowsAffected int, err error)

should be a pointer like so;

Updates(ctx context.Context, t *T) (rowsAffected int, err error)

This doesn't show up as a mistake because the internal Updates receives an interface{}

@NeariX67
Copy link
Copy Markdown

Noticed that as well. With some objects it will work fine, but some require a pointer and it will throw an error. I have not yet figured out what caused this.
Workaround:

gorm.G[*User](DB).Updates(ctx, &user)

@Allendar
Copy link
Copy Markdown

Transaction would logically also be a feature for Generics? So that the tx is referring to the generic Interface object instead of the whole db object. Or am I looking at this incorrectly?

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 28, 2025

@jinzhu A small bug that prevents the correct execution of Updates;

Updates(ctx context.Context, t T) (rowsAffected int, err error)

should be a pointer like so;

Updates(ctx context.Context, t *T) (rowsAffected int, err error)

This doesn't show up as a mistake because the internal Updates receives an interface{}

What issue might occur if a pointer is not passed in?

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 28, 2025

Transaction would logically also be a feature for Generics? So that the tx is referring to the generic Interface object instead of the whole db object. Or am I looking at this incorrectly?

Transactions are used like this: https://github.com/go-gorm/gorm/blob/master/tests/generics_test.go#L813-L819

or

db.Transaction(func(tx *DB) error {
    users, err gorm.G[User](tx).Find(ctx)
    return err
})

@Allendar
Copy link
Copy Markdown

Allendar commented May 28, 2025

@jinzhu A small bug that prevents the correct execution of Updates;
Updates(ctx context.Context, t T) (rowsAffected int, err error)
should be a pointer like so;
Updates(ctx context.Context, t *T) (rowsAffected int, err error)
This doesn't show up as a mistake because the internal Updates receives an interface{}

What issue might occur if a pointer is not passed in?

The error that the library throws is explicitly that it expects to receive a pointer to a struct in the Updates method internals. For example my User Store's Upsert method;

func (s *Store) Upsert(data *model.User, userID uint) error {
	ctx := context.WithValue(context.Background(), model.UserIDKey, userID)

	if data.ID == 0 {
		return errors.Trace(s.creatorDB.Create(ctx, data))
	}

	rowsAffected, err := s.updaterDB.Updates(ctx, data)
	if err != nil {
		return errors.Trace(err)
	}

	if rowsAffected == 0 {
		return errors.Trace(ErrUpsertDidntModifyData)
	}

	return nil
}

If I call s.updaterDB.Updates(ctx, data) as s.updaterDB.Updates(ctx, *data) then it will throw that error.

@Allendar
Copy link
Copy Markdown

Also the Association is not available on purpose in Generics? Like so in the normal style;

	if err := s.osDeleterDB.WithContext(ctx).Model(&data).Association("GameGameOSs").
		Delete(operatingSystem); err != nil {
		return errors.Trace(err)
	}

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 29, 2025

@jinzhu A small bug that prevents the correct execution of Updates;
Updates(ctx context.Context, t T) (rowsAffected int, err error)
should be a pointer like so;
Updates(ctx context.Context, t *T) (rowsAffected int, err error)
This doesn't show up as a mistake because the internal Updates receives an interface{}

What issue might occur if a pointer is not passed in?

The error that the library throws is explicitly that it expects to receive a pointer to a struct in the Updates method internals. For example my User Store's Upsert method;

func (s *Store) Upsert(data *model.User, userID uint) error {
	ctx := context.WithValue(context.Background(), model.UserIDKey, userID)

	if data.ID == 0 {
		return errors.Trace(s.creatorDB.Create(ctx, data))
	}

	rowsAffected, err := s.updaterDB.Updates(ctx, data)
	if err != nil {
		return errors.Trace(err)
	}

	if rowsAffected == 0 {
		return errors.Trace(ErrUpsertDidntModifyData)
	}

	return nil
}

If I call s.updaterDB.Updates(ctx, data) as s.updaterDB.Updates(ctx, *data) then it will throw that error.

Can you write a playground PR that reproduce the issue? https://github.com/go-gorm/playground/pulls

Seems it works for this test case https://github.com/go-gorm/gorm/blob/master/tests/generics_test.go#L144

@jinzhu
Copy link
Copy Markdown
Member Author

jinzhu commented May 29, 2025

Also the Association is not available on purpose in Generics? Like so in the normal style;

	if err := s.osDeleterDB.WithContext(ctx).Model(&data).Association("GameGameOSs").
		Delete(operatingSystem); err != nil {
		return errors.Trace(err)
	}

Yes, it’s not included yet—I’m still considering if there’s a better way to handle associations.

phroggyy pushed a commit to incident-io/gorm that referenced this pull request Jan 23, 2026
* Implement Generics API

* Add more generics tests

* Add more tests and Take method

* use delayed‑ops pipeline for generics API

* fix generics tests for mysql

* Support SubQuery for Generics

* Add clause.JoinTable helper method

* Fix golangci-lint error

* Complete the design and implementation of generic version Join

* improve generics version Joins support

* allow configuring select/omit columns for joins via subqueries

* finish generic version Preload

* handle error of generics Joins/Preload

* fix tests

* Add LimitPerRecord for generic version Preload

* fix tests for mysql 5.7

* test for nested generic version Join/Preload

* Add WithResult support for generics API

* test reuse generics db conditions

* fix data race

* remove ExampleLRU test

* Add default transaction timeout support

* fix test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants