In what area(s)?

/area runtime
/area docs

Describe the proposal

This proposal is a follow-up on #1339

This is a proposed design of the API for querying the state store.

❗ Note: The purpose of this API is to provide the means for querying the state in a state store. This is not a generic database query API.

This design is split into 3 independent parts:

1. Syntax of the query request/response
2. Intermediate query representation
3. Interface for constructing final native query in the state component

These parts could be changed/updated independently of each other, providing flexibility in further development.
Below is mode detailed description.

1. Syntax of the query request

A user submits a query request via HTTP POST or gRPC.
The body of the request is the JSON map with 3 entries: filter, sort, and pagination
The filter section specifies the query conditions in the form of a tree of key/values operations, where the key is the operator and the value is the operands.

Currently we support four operations:

This design allows to add more operations without breaking changes.

The sort is an optional section is an ordered array of key:order pairs, where key is a key in the state store, and the order is an optional string indicating sorting order: "ASC" for ascending and "DESC" for descending. If omitted, ascending order is assumed.

The pagination is an optional section containing limit and token parameters. limit sets the page size. token is an iteration token returned by the component, and used in consecutive queries.

type QueryRequest struct {
	Query Query `json:"query"`
}

type Query struct {
	Filters map[string]interface{} `json:"filter"`
	Sort    []Sorting              `json:"sort"`
	Page    Pagination             `json:"page"`

	// derived from Filters
	Filter Filter
}

type Sorting struct {
	Key   string `json:"key"`
	Order string `json:"order,omitempty"`
}

type Pagination struct {
	Limit int    `json:"limit"`
	Token string `json:"token,omitempty"`
}

Here is an example of the query request JSON body:

{
    "query": {
        "filter": {
            "OR": [
                {
                    "EQ": {"person.org": "Dev Ops"}
                },
                {
                    "AND": [
                        {
                            "EQ": {"person.org": "Finance"}
                        },
                        {
                            "IN": {"state": ["CA", "WA"]}
                        }
                    ]
                }
            ]
        },
        "sort": [
            {
               "key": "state",
                "order": "DESC"
            },
            {
                "key": "person.name"
            }
        ],
        "pagination": {
            "limit": 10
        }
    }
}

The query request will be eventually translated into a native query language and executed in the state store component.

In the case of SQL, the aforementioned query will be translated into

SELECT * FROM c WHERE
  person.org = "Dev Ops" OR
  (person.org = "Finance" AND state IN ("CA", "WA"))
ORDER BY
  state DESC,
  person.name ASC
LIMIT 10

Upon successful execution, the component will return a JSON object with query results and pagination token:

type QueryResponse struct {
	Results []QueryResult `json:"results"`
	Token   string        `json:"token,omitempty"`
}

type QueryResult struct {
	Key   string  `json:"key"`
	Data  []byte  `json:"data"`
	ETag  *string `json:"etag,omitempty"`
	Error string  `json:"error,omitempty"`
}

2. Intermediate query representation

Upon receiving the query request, Dapr will validate it and transform into Intermediate query object MidQuery,
which will be passed on to the state store component.

To leverage JSON query format, MidQuery uses custom JSON marshaling to populate tree of Filters.

type Filter interface {
	Parse(interface{}) error
}

type EQ struct {
	Key string
	Val interface{}
}

type IN struct {
	Key  string
	Vals []interface{}
}

type AND struct {
	Filters []Filter
}

type OR struct {
	Filters []Filter
}

3. An interface for constructing final native query in the state component

To simplify the process of query translation, we propose using visitor design pattern. A state store component developer would need to implement the visit method, and the runtime will use it to construct the native query statement.

type Visitor interface {
	// returns "equal" expression
	VisitEQ(*EQ) (string, error)
	// returns "in" expression
	VisitIN(*IN) (string, error)
	// returns "and" expression
	VisitAND(*AND) (string, error)
	// returns "or" expression
	VisitOR(*OR) (string, error)
	// receives concatenated filters and finalizes the native query
	Finalize(string, *Query) error
}

The Dapr runtime implements QueryBuilder object that takes in Visitor interface and constructs the native query.

type QueryBuilder struct {
	visitor Visitor
}

func (h *QueryBuilder) BuildQuery(q *Query) error {...}

The last part is to implement Querier interface in the component:

type Querier interface {
	Query(req *QueryRequest) (*QueryResponse, error)
}

A sample implementation might look like that:

func (m *MyComponent) Query(req *state.QueryRequest) (*state.QueryRespond, error) {
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	q := &Query{} // Query implements Visitor interface
	qbuilder := query.NewQueryBuilder(q)
	if err := qbuilder.BuildQuery(&req.Query); err != nil {
		return &state.QueryRespond{}, err
	}
	res, token, err := q.execute(ctx)
	if err != nil {
		return &state.QueryRespond{}, err
	}
	return &state.QueryRespond{
		Results:  res,
		Token: token,
	}, nil
}