feat: refactor `manager api` by nic-chen · Pull Request #564 · apache/apisix-dashboard

nic-chen · 2020-10-20T06:51:26Z

Please answer these questions before submitting a pull request

Why submit this pull request?
Bug fix
New feature provided
Improve performance

New feature or improvement

refactor manager api to remove the dependency on MySQL

* add upstream struct * fix structures

* feat: add consumer CURD refactoring * remove debug * remove useless slashes

Co-authored-by: Vinci Xu <[email protected]>

* feat: add Service and Upstream refactoring * fix add license

* feat: add store hub and flake id * feat: add store interface to easy est

* test: delete mysql version test cases. * fix: add ssl test cases and bug fix * fix: list api should return an empty array not a null for client * test: consumer test cases * fix: code style * test: init etcd in github action * fix: skip checking generated file's license * feat: add store hub and flake id (#534) * feat: add store hub and flake id * feat: add store interface to easy est * fix CI failed Co-authored-by: Vinci Xu <[email protected]>

* fix code style * fix code style * feat support query * feat: support query * change: `like` to `equal` * fix api status * feat: upstream existing check * feat: refactor api for upstream names * fix: license * test: add unit test cases * fix: update bug * test: add test cases for route * fix: unified respond format * test: remove test bug * feat: ssl existing check * fix bug: auto generate id * fix: improve consumer * fix: remove key and keys in ssl respond * fix: when list is empty, should respond an empty array * fix code style * feat: plugin orchestration * fix delete bug * fix bug * fix: keep the same request params and respond with the old format

… update resource (#551) * feat: json schema check * fix: don't need to define struct for each resource, because that may cause json schema check fail. * test: add handler test cases * test: complete consumer test cases * test: add test cases for schema check * fix code style and license * feat: add schema check for plugins * test: add ssl handler test cases * test: add test cases for upstream and service * test: add test cases for route * test: add note for route create * test: update CI * fix: remove useless file * test: fix CI * fix: ci fail * test: fix lib `dag-to-lua`'s path in CI * fix: URI for route may be empty * fix: remove empty lines * fix: refactor validator of json schema * fix code style * fix cicd * chore: update docker file * fix: should check schema after id generated * fix code style * chore: page_number -> page * fix: schema sync script

* feat: support search for resource list * fix ci

* feat: refactor plugin and healthy api * feat: refactor authentication api * fix: remove useless files * chore: update json schema * test: add login test * test: add test for plugin * fix: license

…#561) * feat: support labels * feat: compatible with PUT method of `admin api` * fix mock test fail * feat: upstream nodes format * test: add test case * fix code style * fix: update schema sync tool

* feat: compatible with HTTP status of `admin api` * test cases and improve

# Conflicts: # api/conf/conf.go # api/route/route.go # api/route/route_test.go # api/service/consumer.go # api/service/route.go

juzhiyuan

Not sure about those logics, only check those files' license.

juzhiyuan · 2020-10-20T07:12:05Z

@@ -0,0 +1,116 @@
+// Code generated by mockery v1.0.0. DO NOT EDIT.


There should not be any automatically generated code in the code base

juzhiyuan · 2020-10-20T07:13:29Z

@@ -0,0 +1,24 @@
+// Code generated by mockery v1.0.0. DO NOT EDIT.


moonming · 2020-10-20T07:24:57Z

?   	github.com/apisix/manager-api	[no test files]
?   	github.com/apisix/manager-api/conf	[no test files]
?   	github.com/apisix/manager-api/filter	[no test files]
?   	github.com/apisix/manager-api/internal	[no test files]
ok  	github.com/apisix/manager-api/internal/core/entity	0.004s
?   	github.com/apisix/manager-api/internal/core/storage	[no test files]
ok  	github.com/apisix/manager-api/internal/core/store	1.015s
?   	github.com/apisix/manager-api/internal/handler	[no test files]
ok  	github.com/apisix/manager-api/internal/handler/authentication	0.013s
ok  	github.com/apisix/manager-api/internal/handler/consumer	0.334s
?   	github.com/apisix/manager-api/internal/handler/healthz	[no test files]
ok  	github.com/apisix/manager-api/internal/handler/plugin	0.010s
ok  	github.com/apisix/manager-api/internal/handler/route	0.724s
ok  	github.com/apisix/manager-api/internal/handler/service	0.227s
ok  	github.com/apisix/manager-api/internal/handler/ssl	0.160s
ok  	github.com/apisix/manager-api/internal/handler/upstream	0.223s
ok  	github.com/apisix/manager-api/internal/utils	0.005s
?   	github.com/apisix/manager-api/internal/utils/consts	[no test files]
?   	github.com/apisix/manager-api/log	[no test files]

why so many files no test cases cover?

moonming · 2020-10-20T07:31:50Z

@nic-chen do you check licenses for all go dependencies?

nic-chen · 2020-10-20T08:06:24Z

@nic-chen do you check licenses for all go dependencies?

here is the result:

gopkg.in/yaml.v2,Unknown,Apache-2.0
github.com/coreos/go-systemd/journal,https://github.com/coreos/go-systemd/blob/master/journal/LICENSE,Apache-2.0
github.com/gorilla/securecookie,https://github.com/gorilla/securecookie/blob/master/LICENSE,BSD-3-Clause
github.com/go-playground/validator/v10,https://github.com/go-playground/validator/blob/master/v10/LICENSE,MIT
github.com/leodido/go-urn,https://github.com/leodido/go-urn/blob/master/LICENSE,MIT
github.com/sony/sonyflake,https://github.com/sony/sonyflake/blob/master/LICENSE,MIT
go.uber.org/atomic,Unknown,MIT
github.com/sirupsen/logrus,https://github.com/sirupsen/logrus/blob/master/LICENSE,MIT
github.com/shiningrush/droplet,https://github.com/shiningrush/droplet/blob/master/LICENSE,MIT
github.com/coreos/pkg/capnslog,https://github.com/coreos/pkg/blob/master/capnslog/LICENSE,Apache-2.0
github.com/coreos/go-semver/semver,https://github.com/coreos/go-semver/blob/master/semver/LICENSE,Apache-2.0
github.com/api7/go-jsonpatch,https://github.com/api7/go-jsonpatch/blob/master/LICENSE,MIT
golang.org/x/sys,Unknown,BSD-3-Clause
github.com/coreos/etcd,https://github.com/coreos/etcd/blob/master/LICENSE,Apache-2.0
go.uber.org/zap,Unknown,MIT
github.com/gin-contrib/pprof,https://github.com/gin-contrib/pprof/blob/master/LICENSE,MIT
github.com/spf13/cast,https://github.com/spf13/cast/blob/master/LICENSE,MIT
github.com/tidwall/match,https://github.com/tidwall/match/blob/master/LICENSE,MIT
github.com/gin-gonic/gin,https://github.com/gin-gonic/gin/blob/master/LICENSE,MIT
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
github.com/satori/go.uuid,https://github.com/satori/go.uuid/blob/master/LICENSE,MIT
go.etcd.io/etcd/clientv3,Unknown,Apache-2.0
google.golang.org/genproto/googleapis,Unknown,Apache-2.0
github.com/spf13/jwalterweatherman,https://github.com/spf13/jwalterweatherman/blob/master/LICENSE,MIT
gopkg.in/ini.v1,Unknown,Apache-2.0
google.golang.org/grpc,Unknown,Apache-2.0
github.com/tidwall/gjson,https://github.com/tidwall/gjson/blob/master/LICENSE,MIT
github.com/dgrijalva/jwt-go,https://github.com/dgrijalva/jwt-go/blob/master/LICENSE,MIT
github.com/gin-contrib/sse,https://github.com/gin-contrib/sse/blob/master/LICENSE,MIT
google.golang.org/protobuf,Unknown,BSD-3-Clause
github.com/mattn/go-isatty,https://github.com/mattn/go-isatty/blob/master/LICENSE,MIT
github.com/mitchellh/mapstructure,https://github.com/mitchellh/mapstructure/blob/master/LICENSE,MIT
github.com/pmezard/go-difflib/difflib,https://github.com/pmezard/go-difflib/blob/master/difflib/LICENSE,BSD-3-Clause
github.com/stretchr/objx,https://github.com/stretchr/objx/blob/master/LICENSE,MIT
github.com/subosito/gotenv,https://github.com/subosito/gotenv/blob/master/LICENSE,MIT
github.com/apisix/manager-api,Unknown,Apache-2.0
github.com/shiningrush/droplet/wrapper/gin,https://github.com/shiningrush/droplet/blob/master/wrapper/gin/LICENSE,MIT
github.com/gogo/protobuf,https://github.com/gogo/protobuf/blob/master/LICENSE,BSD-3-Clause
go.uber.org/multierr,Unknown,MIT
github.com/fsnotify/fsnotify,https://github.com/fsnotify/fsnotify/blob/master/LICENSE,BSD-3-Clause
github.com/magiconair/properties,https://github.com/magiconair/properties/blob/master/LICENSE,BSD-2-Clause
github.com/google/uuid,https://github.com/google/uuid/blob/master/LICENSE,BSD-3-Clause
github.com/spf13/pflag,https://github.com/spf13/pflag/blob/master/LICENSE,BSD-3-Clause
github.com/tidwall/pretty,https://github.com/tidwall/pretty/blob/master/LICENSE,MIT
github.com/go-playground/universal-translator,https://github.com/go-playground/universal-translator/blob/master/LICENSE,MIT
golang.org/x/text,Unknown,BSD-3-Clause
github.com/gin-contrib/sessions,https://github.com/gin-contrib/sessions/blob/master/LICENSE,MIT
github.com/go-playground/locales,https://github.com/go-playground/locales/blob/master/LICENSE,MIT
github.com/ugorji/go/codec,https://github.com/ugorji/go/blob/master/codec/LICENSE,MIT
github.com/stretchr/testify,https://github.com/stretchr/testify/blob/master/LICENSE,MIT
github.com/gorilla/sessions,https://github.com/gorilla/sessions/blob/master/LICENSE,BSD-3-Clause
github.com/spf13/afero,https://github.com/spf13/afero/blob/master/LICENSE.txt,Apache-2.0
gopkg.in/yaml.v3,Unknown,MIT
golang.org/x/net,Unknown,BSD-3-Clause
github.com/gorilla/context,https://github.com/gorilla/context/blob/master/LICENSE,BSD-3-Clause
github.com/pelletier/go-toml,https://github.com/pelletier/go-toml/blob/master/LICENSE,MIT
github.com/xeipuuv/gojsonschema,Unknown,Apache-2.0
github.com/xeipuuv/gojsonreference,Unknown,Apache-2.0
github.com/xeipuuv/gojsonpointer,Unknown,Apache-2.0
github.com/spf13/viper,https://github.com/spf13/viper/blob/master/LICENSE,MIT


github.com/davecgh/go-spew/spew,https://github.com/davecgh/go-spew/blob/master/spew/LICENSE,ISC
github.com/hashicorp/hcl,https://github.com/hashicorp/hcl/blob/master/LICENSE,MPL-2.0

The last two are not directly dependent. I don't know whether this will be a problem

nic-chen · 2020-10-20T08:20:12Z

?   	github.com/apisix/manager-api	[no test files]
?   	github.com/apisix/manager-api/conf	[no test files]
?   	github.com/apisix/manager-api/filter	[no test files]
?   	github.com/apisix/manager-api/internal	[no test files]
ok  	github.com/apisix/manager-api/internal/core/entity	0.004s
?   	github.com/apisix/manager-api/internal/core/storage	[no test files]
ok  	github.com/apisix/manager-api/internal/core/store	1.015s
?   	github.com/apisix/manager-api/internal/handler	[no test files]
ok  	github.com/apisix/manager-api/internal/handler/authentication	0.013s
ok  	github.com/apisix/manager-api/internal/handler/consumer	0.334s
?   	github.com/apisix/manager-api/internal/handler/healthz	[no test files]
ok  	github.com/apisix/manager-api/internal/handler/plugin	0.010s
ok  	github.com/apisix/manager-api/internal/handler/route	0.724s
ok  	github.com/apisix/manager-api/internal/handler/service	0.227s
ok  	github.com/apisix/manager-api/internal/handler/ssl	0.160s
ok  	github.com/apisix/manager-api/internal/handler/upstream	0.223s
ok  	github.com/apisix/manager-api/internal/utils	0.005s
?   	github.com/apisix/manager-api/internal/utils/consts	[no test files]
?   	github.com/apisix/manager-api/log	[no test files]

why so many files no test cases cover?

github.com/apisix/manager-api	                         --> just a main.go
github.com/apisix/manager-api/conf	                     --> config
github.com/apisix/manager-api/filter	                 --> 
github.com/apisix/manager-api/internal	                 --> no files
github.com/apisix/manager-api/internal/core/storage	     -->  
github.com/apisix/manager-api/internal/handler	         --> no files
github.com/apisix/manager-api/internal/handler/healthz	 -->
github.com/apisix/manager-api/internal/utils/consts	     --> const defined
github.com/apisix/manager-api/log	                     -->

@moonming
The main code is in the subdirectory of handler and the store directory. I think the missing part could be added later.

…factor

moonming · 2020-10-20T09:00:34Z

github.com/davecgh/go-spew/spew,https://github.com/davecgh/go-spew/blob/master/spew/LICENSE,ISC
github.com/hashicorp/hcl,https://github.com/hashicorp/hcl/blob/master/LICENSE,MPL-2.0

ISC is ok. MPL-2.0 maybe is a issue. Can we remove this dependent?

nic-chen · 2020-10-20T09:42:09Z

github.com/davecgh/go-spew/spew,https://github.com/davecgh/go-spew/blob/master/spew/LICENSE,ISC
github.com/hashicorp/hcl,https://github.com/hashicorp/hcl/blob/master/LICENSE,MPL-2.0

ISC is ok. MPL-2.0 maybe is a issue. Can we remove this dependent?

removed.

…factor

vincixu and others added 26 commits September 9, 2020 12:51

feat: refactor some codes and append store core

b2dcf5e

resolve conflict

13d0e98

refference droplet and write a API demo

2ee78c7

Merge remote-tracking branch 'origin/master' into refactor

b1471f9

fmt project

24c627a

feat: add validator for generic store; add demo error

dda1cb9

feat: upgrade droplet

a5956e7

chore: add structures (#484)

7490f7f

* add upstream struct * fix structures

feat: append stock check for generic store

4df9d6d

feat: add consumer CURD refactoring (#486)

33623e8

* feat: add consumer CURD refactoring * remove debug * remove useless slashes

feat: add SSL refactoring (#488)

075c1c3

Co-authored-by: Vinci Xu <[email protected]>

feat: add Service and Upstream refactoring (#497)

094c5ee

* feat: add Service and Upstream refactoring * fix add license

feat: add store hub and flake id (#534)

79348d9

* feat: add store hub and flake id * feat: add store interface to easy est

fix: append sort for list and using sync.Map instead of map

e7c00d2

fix: code style

5152b3a

feat: support search for resource list (#557)

c3dcefb

* feat: support search for resource list * fix ci

feature: refactor plugin api and auth api (#556)

03d39f8

* feat: refactor plugin and healthy api * feat: refactor authentication api * fix: remove useless files * chore: update json schema * test: add login test * test: add test for plugin * fix: license

fix auth bug

65ed623

fix route search by uri

061d9a2

feat: compatible with PUT method of admin api and nodes of upstream (…

0bbcc75

…#561) * feat: support labels * feat: compatible with PUT method of `admin api` * fix mock test fail * feat: upstream nodes format * test: add test case * fix code style * fix: update schema sync tool

feat: compatible with HTTP status of admin api (#563)

2088156

* feat: compatible with HTTP status of `admin api` * test cases and improve

fix: check input.ID before using it

be1edf6

Merge branch 'master' into refactor

9dbfaa0

# Conflicts: # api/conf/conf.go # api/route/route.go # api/route/route_test.go # api/service/consumer.go # api/service/route.go

nic-chen requested review from gxthrj, juzhiyuan, membphis and moonming October 20, 2020 07:03

nic-chen requested a review from ShiningRush October 20, 2020 07:03

juzhiyuan requested changes Oct 20, 2020

View reviewed changes

juzhiyuan added this to the 1.6 milestone Oct 20, 2020

johzchen and others added 4 commits October 20, 2020 16:21

fix: remove CI branch

e0f13fb

fix: remove useless dependences

63f8f17

add license for json.lua

162c0d9

Merge branch 'refactor' of github.com:apache/apisix-dashboard into re…

496d390

…factor

moonming and others added 3 commits October 20, 2020 17:26

fix license issue

1b19a94

remove

2269b53

fix: remove viper that depend github.com/hashicorp/hcl

a985337

moonming and others added 3 commits October 20, 2020 17:58

fix license issue

394a864

Merge branch 'refactor' of github.com:apache/apisix-dashboard into re…

228086f

…factor

fix: skip license check temporarily for CI

1bc5d82

moonming approved these changes Oct 20, 2020

View reviewed changes

juzhiyuan approved these changes Oct 20, 2020

View reviewed changes

juzhiyuan merged commit a1f96d4 into master Oct 20, 2020

juzhiyuan deleted the refactor branch October 20, 2020 15:41

		@@ -0,0 +1,116 @@
		// Code generated by mockery v1.0.0. DO NOT EDIT.

		@@ -0,0 +1,24 @@
		// Code generated by mockery v1.0.0. DO NOT EDIT.

Uh oh!

Conversation

nic-chen commented Oct 20, 2020

New feature or improvement

Uh oh!

juzhiyuan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juzhiyuan Oct 20, 2020

Choose a reason for hiding this comment

Uh oh!

moonming Oct 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juzhiyuan Oct 20, 2020

Choose a reason for hiding this comment

Uh oh!

moonming Oct 20, 2020

Choose a reason for hiding this comment

Uh oh!

moonming commented Oct 20, 2020

Uh oh!

moonming commented Oct 20, 2020

Uh oh!

nic-chen commented Oct 20, 2020

Uh oh!

nic-chen commented Oct 20, 2020

Uh oh!

moonming commented Oct 20, 2020

Uh oh!

nic-chen commented Oct 20, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

moonming Oct 20, 2020 •

edited

Loading