Skip to content

Commit 3510c83

Browse files
jzhou59jzhou59
authored
[ISSUE #339] Add a skeleton for separated openschema registry service and implement most APIs that doesn't need compatibility check. (#525)
* [ISSUE #339] a skeleton of independent openschema registry service * add license for build.gradle add license for ```incubator-eventmesh/eventmesh-openschema/build.gradle``` * modify license modify license of ```incubator-eventmesh/eventmesh-openschema/build.gradle``` * update license update license of ```build.gradle``` in ```incubator-eventmesh/eventmesh-openschema/eventmesh-openschema-registry``` * Update application.yml update license of ```application.yml``` in ```incubator-eventmesh/eventmesh-openschema/eventmesh-openschema-registry``` * [ISSUE #339] add FLIP-like progress * [ISSUE #339] fix allowed-licenses.txt and remove unused dependencies * [ISSUE #339] remove null in allowed-licenses.txt and move dependency version in root-build.gradle
1 parent f12bf52 commit 3510c83

34 files changed

+1754
-0
lines changed

‎build.gradle‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -386,6 +386,13 @@ subprojects {
386386
dependency "org.mockito:mockito-core:2.23.0"
387387
dependency "org.powermock:powermock-module-junit4:2.0.2"
388388
dependency "org.powermock:powermock-api-mockito2:2.0.2"
389+
390+
dependency 'org.springframework.boot:spring-boot-starter-data-jdbc:2.5.4'
391+
dependency 'org.springframework.boot:spring-boot-starter-data-jpa:2.5.4'
392+
dependency 'org.springframework.boot:spring-boot-starter-jdbc:2.5.4'
393+
dependency 'org.springframework:spring-beans:5.1.8.RELEASE'
394+
dependency 'org.projectlombok:lombok:1.18.20'
395+
dependency 'com.h2database:h2:1.4.200'
389396
}
390397
}
391398
}

‎docs/en/features/eventmesh-schemaregistry-design.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,66 @@ The highlevel process of messages transmission undergoes 10 steps as follows:
6363
- step9: Schema Registry returns schema and EventMesh caches it.
6464
- step10: EventMesh patches schema in front of messages and push it to consumer.
6565

66+
## Current Progress
67+
### Status
68+
**Current state** : Developing
69+
70+
**Discussion thread** : ISSUE #339
71+
72+
### Proposed Changes
73+
The proposal has two aspects.
74+
75+
First is a separated Open Schema Registry, which includes storage and compatibility check for schema.
76+
This proposal is under developing.
77+
78+
Second is the integration of Open Schema in Eventmesh, which includes validation for schema. This proposal is to be developed.
79+
80+
As for the first proposal, some developing statuses are as follows.
81+
82+
**Status Code and Exception Code**
83+
84+
No. | Status Code | Exception Code | Description | status
85+
--- | :---: | :---: | :---: | :---:
86+
1 | 401 | 40101 | Unauthorized Exception | ✔
87+
2 | 404 | 40401 | Schema Non- Exception | ✔
88+
3 | ^ | 40402 | Subject Non-exist Exception | ✔
89+
4 | ^ | 40403 | Version Non-exist Exception | ✔
90+
5 | 409 | 40901 | Compatibility Exception | ✔
91+
6 | 422 | 42201 | Schema Format Exception | ✔
92+
7 | ^ | 42202 | Subject Format Exception | ✔
93+
8 | ^ | 42203 | Version Format Exception | ✔
94+
9 | ^ | 42204 | Compatibility Format Exception | ✔
95+
10 | 500 | 50001 | Storage Service Exception | ✔
96+
11 | ^ | 50002 | Timeout Exception | ✔
97+
98+
**API developing status** :
99+
100+
No. | Type | URL | response | exception | code | test
101+
--- | --- | --- | --- | --- | --- | ---
102+
1 | GET | /schemas/ids/{string: id} | Schema.class | 40101\40401\50001 | ✔ | ❌
103+
2 | GET | /schemas/ids/{string: id}/subjects | SubjectAndVersion.class | 40101\40401\50001 | ✔ | ❌
104+
3 | GET | /subjects | List\<String> | 40101\50001 | ✔ | ❌
105+
4 | GET | /subjects/{string: subject}/versions | List\<Integer> | 40101\40402\50001 | ✔ | ❌
106+
5 | DELETE | /subjects/(string: subject) | List\<Integer> | 40101\40402\50001 | ✔ | ❌
107+
6 | GET | /subjects/(string: subject) | Subject.class | 40101\40402\50001 | ✔ | ❌
108+
7 | GET | /subjects/(string: subject)/versions/(version: version)/schema | SubjectWithSchema.class | 40101\40402\40403\50001 | ✔ | ❌
109+
8 | POST | /subjects/(string: subject)/versions | SchemaIdResponse.class | 40101\40901\42201\50001\50002 | - | ❌
110+
9 | POST | /subjects/(string: subject)/ | Subject.class | 40101\40901\42202\50001\50002 | ✔ | ❌
111+
10 | DELETE | /subjects/(string: subject)/versions/(version: version) | int | 40101\40402\40403\40901\50001| - | ❌
112+
11 | POST | /compatibility/subjects/(string: subject)/versions/(version: version) | CompatibilityResultResponse.class | 40101\40402\40403\42201\42203\50001| - | ❌
113+
12 | GET | /compatibility/(string: subject) | Compatibility.class | 40101\40402\50001 | ✔ | ❌
114+
13 | PUT | /compatibility/(string: subject) | Compatibility.class | 40101\40402\40901\42204\50001 | - | ❌
115+
116+
**Overall Project Structure**
117+
118+
```SchemaController.java```+```SchemaService.java``` : ```OpenSchema 7.1.1~7.1.2 (API 1~2)```
119+
120+
```SubjectController.java```+```SubjectService.java``` : ```OpenSchema 7.2.1~7.2.8 (API 3~10)```
121+
122+
```CompatibilityController.java```+```CompatibilityService.java``` : ```OpenSchema 7.3.1~7.3.3 (API 11~13)``` + ```Check for Compatibility```
123+
124+
![Project_Structure](../../images/features/eventmesh-schemaregistry-projectstructure.png)
125+
66126

67127
## References
68128
[1] [schema validator (github.com)](https://github.com/search?q=schema+validator)

‎docs/images/features/eventmesh-schemaregistry-projectstructure.png‎

44.2 KB
Loading

‎eventmesh-openschema/build.gradle‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
plugins {
19+
}
20+
21+
group 'org.apache.eventmesh'
22+
version '1.2.0-SNAPSHOT'
23+
24+
repositories {
25+
mavenCentral()
26+
}
27+
28+
dependencies {
29+
}
30+
31+
test {
32+
useJUnitPlatform()
33+
}

‎eventmesh-openschema/eventmesh-openschema-registry/build.gradle‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
group 'org.apache.eventmesh'
19+
version '1.2.0-SNAPSHOT'
20+
21+
configurations {
22+
compileOnly {
23+
extendsFrom annotationProcessor
24+
}
25+
}
26+
27+
repositories {
28+
mavenCentral()
29+
}
30+
31+
dependencies {
32+
implementation 'org.springframework.boot:spring-boot-starter-data-jdbc'
33+
implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
34+
implementation 'org.springframework.boot:spring-boot-starter-jdbc'
35+
implementation 'org.springframework.boot:spring-boot-starter-web'
36+
implementation 'org.springframework:spring-beans'
37+
compileOnly 'org.projectlombok:lombok'
38+
runtimeOnly 'com.h2database:h2'
39+
annotationProcessor 'org.projectlombok:lombok'
40+
}
41+
42+
test {
43+
useJUnitPlatform()
44+
}

‎eventmesh-openschema/eventmesh-openschema-registry/src/main/java/org/apache/eventmesh/openschemaregistry/OpenSchemaRegistryApplication.java‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package org.apache.eventmesh.openschemaregistry;
18+
19+
import org.springframework.boot.SpringApplication;
20+
import org.springframework.boot.autoconfigure.SpringBootApplication;
21+
22+
@SpringBootApplication
23+
public class OpenSchemaRegistryApplication {
24+
25+
public static void main(String[] args) {
26+
SpringApplication.run(OpenSchemaRegistryApplication.class, args);
27+
}
28+
29+
}

‎eventmesh-openschema/eventmesh-openschema-registry/src/main/java/org/apache/eventmesh/openschemaregistry/controller/CompatibilityController.java‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package org.apache.eventmesh.openschemaregistry.controller;
18+
19+
import org.apache.eventmesh.openschemaregistry.domain.Compatibility;
20+
import org.apache.eventmesh.openschemaregistry.domain.Schema;
21+
import org.apache.eventmesh.openschemaregistry.response.CompatibilityResultResponse;
22+
import org.apache.eventmesh.openschemaregistry.service.CompatibilityService;
23+
import org.springframework.beans.factory.annotation.Autowired;
24+
import org.springframework.http.ResponseEntity;
25+
import org.springframework.web.bind.annotation.*;
26+
27+
@RestController
28+
@RequestMapping("/compatibility")
29+
public class CompatibilityController {
30+
31+
@Autowired
32+
CompatibilityService compatibilityService;
33+
34+
@PostMapping("/subjects/{subject}/versions/{version}")
35+
public ResponseEntity<CompatibilityResultResponse> checkWhetherCompatible(@PathVariable("subject") String subject,
36+
@PathVariable("version") Integer version,
37+
@RequestBody Schema schema){
38+
CompatibilityResultResponse resultResponse = compatibilityService.checkWhetherCompatible(subject, version, schema);
39+
return ResponseEntity.ok(resultResponse);
40+
}
41+
42+
@GetMapping("/{subject}")
43+
public ResponseEntity<Compatibility> getCompatibilityBySubject(@PathVariable("subject") String subject){
44+
Compatibility compatibility = compatibilityService.getCompatibilityBySubject(subject);
45+
return ResponseEntity.ok(compatibility);
46+
}
47+
48+
@PutMapping("/{subject}")
49+
public ResponseEntity<Compatibility> updateCompatibilityBySubject(@PathVariable("subject") String subject,
50+
@RequestBody Compatibility compatibility){
51+
Compatibility updatedCompatibility = compatibilityService.updateCompatibilityBySubject(subject, compatibility);
52+
return ResponseEntity.ok(updatedCompatibility);
53+
}
54+
}

‎eventmesh-openschema/eventmesh-openschema-registry/src/main/java/org/apache/eventmesh/openschemaregistry/controller/SchemaController.java‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package org.apache.eventmesh.openschemaregistry.controller;
18+
19+
import org.apache.eventmesh.openschemaregistry.domain.Schema;
20+
import org.apache.eventmesh.openschemaregistry.response.SubjectAndVersionResponse;
21+
import org.apache.eventmesh.openschemaregistry.service.SchemaService;
22+
import org.springframework.beans.factory.annotation.Autowired;
23+
import org.springframework.http.ResponseEntity;
24+
import org.springframework.web.bind.annotation.*;
25+
26+
@RestController
27+
@RequestMapping("/schemas")
28+
public class SchemaController {
29+
@Autowired
30+
SchemaService schemaService;
31+
32+
@GetMapping("/ids/{id}")
33+
public ResponseEntity<Schema> fetchSchemaById(@PathVariable("id") long id){
34+
Schema schema = schemaService.getSchemaById(id);
35+
return ResponseEntity.ok(schema);
36+
}
37+
38+
@GetMapping("/ids/{id}/subjects")
39+
public ResponseEntity<SubjectAndVersionResponse> fetchSubjectAndVersionById(@PathVariable("id") long id){
40+
SubjectAndVersionResponse subjectAndVersionResponse = schemaService.getSubjectAndVersionById(id);
41+
return ResponseEntity.ok(subjectAndVersionResponse);
42+
}
43+
}

‎eventmesh-openschema/eventmesh-openschema-registry/src/main/java/org/apache/eventmesh/openschemaregistry/controller/SubjectController.java‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.eventmesh.openschemaregistry.controller;
19+
20+
import org.apache.eventmesh.openschemaregistry.domain.Schema;
21+
import org.apache.eventmesh.openschemaregistry.domain.Subject;
22+
import org.apache.eventmesh.openschemaregistry.domain.SubjectWithSchema;
23+
import org.apache.eventmesh.openschemaregistry.response.SchemaIdResponse;
24+
import org.apache.eventmesh.openschemaregistry.service.SubjectService;
25+
import org.springframework.beans.factory.annotation.Autowired;
26+
import org.springframework.http.ResponseEntity;
27+
import org.springframework.web.bind.annotation.*;
28+
29+
import java.util.List;
30+
31+
@RestController
32+
@RequestMapping("/subjects")
33+
public class SubjectController {
34+
35+
@Autowired
36+
SubjectService subjectService;
37+
38+
@GetMapping({"","/"})
39+
public ResponseEntity<List<String>> getAllSubjects(){
40+
List<String> subjects = subjectService.getAllSubjects();
41+
return ResponseEntity.ok(subjects);
42+
}
43+
44+
@GetMapping("/{subject}/versions")
45+
public ResponseEntity<List<Integer>> getAllVersionBySubject(@PathVariable("subject") String subject){
46+
List<Integer> versions = subjectService.getAllVersionsBySubject(subject);
47+
return ResponseEntity.ok(versions);
48+
}
49+
50+
@DeleteMapping("/subjects/{subject}")
51+
public ResponseEntity<List<Integer>> deleteSubjectAndAllSchemaBySubject(@PathVariable("subject") String subject){
52+
List<Integer> versions = subjectService.deleteSubjectAndAllSchemaBySubject(subject);
53+
return ResponseEntity.ok(versions);
54+
}
55+
56+
@GetMapping("/{subject}")
57+
public ResponseEntity<Subject> getSubjectByName(@PathVariable("subject") String subject){
58+
Subject getSubject = subjectService.getSubjectByName(subject);
59+
return ResponseEntity.ok(getSubject);
60+
}
61+
62+
@GetMapping("/{subject}/versions/{version}/schema")
63+
public ResponseEntity<SubjectWithSchema> getSchemaBySubjectAndVersion(@PathVariable("subject")String subject, @PathVariable("version")int version){
64+
SubjectWithSchema subjectWithSchema = subjectService.getSchemaBySubjectAndVersion(subject, version);
65+
return ResponseEntity.ok(subjectWithSchema);
66+
}
67+
68+
@PostMapping("/subjects/{subject}/versions")
69+
public ResponseEntity<SchemaIdResponse> checkOrRegisterSchema(@PathVariable("subject") String subject, @RequestBody Schema schema){
70+
SchemaIdResponse schemaIdResponse = subjectService.checkOrRegisterSchema(subject, schema);
71+
return ResponseEntity.ok(schemaIdResponse);
72+
}
73+
74+
@PostMapping("/subjects/{subject}")
75+
public ResponseEntity<Subject> updateSubjectIfDifferent(@PathVariable("subject") String subjectName, @RequestBody Subject subject){
76+
Subject updatedSubject = subjectService.updateSubjectIfDifferent(subjectName, subject);
77+
return ResponseEntity.ok(updatedSubject);
78+
}
79+
80+
@DeleteMapping("/subjects/{subject}/versions/{version}")
81+
public ResponseEntity<Integer> deleteSchemaBySubjectAndVersion(@PathVariable("subject")String subject, @PathVariable("version") int version){
82+
Integer deletedId = subjectService.deleteSchemaBySubjectAndVersion(subject, version);
83+
return ResponseEntity.ok(deletedId);
84+
}
85+
}

0 commit comments

Comments
 (0)