OBPIH-5936 Add tests for REST APIs to avoid regression#4360
OBPIH-5936 Add tests for REST APIs to avoid regression#4360chetanmaharishi wants to merge 11 commits intofeature/upgrade-to-grails-3.3.10from
Conversation
awalkowiak
left a comment
There was a problem hiding this comment.
@chetanmaharishi please don't use ; at the end of lines and add spaces around operators, before opening {, and after the if keyword.
kchelstowski
left a comment
There was a problem hiding this comment.
and as @awalkowiak said, add indents between {, (, ), + and remove ; as it is not necessary to write them in Groovy.
src/test/groovy/unit/org/pih/warehouse/api/ProductApiTest.groovy
Outdated
Show resolved
Hide resolved
| import org.testng.annotations.BeforeClass; | ||
| import org.testng.annotations.Test; | ||
|
|
||
| import static io.restassured.RestAssured.*; |
There was a problem hiding this comment.
I would say importing everything from a package is not a good practice. We should import only what is needed. (Remove * and replace it with imports that are needed)
There was a problem hiding this comment.
ok i will make this changes
| class ProductApiTest{ | ||
|
|
||
| Map headers = [:] | ||
| String baseURI = "http://localhost:8080/openboxes"; |
There was a problem hiding this comment.
I don't think if hardcoding this url is a good idea, but since I am not a good test-writer, I don't know if we can get an access to context path at this point.
There was a problem hiding this comment.
Thanks @kchelstowski I was just going to mention this.
There was a problem hiding this comment.
Note that the Rest Assured tests are executed in an isolated execution context (either as a separate process or on a separate machine) so it's sort of like running postman tests. Therefore, the API instance will be up and running before these tests are run which means we have the ability to change this URL when we execute the tests. We could even move these tests to a separate github repository to make it more clear that the tests are somewhat independent from the API. However, I don't want to do that because I want them to evolve together so we can create new tests in feature branches that live alongside the code.
I'm currently reading Testing Web APIs to see if there are any best practices to keep in mind when running these Rest Assured tests https://www.manning.com/books/testing-web-apis but I suspect that we should at least provide a default API URL in configuration that can be overridden by a system environment variable or external configuration file.
src/test/groovy/unit/org/pih/warehouse/api/ProductApiTest.groovy
Outdated
Show resolved
Hide resolved
| Assert.assertEquals(statusCode, 200); | ||
| Assert.assertEquals(data, []); | ||
| } | ||
| } No newline at end of file |
There was a problem hiding this comment.
add an empty line at the end of the file
jmiranda
left a comment
There was a problem hiding this comment.
Added some preliminary comments for now.
| class ProductApiTest{ | ||
|
|
||
| Map headers = [:] | ||
| String baseURI = "http://localhost:8080/openboxes"; |
There was a problem hiding this comment.
Thanks @kchelstowski I was just going to mention this.
| class ProductApiTest{ | ||
|
|
||
| Map headers = [:] | ||
| String baseURI = "http://localhost:8080/openboxes"; |
There was a problem hiding this comment.
Note that the Rest Assured tests are executed in an isolated execution context (either as a separate process or on a separate machine) so it's sort of like running postman tests. Therefore, the API instance will be up and running before these tests are run which means we have the ability to change this URL when we execute the tests. We could even move these tests to a separate github repository to make it more clear that the tests are somewhat independent from the API. However, I don't want to do that because I want them to evolve together so we can create new tests in feature branches that live alongside the code.
I'm currently reading Testing Web APIs to see if there are any best practices to keep in mind when running these Rest Assured tests https://www.manning.com/books/testing-web-apis but I suspect that we should at least provide a default API URL in configuration that can be overridden by a system environment variable or external configuration file.
src/test/groovy/unit/org/pih/warehouse/api/ProductApiTest.groovy
Outdated
Show resolved
Hide resolved
3 more test case written 1. get demand of product 2. get demand summary of product 3. get summary of product
| void setup() { | ||
| if (!setupInitialized) { | ||
| RequestSpecification httpRequest = given().formParam("username", "admin").formParam("password", "password"); | ||
| Response response = httpRequest.post(baseURI + "/auth/handleLogin"); |
There was a problem hiding this comment.
Use the /api/login so you can pass the location ID there.
|
|
||
| void "read product list"() { | ||
| when: "/api/products end point called" | ||
| RequestSpecification httpRequest = given().header("Cookie", "${headers.Cookie}"); |
There was a problem hiding this comment.
Make sure you pass an Accept header to ensure you get a JSON response.
RequestSpecification httpRequest = given().header("Accept", "application/json")
.header("Cookie", "${headers.Cookie}")
Otherwise we get this when running tests
io.restassured.path.json.exception.JsonPathException: Failed to parse the JSON document
at io.restassured.path.json.JsonPath$ExceptionCatcher.invoke(JsonPath.java:930)
at io.restassured.path.json.JsonPath$4.doParseWith(JsonPath.java:895)
at io.restassured.path.json.JsonPath$JsonParser.parseWith(JsonPath.java:975)
at io.restassured.path.json.JsonPath.get(JsonPath.java:201)
at unit.org.pih.warehouse.api.ProductApiTest.get demand of product(ProductApiTest.groovy:76)
Caused by: groovy.json.JsonException: Lexing failed on line: 1, column: 1, while reading '<', no possible valid JSON value or punctuation could be recognized.
at groovy.json.JsonLexer.nextToken(JsonLexer.java:87)
at io.restassured.internal.path.json.ConfigurableJsonSlurper.parse(ConfigurableJsonSlurper.groovy:97)
at io.restassured.internal.path.json.ConfigurableJsonSlurper.parseText(ConfigurableJsonSlurper.groovy:83)
at io.restassured.path.json.JsonPath$4$1.method(JsonPath.java:893)
at io.restassured.path.json.JsonPath$ExceptionCatcher.invoke(JsonPath.java:928)
... 4 more
|
|
||
| void setup() { | ||
| if (!setupInitialized) { | ||
| RequestSpecification httpRequest = given().formParam("username", "admin").formParam("password", "password"); |
There was a problem hiding this comment.
Username and password should be @shared and should probably be configurable like the URL.
|
Can you guys review this PR again? And if possible, help us figure out what to do about the failing unit tests. Chetan thinks these REST API tests should be executed as unit tests because they don't require the application to be started i.e. we can tests another OpenBoxes instance somewhere else. Moving the tests to the integration test phase will create a lot more overhead, but I can't see another way of segmenting these REST API tests from the unit tests. |
alannadolny
left a comment
There was a problem hiding this comment.
I will get back to this later to try to check the failing unit tests. (Unless someone else comes up with the solution 😄)
| class ProductApiTest extends Specification { | ||
|
|
||
| @Shared | ||
| String baseURI = System.getProperty("server.host")?:"http://localhost" |
There was a problem hiding this comment.
Please add whitespace after and before the Elvis operators
| String baseURI = System.getProperty("server.host")?:"http://localhost" | |
| String baseURI = System.getProperty("server.host") ?: "http://localhost" |
There was a problem hiding this comment.
I actually hate the extra whitespace, but fine.
| String basePath = System.getProperty("server.path")?:"/openboxes" | ||
|
|
||
| @Shared | ||
| int port = System.getProperty("server.port")?Integer.valueOf(System.getProperty("server.port")):8080 |
There was a problem hiding this comment.
Add the whitespace after and before the ternary operators
| int port = System.getProperty("server.port")?Integer.valueOf(System.getProperty("server.port")):8080 | |
| int port = System.getProperty("server.port") ? Integer.valueOf(System.getProperty("server.port")) : 8080 |
There was a problem hiding this comment.
remember about the other two below also
| JSONObject jsonObject = new JSONObject(). | ||
| put("username", username). | ||
| put("password", password). | ||
| put("location", 1) |
There was a problem hiding this comment.
Those dots shouldn't be at the beginning of the next lines?
| JSONObject jsonObject = new JSONObject(). | |
| put("username", username). | |
| put("password", password). | |
| put("location", 1) | |
| JSONObject jsonObject = new JSONObject() | |
| .put("username", username) | |
| .put("password", password) | |
| .put("location", 1) |
There was a problem hiding this comment.
Hmm, I've never seen that syntax. Do you have a reference for that somewhere in Java / Groovy docs?
The rest assured docs provide a bunch of examples that use the syntax I used above.
https://github.com/rest-assured/rest-assured/wiki/Usage
And it looks like IntelliJ sometimes breaks when using the period at the beginning of the line (this is for a package name, but still makes me wary)
https://youtrack.jetbrains.com/issue/IDEA-242633/Groovy-code-formatting-incorrectly-wraps-qualified-new-expression-which-leads-to-compiler-error
There was a problem hiding this comment.
I've never seen that syntax 😅
I tried to search for breaking chain calls with the dot at the beginning, and I found a bunch of pages with this.
- https://unibeautify.com/docs/option-break-chained-methods
- Break on multiple chained calls prettier/prettier#1565
- https://www.baeldung.com/java-config-spring-security
- I can see it also in the spring security official docs https://spring.io/guides/gs/securing-web/
There was a problem hiding this comment.
Ok, so there's some evidence for leading over trailing, so we should have a discussion about this. I guess I've mostly stayed to one line when chaining but I've only ever remembered seeing a trailing dot whenever I've seen chaining. I honestly don't recall ever seeing a leading dot.
Interesting discussion in the Ruby with mixed feelings.
standardrb/standard#75
https://www.reddit.com/r/ruby/comments/8sc7gx/leading_vs_trailing_dots_why_to_try_trailing_dots/
Phew I'm not insane
There was a problem hiding this comment.
@kchelstowski @alannadolny Ok, after giving this a few days to marinate, I think I'm starting to warm up to the leading dot. I'm not 100% sold yet and I don't necessarily want to apply the line per method convention (like in the following example). But the leading dot does make it more readable.
|
|
||
| // Get a single product to pass to the next test | ||
| when: "we attempt to share a product with the next test" | ||
| ProductResponse product = response.then().body().extract().jsonPath().getObject("data[0]", ProductResponse) |
There was a problem hiding this comment.
Shouldn't we add new lines when we are calling multiple methods at once? I think we should try to avoid long lines like that (for example, on the frontend we have configured eslint with a maximum length of 100). It is harder to read for example in code review, when the line is too long.
There was a problem hiding this comment.
I don't mind breaking it up if there's a logical break (maybe after body and then again after jsonPath) but I also don't think it's necessary unless we go over the 100-120 character limit of the line. Which I'm guessing we are.
But I don't have a strong feeling. In fact, I don't mind the one line per method (example below). I just don't want these tests to be 1000s of lines unnecessarily.
https://www.baeldung.com/rest-assured-response
| // then: "data should not be null" | ||
| // Assert.assertNotNull(data) | ||
| // and: "product should not be null" | ||
| // Assert.assertNotNull(data.product, "Response should have product object") | ||
| // and: "product ID should not be null" | ||
| // Assert.assertEquals(data.product.id, productId, "Product id should match with requested id:${productId}") | ||
| // and: "demand should not be null" | ||
| // Assert.assertNotNull(data.demand, "Demand Object should not be null") |
There was a problem hiding this comment.
Should it be commented?
There was a problem hiding this comment.
Chetan is supposed to turn those into rest assured asserts.
| class ProductResponse { | ||
| String id |
There was a problem hiding this comment.
Add new line
| class ProductResponse { | |
| String id | |
| class ProductResponse { | |
| String id |
There was a problem hiding this comment.
I think there is no need to add an empty line here.
Lately I've been trying to do a research whether the properties should be even separated by an empty line if they don't contain any annotations above, but can't find any good resource about that.
I meant:
class ProductResponse {
String id
String productCode
String name
}vs
class ProductResponse {
String id
String productCode
String name
}There was a problem hiding this comment.
I am a fan of adding a new line before each property (the first version), but if u think there is no need, it can stay
There was a problem hiding this comment.
Can we spend some more time on the substance? This class is going to be removed once we deal with the inconsistences within our JSON responses. We just needed a temporary solution to write these reference implementation tests.
| import static io.restassured.RestAssured.* | ||
| import io.restassured.response.Response | ||
| import static org.hamcrest.MatcherAssert.assertThat | ||
| import static org.hamcrest.Matchers.is | ||
| import static org.hamcrest.Matchers.notNullValue | ||
| import static org.hamcrest.Matchers.lessThanOrEqualTo |
There was a problem hiding this comment.
I think import static should be placed below the "normal" imports. This is also the default behavior when we are adding new imports through Intellij. And also we shouldn't import things using *, because there can be a lot of unnecessary imported things (I think this change request was mentioned before, but I am not sure)
| import static io.restassured.RestAssured.* | |
| import io.restassured.response.Response | |
| import static org.hamcrest.MatcherAssert.assertThat | |
| import static org.hamcrest.Matchers.is | |
| import static org.hamcrest.Matchers.notNullValue | |
| import static org.hamcrest.Matchers.lessThanOrEqualTo | |
| import io.restassured.response.Response | |
| import static io.restassured.RestAssured.* | |
| import static org.hamcrest.MatcherAssert.assertThat | |
| import static org.hamcrest.Matchers.is | |
| import static org.hamcrest.Matchers.notNullValue | |
| import static org.hamcrest.Matchers.lessThanOrEqualTo |
There was a problem hiding this comment.
Check style seems to suggest the opposite.
https://checkstyle.sourceforge.io/checks/imports/importorder.html#:~:text=group%20of%20static%20imports%20is,and%20aren't%20separated%20internally
So let's find a definitive source or just ignore it.
There was a problem hiding this comment.
@alannadolny I honestly don't know what's best. When I saw the recommendation for putting static imports up top I was surprised. Just adding this to "we don't yet know the definitive answer" so we'll need to research it. I just don't want to spend too much time researching some of these syntax issues when we could be spending that time actually making tests work.
| import org.grails.web.json.JSONObject | ||
| import spock.lang.Shared | ||
| import spock.lang.Specification | ||
| import static io.restassured.RestAssured.* |
There was a problem hiding this comment.
the * imports have not been fixed (I've mentioned that in my initial review)
There was a problem hiding this comment.
I wish we had something like pmd checkstyle (I don't know if it is available in Groovy/Grails), but it would catch those things.
I also think that the imports should be grouped differently - the static imports should be at the beginning and be separated with one empty line, but since we don't have any strict rules regarding that, you can skip that grouping - just wanted to mention that.
| String basePath = System.getProperty("server.path")?:"/openboxes" | ||
|
|
||
| @Shared | ||
| int port = System.getProperty("server.port")?Integer.valueOf(System.getProperty("server.port")):8080 |
There was a problem hiding this comment.
remember about the other two below also
| Response response = given().log().all(). | ||
| contentType(ContentType.JSON). | ||
| body(jsonObject.toString()). | ||
| post("/api/login") |
There was a problem hiding this comment.
here dots also should be at the beginning of the line
There was a problem hiding this comment.
Again, where have you seen this syntax? Is that a javascript thing?
| post("/api/login") | ||
|
|
||
| // Make sure we were successfully authenticated | ||
| response.then().assertThat().statusCode(200) |
There was a problem hiding this comment.
it would be good to do it by one line -> one call:
response
.then()
.assertThat()
.statusCode(200)| setBasePath(basePath). | ||
| setPort(port). | ||
| addCookie(cookie). | ||
| setAccept(ContentType.JSON).build() |
|
|
||
| when: "we get products from the API" | ||
| // FIXME figure out better way to set up baseURI using RestAssured API | ||
| Response response = request.when().get("/api/products?max=10") |
| Response response = request.when().get("/api/products?max=10") | ||
|
|
||
| // FIXME Should be replaced by Product domain class once we | ||
| List<ProductResponse> products = response.then().log().all().extract().body().jsonPath(). |
| response.then().assertThat().statusCode(200) | ||
| // Assert.that(is(notNullValue(data))) | ||
| // and: | ||
| // Assert.that(is(data.size(), equalTo(0))) |
There was a problem hiding this comment.
should it removed/uncommented?
There was a problem hiding this comment.
@chetanmaharishi can you turn these into valid rest-assured assert thingees like the first example I did.
| // Assert.assertEquals(data.location.id, locationId, "Location id should be ${locationId}") | ||
| // and: | ||
| // Integer quantityOnHand = data.quantityOnHand ?: 0 | ||
| // Assert.assertEquals(quantityOnHand, 0, "QuantityOnHand should be 0") |
There was a problem hiding this comment.
should it be removed/uncommented?
| class ProductResponse { | ||
| String id |
There was a problem hiding this comment.
I think there is no need to add an empty line here.
Lately I've been trying to do a research whether the properties should be even separated by an empty line if they don't contain any annotations above, but can't find any good resource about that.
I meant:
class ProductResponse {
String id
String productCode
String name
}vs
class ProductResponse {
String id
String productCode
String name
}|
We are temporarily archiving / ice-boxing this until we make a decision on where these tests go |


First commit of Rest assured testing for ProductApi list