Showing posts with label swagger. Show all posts
Showing posts with label swagger. Show all posts

Tuesday, November 6, 2018

Building Enterprise Java applications, the Spring way

I think it is fair to say that Java EE has gained pretty bad reputation among Java developers. Despite the fact that it has certainly improved on all fronts over the years, even changed home to Eclipse Foundation to become Jakarta EE, its bitter taste is still quite strong. On the other side we have Spring Framework (or to reflect the reality better, a full-fledged Spring Platform): brilliant, lightweight, fast, innovative and hyper-productive Java EE replacement. So why to bother with Java EE?

We are going to answer this question by showing how easy it is to build modern Java applications using most of Java EE specs. And the key ingredient to succeed here is Eclipse Microprofile: enterprise Java in the age of microservices.

The application we are going to build is RESTful web API to manage people, as simple as that. The standard way to build RESTful web services in Java is by using JAX-RS 2.1 (JSR-370). Consequently, CDI 2.0 (JSR-365) is going to take care of dependency injection whereas JPA 2.0 (JSR-317) is going to cover the data access layer. And certainly, Bean Validation 2.0 (JSR-380) is helping us to deal with input verification.

The only non-Java EE specification we would be relying on is OpenAPI v3.0 which helps to provide the usable description of our RESTful web APIs. With that, let us get started with the PersonEntity domain model (omitting getters and setters as not very relevant details):

@Entity
@Table(name = "people")
public class PersonEntity {
    @Id @Column(length = 256) 
    private String email;

    @Column(nullable = false, length = 256, name = "first_name")
    private String firstName;

    @Column(nullable = false, length = 256, name = "last_name")
    private String lastName;

    @Version
    private Long version;
}

It just has the absolute minimum set of properties. The JPA repository is pretty straightforward and implements typical set of CRUD methods.

@ApplicationScoped
@EntityManagerConfig(qualifier = PeopleDb.class)
public class PeopleJpaRepository implements PeopleRepository {
    @Inject @PeopleDb private EntityManager em;

    @Override
    @Transactional(readOnly = true)
    public Optional<PersonEntity> findByEmail(String email) {
        final CriteriaBuilder cb = em.getCriteriaBuilder();
    
        final CriteriaQuery<PersonEntity> query = cb.createQuery(PersonEntity.class);
        final Root<PersonEntity> root = query.from(PersonEntity.class);
        query.where(cb.equal(root.get(PersonEntity_.email), email));
        
        try {
            final PersonEntity entity = em.createQuery(query).getSingleResult();
            return Optional.of(entity);
        } catch (final NoResultException ex) {
            return Optional.empty();
        }
    }

    @Override
    @Transactional
    public PersonEntity saveOrUpdate(String email, String firstName, String lastName) {
        final PersonEntity entity = new PersonEntity(email, firstName, lastName);
        em.persist(entity);
        return entity;
    }

    @Override
    @Transactional(readOnly = true)
    public Collection<PersonEntity> findAll() {
        final CriteriaBuilder cb = em.getCriteriaBuilder();
        final CriteriaQuery<PersonEntity> query = cb.createQuery(PersonEntity.class);
        query.from(PersonEntity.class);
        return em.createQuery(query).getResultList();
    }

    @Override
    @Transactional
    public Optional<PersonEntity> deleteByEmail(String email) {
        return findByEmail(email)
            .map(entity -> {
                em.remove(entity);
                return entity;
            });
    }
}

The transaction management (namely, the @Transactional annotation) needs some explanation. In the typical Java EE application, the container runtime is responsible for managing the transactions. Since we don't want to onboard the application container but stay lean, we could have used EntityManager to start / commit / rollback transactions. It would certainly work out but pollute the code with the boilerplate. Arguably, the better option is to use Apache DeltaSpike CDI extensions for declarative transaction management (this is where @Transactional and @EntityManagerConfig annotations are coming from). The snippet below illustrates how it is being integrated.

@ApplicationScoped
public class PersistenceConfig {
    @PersistenceUnit(unitName = "peopledb")
    private EntityManagerFactory entityManagerFactory;

    @Produces @PeopleDb @TransactionScoped
    public EntityManager create() {
        return this.entityManagerFactory.createEntityManager();
    }

    public void dispose(@Disposes @PeopleDb EntityManager entityManager) {
        if (entityManager.isOpen()) {
            entityManager.close();
        }
    }
}

Awesome, the hardest part is already behind! The Person data transfer object and the service layer are coming next.

public class Person {
    @NotNull private String email;
    @NotNull private String firstName;
    @NotNull private String lastName;
}

Honestly, for the sake of keeping the example application as small as possible we could skip the service layer altogether and go to the repository directly. But this is, in general, not a very good practice so let us introduce PeopleServiceImpl anyway.

@ApplicationScoped
public class PeopleServiceImpl implements PeopleService {
    @Inject private PeopleRepository repository;

    @Override
    public Optional<Person> findByEmail(String email) {
        return repository
            .findByEmail(email)
            .map(this::toPerson);
    }

    @Override
    public Person add(Person person) {
        return toPerson(repository.saveOrUpdate(person.getEmail(), person.getFirstName(), person.getLastName()));
    }

    @Override
    public Collection<Person> getAll() {
        return repository
            .findAll()
            .stream()
            .map(this::toPerson)
            .collect(Collectors.toList());
    }

    @Override
    public Optional<Person> remove(String email) {
        return repository
            .deleteByEmail(email)
            .map(this::toPerson);
    }
    
    private Person toPerson(PersonEntity entity) {
        return new Person(entity.getEmail(), entity.getFirstName(), entity.getLastName());
    }
}

The only part left is the definition of the JAX-RS application and resources.

@Dependent
@ApplicationPath("api")
@OpenAPIDefinition(
    info = @Info(
        title = "People Management Web APIs", 
        version = "1.0.0", 
        license = @License(
            name = "Apache License", 
            url = "https://www.apache.org/licenses/LICENSE-2.0"
        )
    )
)
public class PeopleApplication extends Application {
}

Not much to say, as simple as it could possibly be. The JAX-RS resource implementation is a bit more interesting though (the OpenAPI annotations are taking most of the place).

@ApplicationScoped
@Path( "/people" ) 
@Tag(name = "people")
public class PeopleResource {
    @Inject private PeopleService service;
    
    @Produces(MediaType.APPLICATION_JSON)
    @GET
    @Operation(
        description = "List all people", 
        responses = {
            @ApiResponse(
                content = @Content(array = @ArraySchema(schema = @Schema(implementation = Person.class))),
                responseCode = "200"
            )
        }
    )
    public Collection<Person> getPeople() {
        return service.getAll();
    }

    @Produces(MediaType.APPLICATION_JSON)
    @Path("/{email}")
    @GET
    @Operation(
        description = "Find person by e-mail", 
        responses = {
            @ApiResponse(
                content = @Content(schema = @Schema(implementation = Person.class)), 
                responseCode = "200"
            ),
            @ApiResponse(
                responseCode = "404", 
                description = "Person with such e-mail doesn't exists"
            )
        }
    )
    public Person findPerson(@Parameter(description = "E-Mail address to lookup for", required = true) @PathParam("email") final String email) {
        return service
            .findByEmail(email)
            .orElseThrow(() -> new NotFoundException("Person with such e-mail doesn't exists"));
    }

    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    @POST
    @Operation(
        description = "Create new person",
        requestBody = @RequestBody(
            content = @Content(schema = @Schema(implementation = Person.class)),
        ), 
        responses = {
            @ApiResponse(
                 content = @Content(schema = @Schema(implementation = Person.class)),
                 headers = @Header(name = "Location"),
                 responseCode = "201"
            ),
            @ApiResponse(
                responseCode = "409", 
                description = "Person with such e-mail already exists"
            )
        }
    )
    public Response addPerson(@Context final UriInfo uriInfo,
            @Parameter(description = "Person", required = true) @Valid Person payload) {

        final Person person = service.add(payload);
        return Response
             .created(uriInfo.getRequestUriBuilder().path(person.getEmail()).build())
             .entity(person)
             .build();
    }
    
    @Path("/{email}")
    @DELETE
    @Operation(
        description = "Delete existing person",
        responses = {
            @ApiResponse(
                responseCode = "204",
                description = "Person has been deleted"
            ),
            @ApiResponse(
                responseCode = "404", 
                description = "Person with such e-mail doesn't exists"
            )
        }
    )
    public Response deletePerson(@Parameter(description = "E-Mail address to lookup for", required = true ) @PathParam("email") final String email) {
        return service
            .remove(email)
            .map(r -> Response.noContent().build())
            .orElseThrow(() -> new NotFoundException("Person with such e-mail doesn't exists"));
    }
}

And with that, we are done! But how could we assemble and wire all these pieces together? Here is the time for Microprofile to enter the stage. There are many implementations to chose from, the one we are going to use in this post is Project Hammock. The only thing we have to do is to specify the CDI 2.0, JAX-RS 2.1 and JPA 2.0 implementations we would like to use, which translates to Weld, Apache CXF, and OpenJPA respectively (expressed through the Project Hammock dependencies). Let us take a look on the Apache Maven pom.xml file.

<properties>
    <deltaspike.version>1.8.1</deltaspike.version>
    <hammock.version>2.1</hammock.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.apache.deltaspike.modules</groupId>
        <artifactId>deltaspike-jpa-module-api</artifactId>
        <version>${deltaspike.version}</version>
        <scope>compile</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.deltaspike.modules</groupId>
        <artifactId>deltaspike-jpa-module-impl</artifactId>
        <version>${deltaspike.version}</version>
        <scope>runtime</scope>
    </dependency>

    <dependency>
        <groupId>ws.ament.hammock</groupId>
        <artifactId>dist-microprofile</artifactId>
        <version>${hammock.version}</version>
    </dependency>

    <dependency>
        <groupId>ws.ament.hammock</groupId>
        <artifactId>jpa-openjpa</artifactId>
        <version>${hammock.version}</version>
    </dependency>

    <dependency>
        <groupId>ws.ament.hammock</groupId>
        <artifactId>util-beanvalidation</artifactId>
        <version>${hammock.version}</version>
    </dependency>

    <dependency>
        <groupId>ws.ament.hammock</groupId>
        <artifactId>util-flyway</artifactId>
        <version>${hammock.version}</version>
    </dependency>

    <dependency>
        <groupId>ws.ament.hammock</groupId>
        <artifactId>swagger</artifactId>
        <version>${hammock.version}</version>
    </dependency>
</dependencies>

Without further ado, let us build and run the application right away (if you are curious what relational datastore the application is using, it is H2 with the database configured in-memory).

> mvn clean package
> java -jar target/eclipse-microprofile-hammock-0.0.1-SNAPSHOT-capsule.jar 

The best way to ensure that our people management RESTful web APIs are fully functional is to send a couple of requests to it:

>  curl -X POST http://localhost:10900/api/people -H "Content-Type: application\json" \
     -d '{"email": "[email protected]", "firstName": "John", "lastName": "Smith"}'

HTTP/1.1 201 Created
Location: http://localhost:10900/api/people/[email protected]
Content-Type: application/json

{
    "firstName":"John","
    "lastName":"Smith",
    "email":"[email protected]"
}

What about making sure the Bean Validation is working fine? To trigger that, let us send the partially prepared request.

>  curl  --X POST http://localhost:10900/api/people -H "Content-Type: application\json" \
     -d '{"firstName": "John", "lastName": "Smith"}'

HTTP/1.1 400 Bad Request
Content-Length: 0

The OpenAPI specification and pre-bundled Swagger UI distribution are also available at http://localhost:10900/index.html?url=http://localhost:10900/api/openapi.json.

So far so good but fairly speaking we have not talked about testing our application at all. How hard it would be to come up with the integration test for, let say, the scenario of adding a person? It turns out that the frameworks around testing Java EE applications have improved a lot. In particular, it is exceptionally easy to accomplish with Arquillian test framework (along with beloved JUnit and REST Assured). One real example is worth thousand words.

@RunWith(Arquillian.class)
@EnableRandomWebServerPort
public class PeopleApiTest {
    @ArquillianResource private URI uri;
    
    @Deployment
    public static JavaArchive createArchive() {
        return ShrinkWrap
            .create(JavaArchive.class)
            .addClasses(PeopleResource.class, PeopleApplication.class)
            .addClasses(PeopleServiceImpl.class, PeopleJpaRepository.class, PersistenceConfig.class)
            .addPackages(true, "org.apache.deltaspike");
    }
            
    @Test
    public void shouldAddNewPerson() throws Exception {
        final Person person = new Person("[email protected]", "John", "Smith");
        
        given()
            .contentType(ContentType.JSON)
            .body(person)
            .post(uri + "/api/people")
            .then()
            .assertThat()
            .statusCode(201)
            .body("email", equalTo("[email protected]"))
            .body("firstName", equalTo("John"))
            .body("lastName", equalTo("Smith"));
    }
}

Amazing, isn't it? It is actually a lot of fun to develop modern Java EE applications, someone may say, the Spring way! And in fact, the parallels with Spring are not coincidental since it was inspiring, is inspiring and undoubtedly is going to continue inspire a lot of innovations in the Java EE ecosystem.

How the future is looking like? I think, by all means bright, both for Jakarta EE and Eclipse Microprofile. The latter just approached the version 2.0 with tons of new specifications included, oriented to address the needs of the microservice architectures. It is awesome to witness these transformations happening.

The complete source of the project is available on Github.

Sunday, June 17, 2018

In the shoes of the consumer: do you really need to provide the client libraries for your APIs?

The beauty of the RESTful web services and APIs is that any consumer which speaks HTTP protocol will be able to understand and use it. Nonetheless, the same dilemma pops up over and over again: should you accompany your web APis with the client libraries or not? If yes, what languages or/and frameworks should you support?

Quite often this is not really an easy question to answer. So let us take a step back and think about the overall idea for a moment: what are the values which client libraries may bring to the consumers?

Someone may say to lower the barrier for adoption. Indeed, specifically in the case of strongly typed languages, exploring the API contracts from your favorite IDE (syntax highlighting and auto-completion please!) is quite handy. But by and large, the RESTful web APIs are simple enough to start with and good documentation would be certainly more valuable here.

Others may say it is good to shield the consumers from dealing with multiple API versions or rough edges. Also kind of make sense but I would argue it just hides the flaws with the way the web APIs in question are designed and evolve over time.

All in all, no matter how many clients you decide to bundle, the APIs are still going to be accessible by any generic HTTP consumer (curl, HttpClient, RestTemplate, you name it). Giving a choice is great but the price to pay for maintenance could be really high. Could we do it better? And as you may guess already, we certainly have quite a few options hence this post.

The key ingredient of the success here is to maintain an accurate specification of your RESTful web APIs, using OpenAPI v3.0 or even its predecessor, Swagger/OpenAPI 2.0 (or RAML, API Blueprint, does not really matter much). In case of OpenAPI/Swagger, the tooling is the king: one could use Swagger Codegen, a template-driven engine, to generate API clients (and even server stubs) in many different languages, and this is what we are going to talk about in this post.

To simplify the things, we are going to implement the consumer of the people management web API which we have built in the previous post. To begin with, we need to get its OpenAPI v3.0 specification in the YAML (or JSON) format.

java -jar server-openapi/target/server-openapi-0.0.1-SNAPSHOT.jar

And then:

wget http://localhost:8080/api/openapi.yaml

Awesome, the half of the job is done, literally. Now, let us allow Swagger Codegen to take a lead. In order to not complicate the matter, let's assume that consumer is also Java application, so we could understand the mechanics without any difficulties (but Java is only one of the options, the list of supported languages and frameworks is astonishing).

Along this post we are going to use OpenFeign, one of the most advanced Java HTTP client binders. Not only it is exceptionally simple to use, it offers quite a few integrations we are going to benefit from soon.

<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-core</artifactId>
    <version>9.7.0</version>
</dependency>

<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-jackson</artifactId>
    <version>9.7.0</version>
</dependency>

The Swagger Codegen could be run as stand-alone application from command-line, or Apache Maven plugin (the latter is what we are going to use).

<plugin>
    <groupId>io.swagger</groupId>
    <artifactId>swagger-codegen-maven-plugin</artifactId>
    <version>3.0.0-rc1</version>
    <executions>
        <execution>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <inputSpec>/contract/openapi.yaml</inputSpec>
                <apiPackage>com.example.people.api</apiPackage>
                <language>java</language>
                <library>feign</library>
                <modelPackage>com.example.people.model</modelPackage>
                <generateApiDocumentation>false</generateApiDocumentation>
                <generateSupportingFiles>false</generateSupportingFiles>
                <generateApiTests>false</generateApiTests>
                <generateApiDocs>false</generateApiDocs>
                <addCompileSourceRoot>true</addCompileSourceRoot>
                <configOptions>
                    <sourceFolder>/</sourceFolder>
                    <java8>true</java8>
                    <dateLibrary>java8</dateLibrary>
                    <useTags>true</useTags>
                </configOptions>
            </configuration>
        </execution>
    </executions>
</plugin>

If some of the options are not very clear, the Swagger Codegen has pretty good documentation to look for clarifications. The important ones to pay attention to is language and library, which are set to java and feign respectively. The one thing to note though, the support of the OpenAPI v3.0 specification is mostly complete but you may encounter some issues nonetheless (as you noticed, the version is 3.0.0-rc1).

What you will get when your build finishes is the plain old Java interface, PeopleApi, annotated with OpenFeign annotations, which is direct projection of the people management web API specification (which comes from /contract/openapi.yaml). Please notice that all model classes are generated as well.

@javax.annotation.Generated(
    value = "io.swagger.codegen.languages.java.JavaClientCodegen",
    date = "2018-06-17T14:04:23.031Z[Etc/UTC]"
)
public interface PeopleApi extends ApiClient.Api {
    @RequestLine("POST /api/people")
    @Headers({"Content-Type: application/json", "Accept: application/json"})
    Person addPerson(Person body);

    @RequestLine("DELETE /api/people/{email}")
    @Headers({"Content-Type: application/json"})
    void deletePerson(@Param("email") String email);

    @RequestLine("GET /api/people/{email}")
    @Headers({"Accept: application/json"})
    Person findPerson(@Param("email") String email);

    @RequestLine("GET /api/people")
    @Headers({"Accept: application/json"})
    List<Person> getPeople();
}

Let us compare it with Swagger UI interpretation of the same specification, available at http://localhost:8080/api/api-docs?url=/api/openapi.json:

It looks right at the first glance but we have better ensure things work out as expected. Once we have OpenFeign-annotated interface, it could be made functional (in this case, implemented through proxies) using the family of Feign builders, for example:

final PeopleApi api = Feign
    .builder()
    .client(new OkHttpClient())
    .encoder(new JacksonEncoder())
    .decoder(new JacksonDecoder())
    .logLevel(Logger.Level.HEADERS)
    .options(new Request.Options(1000, 2000))
    .target(PeopleApi.class, "http://localhost:8080/");

Great, fluent builder style rocks. Assuming our people management web APIs server is up and running (by default, it is going to be available at http://localhost:8080/):

java -jar server-openapi/target/server-openapi-0.0.1-SNAPSHOT.jar

We could communicate with it by calling freshly built PeopleApi instance methods, as in the code snippet below.:

final Person person = api.addPerson(
        new Person()
            .email("[email protected]")
            .firstName("John")
            .lastName("Smith"));

It is really cool, if we rewind it back a bit, we actually did nothing. Everything is given to us for free with only web API specification available! But let us not stop here and remind ourselves that using Java interfaces will not eliminate the reality that we are dealing with remote systems. And things are going to fail here, sooner or later, no doubts.

Not so long ago we have learned about circuit breakers and how useful they are when properly applied in the context of distributed systems. It would be really awesome to somehow introduce this feature into our OpenFeign-based client. Please welcome another member of the family, HystrixFeign builder, the seamless integration with Hytrix library:

final PeopleApi api = HystrixFeign
    .builder()
    .client(new OkHttpClient())
    .encoder(new JacksonEncoder())
    .decoder(new JacksonDecoder())
    .logLevel(Logger.Level.HEADERS)
    .options(new Request.Options(1000, 2000))
    .target(PeopleApi.class, "http://localhost:8080/");

The only thing we need to do is just to add these two dependencies (strictly speaking hystrix-core is not really needed if you do not mind to stay on older version) to consumer's pom.xml file.

<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-hystrix</artifactId>
    <version>9.7.0</version>
</dependency>

<dependency>
    <groupId>com.netflix.hystrix</groupId>
    <artifactId>hystrix-core</artifactId>
    <version>1.5.12</version>
</dependency>

Arguably, this is one of the best examples of how easy and straightforward integration could be. But even that is not the end of the story. Observability in the distributed systems is as important as never and as we have learned a while ago, distributed tracing is tremendously useful in helping us out here. And again, OpenFeign has support for it right out of the box, let us take a look.

OpenFeign fully integrates with OpenTracing-compatible tracer. The Jaeger tracer is one of those, which among other things has really nice web UI front-end to explore traces and dependencies. Let us run it first, luckily it is fully Docker-ized.

docker run -d -e \
    COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
    -p 5775:5775/udp \
    -p 6831:6831/udp \
    -p 6832:6832/udp \
    -p 5778:5778 \
    -p 16686:16686 \
    -p 14268:14268 \
    -p 9411:9411 \
    jaegertracing/all-in-one:latest

A couple of additional dependencies have to be introduced in order to OpenFeign client to be aware of the OpenTracing capabilities.

<dependency>
    <groupId>io.github.openfeign.opentracing</groupId>
    <artifactId>feign-opentracing</artifactId>
    <version>0.1.0</version>
</dependency>

<dependency>
    <groupId>io.jaegertracing</groupId>
    <artifactId>jaeger-core</artifactId>
    <version>0.29.0</version>
</dependency>

From the Feign builder side, the only change (besides the introduction of the tracer instance) is to wrap up the client into TracingClient, like the snippet below demonstrates:

final Tracer tracer = new Configuration("consumer-openapi")
    .withSampler(
        new SamplerConfiguration()
            .withType(ConstSampler.TYPE)
            .withParam(new Float(1.0f)))
    .withReporter(
        new ReporterConfiguration()
            .withSender(
                new SenderConfiguration()
                    .withEndpoint("http://localhost:14268/api/traces")))
    .getTracer();
            
final PeopleApi api = Feign
    .builder()
    .client(new TracingClient(new OkHttpClient(), tracer))
    .encoder(new JacksonEncoder())
    .decoder(new JacksonDecoder())
    .logLevel(Logger.Level.HEADERS)
    .options(new Request.Options(1000, 2000))
    .target(PeopleApi.class, "http://localhost:8080/");

On the server-side we also need to integrate with OpenTracing as well. The Apache CXF has first-class support for it, bundled into cxf-integration-tracing-opentracing module. Let us include it as the dependency, this time to server's pom.xml.

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-integration-tracing-opentracing</artifactId>
    <version>3.2.4</version>
</dependency>

Depending on the way your configure your applications, there should be an instance of the tracer available which should be passed later on to the OpenTracingFeature, for example.

// Create tracer
final Tracer tracer = new Configuration(
        "server-openapi", 
        new SamplerConfiguration(ConstSampler.TYPE, 1),
        new ReporterConfiguration(new HttpSender("http://localhost:14268/api/traces"))
    ).getTracer();

// Include OpenTracingFeature feature
final JAXRSServerFactoryBean factory = new JAXRSServerFactoryBean();
factory.setProvider(new OpenTracingFeature(tracer()));
...
factory.create()

From now on, the invocation of the any people management API endpoint through generated OpenFeign client will be fully traceable in the Jaeger web UI, available at http://localhost:16686/search (assuming your Docker host is localhost).

Our scenario is pretty simple but imagine the real applications where dozen of external service calls could happen while the single request travels through the system. Without distributed tracing in place, every issue has a chance to turn into a mystery.

As a side note, if you look closer to the trace from the picture, you may notice that server and consumer use different versions of the Jaeger API. This is not a mistake since the latest released version of Apache CXF is using older OpenTracing API version (and as such, older Jaeger client API) but it does not prevent things to work as expected.

With that, it is time to wrap up. Hopefully, the benefits of contract-based (or even better, contract-first) development in the world of RESTful web services and APIs become more and more apparent: generation of the smart clients, consumer-driven contract test, discoverability and rich documentation are just a few to mention. Please make use of it!

The complete project sources are available on Github.

Monday, April 30, 2018

Moving With The Times: Towards OpenAPI v3.0.0 adoption in JAX-RS APIs

It is terrifying to see how fast time passes! The OpenAPI specification 3.0.0, a major revamp of so-get-used-to Swagger specification, has been released mostly one year ago but it took awhile for tooling to catch up. However, with the recent official release of the Swagger Core 2.0.0 things are going to accelerate for sure.

To prove the point, Apache CXF, the well-known JAX-RS 2.1 implementation, is one of the first adopters of the OpenAPI 3.0.0 and in today's post we are going to take a look how easy your JAX-RS 2.1 APIs could benefit from it.

As always, to keep things simple we are going to design a people management web APIs with just a handful set of resources to support it, nothing too exciting here.

POST   /api/people
GET    /api/people/{email}
GET    /api/people
DELETE /api/people/{email}

Our model would consist of a single Person class.

public class Person {
    private String email;
    private String firstName;
    private String lastName;
}

To add a bit of magic, we would be using Spring Boot to get us up and running as fast as possible. With that, let us start to fill in the dependencies (assuming we are using Apache Maven for build management).

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-spring-boot-starter-jaxrs</artifactId>
    <version>3.2.4</version>
</dependency>

In the recent 3.2.x releases Apache CXF introduces a new module cxf-rt-rs-service-description-openapi-v3 dedicated to OpenAPI 3.0.0, based on Swagger Core 2.0.0.

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-rt-rs-service-description-openapi-v3</artifactId>
    <version>3.2.4</version>
</dependency>

<dependency>
    <groupId>org.webjars</groupId>
    <artifactId>swagger-ui</artifactId>
    <version>3.13.6</version>
</dependency>

The presence of the Swagger UI is not strictly necessary, but this is exceptionally useful and beautiful tool to explore your APIs (and if it is available on classpath, Apache CXF seamlessly integrates it into your applications as we are going to see in a bit).

The prerequisites are in place, let us do some coding! Before we begin, it is worth to note that the Swagger Core 2.0.0 has many ways to populate the OpenAPI 3.0.0 definitions for your services, including property files, annotations or programmatically. Along this post we are going to use annotations only.

@OpenAPIDefinition(
    info = @Info(
        title = "People Management API",
        version = "0.0.1-SNAPSHOT",
        license = @License(
            name = "Apache 2.0 License",
            url = "http://www.apache.org/licenses/LICENSE-2.0.html"
        )
    )
)
@ApplicationPath("api")
public class JaxRsApiApplication extends Application {
}

Looks pretty simple, the @OpenAPIDefinition sets the top-level definition for all our web APIs. Moving on to the PeopleRestService, we just add the @Tag annotation to, well, tag our API.

@Path( "/people" ) 
@Tag(name = "people")
public class PeopleRestService {
    // ...
}

Awesome, nothing complicated so far. The meaty part starts with web API operation definitions, so let us take a look on the first example, the operation to fetch everyone.

@Produces(MediaType.APPLICATION_JSON)
@GET
@Operation(
    description = "List all people", 
    responses = {
        @ApiResponse(
            content = @Content(
                array = @ArraySchema(schema = @Schema(implementation = Person.class))
            ),
            responseCode = "200"
        )
    }
)
public Collection<Person> getPeople() {
    // ...
}

Quite a few annotations but by and large, looks pretty clean and straightforward. Let us take a look on another one, the endpoint to find the person by its e-mail address.

@Produces(MediaType.APPLICATION_JSON)
@Path("/{email}")
@GET
@Operation(
    description = "Find person by e-mail", 
    responses = {
        @ApiResponse(
            content = @Content(schema = @Schema(implementation = Person.class)), 
            responseCode = "200"
        ),
        @ApiResponse(
            responseCode = "404", 
            description = "Person with such e-mail doesn't exists"
        )
    }
)
public Person findPerson(
        @Parameter(description = "E-Mail address to lookup for", required = true) 
        @PathParam("email") final String email) {
    // ...
}

In the same vein, the operation to remove the person by e-mail looks mostly identical.

@Path("/{email}")
@DELETE
@Operation(
    description = "Delete existing person",
    responses = {
        @ApiResponse(
            responseCode = "204",
            description = "Person has been deleted"
        ),
        @ApiResponse(
            responseCode = "404", 
            description = "Person with such e-mail doesn't exists"
        )
     }
)
public Response deletePerson(
        @Parameter(description = "E-Mail address to lookup for", required = true ) 
        @PathParam("email") final String email) {
    // ...
}

Great, let us wrap up by looking into last, arguably the most interesting endpoint which adds a new person (the choice to use the @FormParam is purely to illustrate the different flavor of the APIs).

@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
@Produces(MediaType.APPLICATION_JSON)
@POST
@Operation(
    description = "Create new person",
    responses = {
        @ApiResponse(
            content = @Content(
                schema = @Schema(implementation = Person.class), 
                mediaType = MediaType.APPLICATION_JSON
            ),
            headers = @Header(name = "Location"),
            responseCode = "201"
        ),
        @ApiResponse(
            responseCode = "409", 
            description = "Person with such e-mail already exists"
        )
    }
)
public Response addPerson(@Context final UriInfo uriInfo,
        @Parameter(description = "E-Mail", required = true) 
        @FormParam("email") final String email, 
        @Parameter(description = "First Name", required = true) 
        @FormParam("firstName") final String firstName, 
        @Parameter(description = "Last Name", required = true) 
        @FormParam("lastName") final String lastName) {
    // ...
}

If you have an experience with documenting web APIs using older Swagger specifications, you might find the approach quite familiar but more verbose (or better to say, formalized). It is the result of the tremendous work which specification leads and community has done to make it as complete and extensible as possible.

The APIs are defined and documented, it is time to try them out! The missing piece though is the Spring configuration where we would initialize and expose our JAX-RS web services.

@Configuration
@EnableAutoConfiguration
@ComponentScan(basePackageClasses = PeopleRestService.class)
public class AppConfig {
    @Autowired private PeopleRestService peopleRestService;
 
    @Bean(destroyMethod = "destroy")
    public Server jaxRsServer(Bus bus) {
        final JAXRSServerFactoryBean factory = new JAXRSServerFactoryBean();

        factory.setApplication(new JaxRsApiApplication());
        factory.setServiceBean(peopleRestService);
        factory.setProvider(new JacksonJsonProvider());
        factory.setFeatures(Arrays.asList(new OpenApiFeature()));
        factory.setBus(bus);
        factory.setAddress("/");

        return factory.create();
    }

    @Bean
    public ServletRegistrationBean cxfServlet() {
        final ServletRegistrationBean servletRegistrationBean = new ServletRegistrationBean(new CXFServlet(), "/api/*");
        servletRegistrationBean.setLoadOnStartup(1);
        return servletRegistrationBean;
    }
}

The OpenApiFeature is a key ingredient here which takes care of all the integration and introspection. The Spring Boot application is the last touch to finish the picture.

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(AppConfig.class, args);
    }
}

Let us build and run it right away:

mvn clean package 
java -jar target/jax-rs-2.1-openapi-0.0.1-SNAPSHOT.jar

Having the application started, the OpenAPI 3.0.0 specification of our web APIs should be live and available for consumption in the JSON format at:

http://localhost:8080/api/openapi.json

Or in YAML format at:

http://localhost:8080/api/openapi.json

Wouldn't it be great to explore our web APIs and play with it? Since we included Swagger UI dependency, this is no brainer, just navigate to http://localhost:8080/api/api-docs?url=/api/openapi.json:

A special attention to be made to the small icon Swagger UI places alongside your API version, hinting about its conformity to OpenAPI 3.0.0 specification.

Just to note here, there is nothing special in using Spring Boot. In case you are using Apache CXF inside the OSGi container (like Apache Karaf for example), the integration with OpenAPI 3.0.0 is also available (please check out the official documentation and samples if you are interested in the subject).

It all looks easy and simple, but what about migrating to OpenAPI 3.0.0 from older versions of the Swagger specifications? The Apache CXF has an powerful feature to convert older the specifications on the fly but in general the OpenApi.Tools portal is a right place to evaluate your options.

Should you migrate to OpenAPI 3.0.0? I honestly believe you should, at least should try experimenting with it, but please be aware that the tooling is still not mature enough, expect a few roadblocks along the way (which you would be able to overcome by contributing the patches, by the way). But undoubtedly, the future is bright!

The complete project sources are available on Github.

Wednesday, April 26, 2017

When following REST(ful) principles might look impractical, GraphQL could come on the resque

I am certainly late with jumping on the trendy train, but today we are going to talk about GraphQL, a very interesting approach to build REST(ful) web services and APIs. In my opinion, it would be fair to restate that REST(ful) architecture is built on top of quite reasonable principles and constraints (although the debates over that are never ending in the industry).

So ... what is GraphQL? By and large, it is yet another kind of the query language. But what makes it interesting, it is designed to give the clients (f.e., the frontends) the ability to express their needs (f.e., to the backends) in terms of data they are expecting. Frankly speaking, GraphQL goes much further than that but this is the one of its most compelling features.

GraphQL is just a specification, without any particular requirements to the programming language or technology stack but, not surprisingly, it got the widespread acceptance in modern web development, both on client-side (Apollo, Relay) and server-side (the class of APIs often called BFFs these days). In today's post we are going to give GraphQL a ride, discuss where it could be a good fit and why you may consider adopting it. Although there quite a few options on the table, Sangria, terrific Scala-based implementation of GraphQL specification, would be the foundation we are going to build atop.

Essentially, our goal is to develop a simple user management web API. The data model is far from being complete but good enough to serve its purpose. Here is the User class.

case class User(id: Long, email: String, created: Option[LocalDateTime], 
  firstName: Option[String], lastName: Option[String], roles: Option[Seq[Role.Value]], 
    active: Option[Boolean], address: Option[Address])

The Role is a hardcoded enumeration:

object Role extends Enumeration {
  val USER, ADMIN = Value
}

While the Address is another small class in the data model:

case class Address(street: String, city: String, state: String, 
  zip: String, country: String)

In its core, GraphQL is strongly typed. That means, the application specific models should be somehow represented in GraphQL. To speak naturally, we need to define schema. In Sangria, schema definitions are pretty straightforward and consist of three main categories, borrowed from GraphQL specification: object types, query types and mutation types. All of these we are going to touch upon, but the object type definitions sounds like a logical point to start with.

val UserType = ObjectType(
  "User",
  "User Type",
  interfaces[Unit, User](),
  fields[Unit, User](
    Field("id", LongType, Some("User id"), tags = ProjectionName("id") :: Nil, 
      resolve = _.value.id),
    Field("email", StringType, Some("User email address"), 
      resolve = _.value.email),
    Field("created", OptionType(LocalDateTimeType), Some("User creation date"), 
      resolve = _.value.created),
    Field("address", OptionType(AddressType), Some("User address"), 
      resolve = _.value.address),
    Field("active", OptionType(BooleanType), Some("User is active or not"), 
      resolve = _.value.active),
    Field("firstName", OptionType(StringType), Some("User first name"), 
      resolve = _.value.firstName),
    Field("lastName", OptionType(StringType), Some("User last name"), 
      resolve = _.value.lastName),
    Field("roles", OptionType(ListType(RoleEnumType)), Some("User roles"), 
      resolve = _.value.roles)
  ))

In many respects, it is just direct mapping from the User to UserType. Sangria can easy you off from doing that by providing macros support so you may get the schema generated at compile time. The AddressType definition is very similar, let us skip over it and look on how to deal with enumeration, like Roles.

val RoleEnumType = EnumType(
  "Role",
  Some("List of roles"),
  List(
    EnumValue("USER", value = Role.USER, description = Some("User")),
    EnumValue("ADMIN", value = Role.ADMIN, description = Some("Administrator"))
  ))

Easy, simple and compact ... In traditional REST(ful) web services the metadata about the resources is not generally available out of the box. However, several complimentary specifications, like JSON Schema, could fill this gap with a bit of work.

Good, so types are there, but what are these queries and mutations? Query is a special type within GraphQL specification which basically describes how you would like to fetch the data and the shape of it. For example, there is often a need to get user details by its identifier, which could be expressed by following GraphQL query:

query {
  user(id: 1) {
    email
    firstName
    lastName
    address {
      country
    }
  }
}

You can literally read it as-is: lookup the user with identifier 1 and return only his email, first and last names, and address with the country only. Awesome, not only GraphQL queries are exceptionally powerful, but they are giving the control of what to return back to the interested parties. Priceless feature if you have to support a diversity of different clients without exploding the amount of API endpoints. Defining the query types in Sangria is also a no-brainer, for example:

val Query = ObjectType(
  "Query", fields[Repository, Unit](
    Field("user", OptionType(UserType), arguments = ID :: Nil, 
      resolve = (ctx) => ctx.ctx.findById(ctx.arg(ID))),
    Field("users", ListType(UserType), 
      resolve = Projector((ctx, f) => ctx.ctx.findAll(f.map(_.name))))
  ))

There are two queries in fact which the code snippet above describes. The one we have seen before, fetching user by identifier, and another one, fetching all users. Here is a quick example of latter:

query {
  users {
    id 
    email 
  }
}

Hopefully you would agree that no explanations needed, the intent is clear. Queries arguably are the strongest argument in favor of adopting GraphQL, the value proposition is really tremendous. With Sangria you do have access to the fields which clients want back, so the data store could be told to return only these subsets, using projections, selects, or similar concepts. To be closer to reality, our sample application stores data in MongoDB so we could ask it to return only fields the client is interested in.

def findAll(fields: Seq[String]): Future[Seq[User]] = collection.flatMap(
   _.find(document())
    .projection(
      fields
       .foldLeft(document())((doc, field) => doc.merge(field -> BSONInteger(1)))
    )
    .cursor[User]()
    .collect(Int.MaxValue, Cursor.FailOnError[Seq[User]]())
  )

If we get back to the typical REST(ful) web APIs, the approach most widely used these days to outline the shape of the desired response is to pass a query string parameter, for example /api/users?fields=email,firstName,lastName, .... However, from the implementation perspective, not many frameworks support such features natively, so everyone has to come up with their own way. Regarding the querying capabilities, in case you happen to be the user of terrific Apache CXF framework, you may benefit from its quite powerful search extension, which we have talked about some time ago.

If queries usually just fetch data, mutations are serving the purpose of the data modification. Syntactically they are very similar to queries but their interpretation is different. For example, here is one of the ways we could add new user to the application.

mutation {
  add(email: "[email protected]", firstName: "John", lastName: "Smith", roles: [ADMIN]) {
    id
    active
    email
    firstName
    lastName
    address {
      street
      country
    }
  }
}

In this mutation a new user John Smith with email [email protected] and ADMIN role assigned is going to be added to the system. As with queries, client is always in control which data shape it needs from server when mutation completes. Mutations could be think of as the calls for action and resemble a lot method invocations, for example the activation of the user may be done like that:

mutation {
  activate(id: 1) {
    active
  }
}

In Sangria, mutations are described exactly like queries, for example the ones we have looked at before have the following type definition:

val Mutation = ObjectType(
  "Mutation", fields[Repository, Unit](
    Field("activate", OptionType(UserType), 
      arguments = ID :: Nil,
      resolve = (ctx) => ctx.ctx.activateById(ctx.arg(ID))),
    Field("add", OptionType(UserType), 
      arguments = EmailArg :: FirstNameArg :: LastNameArg :: RolesArg :: Nil,
      resolve = (ctx) => ctx.ctx.addUser(ctx.arg(EmailArg), ctx.arg(FirstNameArg), 
        ctx.arg(LastNameArg), ctx.arg(RolesArg)))
  ))

With that, our GraphQL schema is complete:

val UserSchema = Schema(Query, Some(Mutation))

That's great, however ... what we can do with it? Just in time question, please welcome GraphQL server. As we remember, there is no attachment to particular technology or stack, but in the universe of web APIs you can think of GraphQL server as a single endpoint which is bound to POST HTTP verb. And, once we started to talk about HTTP and Scala, who could do better job than amazing Akka HTTP, luckily Sangria has a seamless integration with it.

val route: Route = path("users") {
  post {
    entity(as[String]) { document =>
      QueryParser.parse(document) match {
        case Success(queryAst) =>
          complete(Executor.execute(SchemaDefinition.UserSchema, queryAst, repository)
            .map(OK -> _)
            .recover {
              case error: QueryAnalysisError => BadRequest -> error.resolveError
              case error: ErrorWithResolver => InternalServerError -> error.resolveError
            })

        case Failure(error) => complete(BadRequest -> Error(error.getMessage))
      }
    }
  } ~ get {
    complete(SchemaRenderer.renderSchema(SchemaDefinition.UserSchema))
  }
}

You may notice that we also expose our schema under GET endpoint as well, what it is here for? Well, if you are familiar with Swagger which we have talked about a lot here, it is a very similar concept. The schema contains all the necessary pieces, enough for external tools to automatically discover the respective GraphQL queries and mutations, along with the types they are referencing. GraphiQL, an in-browser IDE for exploring GraphQL, is one of those (think about Swagger UI in the REST(ful) services world).

We are mostly there, our GraphQL server is ready, let us run it and send off a couple of queries and mutations, to get the feeling of it:

sbt run

[info] Running com.example.graphql.Boot
INFO  akka.event.slf4j.Slf4jLogger  - Slf4jLogger started
INFO  reactivemongo.api.MongoDriver  - No mongo-async-driver configuration found
INFO  reactivemongo.api.MongoDriver  - [Supervisor-1] Creating connection: Connection-2
INFO  r.core.actors.MongoDBSystem  - [Supervisor-1/Connection-2] Starting the MongoDBSystem akka://reactivemongo/user/Connection-2

Very likely our data store (we are using MongoDB run as Docker container) has no users at the moment so it sounds like a good idea to add one right away:

$ curl -vi -X POST http://localhost:48080/users -H "Content-Type: application/json" -d " \
   mutation { \
     add(email: \"[email protected]\", firstName: \"John\", lastName: \"Smith\", roles: [ADMIN]) { \
       id \
       active \
       email \
       firstName \
       lastName \
       address { \
       street \
         country \
       } \
     } \
   }"

HTTP/1.1 200 OK
Server: akka-http/10.0.5
Date: Tue, 25 Apr 2017 01:01:25 GMT
Content-Type: application/json
Content-Length: 123

{
  "data":{
    "add":{
      "email":"[email protected]",
      "lastName":"Smith",
      "firstName":"John",
      "id":1493082085000,
      "address":null,
      "active":false
    }
  }
}

It seems to work perfectly fine. The response details will be always wrapped into data envelop, no matter what kind of query or mutation you are running, for example:

$ curl -vi -X POST http://localhost:48080/users -H "Content-Type: application/json" -d " \                                                
   query { \                                                                                                                           
     users { \                                                                                                                         
       id \                                                                                                                            
       email \                                                                                                                         
     } \                                                                                                                               
   }"                                                                                                                                  

HTTP/1.1 200 OK                                                                                                                                   
Server: akka-http/10.0.5                                                                                                                          
Date: Tue, 25 Apr 2017 01:09:21 GMT                                                                                                               
Content-Type: application/json                                                                                                                    
Content-Length: 98                                                                                                                                
                                                                                                                                                  
{
  "data":{
    "users":[
      {
        "id":1493082085000,
        "email":"[email protected]"
      }
    ]
  }
}

Exactly as we ordered... Honestly, working with GraphQL feels natural, specifically when data querying is involved. And we didn't even talk about fragments, variables, directives, and a lot of other things.

Now it comes to the question: should we abandon all our practices, JAX-RS, Spring MVC, ... and switch to GraphQL? I honestly believe that this is not the case, GraphQL is a good fit to address certain kind of problems, but by and large, traditional REST(ful) web services, combined with Swagger or any other established API specification framework, are here to stay.

And please be warned, along with the benefits, GraphQL comes at a price. For example, HTTP caching and cache control won't apply anymore, HATEOAS does not make much sense either, unified responses no matter what you are calling, reliability as everything is behind single facade, ... With that in mind, GraphQL is indeed a great tool, please use it wisely!

The complete project source is available on Github.

Sunday, November 27, 2016

Keep your promises: contract-based testing for JAX-RS APIs

It's been a while since we talked about testing and applying effective TDD practices, particularly related to REST(ful) web services and APIs. But this topic should have never been forgotten, especially in the world where everyone is doing microservices, whatever it means, implies or takes.

To be fair, there are quite a lot of areas where microservice-based architecture shines and allows organizations to move and innovate much faster. But without a proper discipline, it also makes our systems fragile, as they become very loosely coupled. In today's post we are going to talk about contract-based testing and consumer-driven contracts as a practical and reliable techniques to ensure that our microservices fulfill their promises.

So, how does contract-based testing work? In nutshell, it is surprisingly simple technique and is guided by following steps:

  • provider (let say Service A) publishes its contact (or specification), the implementation may not even be available at this stage
  • consumer (let say Service B) follows this contract (or specification) to implement conversations with Service A
  • additionally, consumer introduces a test suite to verify its expectations regarding Service A contract fulfillment
In case of SOAP web services and APIs, things are obvious as there is an explicit contract in a form of WSDL file. But in case of REST(ful) APIs, there are a lot of different options around the corner (WADL, RAML, Swagger, ...) and still no agreement on the one. It may sound complicated but please don't get upset, because Pact is coming on the rescue!

Pact is family of frameworks for supporting consumer-driven contracts testing. There are many language bindings and implementations available, including JVM ones, JVM Pact and Scala-Pact. To evolve such a polyglot ecosystem, Pact also includes a dedicated specification so to provide interoperability between different implementations.

Great, Pact is there, the stage is set and we are ready to take off with some real code snippets. Let us assume we are developing a REST(ful) web API for managing people, using terrific Apache CXF and JAX-RS 2.0 specification. To keep things simple, we are going to introduce only two endpoints:

  • POST /people/v1 to create new person
  • GET /people/v1?email=<email> to find person by email address
Essentially, we may not bother and just communicate these minimal pieces of our service contract to everyone so let the consumers deal with that themselves (and indeed, Pact supports such a scenario). But surely, we are not like that, we do care and would like to document our APIs comprehensively, likely we are already familiar with Swagger. With that, here is our PeopleRestService.
@Api(value = "Manage people")
@Path("/people/v1")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public class PeopleRestService {
    @GET
    @ApiOperation(value = "Find person by e-mail", 
        notes = "Find person by e-mail", response = Person.class)
    @ApiResponses({
        @ApiResponse(code = 404, 
            message = "Person with such e-mail doesn't exists", 
            response = GenericError.class)
    })
    public Response findPerson(
        @ApiParam(value = "E-Mail address to lookup for", required = true) 
        @QueryParam("email") final String email) {
        // implementation here
    }

    @POST
    @ApiOperation(value = "Create new person", 
        notes = "Create new person", response = Person.class)
    @ApiResponses({
        @ApiResponse(code = 201, 
            message = "Person created successfully", 
            response = Person.class),
        @ApiResponse(code = 409, 
            message = "Person with such e-mail already exists", 
            response = GenericError.class)
    })
    public Response addPerson(@Context UriInfo uriInfo, 
        @ApiParam(required = true) PersonUpdate person) {
        // implementation here
    }
}

The implementation details are not important at the moment, however let us take a look at the GenericError, PersonUpdate and Person classes as they are an integral part of our service contract.

@ApiModel(description = "Generic error representation")
public class GenericError {
    @ApiModelProperty(value = "Error message", required = true)
    private String message;
}

@ApiModel(description = "Person resource representation")
public class PersonUpdate {
    @ApiModelProperty(value = "Person's first name", required = true) 
    private String email;
    @ApiModelProperty(value = "Person's e-mail address", required = true) 
    private String firstName;
    @ApiModelProperty(value = "Person's last name", required = true) 
    private String lastName;
    @ApiModelProperty(value = "Person's age", required = true) 
    private int age;
}

@ApiModel(description = "Person resource representation")
public class Person extends PersonUpdate {
    @ApiModelProperty(value = "Person's identifier", required = true) 
    private String id;
}

Excellent! Once we have Swagger annotations in place and Apache CXF Swagger integration turned on, we could generate swagger.json specification file, bring it to live in Swagger UI and distribute to every partner or interested consumer.

Would be great if we could use this Swagger specification along with Pact framework implementation to serve as a service contract. Thanks to Atlassian, we are certainly able to do that using swagger-request-validator, a library for validating HTTP request/respons against a Swagger/OpenAPI specification which nicely integrates with Pact JVM as well.

Cool, now let us switch sides from provider to consumer and try to figure out what we can do having such Swagger specification in our hands. It turns out, we can do a lot of things. For example, let us take a look at the POST action, which creates new person. As a client (or consumer), we could express our expectations in such a form that having a valid payload submitted along with the request, we expect HTTP status code 201 to be returned by the provider and the response payload should contain a new person with identifier assigned. In fact, translating this statement into Pact JVM assertions is pretty straightforward.

@Pact(provider = PROVIDER_ID, consumer = CONSUMER_ID)
public PactFragment addPerson(PactDslWithProvider builder) {
    return builder
        .uponReceiving("POST new person")
        .method("POST")
        .path("/services/people/v1")
        .body(
            new PactDslJsonBody()
                .stringType("email")
                .stringType("firstName")
                .stringType("lastName")
                .numberType("age")
        )
        .willRespondWith()
        .status(201)
        .matchHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON)
        .body(
            new PactDslJsonBody()
                .uuid("id")
                .stringType("email")
                .stringType("firstName")
                .stringType("lastName")
                .numberType("age")
        )
       .toFragment();
}

To trigger the contract verification process, we are going to use awesome JUnit and very popular REST Assured framework. But before that, let us clarify on what is PROVIDER_ID and CONSUMER_ID from the code snippet above. As you may expect, PROVIDER_ID is the reference to the contract specification. For simplicity, we would fetch Swagger specification from running PeopleRestService endpoint, luckily Spring Boot testing improvements make this task a no-brainer.

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT, 
    classes = PeopleRestConfiguration.class)
public class PeopleRestContractTest {
    private static final String PROVIDER_ID = "People Rest Service";
    private static final String CONSUMER_ID = "People Rest Service Consumer";

    private ValidatedPactProviderRule provider;
    @Value("${local.server.port}")
    private int port;

    @Rule
    public ValidatedPactProviderRule getValidatedPactProviderRule() {
        if (provider == null) {
            provider = new ValidatedPactProviderRule("http://localhost:" + port + 
                "/services/swagger.json", null, PROVIDER_ID, this);
        }

        return provider;
    }
}

The CONSUMER_ID is just a way to identify the consumer, not much to say about it. With that, we are ready to finish up with our first test case:

@Test
@PactVerification(value = PROVIDER_ID, fragment = "addPerson")
public void testAddPerson() {
    given()
        .contentType(ContentType.JSON)
        .body(new PersonUpdate("[email protected]", "Tom", "Smith", 60))
        .post(provider.getConfig().url() + "/services/people/v1");
}

Awesome! As simple as that, just please notice the presence of @PactVerification annotation where we are referencing the appropriate verification fragment by name, in this case it points out to addPerson method we have introduced before.

Great, but ... what the point? Glad you are asking, because from now on any change in the contract which may not be backward compatible will break our test case. For example, if provider decides to remove the id property from the response payload, the test case will fail. Renaming the request payload properties, big no-no, again, test case will fail. Adding new path parameters? No luck, test case won't let it pass. You may go even further than that and fail on every contract change, even if it backward-compatible (using swagger-validator.properties for fine-tuning).

validation.response=ERROR
validation.response.body.missing=ERROR

No a very good idea but still, if you need it, it is there. Similarly, let us add a couple of more test cases for GET endpoint, starting from successful scenario, where person we are looking for exists, for example:

@Pact(provider = PROVIDER_ID, consumer = CONSUMER_ID)
public PactFragment findPerson(PactDslWithProvider builder) {
    return builder
        .uponReceiving("GET find person")
        .method("GET")
        .path("/services/people/v1")
        .query("[email protected]")
        .willRespondWith()
        .status(200)
        .matchHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON)
        .body(
            new PactDslJsonBody()
                .uuid("id")
                .stringType("email")
                .stringType("firstName")
                .stringType("lastName")
                .numberType("age")
        )
        .toFragment();
}

@Test
@PactVerification(value = PROVIDER_ID, fragment = "findPerson")
public void testFindPerson() {
    given()
        .contentType(ContentType.JSON)
        .queryParam("email", "[email protected]")
        .get(provider.getConfig().url() + "/services/people/v1");
}

Please take a note that here we introduced query string verification using query("[email protected]") assertion. Following the possible outcomes, let us also cover the unsuccessful scenario, where person does not exist and we expect some error to be returned, along with 404 status code, for example:

@Pact(provider = PROVIDER_ID, consumer = CONSUMER_ID)
public PactFragment findNonExistingPerson(PactDslWithProvider builder) {
    return builder
        .uponReceiving("GET find non-existing person")
        .method("GET")
        .path("/services/people/v1")
        .query("[email protected]")
        .willRespondWith()
        .status(404)
        .matchHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON)
        .body(new PactDslJsonBody().stringType("message"))
        .toFragment();
}

@Test
@PactVerification(value = PROVIDER_ID, fragment = "findNonExistingPerson")
public void testFindPersonWhichDoesNotExist() {
    given()
        .contentType(ContentType.JSON)
        .queryParam("email", "[email protected]")
        .get(provider.getConfig().url() + "/services/people/v1");
}

Really brilliant, maintainable, understandable and non-intrusive approach to address such a complex and important problems as contract-based testing and consumer-driven contracts. Hopefully, this somewhat new testing technique would help you to catch more issues during the development phase, way before they would have a chance to leak into production.

Thanks to Swagger we were able to take a few shortcuts, but in case you don't have such a luxury, Pact has quite rich specification which you are very welcome to learn and use. In any case, Pact JVM does a really great job in helping you out writing small and concise test cases.

The complete project sources are available on Github.

Monday, September 30, 2013

Swagger: make developers love working with your REST API

As JAX-RS API is evolving, with version 2.0 released earlier this year under JSR-339 umbrella, it's becoming even more easy to create REST services using excellent Java platform.

But with great simplicity comes great responsibility: documenting all these APIs so other developers could quickly understand how to use them. Unfortunately, in this area developers are on their own: the JSR-339 doesn't help much. For sure, it would be just awesome to generate verbose and easy to follow documentation from source code, and not asking someone to write it along the development process. Sounds unreal, right? In certain extent, it really is, but help is coming in a form of Swagger.

Essentially, Swagger does a simple but very powerful thing: with a bit of additional annotations it generates the REST API descriptions (HTTP methods, path / query / form parameters, responses, HTTP error codes, ...) and even provides a simple web UI to play with REST calls to your APIs (not to mention that all this metadata is available over REST as well).

Before digging into implementation details, let's take a quick look what Swagger is from API consumer prospective. Assume you have developed a great REST service to manage people. As a good citizen, this REST service is feature-complete and provides following functionality:

  • lists all people (GET)
  • looks up person by e-mail (GET)
  • adds new person (POST)
  • updates existing person (PUT)
  • and finally removes person (DELETE)
Here is the same API from Swagger's perspective:

It looks quite pretty. Let's do more and call our REST service from Swagger UI, here this awesome framework really shines. The most complicated use-case is adding new person (POST) so this one will be looked closely.

As you can see on the snapshot above, every piece of REST service call is there:

  • description of the service
  • relative context path
  • parameters (form / path / query), required or optional
  • HTTP status codes: 201 CREATED and 409 CONFLICT
  • ready to go Try it out! to call REST service immediately (with out-of-the box parameters validation)

To complete the demo part, let me show yet another example, where REST resource is being involved (in our case, it's a simple class Person). Swagger is able to provide its properties and meaningful description together with expected response content type(s).

Looks nice! Moving on to the next part, it's all about implementation details. Swagger supports seamless integration with JAX-RS services, with just couple of additional annotations required on top of existing ones. Firstly, every single JAX-RS service which supposed to be documented should be annotated with @Api annotation, in our case:

@Path( "/people" ) 
@Api( value = "/people", description = "Manage people" )
public class PeopleRestService {
    // ...
}

Next, the same approach is applicable to REST service operations: every method which supposed to be documented should be annotated with @ApiOperation annotation, and optionally with @ApiResponses/@ApiResponse. If it accepts parameters, those should be annotated with @ApiParam annotation. Couple of examples here:

@Produces( { MediaType.APPLICATION_JSON } )
@GET
@ApiOperation( 
    value = "List all people", 
    notes = "List all people using paging", 
    response = Person.class, 
    responseContainer = "List"
)
public Collection< Person > getPeople(  
        @ApiParam( value = "Page to fetch", required = true ) 
        @QueryParam( "page") @DefaultValue( "1" ) final int page ) {
    // ...
}

And another one:

@Produces( { MediaType.APPLICATION_JSON } )
@Path( "/{email}" )
@GET
@ApiOperation( 
    value = "Find person by e-mail", 
    notes = "Find person by e-mail", 
    response = Person.class 
)
@ApiResponses( {
    @ApiResponse( code = 404, message = "Person with such e-mail doesn't exists" )    
} )
public Person getPeople( 
        @ApiParam( value = "E-Mail address to lookup for", required = true ) 
        @PathParam( "email" ) final String email ) {
    // ...
}

REST resource classes (or model classes) require special annotations: @ApiModel and @ApiModelProperty. Here is how our Person class looks like:

@ApiModel( value = "Person", description = "Person resource representation" )
public class Person {
    @ApiModelProperty( value = "Person's first name", required = true ) 
    private String email;
    @ApiModelProperty( value = "Person's e-mail address", required = true ) 
    private String firstName;
    @ApiModelProperty( value = "Person's last name", required = true ) 
    private String lastName;

    // ...
}

The last steps is to plug in Swagger into JAX-RS application. The example I have developed uses Spring Framework, Apache CXF, Swagger UI and embedded Jetty (complete project is available on Github). Integrating Swagger is a matter of adding configuration bean (swaggerConfig), one additional JAX-RS service (apiListingResourceJson) and two JAX-RS providers (resourceListingProvider and apiDeclarationProvider).

package com.example.config;

import java.util.Arrays;

import javax.ws.rs.ext.RuntimeDelegate;

import org.apache.cxf.bus.spring.SpringBus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.DependsOn;
import org.springframework.core.env.Environment;

import com.example.resource.Person;
import com.example.rs.JaxRsApiApplication;
import com.example.rs.PeopleRestService;
import com.example.services.PeopleService;
import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;
import com.wordnik.swagger.jaxrs.config.BeanConfig;
import com.wordnik.swagger.jaxrs.listing.ApiDeclarationProvider;
import com.wordnik.swagger.jaxrs.listing.ApiListingResourceJSON;
import com.wordnik.swagger.jaxrs.listing.ResourceListingProvider;

@Configuration
public class AppConfig {
    public static final String SERVER_PORT = "server.port";
    public static final String SERVER_HOST = "server.host";
    public static final String CONTEXT_PATH = "context.path";  
 
    @Bean( destroyMethod = "shutdown" )
    public SpringBus cxf() {
        return new SpringBus();
    }
 
    @Bean @DependsOn( "cxf" )
    public Server jaxRsServer() {
        JAXRSServerFactoryBean factory = RuntimeDelegate.getInstance().createEndpoint( jaxRsApiApplication(), JAXRSServerFactoryBean.class );
        factory.setServiceBeans( Arrays.< Object >asList( peopleRestService(), apiListingResourceJson() ) );
        factory.setAddress( factory.getAddress() );
        factory.setProviders( Arrays.< Object >asList( jsonProvider(), resourceListingProvider(), apiDeclarationProvider() ) );
        return factory.create();
    }
 
    @Bean @Autowired
    public BeanConfig swaggerConfig( Environment environment ) {
        final BeanConfig config = new BeanConfig();

        config.setVersion( "1.0.0" );
        config.setScan( true );
        config.setResourcePackage( Person.class.getPackage().getName() );
        config.setBasePath( 
            String.format( "http://%s:%s/%s%s",
                environment.getProperty( SERVER_HOST ),
                environment.getProperty( SERVER_PORT ),
                environment.getProperty( CONTEXT_PATH ),
                jaxRsServer().getEndpoint().getEndpointInfo().getAddress() 
            ) 
        );
  
        return config;
    }

    @Bean
    public ApiDeclarationProvider apiDeclarationProvider() {
        return new ApiDeclarationProvider();
    }
 
    @Bean
    public ApiListingResourceJSON apiListingResourceJson() {
        return new ApiListingResourceJSON();
    }
 
    @Bean
    public ResourceListingProvider resourceListingProvider() {
        return new ResourceListingProvider();
    }
 
    @Bean 
    public JaxRsApiApplication jaxRsApiApplication() {
        return new JaxRsApiApplication();
    }
 
    @Bean 
    public PeopleRestService peopleRestService() {
        return new PeopleRestService();
    }
   
    // ... 
}

In order to get rid of any possible hard-coded configuration, all parameters are passed through named properties (SERVER_PORT, SERVER_HOST and CONTEXT_PATH). Swagger exposes additional REST endpoint to provide API documentation, in our case it is accessible at: http://localhost:8080/rest/api/api-docs. It is used by Swagger UI which itself is embedded into final JAR archive and served by Jetty as static web resource.

The final piece of the puzzle is to start embedded Jetty container which glues all those parts together and is encapsulated into Starter class:

package com.example;

import org.apache.cxf.transport.servlet.CXFServlet;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.HandlerList;
import org.eclipse.jetty.servlet.DefaultServlet;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.eclipse.jetty.util.resource.Resource;
import org.springframework.core.io.ClassPathResource;
import org.springframework.web.context.ContextLoaderListener;
import org.springframework.web.context.support.AnnotationConfigWebApplicationContext;

import com.example.config.AppConfig;

public class Starter {
    private static final int SERVER_PORT = 8080;
    private static final String CONTEXT_PATH = "rest";
 
    public static void main( final String[] args ) throws Exception {
        Resource.setDefaultUseCaches( false );
  
        final Server server = new Server( SERVER_PORT );  
        System.setProperty( AppConfig.SERVER_PORT, Integer.toString( SERVER_PORT ) );
        System.setProperty( AppConfig.SERVER_HOST, "localhost" );
        System.setProperty( AppConfig.CONTEXT_PATH, CONTEXT_PATH );    

        // Configuring Apache CXF servlet and Spring listener  
        final ServletHolder servletHolder = new ServletHolder( new CXFServlet() );      
        final ServletContextHandler context = new ServletContextHandler();   
        context.setContextPath( "/" );
        context.addServlet( servletHolder, "/" + CONTEXT_PATH + "/*" );     
        context.addEventListener( new ContextLoaderListener() ); 
     
        context.setInitParameter( "contextClass", AnnotationConfigWebApplicationContext.class.getName() );
        context.setInitParameter( "contextConfigLocation", AppConfig.class.getName() );
   
        // Configuring Swagger as static web resource
        final ServletHolder swaggerHolder = new ServletHolder( new DefaultServlet() );
        final ServletContextHandler swagger = new ServletContextHandler();
        swagger.setContextPath( "/swagger" );
        swagger.addServlet( swaggerHolder, "/*" );
        swagger.setResourceBase( new ClassPathResource( "/webapp" ).getURI().toString() );

        final HandlerList handlers = new HandlerList();
        handlers.addHandler( context );
        handlers.addHandler( swagger );
   
        server.setHandler( handlers );
        server.start();
        server.join(); 
    }
}

Couple of comments make thing a bit more clear: our JAX-RS services will be available under /rest/* context path while Swagger UI is available under /swagger context path. The one important note concerning Resource.setDefaultUseCaches( false ): because we are serving static web content from JAR file, we have to set this property to false as workaround for this bug.

Now, let's build and run our JAX-RS application by typing:

mvn clean package
java -jar target/jax-rs-2.0-swagger-0.0.1-SNAPSHOT.jar

In a second, Swagger UI should be available in your browser at: http://localhost:8080/swagger/

As a final note, there are a lot more to say about Swagger but I hope this simple example shows the way to make our REST services self-documented and easily consumable with minimal efforts. Many thanks to Wordnik team for that.

Source code is available on Github.