The beauty of the RESTful web services and APIs is that any consumer which speaks HTTP protocol will be able to understand and use it. Nonetheless, the same dilemma pops up over and over again: should you accompany your web APis with the client libraries or not? If yes, what languages or/and frameworks should you support?
Quite often this is not really an easy question to answer. So let us take a step back and think about the overall idea for a moment: what are the values which client libraries may bring to the consumers?
Someone may say to lower the barrier for adoption. Indeed, specifically in the case of strongly typed languages, exploring the API contracts from your favorite IDE (syntax highlighting and auto-completion please!) is quite handy. But by and large, the RESTful web APIs are simple enough to start with and good documentation would be certainly more valuable here.
Others may say it is good to shield the consumers from dealing with multiple API versions or rough edges. Also kind of make sense but I would argue it just hides the flaws with the way the web APIs in question are designed and evolve over time.
All in all, no matter how many clients you decide to bundle, the APIs are still going to be accessible by any generic HTTP consumer (curl, HttpClient, RestTemplate, you name it). Giving a choice is great but the price to pay for maintenance could be really high. Could we do it better? And as you may guess already, we certainly have quite a few options hence this post.
The key ingredient of the success here is to maintain an accurate specification of your RESTful web APIs, using OpenAPI v3.0 or even its predecessor, Swagger/OpenAPI 2.0 (or RAML, API Blueprint, does not really matter much). In case of OpenAPI/Swagger, the tooling is the king: one could use Swagger Codegen, a template-driven engine, to generate API clients (and even server stubs) in many different languages, and this is what we are going to talk about in this post.
To simplify the things, we are going to implement the consumer of the people management web API which we have built in the previous post. To begin with, we need to get its OpenAPI v3.0 specification in the YAML (or JSON) format.
java -jar server-openapi/target/server-openapi-0.0.1-SNAPSHOT.jar
And then:
wget http://localhost:8080/api/openapi.yaml
Awesome, the half of the job is done, literally. Now, let us allow Swagger Codegen to take a lead. In order to not complicate the matter, let's assume that consumer is also Java application, so we could understand the mechanics without any difficulties (but Java is only one of the options, the list of supported languages and frameworks is astonishing).
Along this post we are going to use OpenFeign, one of the most advanced Java HTTP client binders. Not only it is exceptionally simple to use, it offers quite a few integrations we are going to benefit from soon.
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-core</artifactId>
<version>9.7.0</version>
</dependency>
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-jackson</artifactId>
<version>9.7.0</version>
</dependency>
The Swagger Codegen could be run as stand-alone application from command-line, or Apache Maven plugin (the latter is what we are going to use).
<plugin>
<groupId>io.swagger</groupId>
<artifactId>swagger-codegen-maven-plugin</artifactId>
<version>3.0.0-rc1</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>/contract/openapi.yaml</inputSpec>
<apiPackage>com.example.people.api</apiPackage>
<language>java</language>
<library>feign</library>
<modelPackage>com.example.people.model</modelPackage>
<generateApiDocumentation>false</generateApiDocumentation>
<generateSupportingFiles>false</generateSupportingFiles>
<generateApiTests>false</generateApiTests>
<generateApiDocs>false</generateApiDocs>
<addCompileSourceRoot>true</addCompileSourceRoot>
<configOptions>
<sourceFolder>/</sourceFolder>
<java8>true</java8>
<dateLibrary>java8</dateLibrary>
<useTags>true</useTags>
</configOptions>
</configuration>
</execution>
</executions>
</plugin>
If some of the options are not very clear, the Swagger Codegen has pretty good documentation to look for clarifications. The important ones to pay attention to is language and library, which are set to java and feign respectively. The one thing to note though, the support of the OpenAPI v3.0 specification is mostly complete but you may encounter some issues nonetheless (as you noticed, the version is 3.0.0-rc1).
What you will get when your build finishes is the plain old Java interface, PeopleApi, annotated with OpenFeign annotations, which is direct projection of the people management web API specification (which comes from /contract/openapi.yaml). Please notice that all model classes are generated as well.
@javax.annotation.Generated(
value = "io.swagger.codegen.languages.java.JavaClientCodegen",
date = "2018-06-17T14:04:23.031Z[Etc/UTC]"
)
public interface PeopleApi extends ApiClient.Api {
@RequestLine("POST /api/people")
@Headers({"Content-Type: application/json", "Accept: application/json"})
Person addPerson(Person body);
@RequestLine("DELETE /api/people/{email}")
@Headers({"Content-Type: application/json"})
void deletePerson(@Param("email") String email);
@RequestLine("GET /api/people/{email}")
@Headers({"Accept: application/json"})
Person findPerson(@Param("email") String email);
@RequestLine("GET /api/people")
@Headers({"Accept: application/json"})
List<Person> getPeople();
}
Let us compare it with Swagger UI interpretation of the same specification, available at http://localhost:8080/api/api-docs?url=/api/openapi.json:
It looks right at the first glance but we have better ensure things work out as expected. Once we have OpenFeign-annotated interface, it could be made functional (in this case, implemented through proxies) using the family of Feign builders, for example:
final PeopleApi api = Feign
.builder()
.client(new OkHttpClient())
.encoder(new JacksonEncoder())
.decoder(new JacksonDecoder())
.logLevel(Logger.Level.HEADERS)
.options(new Request.Options(1000, 2000))
.target(PeopleApi.class, "http://localhost:8080/");
Great, fluent builder style rocks. Assuming our people management web APIs server is up and running (by default, it is going to be available at http://localhost:8080/):
java -jar server-openapi/target/server-openapi-0.0.1-SNAPSHOT.jar
We could communicate with it by calling freshly built PeopleApi instance methods, as in the code snippet below.:
final Person person = api.addPerson(
new Person()
.email("[email protected]")
.firstName("John")
.lastName("Smith"));
It is really cool, if we rewind it back a bit, we actually did nothing. Everything is given to us for free with only web API specification available! But let us not stop here and remind ourselves that using Java interfaces will not eliminate the reality that we are dealing with remote systems. And things are going to fail here, sooner or later, no doubts.
Not so long ago we have learned about circuit breakers and how useful they are when properly applied in the context of distributed systems. It would be really awesome to somehow introduce this feature into our OpenFeign-based client. Please welcome another member of the family, HystrixFeign builder, the seamless integration with Hytrix library:
final PeopleApi api = HystrixFeign
.builder()
.client(new OkHttpClient())
.encoder(new JacksonEncoder())
.decoder(new JacksonDecoder())
.logLevel(Logger.Level.HEADERS)
.options(new Request.Options(1000, 2000))
.target(PeopleApi.class, "http://localhost:8080/");
The only thing we need to do is just to add these two dependencies (strictly speaking hystrix-core is not really needed if you do not mind to stay on older version) to consumer's pom.xml file.
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-hystrix</artifactId>
<version>9.7.0</version>
</dependency>
<dependency>
<groupId>com.netflix.hystrix</groupId>
<artifactId>hystrix-core</artifactId>
<version>1.5.12</version>
</dependency>
Arguably, this is one of the best examples of how easy and straightforward integration could be. But even that is not the end of the story. Observability in the distributed systems is as important as never and as we have learned a while ago, distributed tracing is tremendously useful in helping us out here. And again, OpenFeign has support for it right out of the box, let us take a look.
OpenFeign fully integrates with OpenTracing-compatible tracer. The Jaeger tracer is one of those, which among other things has really nice web UI front-end to explore traces and dependencies. Let us run it first, luckily it is fully Docker-ized.
docker run -d -e \
COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
-p 5775:5775/udp \
-p 6831:6831/udp \
-p 6832:6832/udp \
-p 5778:5778 \
-p 16686:16686 \
-p 14268:14268 \
-p 9411:9411 \
jaegertracing/all-in-one:latest
A couple of additional dependencies have to be introduced in order to OpenFeign client to be aware of the OpenTracing capabilities.
<dependency>
<groupId>io.github.openfeign.opentracing</groupId>
<artifactId>feign-opentracing</artifactId>
<version>0.1.0</version>
</dependency>
<dependency>
<groupId>io.jaegertracing</groupId>
<artifactId>jaeger-core</artifactId>
<version>0.29.0</version>
</dependency>
From the Feign builder side, the only change (besides the introduction of the tracer instance) is to wrap up the client into TracingClient, like the snippet below demonstrates:
final Tracer tracer = new Configuration("consumer-openapi")
.withSampler(
new SamplerConfiguration()
.withType(ConstSampler.TYPE)
.withParam(new Float(1.0f)))
.withReporter(
new ReporterConfiguration()
.withSender(
new SenderConfiguration()
.withEndpoint("http://localhost:14268/api/traces")))
.getTracer();
final PeopleApi api = Feign
.builder()
.client(new TracingClient(new OkHttpClient(), tracer))
.encoder(new JacksonEncoder())
.decoder(new JacksonDecoder())
.logLevel(Logger.Level.HEADERS)
.options(new Request.Options(1000, 2000))
.target(PeopleApi.class, "http://localhost:8080/");
On the server-side we also need to integrate with OpenTracing as well. The Apache CXF has first-class support for it, bundled into cxf-integration-tracing-opentracing module. Let us include it as the dependency, this time to server's pom.xml.
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-integration-tracing-opentracing</artifactId>
<version>3.2.4</version>
</dependency>
Depending on the way your configure your applications, there should be an instance of the tracer available which should be passed later on to the OpenTracingFeature, for example.
// Create tracer
final Tracer tracer = new Configuration(
"server-openapi",
new SamplerConfiguration(ConstSampler.TYPE, 1),
new ReporterConfiguration(new HttpSender("http://localhost:14268/api/traces"))
).getTracer();
// Include OpenTracingFeature feature
final JAXRSServerFactoryBean factory = new JAXRSServerFactoryBean();
factory.setProvider(new OpenTracingFeature(tracer()));
...
factory.create()
From now on, the invocation of the any people management API endpoint through generated OpenFeign client will be fully traceable in the Jaeger web UI, available at http://localhost:16686/search (assuming your Docker host is localhost).
Our scenario is pretty simple but imagine the real applications where dozen of external service calls could happen while the single request travels through the system. Without distributed tracing in place, every issue has a chance to turn into a mystery.
As a side note, if you look closer to the trace from the picture, you may notice that server and consumer use different versions of the Jaeger API. This is not a mistake since the latest released version of Apache CXF is using older OpenTracing API version (and as such, older Jaeger client API) but it does not prevent things to work as expected.
With that, it is time to wrap up. Hopefully, the benefits of contract-based (or even better, contract-first) development in the world of RESTful web services and APIs become more and more apparent: generation of the smart clients, consumer-driven contract test, discoverability and rich documentation are just a few to mention. Please make use of it!
The complete project sources are available on Github.
Posts
