Run away from 'null' checks feast: doing PATCH properly with JSON Patch (RFC-6902)

Today we are going to have a conversation about REST(ful) services and APIs, more precisely, around one peculiar subject many experienced developers are struggling with. To put things into perspective, we are going to talk about web APIs, where the REST(ful) principles adhere to HTTP protocol and heavily exploit the semantics of HTTP methods and (usually but not necessarily) use JSON to represent the state.

One particular HTTP method stands out, and although its meaning sounds pretty straightforward, the implementation is far from that. Yes, we are looking at you, the PATCH. So what is the problem, really? It is just an update, right? Yes, in the essence the semantics of the PATCH method in the context of the HTTP-based REST(ful) web services is partial update of the resource. Now, how would you do that, Java developer? Here is where the fun begins.

Let us go over a very simple example of book management API, modeled using latest JSR 370: Java API for RESTful Web Services (JAX-RS 2.1) specification (which finally includes the @PATCH annotation!) and terrific Apache CXF framework. Our resource is just a very simplistic Book class.

public class Book {
    private String title;
    private Collection>String< authors;
    private String isbn;
}

How would you implement the partial update using the PATCH method? Sadly, the brute force solution, the null feast, is the clear winner here.

@PATCH
@Path("/{isbn}")
@Consumes(MediaType.APPLICATION_JSON)
public void update(@PathParam("isbn") String isbn, Book book) {
    final Book existing = bookService.find(isbn).orElseThrow(NotFoundException::new);
        
    if (book.getTitle() != null) {
        existing.setTitle(book.getTitle());
    }

    if (book.getAuthors() != null) {
        existing.setAuthors(book.getAuthors());
    }
        
    // And here it goes on and on ...
    // ...
}

In the nutshell, this is null-guarded PUT clone. Probably, someone could claim that it kind of works and declare the victory here. But hopefully for majority of us this approach clearly has a lot of flaws and should be never taken. Alternatives? Yes, absolutely, RFC-6902: JSON Patch, not an official standard just yet but it is getting there.

The RFC-6902: JSON Patch drastically changes the game by expressing a sequence of operations to apply to a JSON document. To illustrate the idea in action, let us start from a simple example of changing book's title, described in the terms of desired outcome.

{ "op": "replace", "path": "/title", "value": "..." }

Looks clean, what about adding the authors? Easy ...

{ "op": "add", "path": "/authors", "value": ["...", "..."] }

Awesome, sold out, but ... implementation-wise it seems to require quite a lot of work, isn't it? Not really if we rely on the latest and greatest JSR 374: Java API for JSON Processing 1.1 which fully supports RFC-6902: JSON Patch. Armed with the right tools, this time let us do it right.

    org.glassfish
    javax.json
    1.1.2

Interestingly, not many are aware of that Apache CXF, and in general any JAX-RS-complaint framework, closely integrates with JSON-P and supports its basic data types. In case of Apache CXF, it is just a matter of adding cxf-rt-rs-extension-providers module dependency:

    org.apache.cxf
    cxf-rt-rs-extension-providers
    3.2.2

And registering JsrJsonpProvider with your server factory bean, for example:

@Configuration
public class AppConfig {
    @Bean
    public Server rsServer(Bus bus, BookRestService service) {
        JAXRSServerFactoryBean endpoint = new JAXRSServerFactoryBean();
        endpoint.setBus(bus);
        endpoint.setAddress("/");
        endpoint.setServiceBean(service);
        endpoint.setProvider(new JsrJsonpProvider());
        return endpoint.create();
    }
}

With all the pieces wired together, our PATCH operation could be implemented using JSR 374: Java API for JSON Processing 1.1 alone, in just a few lines:

@Service
@Path("/catalog")
public class BookRestService {
    @Inject private BookService bookService;
    @Inject private BookConverter converter;

    @PATCH
    @Path("/{isbn}")
    @Consumes(MediaType.APPLICATION_JSON)
    public void apply(@PathParam("isbn") String isbn, JsonArray operations) {
        final Book book = bookService.find(isbn).orElseThrow(NotFoundException::new);
        final JsonPatch patch = Json.createPatch(operations);
        final JsonObject result = patch.apply(converter.toJson(book));
        bookService.update(isbn, converter.fromJson(result));
    }
}

The BookConverter performs the conversion between Book class and its JSON representation (and vise versa), which we are doing by hand to illustrate another capabilities which JSR 374: Java API for JSON Processing 1.1 provides.

@Component
public class BookConverter {
    public Book fromJson(JsonObject json) {
        final Book book = new Book();
        book.setTitle(json.getString("title"));
        book.setIsbn(json.getString("isbn"));
        book.setAuthors(
            json
                .getJsonArray("authors")
                .stream()
                .map(value -> (JsonString)value)
                .map(JsonString::getString)
                .collect(Collectors.toList()));
        return book;
    }

    public JsonObject toJson(Book book) {
        return Json
            .createObjectBuilder()
            .add("title", book.getTitle())
            .add("isbn", book.getIsbn())
            .add("authors", Json.createArrayBuilder(book.getAuthors()))
            .build();
    }
}

To finish up, let is wrap this simple JAX-RS 2.1 web API into the beautiful Spring Boot envelope.

@SpringBootApplication
public class BookServerStarter {    
    public static void main(String[] args) {
        SpringApplication.run(BookServerStarter.class, args);
    }
}

And run it.

mvn spring-boot:run

To conclude the discussion, let us play a bit with more realistic examples by deliberately adding an incomplete book into our catalog.

$ curl -i -X POST http://localhost:19091/services/catalog -H "Content-Type: application\json" -d '{
       "title": "Microservice Architecture",
       "isbn": "978-1491956250",
       "authors": [
           "Ronnie Mitra",
           "Matt McLarty"
       ]
   }'

HTTP/1.1 201 Created
Date: Tue, 20 Feb 2018 02:30:18 GMT
Location: http://localhost:19091/services/catalog/978-1491956250
Content-Length: 0

There are a couple of inaccuracies we would like to fix in this book description, namely set the title to be complete, "Microservice Architecture: Aligning Principles, Practices, and Culture", and include missing co-authors, Irakli Nadareishvili and Mike Amundsen. With the API we have developed a moment ago, it is a no-brainer.

$ curl -i -X PATCH http://localhost:19091/services/catalog/978-1491956250 -H "Content-Type: application\json" -d '[
       { "op": "add", "path": "/authors/0", "value": "Irakli Nadareishvili" },
       { "op": "add", "path": "/authors/-", "value": "Mike Amundsen" },
       { "op": "replace", "path": "/title", "value": "Microservice Architecture: Aligning Principles, Practices, and Culture" }
   ]'

HTTP/1.1 204 No Content
Date: Tue, 20 Feb 2018 02:38:48 GMT

The path reference of the first two operations may look confusing a bit but fear no more, let us clarify that. Because authors is a collection (or in terms of JSON data types, an array) we could use RFC-6902: JSON Patch array index notation to specify exactly where we would like the new element to be inserted. The first operations uses index '0' to denote the head position, while the second one uses '-' placeholder to simplify say "add to the end of the collection". If we retrieve the book right after the update, we should see our modifications to be applied exactly as we asked.

$ curl http://localhost:19091/services/catalog/978-1491956250

{
    "title": "Microservice Architecture: Aligning Principles, Practices, and Culture",
    "isbn": "978-1491956250",
    "authors": [
        "Irakli Nadareishvili",
        "Ronnie Mitra",
        "Matt McLarty",
        "Mike Amundsen"
    ]
}

Clean, simple and powerful. To be fair, there is a price to pay is a form of additional JSON manipulations (in order to apply the patch) but is it worth the effort? I believe it is ...

Next time you are going to design new shiny REST(ful) web APIs, please seriously consider RFC-6902: JSON Patch to back the PATCH implementation of your resources. I believe more close integration with JAX-RS is also coming (if not there yet) to directly support JSONPatch class and its family.

And last, but not least, in this post we have touched on server-side implementation only, but JSR 374: Java API for JSON Processing 1.1 includes convenient client-side scaffolding as well, giving full-fledged programmatic control over the patches.

final JsonPatch patch = Json.createPatchBuilder()
    .add("/authors/0", "Irakli Nadareishvili")
    .add("/authors/-", "Mike Amundsen")
    .replace("/title", "Microservice Architecture: Aligning Principles, Practices, and Culture")
    .build();

The complete project sources are available on Github.

Better late than never: SSE, or Server-Sent Events, are now in JAX-RS

Server-Sent Events (or just SSE) is quite useful protocol which allows the servers to push data to the clients over HTTP. This is something our web browsers support for ages but, surprisingly, neglected by JAX-RS specification for quite a long time. Although Jersey had an extension available for SSE media type, the API has never been formalized and as such, was not portable to other JAX-RS implementations.

Luckily, JAX-RS 2.1, also known as JSR-370, has changed that by making SSE support, both client-side and server-side, a part of the official specification. In today's post we are going to look at how to integrate SSE support into the existing Java REST(ful) web services, using recently released version 3.2.0 of the terrific Apache CXF framework. In fact, beside the bootstrapping, there is nothing CXF-specific really, all the examples should work in any other framework which implements JAX-RS 2.1 specification.

Without further ado, let us get started. As the significant amount of Java projects these days are built on top of awesome Spring Framework, our sample application would use Spring Boot and Apache CXF Spring Boot Integration to get us off the ground quickly. The old good buddy Apache Maven would help us as well by managing our project dependencies.

    org.springframework.boot
    spring-boot-starter
    1.5.8.RELEASE



    org.apache.cxf
    cxf-rt-frontend-jaxrs
    3.2.0



    org.apache.cxf
    cxf-spring-boot-starter-jaxrs
    3.2.0



    org.apache.cxf
    cxf-rt-rs-client
    3.2.0



     org.apache.cxf
     cxf-rt-rs-sse
     3.2.0

Under the hood Apache CXF is using Atmosphere framework to implement SSE transport so this is another dependency we have to include.

    org.atmosphere
    atmosphere-runtime
    2.4.14

The specifics around relying on Atmosphere framework introduces a need to provide additional configuration settings, namely transportId, so to ensure that SSE-capable transport will be picked up at runtime. The relevant details could be added into application.yml file:

cxf:
  servlet:
    init:
      transportId: http://cxf.apache.org/transports/http/sse

Great, so the foundation is there, moving on. The REST(ful) web service we are going to build would expose imaginary CPU load averages (for simplicity randomly generated) as the SSE streams. The Stats class would constitute our data model.

public class Stats {
    private long timestamp;
    private int load;

    public Stats() {
    }

    public Stats(long timestamp, int load) {
        this.timestamp = timestamp;
        this.load = load;
    }

    // Getters and setters are omitted
    ...
}

Speaking of streams, the Reactive Streams specification made its way into Java 9 and hopefully we are going to see the accelerated adoption of the reactive programming models by Java community. Moreover, developing SSE-enabled REST(ful) web services would be so much easier and straightforward when backed by Reactive Streams. To make the case, let us onboard RxJava 2 into our sample application.

    io.reactivex.rxjava2
    rxjava
    2.1.6

This is a good moment to start with our StatsRestService class, the typical JAX-RS resource implementation. The key SSE capabilities in JAX-RS 2.1 are centered around Sse contextual object which could be inject like this.

@Service
@Path("/api/stats")
public class StatsRestService {
    @Context 
    public void setSse(Sse sse) {
        // Access Sse context here
    }

Out of the Sse context we could get access to two very useful abstractions: SseBroadcaster and OutboundSseEvent.Builder, for example:

private SseBroadcaster broadcaster;
private Builder builder;
    
@Context 
public void setSse(Sse sse) {
    this.broadcaster = sse.newBroadcaster();
    this.builder = sse.newEventBuilder();
}

As you may already guess, the OutboundSseEvent.Builder constructs the instances of the OutboundSseEvent classes which could be sent over the wire, while SseBroadcaster broadcasts the same SSE stream to all the connected clients. With that being said, we could generate the stream of OutboundSseEvents and distribute it to everyone who is interested:

private static void subscribe(final SseBroadcaster broadcaster, final Builder builder) {
    Flowable
        .interval(1, TimeUnit.SECONDS)
        .zipWith(eventsStream(builder), (id, bldr) -> createSseEvent(bldr, id))
        .subscribeOn(Schedulers.single())
        .subscribe(broadcaster::broadcast);
}

private static Flowable<OutboundSseEvent.Builder> eventsStream(final Builder builder) {
    return Flowable.generate(emitter -> emitter.onNext(builder.name("stats")));
}

If you are not familiar with RxJava 2, no worries, this is what is happening here. The eventsStream method returns an effectively infinite stream of OutboundSseEvent.Builder instances for the SSE events of type stats. The subscribe method is a little bit more complicated. We start off by creating a stream which emits sequential number every second, f.e. 0,1,2,3,4,5,6,... and so on. Later, we combine this stream with the one returned by eventsStream method, essentially merging both streams to a single one which emits a tuple of (number, OutboundSseEvent.Builder) every second. Fairly speaking, this tuple is not very useful to us so we transform it to the instance of OutboundSseEvent class, treating the number as SSE event identifier:

private static final Random RANDOM = new Random();

private static OutboundSseEvent createSseEvent(OutboundSseEvent.Builder builder, long id) {
    return builder
        .id(Long.toString(id))
        .data(Stats.class, new Stats(new Date().getTime(), RANDOM.nextInt(100)))
        .mediaType(MediaType.APPLICATION_JSON_TYPE)
        .build();
}

The OutboundSseEvent may carry any payload in the data property which will be serialized with respect to the mediaType specified, using the usual MessageBodyWriter resolution strategy. Once we get our OutboundSseEvent instance, we send it off using SseBroadcaster::broadcast method. Please notice that we handed off the control flow to another thread using subscribeOn operator, this is usually what you would do all the time.

Good, hopefully the stream part is cleared out now but how could we actually subscribe to the SSE events emitted by SseBroadcaster? That is easier than you might think:

@GET
@Path("broadcast")
@Produces(MediaType.SERVER_SENT_EVENTS)
public void broadcast(@Context SseEventSink sink) {
    broadcaster.register(sink);
}

And we are all set. The most important piece here is content type being produced, which should be set to MediaType.SERVER_SENT_EVENTS. In this case, the contextual instance of the SseEventSink becomes available and could be registered with SseBroadcaster instance.

To see our JAX-RS resource in action, we need to bootstrap the server instance using, for example, JAXRSServerFactoryBean, configuring all the necessary providers along the way. Please take a note that we are also explicitly specifying transport to be used, in this case SseHttpTransportFactory.TRANSPORT_ID.

@Configuration
@EnableWebMvc
public class AppConfig extends WebMvcConfigurerAdapter {
    @Bean
    public Server rsServer(Bus bus, StatsRestService service) {
        JAXRSServerFactoryBean endpoint = new JAXRSServerFactoryBean();
        endpoint.setBus(bus);
        endpoint.setAddress("/");
        endpoint.setServiceBean(service);
        endpoint.setTransportId(SseHttpTransportFactory.TRANSPORT_ID);
        endpoint.setProvider(new JacksonJsonProvider());
        return endpoint.create();
    }
    
    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {
        registry
          .addResourceHandler("/static/**")
          .addResourceLocations("classpath:/web-ui/"); 
    }
}

To close the loop, we just need to supply the runner for our Spring Boot application:

@SpringBootApplication
public class SseServerStarter {    
    public static void main(String[] args) {
        SpringApplication.run(SseServerStarter.class, args);
    }
}

Now, if we run the application and navigate to http://localhost:8080/static/broadcast.html using multiple web browsers or different tabs within the same browser, we would observe the identical stream of events charted inside all of them:

Nice, broadcasting is certainly a valid use case, but what about returning an independent SSE stream on each endpoint invocation? Easy, just use SseEventSink methods, like send and close, to manipulate the SSE stream directly.

@GET
@Path("sse")
@Produces(MediaType.SERVER_SENT_EVENTS)
public void stats(@Context SseEventSink sink) {
    Flowable
        .interval(1, TimeUnit.SECONDS)
        .zipWith(eventsStream(builder), (id, bldr) -> createSseEvent(bldr, id))
        .subscribeOn(Schedulers.single())
        .subscribe(sink::send, ex -> {}, sink::close);
}

This time, if we run the application and navigate to http://localhost:8080/static/index.html using multiple web browsers or different tabs within the same browser, we would observe absolutely different charts:

Excellent, the server-side APIs are indeed very concise and easy to use. But what about client side, could we consume SSE streams from the Java applications? The answer is yes, absolutely. The JAX-RS 2.1 outlines the client-side API as well, with SseEventSource in the heart of it.

final WebTarget target = ClientBuilder
    .newClient()
    .register(JacksonJsonProvider.class)
    .target("http://localhost:8080/services/api/stats/sse");
        
try (final SseEventSource eventSource =
            SseEventSource
                .target(target)
                .reconnectingEvery(5, TimeUnit.SECONDS)
                .build()) {

    eventSource.register(event -> {
        final Stats stats = event.readData(Stats.class, MediaType.APPLICATION_JSON_TYPE);
        System.out.println("name: " + event.getName());
        System.out.println("id: " + event.getId());
        System.out.println("comment: " + event.getComment());
        System.out.println("data: " + stats.getLoad() + ", " + stats.getTimestamp());
        System.out.println("---------------");
    });
    eventSource.open();

    // Just consume SSE events for 10 seconds
    Thread.sleep(10000); 
}     

If we run this code snippet (assuming the server is up and running as well) we would see something like that in the console (as you may recall, the data is generated randomly).

name: stats
id: 0
comment: null
data: 82, 1509376080027
---------------
name: stats
id: 1
comment: null
data: 68, 1509376081033
---------------
name: stats
id: 2
comment: null
data: 12, 1509376082028
---------------
name: stats
id: 3
comment: null
data: 5, 1509376083028
---------------

...

As we can see, the OutboundSseEvent from server-side becomes InboundSseEvent for the client side. The client may consume any payload from the data property which could be deserialized by specifying expected media type, using the usual MessageBodyReader resolution strategy.

There are a lot of material squeezed in the single post. And still, there are few more things regarding SSE and JAX-RS 2.1 which we have not covered here, like for example using HttpHeaders.LAST_EVENT_ID_HEADER or configuring reconnect delays. Those could be a great topic for the upcoming post if there would be an interest to learn about.

To conclude, the SSE support in JAX-RS is what many of us have been awaiting for so long. Finally, it is the there, please give it a try!

The complete project sources are available on Github.