{"id":8446,"date":"2013-02-12T13:00:54","date_gmt":"2013-02-12T11:00:54","guid":{"rendered":"http:\/\/www.javacodegeeks.com\/?p=8446"},"modified":"2013-02-11T22:38:55","modified_gmt":"2013-02-11T20:38:55","slug":"groovy-and-http","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2013\/02\/groovy-and-http.html","title":{"rendered":"Groovy and HTTP"},"content":{"rendered":"

This article originally appeared in the December 2012 issue of GroovyMag<\/a>.<\/em><\/p>\n

Some different ways that Groovy makes interacting with the web easier<\/h2>\n

One of the major benefits of Groovy is how it simplifies some of the common scenarios we deal with in Java. Complex code with conditionals, error handling and many other concerns can be expressed in a very concise and easily understandable fashion. This article will touch on some convenient Groovy-isms related to interacting with content over HTTP. First we\u2019ll look at some of the syntactic sugar added to the standard Java classes that simplify GET and POST requests, and then we\u2019ll take a look at how the
\n 
\nHTTPBuilder<\/a> module provides a DSL for using the HttpClient<\/a> library.<\/p>\n

The test project<\/h2>\n

In order to provide an environment for putting up a website and demonstrating various HTTP requests, we\u2019ll be using the Gradle Jetty plugin and some simple Groovlets<\/a>. The full source code is available at https:\/\/github.com\/kellyrob99\/groovy-http<\/a> and I hope you\u2019ll clone a copy to take a closer look. The simple index page contains the \u2018hello world\u2019 content shown in Listing 1.<\/p>\n

<!DOCTYPE html>\r\n<html>\r\n<head>\r\n    <title>Groovy HTTP<\/title>\r\n<\/head>\r\n<body>\r\n<p>hello world<\/p>\r\n<\/body>\r\n<\/html><\/pre>\n
Listing 1: Our \u2018hello world\u2019 index page used for testing<\/h4>\n
We\u2019ll start with the simplest available methods for interacting with HTTP using Groovy and no additional library support.<\/p>\n
Groovy methods added to String and URL<\/h2>\n
The DefaultGroovyMethods<\/a> class provides a couple of very handy methods to enhance the default operation of the String and URL classes. In particular for String we have a new toURL() method and, for URL, the text property. In addition, the URL class is enhanced with convenience methods for working with associated InputStream and OutputStreams.<\/p>\n
String.toURL()<\/h3>\n
This is a small gain as all you\u2019re really doing is avoiding a call to new URL(String spec)<\/a>. The difference in keystrokes isn\u2019t large but, combined with some other MetaClass benefits of Groovy, it can be very helpful for creating fluent and easily understandable code.<\/p>\n
URL.text()<\/h3>\n
This seemingly small addition to the API of the URL class abstracts away a lot of the usual boilerplate involved in streaming content over a URLConnection<\/a>. Underneath the hood is a very sensible implementation that buffers the underlying connection and automatically handles the closing of all resources for you. For most use cases the default behaviour is likely to be sufficient but, if not, there are overloaded URL.text(String charset) and URL.text(Map parameters, String charset) methods that allow for modification and handle more specifics of the connection configuration.
\nThe one line invocation in Listing 2 demonstrates how to load an html page, returning the raw html as a String.<\/p>\n
String html = 'http:\/\/localhost:8081\/groovy-http'.toURL().text<\/pre>\n
Listing 2: One liner to initiate an HTTP GET request for an html page<\/h4>\n
There\u2019s still a lot that could go wrong using this shorthand syntax for an HTTP request, as several exceptions might be thrown depending on whether or not the url is correctly formatted, or if the content specified doesn\u2019t exist. The Spock<\/a> test shown in Listing 3 exercises both of these conditions. Note that a 404 response will result in a FileNotFoundException.<\/p>\n
@Unroll('The url #url should throw an exception of type #exception')\r\ndef 'exceptions can be thrown converting a String to URL and accessing the text'() {\r\n    when:\r\n    String html = url.toURL().text\r\n\r\n    then:\r\n    def e = thrown(exception)\r\n\r\n    where:\r\n    url                          | exception\r\n    'htp:\/\/foo.com'              | MalformedURLException\r\n    'http:\/\/google.com\/notThere' | FileNotFoundException\r\n}<\/pre>\n
Listing 3: Spock test showing some possible failure conditions for our GET request<\/h4>\n
For comparison let\u2019s take a look at what the same GET request looks like using a URL in Java, shown in Listing 4.
 
<\/div> <\/div><\/p>\n
URL html = new URL('http:\/\/localhost:8081\/groovy-http\/index.html');\r\nURLConnection urlConnection = html.openConnection();\r\nBufferedReader reader = new BufferedReader(\r\n\tnew InputStreamReader(urlConnection.getInputStream()));\r\nStringBuffer response = new StringBuffer();\r\nString inputLine;\r\nwhile ((inputLine = reader.readLine()) != null)\r\n{\r\n\tresponse.append(inputLine)\r\n}\r\nreader.close();<\/pre>\n
Listing 4: The Java version of reading from a URLConnection (based on the canonical example from Oracle.com)<\/h4>\n
There\u2019s still no error handling in place in the Java version which is obviously a much more verbose way to load the same data.<\/p>\n
POST with URL streams<\/h2>\n
Similarly to simplifying GET requests, executing a POST using Groovy can take advantage of some of the enhancements to common Java classes. In particular, simplified stream handling allows for tight, correct and expressive coding. Listing 5 shows a Spock test configuring the URLConnection, POSTing some data and reading back the result from the connection.<\/p>\n
private static final String POST_RESPONSE = 'Successfully posted [arg:[foo]] with method POST'    \r\n\r\ndef 'POST from a URLConnection'() {\r\n    when:\r\n    final HttpURLConnection connection = makeURL('post.groovy').toURL().openConnection()\r\n    connection.setDoOutput(true)\r\n    connection.outputStream.withWriter { Writer writer ->\r\n        writer << 'arg=foo'\r\n    }\r\n\r\n    String response = connection.inputStream.withReader { Reader reader -> reader.text }\r\n\r\n    then:\r\n    connection.responseCode == HttpServletResponse.SC_OK\r\n    response == POST_RESPONSE\r\n}<\/pre>\n
Listing 5: POST request using Groovy and a URLConnection<\/h4>\n
Notice that we don\u2019t have to explicitly cast the connection to HttpUrlConnection<\/a> in order to get the responseCode back, and that we don\u2019t have to explicitly close any of the streams used. Also, we don\u2019t need to create local variables for the Reader\/Writer object as we would have to in Java; similarly no calls to \u2018new\u2019 are required, as Object creation is all hidden behind the convenience methods. The equivalent Java code requires four calls to new and two to close(), as well as much more involved code for extracting the result. The canonical example of how to do this in Java can be seen on http:\/\/docs.oracle.com\/javase\/tutorial\/networking\/urls\/readingWriting.html
\n<\/a>
\nNote that you can also parse response content very easily using the XmlSlurper<\/a> \/ XmlParser<\/a> and JsonSlurper<\/a> classes included in the standard Groovy distribution.<\/p>\n
HttpClient and HTTPBuilder make things even easier<\/h2>\n
The reality is that in most modern Java applications developers have some nice alternatives to directly working with URL and URLConnection objects for working with HTTP. One of the more popular libraries available is HttpClient and its successor HttpComponents<\/a>. Wrappers for all of the HTTP verbs are provided which simplifies configuration, execution and consumption of responses. Listing 6 shows a Spock test using HttpClient and mirroring our prior GET examples.<\/p>\n
def 'HttpClient example in Java'() {\r\n    when:\r\n    HttpClient httpclient = new DefaultHttpClient();\r\n    HttpGet httpget = new HttpGet(makeURL('helloWorld.groovy'));\r\n    ResponseHandler<String> responseHandler = new BasicResponseHandler();\r\n    String responseBody = httpclient.execute(httpget, responseHandler);\r\n\r\n    then:\r\n    responseBody == HELLO_WORLD_HTML\r\n}<\/pre>\n
Listing 6: HttpClient GET example<\/h4>\n
This can be further reduced if there is no need for keeping the intermediate variables around. In fact, we can get it down to the single line shown in Listing 7.<\/p>\n
String response = new DefaultHttpClient().execute(new HttpGet(makeURL('helloWorld.groovy')), new BasicResponseHandler())<\/pre>\n
Listing 7: HttpClient GET one-liner<\/h4>\n
This is obviously a lot easier on the eyes and very clear in intent. The HttpClient library also has convenience mechanisms for declaring common behaviour across connections, an API for providing custom response parsing implementations and automatic handling for (most of) the underlying resource streams and connections. For those of us using Groovy, there\u2019s a nice wrapper for HttpClient called HTTPBuilder that adds a DSL-style configuration mechanism and some very nice features in terms of error handling and content parsing. Listing 8 shows our standard GET example again, this time working against an object called http assigned from new HTTPBuilder(Object uri)<\/em>. Note that we\u2019re using Groovy\u2019s multiple assignment feature to return and assign multiple values from our Closure.<\/p>\n
def 'GET with HTTPBuilder'() {\r\n    when:\r\n    def (html, responseStatus) = http.get(path: 'helloWorld.groovy', contentType: TEXT) { resp, reader ->\r\n        [reader.text, resp.status]\r\n    }\r\n\r\n    then:\r\n    responseStatus == HttpServletResponse.SC_OK\r\n    html == HELLO_WORLD_HTML\r\n}<\/pre>\n
Listing 8: Spock test showing Groovy HTTPBuilder GET support<\/h4>\n
If you noticed in Listing 8 I explicitly set the request with contentType: TEXT<\/em>, it\u2019s because HTTPBuilder by default provides automatic response content type detection and parsing. Since I\u2019m requesting an xml document, HTTPBuilder can automatically parse the result with Groovy\u2019s XmlSlurper. HTTPBuilder can also detect that it is an html page and pass the response through NekoHTML first to ensure that you\u2019re working with a well-formed document. Listing 9 shows the slight difference in how we could interact with the parsed response content and the reader in our Closure from Listing 8 is quietly replaced with a GPathResult referring to the parsed content.<\/p>\n
def 'GET with HTTPBuilder and automatic parsing'() {\r\n    when:\r\n    def (html, responseStatus) = http.get(path: 'helloWorld.groovy') { resp, reader ->\r\n        [reader, resp.status]\r\n    }\r\n\r\n    then:\r\n    responseStatus == HttpServletResponse.SC_OK\r\n    html instanceof GPathResult\r\n    html.BODY.P.text() == 'hello world'\r\n}<\/pre>\n
Listing 9: automatic detection and parsing of xml <\/h4>\n
It\u2019s unlikely that you\u2019re going to be parsing a lot of html this way but with the abundance of xml services available nowadays automated parsing can be very helpful. The same applies for JSON and if we give a hint as to the contentType we can get back a parsed JSONObject when interacting with such services as shown in Listing 10.<\/p>\n
def 'GET with HTTPBuilder and automatic JSON parsing'() {\r\n    when:\r\n    def (json, responseStatus) = http.get(path: 'indexJson.groovy', contentType: JSON) { resp, reader ->\r\n        [reader, resp.status]\r\n    }\r\n\r\n    then:\r\n    responseStatus == HttpServletResponse.SC_OK\r\n    json instanceof JSONObject\r\n    json.html.body.p == 'hello world'\r\n}<\/pre>\n
Listing 10: automatic parsing of JSON responses<\/h4>\n
The HTTPBuilder module also has some convenience methods for handling failure conditions. By allowing for specifying both default failure handlers and specific behaviour for individual requests you\u2019ve got lots of options at your disposal. Listing 11 shows how to define a default failure handler that simply traps the response code. Note that the Closure used for hading GET response is never run since in this case the page we\u2019re requesting results in an HTTP 404 Not Found response code.<\/p>\n
def 'GET with HTTPBuilder and error handling'() {\r\n    when:\r\n    int responseStatus\r\n    http.handler.failure = { resp ->\r\n        responseStatus = resp.status\r\n    }\r\n    http.get(path: 'notThere.groovy', contentType: TEXT) { resp, reader ->\r\n        throw new IllegalStateException('should not be executed')\r\n    }\r\n\r\n    then:\r\n    responseStatus == HttpServletResponse.SC_NOT_FOUND\r\n}<\/pre>\n
Listing 11: Defining a failure handler with HTTPBuilder<\/h4>\n
POSTing data with HTTPBuilder is also very straightforward, requiring only an additional body parameter as shown in Listing 12.<\/p>\n
def 'POST with HTTPBuilder'() {\r\n    when:\r\n    def (response, responseStatus) = http.post(path: 'post.groovy', body: [arg: 'foo']) { resp, reader ->\r\n        [reader.text(),resp.status]\r\n    }\r\n\r\n    then:\r\n    responseStatus == HttpServletResponse.SC_OK\r\n    response == POST_RESPONSE\r\n}<\/pre>\n
Listing 12: POST using HTTPBuilder<\/h4>\n
HTTPBuilder also provides some more specific abstractions for dealing with certain scenarios. There\u2019s RESTClient for dealing with RESTful webservices in a simplified manner, there\u2019s AsyncHTTPBuilder for asynchronously executing requests and for the Google App Engine, which doesn\u2019t allow socket based connections, there\u2019s the HttpURLClient which wraps HttpUrlConnection usage.<\/p>\n
Conclusion<\/h2>\n
Hopefully this has given you a taste for what Groovy can do to help you with HTTP interactions and gives you some ideas for making your own HTTP client applications a bit Groovier.<\/p>\n
More reading<\/h2>\n