Showing posts with label NoSQL. Show all posts
Showing posts with label NoSQL. Show all posts

Monday, June 25, 2012

Using Redis with Spring

As NoSQL solutions are getting more and more popular for many kind of problems, more often the modern projects consider to use some (or several) of NoSQLs instead (or side-by-side) of traditional RDBMS. I have already covered my experience with MongoDB in this, this and this posts. In this post I would like to switch gears a bit towards Redis, an advanced key-value store.

Aside from very rich key-value semantics, Redis also supports pub-sub messaging and transactions. In this post I am going just to touch the surface and demonstrate how simple it is to integrate Redis into your Spring application.

As always, we will start with Maven POM file for our project:


 4.0.0
 com.example.spring
 redis
 0.0.1-SNAPSHOT
 jar

 
  UTF-8
  3.1.0.RELEASE
 

 
  
   org.springframework.data
   spring-data-redis
   1.0.0.RELEASE
  

  
   cglib
   cglib-nodep
   2.2
  

  
   log4j
   log4j
   1.2.16
  

  
   redis.clients
   jedis
   2.0.0
   jar
  

  
   org.springframework
   spring-core
   ${spring.version}
  

  
   org.springframework
   spring-context
   ${spring.version}
  
 

Spring Data Redis is the another project under Spring Data umbrella which provides seamless injection of Redis into your application. The are several Redis clients for Java and I have chosen the Jedis as it is stable and recommended by Redis team at the moment of writing this post.

We will start with simple configuration and introduce the necessary components first. Then as we move forward, the configuration will be extended a bit to demonstrated pub-sub capabilities. Thanks to Java config support, we will create the configuration class and have all our dependencies strongly typed, no XML anymore:

package com.example.redis.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.GenericToStringSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

@Configuration
public class AppConfig {
 @Bean
 JedisConnectionFactory jedisConnectionFactory() {
  return new JedisConnectionFactory();
 }

 @Bean
 RedisTemplate< String, Object > redisTemplate() {
  final RedisTemplate< String, Object > template =  new RedisTemplate< String, Object >();
  template.setConnectionFactory( jedisConnectionFactory() );
  template.setKeySerializer( new StringRedisSerializer() );
  template.setHashValueSerializer( new GenericToStringSerializer< Object >( Object.class ) );
  template.setValueSerializer( new GenericToStringSerializer< Object >( Object.class ) );
  return template;
 }
}
That's basically everything we need assuming we have single Redis server up and running on localhost with default configuration. Let's consider several common uses cases: setting a key to some value, storing the object and, finally, pub-sub implementation. Storing and retrieving a key/value pair is very simple:
@Autowired private RedisTemplate< String, Object > template;

public Object getValue( final String key ) {
    return template.opsForValue().get( key );
}

public void setValue( final String key, final String value ) {
    template.opsForValue().set( key, value );
}
Optionally, the key could be set to expire (yet another useful feature of Redis), f.e. let our keys expire in 1 second:
public void setValue( final String key, final String value ) {
    template.opsForValue().set( key, value );
    template.expire( key, 1, TimeUnit.SECONDS );
}
Arbitrary objects could be saved into Redis as hashes (maps), f.e. let save instance of some class User
public class User {
 private final Long id;
 private String name;
 private String email;
       
    // Setters and getters are omitted for simplicity
}
into Redis using key pattern "user:<id>":
public void setUser( final User user ) {
 final String key = String.format( "user:%s", user.getId() );
 final Map< String, Object > properties = new HashMap< String, Object >();

 properties.put( "id", user.getId() );
 properties.put( "name", user.getName() );
 properties.put( "email", user.getEmail() );

 template.opsForHash().putAll( key, properties);
}
Respectively, object could easily be inspected and retrieved using the id.
public User getUser( final Long id ) {
 final String key = String.format( "user:%s", id );

 final String name = ( String )template.opsForHash().get( key, "name" );
 final String email = ( String )template.opsForHash().get( key, "email" );

 return new User( id, name, email );
}
There are much, much more which could be done using Redis, I highly encourage to take a look on it. It surely is not a silver bullet but could solve many challenging problems very easy. Finally, let me show how to use a pub-sub messaging with Redis. Let's add a bit more configuration here (as part of AppConfig class):
@Bean
MessageListenerAdapter messageListener() {
 return new MessageListenerAdapter( new RedisMessageListener() );
}

@Bean
RedisMessageListenerContainer redisContainer() {
 final RedisMessageListenerContainer container = new RedisMessageListenerContainer();

 container.setConnectionFactory( jedisConnectionFactory() );
 container.addMessageListener( messageListener(), new ChannelTopic( "my-queue" ) );

 return container;
}
The style of message listener definition should look very familiar to Spring users: generally, the same approach we follow to define JMS message listeners. The missed piece is our RedisMessageListener class definition:
package com.example.redis.impl;

import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.connection.MessageListener;

public class RedisMessageListener implements MessageListener {
 @Override
 public void onMessage(Message message, byte[] paramArrayOfByte) {
  System.out.println( "Received by RedisMessageListener: " + message.toString() );
 }
}
Now, when we have our message listener, let see how we could push some messages into the queue using Redis. As always, it's pretty simple:
@Autowired private RedisTemplate< String, Object > template;

public void publish( final String message ) {
 template.execute(
  new RedisCallback< Long >() {
   @SuppressWarnings( "unchecked" )
   @Override
   public Long doInRedis( RedisConnection connection ) throws DataAccessException {
    return connection.publish(
     ( ( RedisSerializer< String > )template.getKeySerializer() ).serialize( "queue" ),
     ( ( RedisSerializer< Object > )template.getValueSerializer() ).serialize( message ) );
   }
  }
 );
}
That's basically it for very quick introduction but definitely enough to fall in love with Redis.

Wednesday, January 25, 2012

Storing hierarchical data in MongoDB

Continuing NoSQL journey with MongoDB, I would like to touch one specific use case which comes up very often: storing hierarchical document relations. MongoDB is awesome document data store but what if documents have parent-child relationships? Can we effectively store and query such document hierarchies? The answer, for sure, is yes, we can. MongoDB has several recommendations how to store Trees in MongoDB. The one solution described there as well and quite widely used is using materialized path.

Let me explain how it works by providing very simple examples. As in previous posts, we will build Spring application using recently released version 1.0 of Spring Data MongoDB project. Our POM file contains very basic dependencies, nothing more.


    4.0.0

    mongodb
    com.example.spring
    0.0.1-SNAPSHOT
    jar

    
        UTF-8
        3.0.7.RELEASE
    

    
        
            org.springframework.data
            spring-data-mongodb
            1.0.0.RELEASE
            
                
                    org.springframework
                    spring-beans
                
                
                    org.springframework
                    spring-expression
                
            
        

        
            cglib
            cglib-nodep
            2.2
        

        
            log4j
            log4j
            1.2.16
        

        
            org.mongodb
            mongo-java-driver
            2.7.2
        

        
            org.springframework
            spring-core
            ${spring.version}
        

        
            org.springframework
            spring-context
            ${spring.version}
        

        
            org.springframework
            spring-context-support
            ${spring.version}
        
    

    
        
            
                org.apache.maven.plugins
                maven-compiler-plugin
                2.3.2
                
                    1.6
                    1.6
                
            
        
    

To properly configure Spring context, I will use configuration approach utilizing Java classes. I am more and more advocating to use this style as it provides strong typed configuration and most of the mistakes could be caught on compilation time, no need to inspect your XML files anymore. Here how it looks like:

package com.example.mongodb.hierarchical;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.core.MongoFactoryBean;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.SimpleMongoDbFactory;

@Configuration
public class AppConfig {
    @Bean
    public MongoFactoryBean mongo() {
        final MongoFactoryBean factory = new MongoFactoryBean();
        factory.setHost( "localhost" );
        return factory;
    }

    @Bean
    public SimpleMongoDbFactory mongoDbFactory() throws Exception{
        return new SimpleMongoDbFactory( mongo().getObject(), "hierarchical" );
    }

    @Bean
    public MongoTemplate mongoTemplate() throws Exception {
        return new MongoTemplate( mongoDbFactory() );
    }

    @Bean
    public IDocumentHierarchyService documentHierarchyService() throws Exception {
        return new DocumentHierarchyService( mongoTemplate() );
    }
}

That's pretty nice and clear. Thanks, Spring guys! Now, all boilerplate stuff is ready. Let's move to interesting part: documents. Our database will contain 'documents' collection which stores documents of type SimpleDocument. We describe this using Spring Data MongoDB annotations for SimpleDocument POJO.

package com.example.mongodb.hierarchical;

import java.util.Collection;
import java.util.HashSet;

import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.Transient;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.data.mongodb.core.mapping.Field;

@Document( collection = "documents" )
public class SimpleDocument {
    public static final String PATH_SEPARATOR = ".";

    @Id private String id;
    @Field private String name;
    @Field private String path;

    // We won't store this collection as part of document but will build it on demand
    @Transient private Collection< SimpleDocument > documents = new HashSet< SimpleDocument >();

    public SimpleDocument() {
    }

    public SimpleDocument( final String id, final String name ) {
        this.id = id;
        this.name = name;
        this.path = id;
    }

    public SimpleDocument( final String id, final String name, final SimpleDocument parent ) {
        this( id, name );
        this.path = parent.getPath() + PATH_SEPARATOR + id;
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getPath() {
        return path;
    }

    public void setPath(String path) {
        this.path = path;
    }

    public Collection< SimpleDocument > getDocuments() {
        return documents;
    }
}

Let me explain few things here. First, magic property path: this is a key to construct and query through our hierarchy. Path contains identifiers of all document's parents, usually divided by some kind of separator, in our case just . (dot). Storing document hierarchical relationships in this way allows quickly build hierarchy, search and navigate. Second, notice transient documents collection: this non-persistent collection is constructed by persistent provider and contains all descendant documents (which, in case, also contain own descendants). Let see it in action by looking into find method implementation:

package com.example.mongodb.hierarchical;

import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;

import org.springframework.data.mongodb.core.MongoOperations;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;

public class DocumentHierarchyService {
    private MongoOperations template;

    public DocumentHierarchyService( final MongoOperations template ) {
        this.template = template;
    }

    @Override
    public SimpleDocument find( final String id ) {
        final SimpleDocument document = template.findOne(
            Query.query( new Criteria( "id" ).is( id ) ),
            SimpleDocument.class
        );

        if( document == null ) {
            return document;
        }

        return build(
            document,
            template.find(
                Query.query( new Criteria( "path" ).regex( "^" + id + "[.]" ) ),
                SimpleDocument.class
            )
        );
    }

    private SimpleDocument build( final SimpleDocument root, final Collection< SimpleDocument > documents ) {
        final Map< String, SimpleDocument > map = new HashMap< String, SimpleDocument >();

        for( final SimpleDocument document: documents ) {
            map.put( document.getPath(), document );
        }

        for( final SimpleDocument document: documents ) {
            map.put( document.getPath(), document );

            final String path = document
                .getPath()
                .substring( 0, document.getPath().lastIndexOf( SimpleDocument.PATH_SEPARATOR ) );

            if( path.equals( root.getPath() ) ) {
                root.getDocuments().add( document );
            } else {
                final SimpleDocument parent = map.get( path );
                if( parent != null ) {
                    parent.getDocuments().add( document );
                }
            }
        }

        return root;
    }
}

As you can see, to get single document with a whole hierarchy we need to run just two queries (but more optimal algorithm could reduce it to just one single query). Here is a sample hierarchy and the the result of reading root document from MongoDB


template.dropCollection( SimpleDocument.class );

final SimpleDocument parent = new SimpleDocument( "1", "Parent 1" );
final SimpleDocument child1 = new SimpleDocument( "2", "Child 1.1", parent );
final SimpleDocument child11 = new SimpleDocument( "3", "Child 1.1.1", child1 );
final SimpleDocument child12 = new SimpleDocument( "4", "Child 1.1.2", child1 );
final SimpleDocument child121 = new SimpleDocument( "5", "Child 1.1.2.1", child12 );
final SimpleDocument child13 = new SimpleDocument( "6", "Child 1.1.3", child1 );
final SimpleDocument child2 = new SimpleDocument( "7", "Child 1.2", parent );

template.insertAll( Arrays.asList( parent, child1, child11, child12, child121, child13, child2 ) );

...

final ApplicationContext context = new AnnotationConfigApplicationContext( AppConfig.class );
final IDocumentHierarchyService service = context.getBean( IDocumentHierarchyService.class );

final SimpleDocument document = service.find( "1" );
//  Printing document show following hierarchy:
//
//  Parent 1
//   |-- Child 1.1
//     |-- Child 1.1.1
//     |-- Child 1.1.3
//     |-- Child 1.1.2
//       |-- Child 1.1.2.1
//   |-- Child 1.2

That's it. Simple a powerful concept. Sure, adding index on a path property will speed up query significantly. There are a plenty of improvements and optimizations but basic idea should be clear now.