Showing posts with label configuration. Show all posts
Showing posts with label configuration. Show all posts

Tuesday, 5 March 2019

Enhanced JGroups configuration

Infinispan uses JGroups as its underlying clustering layer. In order to configure the finer details of clustering (discovery, flow control, cross-site, etc) you have to provide a separate XML file with the desired configuration and reference it from your Infinispan XML file as follows:


For simple configurations this is usually fine, but configuring complex setups, such as cross-site replication, means juggling multiple files (one for the local stack, one for the cross-site stack and one for the relay configuration).

Starting with Infinispan 10 Alpha2 we have introduced a number of changes to make your life with JGroups configurations a lot easier.

Default stacks

Infinispan now comes with two pre-declared stacks: tcp and udp. Using them is as simple as just referencing their names in the <transport> element.

Inline stacks

Inlining a stack means you can put the JGroups configuration inside the Infinispan one as follows:

You can use the full JGroups schema, and by using XML namespaces you get full validation.

Stack inheritance

Most of the time you want to reuse one of the pre-declared stacks but just override some of the parameters (e.g. discovery) to suit your environment. The following example creates a new tcpgossip stack which is based on the default tcp stack but replaces the discovery protocol with TCPGOSSIP:


In the above example you can see that we have enhanced the JGroups protocol declarations with two new attributes: ispn:stack.combine and ispn:stack.position which affect how and where protocol changes are applied on the parent configuration to obtain a new configuration. stack.combine can be one of COMBINE (the default, possibly overriding any specified attributes), REPLACE (which completely replaces the protocol and resets all attributes), REMOVE (removes the protocol) and INSERT_AFTER (which places this protocol in the stack immediately after the protocol specified by stack.position).

Multiple stacks and Cross-site

The inline configuration really shows its usefulness in cross-site configurations. In fact, the JGroups stack declaration has been extended with a special element which replaces the need for a separate relay XML file and can reference other stacks just by name. The following configuration uses the default udp stack for the local cluster transport and uses the default tcp stack for connecting to a remote site:

Having the entire configuration in a single place greatly simplifies management. Of course you can combine all of the above features to obtain the configuration you need for your environment. You can find more details and examples in the documentation.
Enjoy !
Tristan

Wednesday, 7 February 2018

Data Container Changes Part 3

Just over a year ago we detailed some improvements to the data container, including the availability of Off Heap storage in part 2. There have been quite a few fixes for Off Heap especially around memory size estimations with Infinispan 9.2. There is also a brand new "eviction" strategy that has a bit of a twist.

Eviction Strategy Resurrected


Some of you may have remembered that Infinispan used to have an eviction strategy. This was originally used to decide what eviction algorithm was used, such as LRU or LIRS. This was removed when the new data container was introduced. Well... it is back again, but it will be used for a slightly different purpose.

The eviction strategy still has NONE & MANUAL which are exactly the same as before.

Remove strategy


There is a new REMOVE strategy that is configured by default if eviction size is greater than 0. This strategy essentially enables eviction and removes old entries as new ones are inserted.

Exception strategy


We have a brand new "eviction" strategy that provides new functionality. It is unique in that it doesn't really evict, but rather prevent entries from being inserted.  This is the EXCEPTION strategy which blocks new entries from being inserted (or updated if they exceed memory size) by throwing a ContainerFullException when the size is reached.

This strategy only works on transactional caches that always have 2 phase commit enabled. This can be useful if you want to always have only so many entries and to give priority to currently inserted entries. This strategy has better performance than REMOVE since it doesn't have to bookkeep all entries to know what to remove as well.

Note this strategy works across all storage types: OBJECT, BINARY and OFFHEAP and works with both MEMORY and SIZE based "eviction types. This makes it just as flexible as the REMOVE eviction strategy and hope it finds some uses by people.

How to Configure EXCEPTION Strategy


This is how you can enable MEMORY based EXCEPTION "eviction" using xml configuration.
This is how you configure the same thing but programmatically.

Off Heap Memory Size Allocations & Estimations


Before the off heap memory based eviction only counted the allocated memory chunks for the stored entries themselves. This unfortunately meant that the size estimate is a bit less than what it should have been. There are a few things that we improved since then, including reducing the overhead of our allocations. Note all of the below things require no configuration changes and users should just get the benefits.

Reduced per object overhead


Prior the overhead for immutable entries with eviction, Infinispan itself use to allocate 2 chunks of memory with one being 28 bytes and adding 8 bytes to the actual object. Now we only allocate an additional 16 bytes to the object memory block itself (saving the extra allocation and requiring less on the object) when using eviction. Due to memory allocation overhead this saves much more than the 20 bytes as the allocator also has its own overhead.

We also shaved off 4 bytes off of all entries if expiration was not used, meaning overhead for an immutable cache entry without eviction only requires 21 bytes of overhead from ISPN when using off heap (retained in the same allocation block).

Per allocation memory sizing estimations


Internally ISPN allocates a new chunk of memory for each object. This is done currently to leverage the underlying OS allocator to handle features such as fragmentation or compaction (if the allocator does so). Unfortunately this means that each object has its own overhead from the allocator. Thus we now take that into account when estimating the memory used by adding 8 bytes overhead and aligning to 16 bytes. This seems to be a pretty common way for allocators to work. If possible we could allow for tweaking these values, but they are hard coded currently.

Accounting for Address Count


As was mentioned in the prior blog post about off heap, we allocate a single block of memory to hold address counters for our lookups when using Off Heap. Unfortunately we didn't account for that in the memory eviction count. We now account for that in the eviction mechanism, thus your memory eviction size must be greater than the address count rounded up to the nearest power of 2, multiplied by 8. What a mouthful...

Wrap up


Off heap has been overhauled quite a bit to try to reduce memory usage, fix bugs and more accurately estimate the memory used. We hope that along with the new eviction strategy are welcome additions to various applications.

Please make sure to contact us if you have any feedback, find any bugs or have any questions! You can get in contact with various places listed on our website.

Friday, 2 February 2018

A different kind of template: wildcards

Infinispan's configuration templates are an extremely flexible way to create multiple caches using the same configuration. Configuration inheritance works by explicitly declaring the configuration a specific cache should use.

This works fine when you know the caches you are going to use upfront, but in more dynamic scenarios, this might not be possible. Additionally, if you are using the JCache API, there is no way for you to specify the configuration template you want to use.

Infinispan 9.2 introduces an alternative way to apply templates to caches: wildcards. By creating a template with a wildcard in its name, e.g. `basecache*`, any cache whose name matches the template name will inherit that configuration.

Let's show an example:

Above, caches `basecache-1` and `basecache-2` will use the `basecache*` configuration. This behaviour also applies when retrieving caches programmatically:


When using the JCache API, using the XML file above and the following code will achieve the same result:


NOTE: If a cache name matches multiple wildcards, i.e. it is ambiguous, an exception will be thrown.

I will be introducing other new features that Infinispan 9.2 brings to cache configuration in an upcoming blog post. Stay tuned !

Thursday, 22 June 2017

Cache configuration inheritance: you're no son of mine

Once upon a time Infinispan cache configurations were all orphans.

Actually, it wasn't as sad as that: they all shared a single parent - the default cache. While this gave caches a limited form of inheritance, it led to confusion as users weren't really aware of it and it was impossible to turn it off: the limited gene pool was propagating possibly unwanted traits to all of its children.

Templates and real configuration inheritance

Infinispan 7.2 finally introduced proper configuration templates and inheritance. But there was a catch. Backwards compatibility dictated that the "default mother of all caches" behaviour survived.

In the above example, the default cache is a replicated cache with a file store. The distributed cache inherits the "transactional" configuration. However, because of default inheritance present up to Infinispan 8.2, the distributed-cache also ended up having a file store. Confusing or what !?!
The best workaround was to never give it a specific configuration, let Infinispan use its internal defaults and essentially avoid it. Just like the black sheep in the family.

Bye bye default cache 

With Infinispan 9.0 we decided it was finally time to cut the umbilical cord between the default cache and all the other caches: if you declare one, it will never be used as default inheritance for every other cache. In the above example, the distributed cache won't have a file store any more.

We've gone even further: unless you declare a default cache, we will not even set one up for you, not even one with default settings!

Aleksandr Sergeevich Serebrovskii, the Russian geneticist  who first formulated the concept of the gene pool and the diversity benefits it brings, would be proud of us.

Monday, 30 March 2015

Infinispan 7.2.0.Beta2 released with better default configuration handling

We've just released Infinispan 7.2.0.Beta2 which adds better support for determining which configuration options have been defined by the user versus those options that contain default values. This is an important stepping stone in our aim to enable partial configuration overlays and proper configuration template support.

On top of that, we've added the capability for Java Hot Rod clients to recover from full cluster restarts and fixed an important topology transfer bug that lead to ArrayIndexOutOfBoundsException exceptions when applying topology changes.

Finally, in order to provide better out-of-the-box experience with Near Caches in Java Hot Rod client, we have made mandatory to define the maximum number of entries Near Caches should have. Previously, Near Caches were configured to be unbounded by default which would have resulted in memory leaks unless the client removed them by calling RemoteCache.remove() or similar. A configuration exceptions is reported now if no maximum size has been defined in the Java Hot Rod client, but the user can still provide a 0 or negative value to indicate the Near Caches should be unbounded.

For a complete list of features and bug fixes included in this release, please refer to the release notes.  

Feel free to join us and shape the future releases on our forums, our mailing lists or our #infinispan IRC channel.

Cheers,
Galder

Friday, 11 April 2014

Infinispan 7.0.0.Alpha3 is out!

Hi,
 
The Alpha3 release of Infinispan 7.0.0 is now available.


Highlights:

  • authorization at both CacheManager and Cache levels
  • some important enhancements for Map/Reduce's usability, like the ability to use an intermediate cache during Map/Reduce execution and for storing the final results of the Map/Reduce tasks
  • a much welcomed revamp of the Infinispan embedded configuration which has been aligned to with the server
For a complete list of features and bug fixes included in this release please refer to the release notesVisit our downloads section to find the latest release.

If you have any questions please check our forums, our mailing lists or ping us directly on IRC.

Cheers,
Mircea

Friday, 31 August 2012

Configuration overhaul

Infinispan 5.2 will sport a much needed configuration overhaul which will affect both the programmatic builder API and the declarative XML parsing.

As you all know by now, 5.1 introduced a new fluent builder-based API with immutable POJOs for configuring Infinispan's core. This coolness however was not extended to all the extra modules available for Infinispan (and there are quite a few of those), leaving them with a simple untyped key/value properties-based configuration. This was especially visible (and painful) when configuring the cache loaders, some of which have a plethora of parameters and options.

In 5.2 modules become first-class citizens and can provide their own builders and can take care of parsing their own XML for which they can provide a custom schema (for editors/IDE which provide content-assist). Modules can retrieve information from either the GlobalConfiguration or the per-cache Configuration objects via the T modules(Class<T> moduleClass) method.

Loaders and Stores also get this treatment. Look at the two before and after configurations below for configuring the JDBC Cache Store.

Before:

After:

You will be able to check-out these features in Infinispan 5.2.0.Alpha3. Bear in mind that at the time of writing not all cache loaders have been migrated to this new configuration style, but they should all be complete by the time 5.2.0.Final is released.

If you want to learn how to extend Infinispan's configuration for your own modules, head over to ExtendingInfinispansConfiguration which should provide all the information you need

Wednesday, 4 January 2012

Configuration Changes in Infinispan

This blog will introduce both Infinispan users, and Infinispan contributors to the new configuration system. First, I'll detail the changes for users, and then for committers.

Users


If you use XML to configure Infinispan, you shouldn't notice any change, except a much faster startup, courtesy of the Stax based parser. However, if you use programmatic configuration, read on for the important differences.
Configuration is now packaged in org.infinispan.configuration, and you must use a builder style:
Configuration c1 = new ConfigurationBuilder()
   // Adjust any configuration defaults you want
   .clustering()
      .l1()
         .disable()
      .mode(DIST_SYNC)
      .hash()
         .numOwners(5)
   .build();
The old bean style configuration is now deprecated and will be removed in a later version.
Configuration properties which can be safely changed at runtime are mutable, and all others are immutable.
To copy a configuration, use the read() method on the builder, for example:
Configuration c2 = new ConfigurationBuilder()
   // Read in C1 to provide defaults
   .read(c1)
   .clustering()
      .l1()
         .enable()
   // This cache is DIST_SYNC, will have 5 owners, with L1 cache enabled
   .build();
This completely replaces the old system of defining a set of overrides on bean properties. Note that this means the behaviour of Infinispan configuration is somewhat different when used programmatically. Whilst before, you could define a default configuration, and any overrides would be applied on top of *your* defaults when defined, now you must explicitly read in your defaults to the builder. This allows for much greater flexibility in your code (you can have a as many "default" configurations as you want), and makes your code more explicit and type safe (finding references works).

The schema is unchanged from before. Infinispan 4.0 configurations are currently not being parsed. Support for these will be added shortly, however a warning message will be printed if they are used. To upgrade, just change the schema definition from:
<infinispan
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="urn:infinispan:config:4.1 http://www.infinispan.org/schemas/infinispan-config-4.1.xsd"
     xmlns="urn:infinispan:config:4.1">
to
<infinispan
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
     xmlns="urn:infinispan:config:5.1">
The schema documentation has changed format, as it is now produced using the standard tool x3p. This should be a significant improvement, as better navigation is offered. Some elements and attributes are missing docs right now, we are working on adding this. As an added benefit, your IDE should now show documentation when an xsd referenced (as above)

We are in the process of adding in support for this configuration style for modules (such as cache stores). In the meantime, please use the old configuration or XML if you require support for cache store module configuration.

Committers


If you are a committer to Infinispan, you may find the following notes useful. Note that currently we still use the old configuration system internally within Infinispan. This makes things a little complicated. This will be switched out soon! For now, you need to also add your property to the old config system as well as the new.

Note, these guides assume you are adding an element to the cache configuration, but apply equally to the global configuration.

Before you start adding a configuration property, identify whether you want to add a property to an existing configuration group/element, or whether you need to create a child object. We call the configuration group XXX in the steps below.

Adding a property

  1. Add the property to the relevant XXXConfiguration class. Add a private final field, add a parameter to the constructor, and assign the value to the field in the constructor body. Add a accessor for the property. If the property should be mutable at runtime, then add a mutator as well. Most configuration properties are not mutable at runtime - if the configuration is runtime mutable, then Infinispan needs to take notice of this update whilst the cache is running (you can't cache the value of the configuration in your implementation class). Mutators and accessors don't use the classic JavaBean pattern of prepending accessors with "get" and mutators with "set". Instead, the name of the property is used for an accessor. A mutator is an overloaded version of the accessor which takes a parameter, the new value.

  2. Add the property to the matching XXXConfigurationBuilder. You'll need to add a mutable field to the class, and initialise it to it's default value in the field declaration. Add a mutator (following the above pattern).

  3. The create() method is called by the parent object in order to instantiate the XXXConfiguration object from the builder. Therefore, make sure to pass the value of the field in the builder to the XXXConfiguration object's constructor here. Additionally, if you require a complex default (for example, the value of a configuration property is defaulted conditionally based on the value of some other configuration property), then this is the place to do this.

  4. The validate() method is called by the parent object to validate the values the user has passed in. This method may also be called directly by user code, should they wish to manually validate a configuration object. You should place any validation logic here related to your configuration property. If you need to "cross-validate" properties (validate the value of your property conditionally upon the value of another property), and the other property is on another builder object, increase the visibility of that other property field to "default", and reference it from this builder, by calling the getBuilder() method, which will gives you a handle on the root configuration builder.

  5. The final step is to add parsing logic to the Parser class. First, add the attribute to name to the Attribute enum (this class simply provides a mapping between the non-type-safe name of the attribute in XML and a type-safe reference to use in the parser). Locate the relevant parseXXX() method on the class, and add a case to the switch statement for the attribute. Call the builder mutator you created above, performing any XML related validation (you are unlikely to need this), and type conversion (using the static methods on the primitive wrapper classes, String class, or relevant enum class).

Adding a group


In some situations you may additionally want to add a configuration grouping object, represented in XML as an element. You might want to do this if you are adding a new area of functionality to Infinispan. Identify the location of the new configuration grouping object. It might be added to the root Configuration object, or it might be added to one it's children, children's children. We'll call the parent YYY in the steps below.
  1. Create the XXXConfiguration object. Add any properties required following the guide for adding properties. The constructors visibility should be "default".

  2. Create the XXXConfigurationBuilder object. It should subclass the relevant configuration child builder -- use the YYYConfigurationChildBuilder as the superclass. This will ensure that all builder methods that allow the user to "escape" are provided correctly (i.e provide access to other grouping elements), and also require you to provide a create() and validate() method. The constructor needs to take the the YYYConfigurationBuilder as an argument, and pass this to the superclass (this simply allows access to the root of the builder tree using the getBuilder() method).

  3. Follow the property adding guide to add any properties you need to the builder. The create() method needs to return a new instance of the XXXConfiguration object. Implement any validation needed in the validate() method.

  4. In the YYYConfiguration object, add your new configuration class as a private final field, add an accessor, and add initialiser assignment in the constructor

  5. In the YYYConfigurationBuilder, add your new configuration builder as a private final field, and initialise it in the constructor with a new instance. Finally, add an accessor for it following the standard pattern discussed in the guide.

  6. In the YYYConfigurationBuilder ensure that your validate method is called in it's validate method, and that result of the XXXConfiguration instances' create method is passed to the constructor of YYYConfiguration

  7. Finally, add this to the parser. First, add the element to the Element class, which provides a type safe representation of the element name in XML. In the Parser class, add a new parseXXX method, copying one of the others that most matches your requirements (parse methods either parse elements only - look for ParseUtils,requireNoAttributes(), attributes only -- look for ParseUtils.requireNoContent() or a combination of both -- look for an iterator over both elements and attributes). Add any attributes as discussed in the adding a property guide. Finally, wire this in by locating the parseYYY() method, and adding an element to the switch statement, that calls your new parseXXX() method.

Bridging to the old configuration


Until we entirely swap out the old configuration you will need to add your property to the old configuration (no need to worry about jaxb mappings though!), and then add some code to the LegacyConfigurationAdaptor to adapt both ways. It's fairly straightforward, just locate the relevant point in the adapt() method (near the configuration group you are using) and map from the legacy configuration to the new configuration, or vs versa. You will need to map both ways, in both adapt methods.

Tuesday, 22 November 2011

Infinispan 5.1.0.BETA5 is out!

Infinispan 5.1.0.BETA5 has just been the released with a few interesting additions and important fixes:
  • Locks acquired within a transaction are now reordered in order to avoid deadlocks. There's no new configuration required to take advantage of this feature. More information on how lock reordering works can be found here.
  • One of the aims of Infinispan 5.1 'Brahma' series is to move away from JAXB and instead use Stax based XML parsing. Ahead of that, a new configuration API based on builders has been developed. Expect to hear more about it and examples on using the API in the next few days.
Amongst the fixes included in this release, it's worth mentioning:
  • The demo paths that were broken in 5.1.0.BETA4 have now been fixed.
  • Some of the Infinispan jars in 5.1.0.BETA4 were showing duplicate classes. This was the result of an OSGI bundle generation bug, and so to avoid the issue 5.1.0.BETA5 OSGI bundle generation has been disabled. This functionality will be re-enabled once the issue has been fixed by the Maven Felix plugin.
As always, please keep the feedback coming. You can download the release from here and you get further details on the issues addressed in the changelog.

Cheers,
Galder

Friday, 18 February 2011

5.0.0.ALPHA3 is here with more goodies!

For those who can't wait to get hold on the newest Infinispan features, we've just released the third alpha release of the 5.0 series, called 5.0.0.ALPHA3.

Apart from including fixes in 4.2.1.CR2, it allows users to prefetch data in advance in parallel thanks to the new getAsync() method. Why is this useful? Imagine the following scenario: A cache is configured with distribution and an app that requires values associated with k1, k2, and k3 in order to do its job. In the worst case scenario assume that all these keys located in remote nodes. Previously, before 5.0.0.ALPHA3, you'd have written something like this:


Value v1 = cache.get("k1");
Value v2 = cache.get("k2");
Value v3 = cache.get("k3");

The problem with this code is that each get() is sequential, so the second get() call must wait for first get() to retrieve data from the remote node before it can execute. This is clearly not very optimal. From 5.0.0.ALPHA3 onwards, you can do this instead:


NotifyingFuture<Value> f1 = cache.getAsync("k1");
NotifyingFuture<Value> f2 = cache.getAsync("k2");
NotifyingFuture<Value> f3 = cache.getAsync("k3");
...
Value v1 = f1.get(); // blocks until value has been returned
Value v2 = f2.get();
Value v3 = f3.get();

The clear advantage here is that the get requests, via getAsync(), can be fired in parallel and they don't need to wait for each other. Afterwards, when the value is needed, simply call get() on the future received.

Amongst other API enhancements, such as RemoteCache implementing size() and isEmpty(), or the ability to delete AtomicMap instances via AtomicMapLookup, it's worth highlighting that we've taken your feedback on board with regards to how Infinispan is configured programmatically, and from 5.0.0.ALPHA3 onwards, we provide a more fluent way of configuring Infinispan. For example:


GlobalConfiguration gc = new GlobalConfiguration();

GlobalJmxStatisticsConfig jmxStatistics = gc.configureGlobalJmxStatistics();
jmxStatistics.allowDuplicateDomains(true).enabled(true);

TransportConfig transport = gc.configureTransport();
transport.clusterName("blah").machineId("id").rackId("rack").strictPeerToPeer(true);

You can find more examples of this new configuration here. Note that this fluent API is likely to evolve further before we go final with 5.0.0 as shown in this forum discussion, but please keep your feedback coming! Finally, details of what's fixed is in the release notes. As always, please use the user forums to report back, grab the release here, enjoy and keep the feedback coming.

Cheers,
Galder