Category Archives: springframework

Spring Migration Analyzer – The Beginning

Spring Team surprised us once again with Spring Migration Analyzer tool. Migration with Spring wasn’t so painful, BUT this one create report for us. Now it easier to estimate effort and we now spots where we should put our attention.

I read InfoQ article: “Spring Migration Analyzer: An Assistant For JavaEE To Spring Conversion” about great features and possibilities, but I rather do not belief until I touch :).

Now guys have a lot to do but in some point SMA should be able to run just on sources (especially if this is ant, ivy, maven or gradle project).

I ran this tool in this projects:

  1. open source project (war – 30Mb) with spring 3.1, guava, hibernate – nothing fancy
    • First run: java.lang.OutOfMemoryError: Java heap space -> so I add more heap, and rerun.
    • Second run (512M) - ”Poller SunPKCS11-Darwin” java.lang.OutOfMemoryError: Java heap space -> so I found SMA-21 - fix in M2 – let’s switch do suggested nightly build
    • While third build was running I checked M2 fixes – so my advice is use nightly build.
    • Third run (1Gb – nighty) : 1m10.842s (real):  Nothing special, and a lot of false positives, as example the migrator assumes that I handle transaction programmatically because in org.springframework.transaction.jta.ManagedTransactionAdapter I’m using void public commit() and that is quite opposite ;).
    • OK – let assume it will work better when, there is no spring yet.
  2. some secret project (zip 10Mb) guice no j2ee, no spring at all.
    • First run : 0m14.807s : look promising.
    • Two findings
      • The application utilises the JMS API. Typically, little or no migration is required as JMS can be used within Tomcat. During the migration, it may be worth considering using Spring’s JMS support, for example JmsTemplate, to simplify the code.
      • The application utilises the JNDI API. JNDI is available in Tomcat and therefore no work is required to migrate the application. However, it may be worth considering using Spring’s <jee-jndi-lookup> support to perform any JNDI lookups, rather than using the JNDI API directly.
    • BUT grep -rial “jms” src/  -> nothing same for jndi
    • Fail once again

I’ve skipped checking another one, as the project started, there is no point to kick the Younglings

They’are just started (SMA-24 do not exists) and  from my point of view it might be a good tool in the future: My ideas are:

  • automatic exclusion on common names: eg. org.springframework
  • working with source code
  • understanding dependencies (most java project are managed by ivy,maven,gradle)

Anyway, it worth to now that such tool as Spring Migration Analyzer is rising, or now it is the best time to join and help a little bit:

Spring Template Objects Testing

My friend asked me, how we should test this piece of code

public void fire(final NotificationMessage message) {
  jmsTemplate.send(new MessageCreator() {
    @Override
    public Message createMessage(Session session) throws JMSException {
      return session.createObjectMessage(message);
    }
  });
}

Mock, stubs, arguments matchers are not successful here. Of course we can mock jmsTemplate, verify if send method was executed, but this way we only test if springframework works properly. Should we leave this to integration test?

Of course not, and our business part is: session.createObjectMessage(message). We have to change our code. First at all we provide second method.

public void fire(final NotificationMessage message) {
  this.fire(message, getMessageCreator(message));
}
public|protected| void fire(final NotificationMessage message, MessageCreator mc) {
  jmsTemplate.send(mc);
}

This two methods are easily testable by using mocks and verifying object interactions. We can decide about method modifier:

  • public – if we want second function to be available in our API.
  • protected – for future subclass usage.
  • package public – for use in package.
  • or even private

Finally we have decide about getMessageCreator(message).

  1. Provide Factory Method for it: MessageCrator getMessageCreator(NotificationMessage message);
    MessageCreator getMessageCreator(final NotificationMessage message) {
      return new MessageCreator() {
        @Override
        public Message createMessage(Session session) throws JMSException {
          return session.createObjectMessage(message);
        }
      };
    }
    
  2. Create Class which implements MessageCreator interface
    public class NotificationMessageCreator implements MessageCreator {
        private final NotificationMessage message;
    
        public NotificationMessageCreator(NotificationMessage message) {
            this.message = message;
        }
        @Override
        public Message createMessage(Session session) throws JMSException {
            return session.createObjectMessage(message);
        }
    }
    

I think both approaches are ok, so choose better one for you.

Of course you may argue that you don’t need unit test, you will have integration test for that part of the code. Unfortunately integration tests are slow and from time to time they fail, so for simple check I prefer unit test.

Summary

We use a lot of Spring Template Object helpers, and most of the time we are tempted to create anonymous class in place. This way we make our code hard to unit test. Consider this approach to make your code easier to test and maintain. What do you think about that?


Spring Data – Redis – tutorial

Spring Data project provides a solution to access data stored in new emerging technologies like NoSQL databases, cloud based services etc. When we look into SpringSource git repository we see a lot of spring-data projects:

  • spring-data-commons: common interfaces and utility class for other spring-data projects.
  • spring-data-column: support for column based databases. It has not started yet, but there will be support for Cassandra and HBase
  • spring-data-document: support for document databases. Currently MongoDB and CouchDB are supported.
  • spring-data-graph: support for graph based databases. Currently Neo4j is supported.
  • spring-data-keyvalue: support for key-value databases. Currently Redis and Riak are supported and probably Membase will be supported in future.
  • spring-data-jdbc-ext: JDBC extensions, as example Oracle RAC connection failover is implemented.
  • spring-data-jpa: simplifies JPA based data access layer.

I would like to share with you how you can use Redis. First step is to  download it from redis.io web page. Here is useful site try.redis-db.com where we can run Redis commands and also read step by step tutorial. This tutorial shows us all structures Redis supports (list,set,sorted set and hashes) and some useful commands. A lot of reputable sites use Redis today .

After download and unpacking we should compile Redis (version 2.2 (it is release candidate) is preferable one to use since some commands do not work in version 2.0.4).

> make

> sudo make install

Once we run this commands we are all set to run the following five commands:

  • redis-benchmark – for benchmarking Redis server
  • redis-check-aof – check the AOF (Aggregate Objective Function), and it can repair that.
  • redis-check-dump – check rdb files for unprocessable opcodes.
  • redis-cli – Redis client.
  • redis-server – Redis server.

We can test Redis server.

>redis-server

[1055] 06 Jan 18:19:15 # Warning: no config file specified, using the default config. In order to specify a config file use ‘redis-server /path/to/redis.conf’

[1055] 06 Jan 18:19:15 * Server started, Redis version 2.0.4

[1055] 06 Jan 18:19:15 * The server is now ready to accept connections on port 6379

[1055] 06 Jan 18:19:15 – 0 clients connected (0 slaves), 1074272 bytes in use

and Redis client.

> redis-cli

redis> set my-super-key “my-super-value”

OK

Now we create a simple Java project in order to show how simple a spring-data-redis module essentially is.

> mvn archetype:create -DgroupId=info.pietrowski -DpackageName=info.pietrowski.redis -DartifactId=spring-data-redis -Dpackage=jar

Next we have to add in pom.xml milestone spring repository, and add spring-data-redis as dependence, after that all needed dependencies will be fetched.

Next we are creating resources folder under main folder, and create application.xml which will have all the configuration.

We can configure the JedisConnectionFactory,  in two different ways, One – we can provide JedisShardInfo object in shardInfo property or second – we can provide host (default localhost), port (default 6379), password (default empty) and timeout (default 2000) properties. One think to keep in mind is  that JedisShardInfo object has precedence and allows to setup weight, but only allows constructor injection.

We can set factory to use connection pooling by setting the value of propery  pooling to ‘true’ (default).

See application.xml comments to see three different way of configuration.

Note: There are two different libraries supported Jedis and JRedis, there have very similar names and both have the same factory name. See the difference (two r isn’t enough for me):

  • org.springframework.data.keyvalue.redis.connection.jedis.JedisConnectionFactory
  • org.springframework.data.keyvalue.redis.connection.jredis.JredisConnectionFactory

Similar to what we do in Spring, we configure template object providing it with connection factory. We will perform all the operations through this template object. By default we need to provide only Connection Factory, but there are more properties we can provide:

  • exposeConnection (default false) – if we return real connection or proxy object.
  • keySerializer, hashKeySerializer, valueSerializer, hashValueSerializer (default JdkSerializationRedisSerializer) which delegates serialization to default Java serialization mechanism.
  • stringSerializer (default StringRedisSerializer) which is simple String to byte[] (and back) serializer with UTF8 encoding.

We are ready to execute some code which will be cooperating with Redis instance. Spring-Data provide us with two ways of interaction, First is by using execute method and providing RedisCallback object. Second is by using *Operations helpers (it will be explained later)

When we are using RedisCallback we have access to low level Redis commands, see this list of interface (I won’t put all the method here because it is huge list):

Check RedisCallbackExample class, this was the hard way and the problem is we have to convert our objects into byte arrays in both directions, the second way is easier. Spring Data provides for us Operations objects, than we have much more simpler API and all byte<->object conversion is made by serializer we setup (or the default one). Higher level API (you will easily recognize *Operation *Commands equivalents):

Most of methods get key as first parameters so we have even better API for multiple operation on the same key:

Check RedisCallbackExample class to see some easy examples of *Operations usage. One important thing to mention is that you should use stringSerializers for keys, otherwise you will have problems from other clients, because standard serialization adds class information. Otherwise you end with such keys:

  1. “\xac\xed\x00\x05t\x00\x05atInt”
  2. “\xac\xed\x00\x05t\x00\nmySuperKey”
  3. “\xac\xed\x00\x05t\x00\bsuperKey”

Till now we just check API for Redis, but Spring Data offers more for us. All cool stuff is in org.springframework.data.keyvalue.redis.support package and all sub-packages. We have than:

  • RedisAtomicInteger – Atomic integer (CAS operation) backed by Redis.
  • RedisAtomicLong – Same as previous for Long.
  • RedisList – Redis extension for List, Queue, Deque, BlockingDeque and BlockingQueue with two additional methods List range(start, end) and RedisList trim(start, end).
  • RedisSet – Redis extension for Set with additional methods: diff, diffAndStore, intersect, intersectAndStore, union, unionAndStore.
  • RedisZSet – Redis extension for SortedSet. Note that Comparator is not applicable here so this interface extends normal Set and provide proper methods similar to SortedSet.
  • RedisMap – Redis extension for Map with additional Long increment(key, delta) method

Every interface currently have one Default implementation. Check application-support.xml for examples of configuration and RedisSupportClassesExample for examples of use. There is lot of useful information in the comments as well.

Summary

The library is first milestone release so there are minor bugs, documentation isn’t as perfect as we used to and current version needs no stable Redis server but this is definitely a great library which allow us to use all this cool NoSQL stuff in a “standard” Spring Data Access manner.

Awesome job!

This post is only useful if you checkout the code: from bitbucket , for the lazy ones here is spring-data-redis zip file as well.


Spring MVC Exception Handler

In previous version of Spring Framework to handle exception we used HandlerExceptionResolver interface. Spring also provides simple implementations which was suitable in most of cases. Since 3.0 version we have few additional exception resolvers plus we can use annotation to specify our exception handler methods or classes without implementing HandleExceptionResolver interface.

  1. AnnotationMethodHandlerExceptionResolver (new in 3.0)
  2. ResponseStatusExceptionResolver (new in 3.0)
  3. DefaultHandlerExceptionResolver (new in 3.0)
  4. SimpleMappingExceptionResolver

SimpleMappingExceptionResolver

Should be configured in XML, there are few properties which drives this resolver behaviors.

  • defaultErrorView -> this view name will be returned if no mapping will be found.
  • defaultStatusCode -> once the view name was resolved if the view hasn’t status code the defaultStatusCode will be applied.
  • statusCodes -> Map with “view name” to “status code” mapping.
  • exceptionAttribute -> the name of attribute where exception is hold by default “exception” is used.
  • exceptionMappings -> Properties which maps Exception “class name” to “view name”. Watch out as you can easily maps more than you want, this is because only String.contains is checked, so a=viewName mapping will catch all exception with “a” in the name.
    Another thing to consider is that if you provide deep Exception hierarchy (n) and you provide a lot of exception mapping (m), you will end up with O(n*m) lookup algorithm. It may seem not problem, but if somebody spots that (Error page which render time is quite long), than it may be used with DDOS to increase load on ours machines.

DefaultHandlerExceptionResolver

First at all it has pageNotFound logger which is org.springframework.web.servlet.PageNotFound logger category, second it decides what to do depending on exception type, it is base class to extend when we want change default behavior:

  • NoSuchRequestHandlingMethodException : no request handler method was found by default it sends 404 error. Override handleNoSuchRequestHandlingMethod.
  • HttpRequestMethodNotSupportedException : no request handler method was found for the particular HTTP request method by default 405 error.Override handleHttpRequestMethodNotSupported.
  • HttpMediaTypeNotSupportedException : no HttpMessageConverter were found for the PUT or POSTed content by default 415 error. Override handleHttpMediaTypeNotSupported.
  • HttpMediaTypeNotAcceptableException : no HttpMessageConverter were found that were acceptable for the client (Accept header) by default 406 error. Override handleHttpMediaTypeNotAcceptable.
  • MissingServletRequestParameterException : required parameter is missing by default 400 error. Override handleMissingServletRequestParameter.
  • ConversionNotSupportedException : WebDataBinder conversion cannot occur by default 500 error. Override handleConversionNotSupported.
  • TypeMismatchException : WebDataBinder conversion error occurs by default 400. Override handleTypeMismatch.
  • HttpMessageNotReadableException : HttpMessageConverter cannot read from HTTP request by default 400 error. Override handleHttpMessageNotReadable.
  • HttpMessageNotWritableException : HttpMessageConverter cannot write to HTTP response by default 500 error. Override handleHttpMessageNotWritable.

ResponseStatusExceptionResolver

This resolver allows us to use @ResponseStatus annotation. If we annotate our Exception with @ResponseStatus annotation than the response will get status code from annotation.

  • ResponseStatus.value as status code (it is HttpStatus enum)
  • ResponseStatus.reason as reason or default HttpResponse reason if not set.

AnnotationMethodHandlerExceptionResolver

Last but not least, and from my point of view this resolver is the most important. This exception resolver allows us to use @ExceptionHandler annotation, every method annotated by @ExceptionHandler will become exception handler. As parameters @ExceptionHandler need an array of Throwable. We also should use @ResponseStatus to indicate status code.

The method signatures possibilities are vary so see documentation to get all the proper combinations of parameters and return types.

The idea is pretty simple, for all the ExceptionHandler.value exception the exception -> Method handler is created, when Exception happens Method will be invoked.

Conclusions

Since 3.0 we rather has no need to setup handlers by ourself, the most useful way is to use @ExceptionHandler annotation. By default DispatcherServlet will setup AnnotationMethodHandlerExceptionResolver, ResponseStatusExceptionResolver and DefaultHandlerExceptionResolver as handlers resolvers. So we are ready to start with annotation based approach.

My approach is that I try to create moduleExceptionHandler class which has all methods annotated by @ExceptionHandler (except utility methods of course) and I always annotate them by @ResponseStatus as well. The methods parameters and return type vary depending on my needs. It a little bit central approach but it avoids problem with more than one handler per Exception.

UPDATE :: I’ve downloaded code to Bitbucket, you can clone it: hg clone https://pedrowaty@bitbucket.org/pedrowaty/spring-tryouts and than go to mvc-error-handlers directory.


How to extend SQLErrorCodesFactory

I found some interesting feature in springframework. Spring offers you to map sql error code to exception type. By default there is in springframework distribution file called sql-error-codes.xml which has definition for common databases ( DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase).

This keys are taken form java.sql.DatabaseMetaData databaseProductName property. Each driver should provide proper implementation. This post will provide you the way how to add another databases or extends definition to more sql codes.

As we read javadoc, the default file is bundled in spring.jar and it’s location is “org/springframework/jdbc/support/sql-error-codes.xml" so first solution is to simple extract this file, add what we want, and than put it back. This solution has many drawbacks e.g. when we upgrade we must repeat our steps once again. If we keep reading javadoc we read

Reads the default file in this package if not overridden by a file in the root of the class path (for example in the “/WEB-INF/classes” directory).

So the second option is to provide file called ‘sql-error-codes.xml’ in the root path. Lets start coding.
Our App.java is very simple.

public class App {
public static void main(String[] args) {
SQLErrorCodesFactory errorCodes = SQLErrorCodesFactory.getInstance();
SQLErrorCodes errorCode = errorCodes.getErrorCodes(“MySQL”);
}
}

sql error codes default

If we run this in debug mode we can see something like this. This is default SQL error codes definition for MySQL database. Now we add the ‘sql-error-codes.xml’. I’ve just copied MySQL bean from default file and added some codes to badSqlGrammarCodes. This time the result look like this:

sql error codes override

And here is execution log, interesting lines in bold:

Sep 2, 2008 9:54:46 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml]
Sep 2, 2008 9:54:46 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [sql-error-codes.xml]
Sep 2, 2008 9:54:46 PM org.springframework.beans.factory.support.DefaultListableBeanFactory registerBeanDefinition
INFO: Overriding bean definition for bean 'MySQL':
    replacing [Generic bean: ... defined in class path resource [org/springframework/jdbc/support/sql-error-codes.xml]]
    with [Generic bean: ...  defined in class path resource [sql-error-codes.xml]]
Sep 2, 2008 9:54:46 PM org.springframework.jdbc.support.SQLErrorCodesFactory 
INFO: Found custom sql-error-codes.xml file at the root of the classpath
Sep 2, 2008 9:54:46 PM org.springframework.jdbc.support.SQLErrorCodesFactory 
INFO: SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase]

The point to remember is that our definition didn’t override all the definition but only definition with the same bean id. This is not so clear in SQLErrorCodesFactory javadoc, which suggest that we override all beans.

We can use this mechanism if we have different database, or if we want to extend or change sql code mapping to exception. The application is really simple but you can
download it, and try by yourself.

Bye.