Category Archives: java

JIRA Tuning – part one

There was a time when I did some JVM tuning. It’s time to reactivate the part of the brain responsible for this activity.

The key principle is to prepare a baseline. What is it?

The baseline refers to:

  • the first measuring mechanisms (np. gc.log) – we should measure on different levels (cpu, I/O, database etc) if in doubt follow thus rule: “more is better”
  • a load test script that makes JIRA work (e.g. Apache JMeter script),
  • a test environment – should be identical as production (different environment create different baseline), and,
  • a desired goal (we tend to tune solutions not having any goal). Never tune just for tuning.

The next required value is time… a lot of time. The reason is simple. We make only one change at a time and when it is completed, we launch load scripts and use metrics to see if we’re going in the right direction. The whole operation is based on making several steps forward followed by a few steps back.

Our recent goal was to make JIRA stable, despite working slow (no restart during the day).

We launched the analysis of gc.loc. Due to short pauses, we used the Concurrent Mark Sweep collector. Unfortunately, there are two sides of a coin. As CMS does not compact the memory, you may not be able to place any object in free spaces despite having free memory.

This usually results in the death of JVM – when looking for some free space, you do some cleaning, which takes pretty long time. This causes a “slashdot effect”. As a result you need more and more time, because of which the main GC works and the remaining part is stuck -> reset!

OK, we update two parameters causing CMS to start working earlier – usually, it is launched when the old generation is filled up to 68% (exact value vary depending on JVM version):

  • -XX:CMSInitiatingOccupancyFraction=40
  • -XX:+UseCMSInitiatingOccupancyOnly

Because of that, the whole commotion happens on the bottom of the stack. On the top, there is a place for larger objects. Obviously, you may (and will) experience a problem with stack fragmentation.

We launch load tests. The situation was better, but….

Unfortunately, we killed JIRA. The next decision is to limit the work of GC by reducing the pipe. We lower the maxThreads parameter on tomcat. We launch tests on one of the machines. Nothing happen. We add the second machine and start acting more aggressively. Result: although slow, Jira responds in a stable manner. This is what we were hoping for!

I may still try two other options:

  • -XX:+UseCompressedStrings
  • -XX:+OptimizeStringConcat

They were added in java 6u21 and 6u20. It is an interesting option, because it optimizes String-type objects, so that – whenever possible – it used 1 byte tables instead of 2 byte char tables. Most of the texts can be squeezed in a single byte. We check the memory usage histogram (previously 98% – char[]). This time a byte[] takes precedence over a char[].

In the meantime, the jConsole operator screams with surprise: “Pedro, what have you done?!”. This draws our attention (we think we’ve f… up something). Fortunately, all this hassle was about a 50% drop in memory usage. Uff, this is yet another confirmation that our switch is working.

Time to finish. We note down our ideas for later. Migration is in just two days. Quick help of our colleague gave us a new machine. We’ve finally got a place to use the G1 collector.

Model Driven Architecture live or dead?

In my wunderlist‘s “watch this” list I found this dinosaur movie to watch: MDA: A forlorn hope. by Uncle Bob. It was posted over two years ago and it was viewed almost 55 thousand times.

MDA is easy isn’t it? The MDA Guid has only 62 pages. We need some modeling tools and another tool for model to code transformation. Sounds easy :)

Few years ago, we’ve been using MDA approach in our project. We used MagicDraw for UML part and AndroMDA for UML to Java code generation (of course there are many other tools). From my point of view it was great experience. I share with my opinion below:

  • (pros) Model and Factory for free.
  • (pros) Hibernate mapping for free.
  • (pros) Documentation is up to date – you have to modify it to generate changes.
  • (pros) We focused on design, before coding.
  • (pros) All you hibernate mapping/DAO/etc is similar ( standardized – we can modify template)
  • (pros/cons) We can/have to change templates to align to ours standards.
  • (cons) Lot of  code you prefer to never read :/
  • (cons) Every time you change something, you have to regenerate code.
  • (cons) You have codebase divided into “read only” and “change here”.
  • (cons) If you forget to put additional metadata into model you’ll be doomed in future.
  • (cons) Once you change template, it is harder to upgrade tools and we have to regenerate code.

Nowadays I compare that experience to modern frameworks such as Rails or Django. I have to add comment here, what I really mean is that by using MDA approach I do not think about database mapping, DAO/repository object, I started at service level, and I have similar fillings when I’m using Rails or Django framework. I’m focusing on business logic, not how to get or save data into persistence storage.

Uncle Bob is talking about analyst as software creators, and this idea fortunately for my salary is impossible ;).

I totally agree, doing software is on much more detail level than model thinking, but … I think it wasn’t so bad to think about design on high level and then generate code and go deep in business logic details. How many times Java programers do the same job: create POJO, annotate or write XML descriptor, create DAO which looks more or less the same as another one, etc.

Let cite some smart guys from Uncle Bob post:

  • Uncle Bob: “Programmers are details managers – sorry MDA” :)
  • Comment: “MDA is actually based on two grand ideas:
    - raising the level of abstraction above programming language.
    - satisfying everyone with universal set of *standard* abstractions. Most of MDA failures is due to the the second idea, which is why MDA (not MDE in general) may be indeed a forlorn hope.”

I don’t want to force you to use MDA, just think about it and in …meantime … find the difference :).

Old times

Blog post before 1st May

nowadays

Blog post after 1st May

Yep, posterous is dead. Is MDA dead? Please share your opinion in comments.

 

Big Data Predictions

I’ve read Forrester’s Big Data Predictions for 2013. Mike gave four predictions:

First prediction: everyone must figure out that BigData is everywhere, surrounds us. We need to find ways of learning from that data. We need to build new tools and methods for easily decision taking based on many different sources of data simultaneously and what is pretty hard do it in real time. It is not easy, so 2nd & 3rd prediction emerge here.

Second prediction: “The algorithm wars will begin.” : machine learning, advanced analytics, natural language processing, business intelligence, predictive analytics etc.. Lots of things to discover and huge data has to be processed.

Third prediction: Real-time technology have to be highly available, scalable, and performant.

Fourth prediction: “Time magazine will name big data its 2013 person of the year”. We will see :).

Another four from me:

  • BigData Appstore: If there is a lot of data we can sell it cheaply.
  • BigData Startups: figure out the “new way of dealing” and you may be the next Google.
  • BigData Consultancy: …. so you want us to install complete BigData solution … Yes we can do this, just show us the money!
  • BigData Solutions: NewSQL, NoSQL, Polyglot Persistence, and probably some others ideas will show up.

I suggest you to design with openness and integration in mind, and be prepared – check this free Datascience books list.

Wondering How Big is BigData - thery funny ;)?

Engineering New Year’s Resolutions

For all engineers seeking what resolutions to choose. This presentation will help: Engineering Change.

Some ideas from presentation:

  • Continuous Deployment: make mistake quick, cheap failures,
  • Metric Driven Development: learning based on knowledge, know the project health, measure, measure and once again measure.
  • Learn new language:  chosen from different programing paradigm.
  • NoSQL: choose NoSQL solution to evaluate or compare few of them (eg. Riak, Cassandra, MongoDB, etc.).
  • BigData: Everyone talks about that, so you have to know something about that. Start with Wiki than use google or twitter to find sources and read a lot.

In presentation you will find, how Etsy do the first two. Third one is obvious. 4th and 5th are today’s future.

Choose at least three and I’m assure you won’t regret that.

In my resolutions I’m going to focus on BigData.

Spring Migration Analyzer – The Beginning

Spring Team surprised us once again with Spring Migration Analyzer tool. Migration with Spring wasn’t so painful, BUT this one create report for us. Now it easier to estimate effort and we now spots where we should put our attention.

I read InfoQ article: “Spring Migration Analyzer: An Assistant For JavaEE To Spring Conversion” about great features and possibilities, but I rather do not belief until I touch :).

Now guys have a lot to do but in some point SMA should be able to run just on sources (especially if this is ant, ivy, maven or gradle project).

I ran this tool in this projects:

  1. open source project (war – 30Mb) with spring 3.1, guava, hibernate – nothing fancy
    • First run: java.lang.OutOfMemoryError: Java heap space -> so I add more heap, and rerun.
    • Second run (512M) - ”Poller SunPKCS11-Darwin” java.lang.OutOfMemoryError: Java heap space -> so I found SMA-21 - fix in M2 – let’s switch do suggested nightly build
    • While third build was running I checked M2 fixes – so my advice is use nightly build.
    • Third run (1Gb – nighty) : 1m10.842s (real):  Nothing special, and a lot of false positives, as example the migrator assumes that I handle transaction programmatically because in org.springframework.transaction.jta.ManagedTransactionAdapter I’m using void public commit() and that is quite opposite ;).
    • OK – let assume it will work better when, there is no spring yet.
  2. some secret project (zip 10Mb) guice no j2ee, no spring at all.
    • First run : 0m14.807s : look promising.
    • Two findings
      • The application utilises the JMS API. Typically, little or no migration is required as JMS can be used within Tomcat. During the migration, it may be worth considering using Spring’s JMS support, for example JmsTemplate, to simplify the code.
      • The application utilises the JNDI API. JNDI is available in Tomcat and therefore no work is required to migrate the application. However, it may be worth considering using Spring’s <jee-jndi-lookup> support to perform any JNDI lookups, rather than using the JNDI API directly.
    • BUT grep -rial “jms” src/  -> nothing same for jndi
    • Fail once again

I’ve skipped checking another one, as the project started, there is no point to kick the Younglings

They’are just started (SMA-24 do not exists) and  from my point of view it might be a good tool in the future: My ideas are:

  • automatic exclusion on common names: eg. org.springframework
  • working with source code
  • understanding dependencies (most java project are managed by ivy,maven,gradle)

Anyway, it worth to now that such tool as Spring Migration Analyzer is rising, or now it is the best time to join and help a little bit: