Saturday, January 4, 2014

Continuous integration problems : Jenkins, JVM Class loader, permgen error and hotdeploy


As I figured out that my previous blog posts are mostly related to front-end technologies, I decided to share some knowledge related to dev operations - project build and deployment automation. I won't go too much in detail about introduction to continuous integration (CI) systems, and assume that reader of this blog is aware of it's role in software development cycle, and rather focus on one particular problem.

Whatever we are using maven, ant or some other build system to build our Java (or any other JVM compatible language like Groovy, Scala etc.), we'll probably want to deploy it to some application server in the spirit of automation, and offloading programmable tasks from humans to machines. Problem arises when doing hot deployment to application server like Tomcat, where after couple of deployments memory reserved in JVM for class definitions and other non-mutable objects (so called PermGen space) becomes full, and we are getting so called 'Permgen error'.

Turns out that there is simple solution to this problem, while in development environment. For production environments I've strongly recommend doing cold deployment. It's simple as restarting Java servlet container (Tomcat, JBoss, Glassfish.) after doing package deployment. I recommend deploying war/jar file using Jenkins Deploy plugin (alternative to this would be to use script to perform this), while restarting tomcat is simple as executing command (on *NIX environment). If you have installed Tomcat as a service:

service tomcat restart

You can execute script on Jenkins using PostBuild script plugin. Note that it's essential for above approach to work to run Jenkins outside of Servlet container that you're restarting, and that jenkins user has permissions to perform restart operation.

With Servlet container restarted, there is overhead in time delay while waiting for  application server to boot, but JVM is essentially restarted, so PermGen memory does not get filled up, and you don't have to login into your development box every-once-in-a-while and reboot it.