Tuesday, November 27, 2012

Compiling Storm (and jzmq) on Mac OSX


I recently setup a new machine, went to compile Storm, and forgot what a PITA it is.  Here is a blog post so future-me knows how to do it.

The biggest hurdle is jzmq.   To get that installed, your going to need some make mojo, and some libraries.  I use Brew.  If you are still using macports, consider a switch.

Anyway, here is the recipe to get jzmq installed assuming you have Brew.
$ git clone https://github.com/nathanmarz/jzmq.git
$ brew install automake
$ brew install libtool
$ brew install zmq
$ cd jzmq
$ ./configure

That resulted in a cryptic message:
cannot find jni.h in /Library/Java/Home/include 

To fix that, I found a suggested fix and created a symbolic link.
$ sudo ln -s /System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/ /Library/Java/Home/include

Then, you hit this:
No rule to make target `classdist_noinst.stamp'

I had to dig through Google cache archives to find a solution for that one:
$ touch src/classdist_noinst.stamp

Then, you hit:
error: cannot access org.zeromq.ZMQ
That's because you haven't compiled zeromq yet!  So:
$ cd src
$ javac -d . org/zeromq/*.java
Wasn't that easy? =)  All that's left is to build and install:
$ cd ..
$ make
$ sudo make install

Now you have jzmq installed.  So we can get on with Storm.  Storm needs lein to build.  Don't go grabbing the latest version of lein either.   You'll need < 2.  There is an explicit check that was added to Storm that will refuse to build with lein >= 2.  You can grab older versions here. (we use 1.7.1)

Unzip that and copy $LEIN_HOME/bin/lein to your bin directory.  Make it executable and put it in your path.  Once you've done that, building Storm isn't so bad.  From the root of the storm source tree:
$ lein deps
$ lein jar 
$ lein install

Happy Storming.

11 comments:

Nick J said...

This helped a tremendous amount. Thanks for the blog post.

Nick J said...

This helped a tremendous amount. Thanks for the blog post.

Scott Stults said...
This comment has been removed by the author.
Scott Stults said...

This may be peculiar to my version of Java (I probably installed 7 before it was well vetted). In addition to what you have, it complained about missing jni_md.h. There was something on the mailing list about this:

http://comments.gmane.org/gmane.network.zeromq.devel/1083

I'm not sure what changed, but I wasn't able to proceed until I re-added the "darwin" folder back to the include path. I did that by adding this to line 108 of configure.in:

CPPFLAGS="$CPPFLAGS -I${JAVA_HOME}/include/darwin"

I haven't tried it out yet to see if I get the errors with device names they mention in the mailing list, but the included Java tests worked with just "mvn test".

mahasiswa teladan said...

hi...Im student from Informatics engineering nice article,
thanks for sharing :)

Unknown said...
This comment has been removed by the author.
Svend Vanderveken said...

Thanks for your article, I tried to build Storm from source this afternoon and that's indeed not as straightforward as it should.

I used the method you described, and then modified a few aspects to make sure the dependencies version were aligned. I posted the result here, for info:

http://svendvanderveken.wordpress.com/2013/09/04/how-to-compile-storm-0-8-2-on-mac-os-x/

Nagendra Addepalli said...

Thank you., Its very helpful for me

numb3rs1x said...

This saved me so much time and confusion. The added details from Svend got me the rest of the way. Thank you both!

numb3rs1x said...

This saved me so much time and confusion. The added details from Svend got me the rest of the way. Thank you both!

desipreneur said...

On running ./configure I get following when install jzmq

/configure: line 15039: syntax error near unexpected token `newline'
./configure: line 15039: ` PKG_CHECK_MODULES('