Rockstars drive Teslas, not F-150’s (Mac vs. Windows for develpment)

I do quite a bit of speaking at technology events.  And these days, when you are up on stage, there is a sea of glowing Apples staring back at you.  Almost every software developer I know cranks code on a Macbook Pro.  It’s become a status symbol, and a recruiting necessity.  Honestly though, it’s just a more productive platform.  With it, I can go further on less gas.  

It wasn’t always that way however...

I spent my elementary school years coding on an Apple IIc, hacking assembly code to get faster graphics routines.  For many of my generation, IIe's and IIc's were what we grew up with.  Every classroom had one.

When it came to buying my first machine however, I saved my nickels and dimes and bought an Intel 8088, with a monochrome graphics card.  Why? Because, that was all I could afford in middle school. It wasn’t powerful or sexy.  I had to work to squeeze every last bit of performance out of it, which is likely why I then spent the next six years, cutting my teeth on every operating system out there: Linux, FreeBSD, Solaris, Windows and OS Warp.  (ooh, ahh, multi-tasking!)

I learned a ton, but I was definitely *not* productive.  I spent so much time fighting with the machinery that I didn’t have time to solve people’s problems.  And evidently, solving people’s problems is how you make money.

Admittedly, I partially blame my OCD for the hours I wasted away tweaking windows managers (much love for fvwm) and xterms so I never had to touch the mouse.  But the fact of the matter is, I was wasting time fighting and customizing my tools instead of using those tools to craft products and solutions.  Minimally, I’d say the operating systems and tools were distractions, but sometimes they were actual impediments.

Then came OS X.   An operating system with the power of unix, with none of the impediments and distractions.  I was freed to focus on problem solving.  And I never looked back.

Now -- these days, when I go to conferences, everyone has a mac.   Or maybe I should say everyone who’s anyone has a mac.  That’s not entirely true, but that is certainly the way it feels.  (* One exception to this rule is the hard-core hackers running the latest flavor of linux, hacking at such low-levels that they are one with the code. For them, I pour a little out.)

There is no doubt that Mac’s are now ubiquitous in the development community. But when I was recently approached by an executive of a local technology company wondering if they should introduce Mac’s into their environment, I had trouble expressing concrete functional differences (especially differences substantial enough to offset the cost of supporting two platforms).  Quite literally, Mac’s have that: ‘je ne sais quoi’.

After thinking it over a bit, I thought it might best to draw an analogy...  

It is true that Windows machines and Mac’s have very similar functional components and capabilities (cpu, storage, memory, etc.).  So do Tesla’s and Ford F-150’s (wheels, air conditioning, engines). Both will comfortably get you from point A to point B. And depending on the driver, it might even take the same amount of time.  There are even situations and terrain for which an F150 is more suited (e.g. ass-backwards country roads).

But between the two, the experience is completely different.  If you are looking to attract rockstar race car drivers, you’ll attract more with Teslas than you will with F150s.  And if you give a Tesla to the right race car driver, they will have you rocketing down the information superhighway, while you both enjoy the ride.

I love my MBP (macbook pro), and while I can’t afford a Tesla yet – it’s the closest thing I have to one. =)

Maybe someday the tables will turn, but for today – fuel your crew with mac’s, get them an assortment of decals (http://www.macdecals.com/), and keep them hydrated.   Sit back and enjoy the ride.

[YARN] : NullPointerException in MRClientService.getHttpPort()

Have you moved your Hadoop jobs over to Yarn?  Are you seeing the following NullPointerException coming out of your job?

Caused by: java.lang.NullPointerException
 at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:167)
 at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:150)
 ... 14 more

This appears to be a poorly handled case in Hadoop, where the webApp fails to load.  You need to look at the full log file.  You will likely see the following in the log preceeding the NPE:

ERROR (org.apache.hadoop.mapreduce.v2.app.client.MRClientService:139) - Webapps failed to start. Ignoring for now:

That line is produced by the following code in YARN:

    LOG.info("Instantiated MRClientService at " + this.bindAddress);
    try {
      webApp = WebApps.$for("mapreduce", AppContext.class, appContext, "ws").with(conf).
          start(new AMWebApp());
    } catch (Exception e) {
      LOG.error("Webapps failed to start. Ignoring for now:", e);
    }
    super.start();

This means your webApp failed to load.  In my case, this was caused by dependency conflicts on my classpath.  (servlet & jasper compiler)  I've also seen it cause by a dependency mssing, such as:

java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer

In my situation, the solution was to cleanup my classpath, but the webApp might fail for lots of reasons, which will result in an NPE later in the log file when MRClientService attempts to getHttpPort without a webApp, which is shown in the following code:

public int getHttpPort() {
    return webApp.port();
  }

Anyway, that is probably more detail than you wanted. Happy yarning.