Java - Accumulo init exception on OSX Mavericks - log4j logger

Question

I am trying to get Accumulo working on OSX Mavericks. I have Java 1.8, and the most recent Zookeeper and Hadoop installed via Homebrew. Both of those appears to be working properly.

I downloaded the binaries for Accumulo and I have edited these lines in accumulo/conf/accumulo-env.sh:

test -z "$HADOOP_PREFIX"      && export HADOOP_PREFIX=/usr/local/Cellar/hadoop/2.4.0
test -z "$HADOOP_CONF_DIR"     && export HADOOP_CONF_DIR="$HADOOP_PREFIX/libexec/etc/hadoop"
test -z "$JAVA_HOME"             && export JAVA_HOME=$(/usr/libexec/java_home)
test -z "$ZOOKEEPER_HOME"        && export ZOOKEEPER_HOME=/usr/local/Cellar/zookeeper/3.4.6

However, when I run bin/accumulo init, I get:

$ bin/accumulo init
Uncaught exception: java.lang.NoClassDefFoundError: org/apache/log4j/Logger
java.lang.NoClassDefFoundError: org/apache/log4j/Logger
    at org.apache.accumulo.start.classloader.AccumuloClassLoader.<clinit>(AccumuloClassLoader.java:78)
    at org.apache.accumulo.start.Main.main(Main.java:39)
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Logger
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 2 more

I cannot find any useful results searching for this. I am not sure what could be improperly configured to cause such an error, so I'm not sure where to start.

I have not made any changes to the configurations for Zookeeper or Hadoop, so Hadoop is in its basic standalone mode, which I have confirmed to be working via one of their examples. Zookeeper starts/stops properly with zkServer. So I'm guessing the problem is in my Accumulo configuration. Unfortunately their README is not helpful at all for my situation, and basically just tells me that the Accumulo binaries I've downloaded should just work out of the box.

I have tried running this with the environment variable JAVA_HOME both set and unset, and with it explicitly defined in accumulo-env.sh.

Thank you for anything that can point me in the right direction. My end-goal is really to be tinkering with GeoMesa, which depends on Accumulo working.

Answer 1

When you configured Accumulo, did you edit conf/accumulo-site.xml?

I am assuming you are using Accumulo 1.5.1 and it looks like you are using Hadoop 2.4.0. Inside of conf/accumulo-site.xml you need to add some paths to the general.classpaths property to pick up all of the JARs used by Hadoop.

Here is what your property for general.classpaths should look like:

<property>
  <name>general.classpaths</name>
  <!--
     Add the following for Hadoop2, actual needs depend on Hadoop installation details. 
     This list may be excessive, but this should cause no issues. Append these values
     after the $HADOOP_PREFIX entries

     $HADOOP_PREFIX/share/hadoop/common/.*.jar,
     $HADOOP_PREFIX/share/hadoop/common/lib/.*.jar,
     $HADOOP_PREFIX/share/hadoop/hdfs/.*.jar,
     $HADOOP_PREFIX/share/hadoop/mapreduce/.*.jar,
     $HADOOP_PREFIX/share/hadoop/yarn/.*.jar,
     /usr/lib/hadoop/.*.jar,
     /usr/lib/hadoop/lib/.*.jar,
     /usr/lib/hadoop-hdfs/.*.jar,
     /usr/lib/hadoop-mapreduce/.*.jar,
     /usr/lib/hadoop-yarn/.*.jar,
  -->
  <value>
    $ACCUMULO_HOME/server/target/classes/,
    $ACCUMULO_HOME/lib/accumulo-server.jar,
    $ACCUMULO_HOME/core/target/classes/,
    $ACCUMULO_HOME/lib/accumulo-core.jar,
    $ACCUMULO_HOME/start/target/classes/,
    $ACCUMULO_HOME/lib/accumulo-start.jar,
    $ACCUMULO_HOME/fate/target/classes/,
    $ACCUMULO_HOME/lib/accumulo-fate.jar,
    $ACCUMULO_HOME/proxy/target/classes/,
    $ACCUMULO_HOME/lib/accumulo-proxy.jar,
    $ACCUMULO_HOME/lib/[^.].*.jar,
    $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
    $HADOOP_CONF_DIR,
    $HADOOP_PREFIX/[^.].*.jar,
    $HADOOP_PREFIX/lib/[^.].*.jar,

    <!-- Added for Hadoop 2 -->
    /usr/lib/hadoop/.*.jar,
    /usr/lib/hadoop/lib/.*.jar,
    /usr/lib/hadoop-hdfs/.*.jar,
    /usr/lib/hadoop-mapreduce/.*.jar,
    /usr/lib/hadoop-yarn/.*.jar,
  </value>
  <description>Classpaths that accumulo checks for updates and class files.
    When using the Security Manager, please remove the ".../target/classes/" values.
  </description>
</property>

After configuring this, log4j should be found.