Spark Dependency Libraries

Spark Dependencies

MLlib uses the linear algebra package Breeze, which depends on netlib-java, and jblas. netlib-java and jblas depend on native Fortran routines. You need to install the gfortran runtime library if it is not already present on your nodes. MLlib will throw a linking error if it cannot detect these libraries automatically. Due to license issues, we do not include netlib-java’s native libraries in MLlib’s dependency set under default settings. If no native library is available at runtime, you will see a warning message. To use native libraries from netlib-java, please build Spark with -Pnetlib-lgpl or include com.github.fommil.netlib:all:1.1.2 as a dependency of your project. If you want to use optimized BLAS/LAPACK libraries such as OpenBLAS, please link its shared libraries to /usr/lib/libblas.so.3 and /usr/lib/liblapack.so.3, respectively. BLAS/LAPACK libraries on worker nodes should be built without multithreading.

To use MLlib in Python, you will need NumPy version 1.4 or newer.

JNI JBlas

jblas is a linear algebra library, created by Mikio Braun, for the Java programming language built upon BLAS and LAPACK. Unlike most other Java linear algebra libraries, jblas is designed to be used with native code through the Java Native Interface (JNI) and comes with pre-compiled binaries. When used on one of the targeted architectures, it will automatically select the correct binary to use and load it. This allows it to be used out of the box and avoid a potentially tedious compilation process. jblas provides an easier to use high level API on top of the archaic API provided by BLAS and LAPACK, removing much of the tediousness.

More information - http://everything.explained.today/Jblas%3A_Linear_Algebra_for_Java/

Main Features:

  • Matrix library for JAVA based on native BLAS and LAPACK Support for single and double precision floats, real and complex matrices Vectors and two-dimensional dense matrices only For large matrix practically native performance Precompiled "fat jar" for Linux(Intel), MacOS, and Windows Parses FORTRAN code to automatically generate JNI stubs.

Jblas User's group - https://groups.google.com/forum/#!forum/jblas-users

How to compile JNI JBlas

The following set of commands work without using -DskipTests while compiling the JBlas JNI libraries

git clone https://github.com/mikiobraun/jblas.git
cd jblas
./configure --download-lapack
./configure --lapack=lapack-lite-3.1.1
make clean all
mvn clean package

The following are the sample logs for compilation and generation of JAR file on ARM64 machine. Note that you need to install some dependency package like libfortran, libfortran3 before doing compilation

ubuntu@arm64:~/LEG-242/jblas$ ./configure --lapack=lapack-lite-3.1.1
checking for java, javac... ok
determining operating system... ok (Linux)
determining architecture... ok (aarch64)
determining architecture flavor... ok
looking for nm... ok
locating the Java Development Kit... ok (/home/ubuntu/jdk8-1602-mvn-3-3-9/jdk8-server-release-1602)
Setting up gcc and flags... ok (gcc)
deciding whether to use g77 or gfortran... ok (gfortran)
looking for version of make... ok (make)
search for lapack sources (configure by --lapack=dir)... ok (lapack-lite-3.1.1)
determining build type... ok (dynamic)
getting library path...... ok (["/usr/lib", "/lib", "/usr/lib/sse2"])
determining whether to build for lapack or atlas... ok (atlas)
looking for libraries......
{"lapack"=>"/usr/lib", "f77blas"=>"/usr/lib", "cblas"=>"/usr/lib", "atlas"=>"/usr/lib"}

ok

Configuration succesfull, writing out results to configure.out
ubuntu@arm64:~/LEG-242/jblas$ make clean all
rm -f native/.o native/.so \
src/main/resources/lib/dynamic/Linux/aarch64/libjblas.so src/main/resources/lib/dynamic/Linux/aarch64/libjblas_arch_flavor.so \
src/main/resources/lib/dynamic/Linux/aarch64/libjblas.so src/main/resources/lib/dynamic/Linux/aarch64/libjblas_arch_flavor.so \
src/main/java/org/jblas/NativeBlas.java src/main/c/NativeBlas.c generated-sources
ruby scripts/fortranwrapper.rb --complexcc c99 org.jblas NativeBlas \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]copy.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]swap.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]axpy.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]scal.f \
lapack-lite-3.1.1/BLAS/SRC/[cz][sd]scal.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]dot*.f \
lapack-lite-3.1.1/BLAS/SRC/[sd]*nrm2.f \
lapack-lite-3.1.1/BLAS/SRC/[sd]*asum.f \
lapack-lite-3.1.1/BLAS/SRC/i[sdcz]amax.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]gemv.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]ger*.f \
lapack-lite-3.1.1/BLAS/SRC/[sdcz]gemm.f \
lapack-lite-3.1.1/SRC/[sd]gesv.f \
lapack-lite-3.1.1/SRC/[sd]sysv.f \
lapack-lite-3.1.1/SRC/[sd]syev.f \
lapack-lite-3.1.1/SRC/[sd]syev[rdx].f \
lapack-lite-3.1.1/SRC/[sd]posv.f \
lapack-lite-3.1.1/SRC/[sdcz]geev.f \
lapack-lite-3.1.1/SRC/[sd]getrf.f \
lapack-lite-3.1.1/SRC/[sd]potrf.f \
lapack-lite-3.1.1/SRC/[sdcz]gesvd.f \
lapack-lite-3.1.1/SRC/[sd]sygvd.f \
lapack-lite-3.1.1/SRC/[sd]gelsd.f \
lapack-lite-3.1.1/SRC/ilaenv.f \
lapack-lite-3.1.1/SRC/[sd]geqrf.f lapack-lite-3.1.1/SRC/[sd]ormqr.f \
lapack-lite-3.1.1/SRC/[sd]orgqr.f \
lapack-lite-3.1.1/SRC/[sd]sygvx.f
complex calling convention = c99
Using dumped routines...
ant javah
Buildfile: /home/ubuntu/LEG-242/jblas/build.xml

prepare:

javah:
[javac] /home/ubuntu/LEG-242/jblas/build.xml:142: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 1 source file to /home/ubuntu/LEG-242/jblas/target/classes

BUILD SUCCESSFUL
Total time: 4 seconds
touch generated-sources
gcc -fPIC -Iinclude -I/home/ubuntu/jdk8-1602-mvn-3-3-9/jdk8-server-release-1602/include -I/home/ubuntu/jdk8-1602-mvn-3-3-9/jdk8-server-release-1602/include/linux -c src/main/c/NativeBlas.c -o target/c/NativeBlas.o
gcc -shared -L/usr/lib -o target/c/NativeBlas.so target/c/NativeBlas.o -llapack -lf77blas -lcblas -latlas
mv "target/c/NativeBlas.so" "src/main/resources/lib/dynamic/Linux/aarch64/libjblas.so"
gcc -fPIC -Iinclude -I/home/ubuntu/jdk8-1602-mvn-3-3-9/jdk8-server-release-1602/include -I/home/ubuntu/jdk8-1602-mvn-3-3-9/jdk8-server-release-1602/include/linux -c src/main/c/jblas_arch_flavor.c -o src/main/c/jblas_arch_flavor.o
gcc -shared -L/usr/lib -o src/main/c/jblas_arch_flavor.so src/main/c/jblas_arch_flavor.o -llapack -lf77blas -lcblas -latlas
mv "src/main/c/jblas_arch_flavor.so" "src/main/resources/lib/dynamic/Linux/aarch64/libjblas_arch_flavor.so"
rm src/main/c/jblas_arch_flavor.o
ubuntu@arm64:~/LEG-242/jblas$ mvn clean package
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building jblas 1.2.4-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] — maven-clean-plugin:2.5:clean (default-clean) @ jblas —
[INFO] Deleting /home/ubuntu/LEG-242/jblas/target
[INFO]
[INFO] — maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jblas —
[INFO]
[INFO] — maven-antrun-plugin:1.3:run (generate-float-sources) @ jblas —
[INFO] Executing tasks

preprocess:

generate-float:
[echo] Generating float version of org.jblas.DoubleMatrix
[echo] Generating float version of org.jblas.DoubleFunction
[echo] Generating float version of org.jblas.ComplexDouble
[echo] Generating float version of org.jblas.TestDoubleMatrix
[echo] Generating float version of org.jblas.TestBlasDouble
[echo] Generating float version of org.jblas.ComplexDoubleMatrix
[echo] Add float versions to class org.jblas.SimpleBlas
[echo] Add float versions to class org.jblas.Solve
[echo] Add float versions to class org.jblas.Eigen
[echo] Add float versions to class org.jblas.Geometry
[echo] Add float versions to class org.jblas.MatrixFunctions
[echo] Add float versions to class org.jblas.JavaBlas
[echo] Add float versions to class org.jblas.Singular
[INFO] Executed tasks
[INFO]
[INFO] — maven-resources-plugin:2.6:resources (default-resources) @ jblas —
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 18 resources
[INFO]
[INFO] — maven-compiler-plugin:2.3.2:compile (default-compile) @ jblas —
[INFO] Compiling 56 source files to /home/ubuntu/LEG-242/jblas/target/classes
[INFO]
[INFO] — maven-resources-plugin:2.6:testResources (default-testResources) @ jblas —
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/ubuntu/LEG-242/jblas/src/test/resources
[INFO]
[INFO] — maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ jblas —
[INFO] Compiling 16 source files to /home/ubuntu/LEG-242/jblas/target/test-classes
[INFO]
[INFO] — maven-surefire-plugin:2.12.4:test (default-test) @ jblas —
[INFO] Surefire report directory: /home/ubuntu/LEG-242/jblas/target/surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.jblas.TestEigen
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.204 sec
Running org.jblas.TestGeometry
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.jblas.TestFloatMatrix
Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.06 sec
Running org.jblas.TestDoubleMatrix
Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec
Running org.jblas.SimpleBlasTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.jblas.TestComplexFloat
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.jblas.TestBlasDoubleComplex
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running org.jblas.TestSolve
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec
Running org.jblas.TestSingular
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec
Running org.jblas.ComplexDoubleMatrixTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.jblas.TestDecompose
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec
Running org.jblas.TestBlasDouble
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
Running org.jblas.ranges.RangeTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
Running org.jblas.TestBlasFloat
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
– org.jblas INFO Deleting /tmp/jblas330364875467356675/libjblas.so
– org.jblas INFO Deleting /tmp/jblas330364875467356675/libjblas_arch_flavor.so
– org.jblas INFO Deleting /tmp/jblas330364875467356675

Results :

Tests run: 120, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] — maven-jar-plugin:2.3.1:jar (default-jar) @ jblas —
[INFO] Building jar: /home/ubuntu/LEG-242/jblas/target/jblas-1.2.4-SNAPSHOT.jar
[INFO]
[INFO] >>> maven-source-plugin:2.2.1:jar (default) > generate-sources @ jblas >>>
[INFO]
[INFO] — maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jblas —
[INFO]
[INFO] — maven-antrun-plugin:1.3:run (generate-float-sources) @ jblas —
[INFO] Executing tasks

preprocess:

generate-float:
[echo] Generating float version of org.jblas.DoubleMatrix
[echo] Generating float version of org.jblas.DoubleFunction
[echo] Generating float version of org.jblas.ComplexDouble
[echo] Generating float version of org.jblas.TestDoubleMatrix
[echo] Generating float version of org.jblas.TestBlasDouble
[echo] Generating float version of org.jblas.ComplexDoubleMatrix
[echo] Add float versions to class org.jblas.SimpleBlas
[echo] Add float versions to class org.jblas.Solve
[echo] Add float versions to class org.jblas.Eigen
[echo] Add float versions to class org.jblas.Geometry
[echo] Add float versions to class org.jblas.MatrixFunctions
[echo] Add float versions to class org.jblas.JavaBlas
[echo] Add float versions to class org.jblas.Singular
[INFO] Executed tasks
[INFO]
[INFO] <<< maven-source-plugin:2.2.1:jar (default) < generate-sources @ jblas <<<
[INFO]
[INFO] — maven-source-plugin:2.2.1:jar (default) @ jblas —
[INFO] Building jar: /home/ubuntu/LEG-242/jblas/target/jblas-1.2.4-SNAPSHOT-sources.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.225 s
[INFO] Finished at: 2016-04-21T10:07:46+00:00
[INFO] Final Memory: 22M/800M
[INFO] ------------------------------------------------------------------------
ubuntu@arm64:~/LEG-242/jblas$
ubuntu@arm64:~/LEG-242/jblas$

You can run the generated JAR file as below

ubuntu@arm64:~/LEG-242/jblas/target$ java -jar /home/ubuntu/LEG-242/jblas/target/jblas-1.2.4-SNAPSHOT.jar
– org.jblas INFO jblas version is 1.2.4
Simple benchmark for jblas

Running sanity benchmarks.

checking vector addition... ok
– org.jblas CONFIG BLAS native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH).
– org.jblas CONFIG ArchFlavor native library not found in path. Copying native library libjblas_arch_flavor from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH).
– org.jblas CONFIG Loading libjblas_arch_flavor.so from /lib/dynamic/Linux/aarch64/, copying to libjblas_arch_flavor.so.
– org.jblas CONFIG Loading libjblas.so from /lib/dynamic/Linux/aarch64/, copying to libjblas.so.
checking matrix multiplication... ok
checking existence of dsyev...... ok
[-0.210656, -0.640445, -0.444736; -0.509085, -0.116445, 0.800107; -0.807515, 0.407556, -0.400053; 0.210656, 0.640445, -0.044682]
[17.233688; 1.414214; 0.000000]
[-0.470605, 0.782218, 0.408248; -0.571449, 0.082339, -0.816497; -0.672293, -0.617540, 0.408248]
[17.233688; 1.414214; 0.000000]
checking existence of dgesvd...... ok
Checking complex return values... (z = -21.0 + 88.0i)
Check whether we're catching XERBLA errors. If you see something like "** On entry to DGEMM parameter number 4 had an illegal value", it didn't work!
checking XERBLA... ok
Sanity checks passed.

Each benchmark will take about 5 seconds...

Running benchmark "Java matrix multiplication, double precision".
n = 10 : 0.216 GFLOPS (538802 iterations in 5.0 seconds)
n = 100 : 0.209 GFLOPS (523 iterations in 5.0 seconds)
n = 1000 : 0.212 GFLOPS (1 iterations in 9.5 seconds)

Running benchmark "Java matrix multiplication, single precision".
n = 10 : 0.216 GFLOPS (541152 iterations in 5.0 seconds)
n = 100 : 0.225 GFLOPS (564 iterations in 5.0 seconds)
n = 1000 : 0.227 GFLOPS (1 iterations in 8.8 seconds)

Running benchmark "native matrix multiplication, double precision".
n = 10 : 0.195 GFLOPS (486989 iterations in 5.0 seconds)
n = 100 : 0.520 GFLOPS (1300 iterations in 5.0 seconds)
n = 1000 : 0.599 GFLOPS (2 iterations in 6.7 seconds)

Running benchmark "native matrix multiplication, single precision".
n = 10 : 0.217 GFLOPS (541664 iterations in 5.0 seconds)
n = 100 : 0.566 GFLOPS (1415 iterations in 5.0 seconds)
n = 1000 : 0.634 GFLOPS (2 iterations in 6.3 seconds)
– org.jblas INFO Deleting /tmp/jblas7843696522370548912/libjblas.so
– org.jblas INFO Deleting /tmp/jblas7843696522370548912/libjblas_arch_flavor.so
– org.jblas INFO Deleting /tmp/jblas7843696522370548912
ubuntu@arm64:~/LEG-242/jblas/target$

How to confirm JNI JBlas is used

Build the spark with following lines added into pom.xml file.

ubuntu@arm64:~/LEG-240/spark$ git diff
diff --git a/pom.xml b/pom.xml
index e2730ee..aab1e24 100644
--- a/pom.xml
+++ b/pom.xml
@@ -250,6 +250,11 @@
     </pluginRepository>
   </pluginRepositories>
   <dependencies>
+    <dependency>
+      <groupId>org.jblas</groupId>
+      <artifactId>jblas</artifactId>
+      <version>1.2.4</version>
+    </dependency>
     <!--
       This is a dummy dependency that is used to trigger the maven-shade plugin so that Spark's
       published POMs are flattened and do not contain variables. Without this dependency, some
ubuntu@arm64:~/LEG-240/spark$ 

To confirm that the jblas jar file is being used. You can do the following: copy the JBlas jar file i.e jblas-1.2.4-SNAPSHOT.jar into the following path /usr/local/hadoop/share/hadoop/common/lib/ on ARM64 system and also I have compiled the spark by enabling the netlib

i.e. use the "mvn -DskipTests -Pnetlib-lgpl clean package"

from the webpage - https://spark.apache.org/docs/latest/mllib-guide.html Now, run the SparkPi, SparkALS examples and check the logs. I can see the below lines,

spark$ ./bin/run-example SparkPi 10
....
..........
INFO SparkContext: Added JAR.....jblas-1.2.4.jar at spark:/...
....
....
INFO Executor: Fetching spark:..../jblas-1.2.4.jar with timestamp
Utils: Fetching spark:..../jblas-1.2.4.jar
INFO Executor: Adding file:..../jblas-1.2.4.jar to class loader
...
....

The above logs I am unable to see it when I compile the spark using "mvn -DskipTests clean package" command i.e note that I have not used the "-Pnetlib-lgpl" switch case in the command and run the SparkPi example.

This confirms that the JBlas JNI libraries are being used while running the Spark examples.

For more information you can refer jira issue LEG-242

JNA Libraries

Java Native Access (JNA) is a community-developed library that provides Java programs easy access to native shared libraries without using the Java Native Interface. JNA's design aims to provide native access in a natural way with a minimum of effort. No boilerplate or generated glue code is required.

How to compile and generate JNA jar file

The below set of commands guide you to compile and generate the jar file of JNA.

ubuntu@arm64:~/LEG-240$ git clone https://github.com/java-native-access/jna.git
ubuntu@arm64:~/LEG-240$ cd jna
ubuntu@arm64:~/LEG-240/jna$ ant native; ant jar
ubuntu@arm64:~/LEG-240/jna$ jar tf /home/ubuntu/LEG-240/jna/build/jna.jar|grep aarch64
com/sun/jna/linux-aarch64/
com/sun/jna/linux-aarch64/libjnidispatch.so
ubuntu@arm64:~/LEG-240/jna$

Verify JNA jar file

You can copy the jar file into some hadoop location e.g. /usr/local/hadoop/share/hadoop/common/lib/ and run JNA example which uses the jar.

ubuntu@arm64:~/LEG-240$ export CLASSPATH=.:/home/ubuntu/LEG-240/jna/build/jna.jar
ubuntu@arm64:~/LEG-240$ javac -g HelloWorld.java
ubuntu@arm64:~/LEG-240$
ubuntu@arm64:~/LEG-240$ java HelloWorld
Hello, World
ubuntu@arm64:~/LEG-240$

You can refer LEG-240 jira issue more information.

LevelDB JNI libraries

LevelDB JNI gives you a Java interface to the LevelDB C++ library which is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

Prerequisites

* GNU compiler toolchain

* Maven 3

* apt-get install autoconf libtool

* apt-get install libxt-dev libffi-dev automake libx11-dev

Build Procedure

Since levedbjni has to be build against multiple platforms, the standard maven release plug-in will not work to do the release. So the build procedure is as below download the snappy, leveldb, and leveldbjni project source code:

Download the latest snappy version from the snappy because this contains the _aarch64_ support patch.

$ wget https://github.com/google/snappy/releases/download/1.1.3/snappy-1.1.3.tar.gz
$ tar -zxvf snappy-1.1.3.tar.gz
$ git clone  https://github.com/google/leveldb
$ git clone git://github.com/fusesource/leveldbjni.git
$ export SNAPPY_HOME=`cd snappy-1.1.3; pwd`
$ export LEVELDB_HOME=`cd leveldb; pwd`
$ export LEVELDBJNI_HOME=`cd leveldbjni; pwd`

Compile the snappy project. This produces a static library.

$ cd ${SNAPPY_HOME}
$ ./configure --disable-shared --with-pic
$ make

Patch and Compile the leveldb project. This produces a static library.

$ cd ${LEVELDB_HOME}
$ export LIBRARY_PATH=${SNAPPY_HOME}
$ export C_INCLUDE_PATH=${LIBRARY_PATH}
$ export CPLUS_INCLUDE_PATH=${LIBRARY_PATH}
$ git apply ../leveldbjni/leveldb.patch
$ make

You may get the below error while compiling on ARM64 machine,

ubuntu@arm64:~/LEG-241/leveldb$ make
g++ -pthread -shared -Wl,-soname -Wl,libleveldb.so.1 out-shared/db/builder.o out-shared/db/c.o out-shared/db/dbformat.o out-shared/db/db_impl.o out-shared/db/db_iter.o out-shared/db/dumpfile.o out-shared/db/filename.o out-shared/db/log_reader.o out-shared/db/log_writer.o out-shared/db/memtable.o out-shared/db/repair.o out-shared/db/table_cache.o out-shared/db/version_edit.o out-shared/db/version_set.o out-shared/db/write_batch.o out-shared/table/block_builder.o out-shared/table/block.o out-shared/table/filter_block.o out-shared/table/format.o out-shared/table/iterator.o out-shared/table/merger.o out-shared/table/table_builder.o out-shared/table/table.o out-shared/table/two_level_iterator.o out-shared/util/arena.o out-shared/util/bloom.o out-shared/util/cache.o out-shared/util/coding.o out-shared/util/comparator.o out-shared/util/crc32c.o out-shared/util/env.o out-shared/util/env_posix.o out-shared/util/filter_policy.o out-shared/util/hash.o out-shared/util/histogram.o out-shared/util/logging.o out-shared/util/options.o out-shared/util/status.o out-shared/port/port_posix.o -o out-shared/libleveldb.so.1.18 -lsnappy
/usr/bin/ld: cannot find -lsnappy
collect2: error: ld returned 1 exit status
make: *** [out-shared/libleveldb.so.1.18] Error 1
ubuntu@arm64:~/LEG-241/leveldb$

Note that please install all the missing packages on your distribution e.g.

ubuntu@arm64:~$ sudo apt-get install libsnappy-dev

LEG/Engineering/BigData/SparkDependencyLibraries (last modified 2016-09-21 05:58:29)