paritoshgote Comments - Answer Overflow

paritoshgote

•Created by paritoshgote on 6/1/2023 in #questions

JanusGraph on Azure HDInsights Spark Cluster

Actually, after copying all the binaries from the /usr/hdp/4.1.14.2/hadoop/ location to the Janus distro lib location, we see this error. Any tips/recommendations appreciated! Exception in thread "main" java.lang.AbstractMethodError: org.apache.hadoop.metrics2.sink.WasbAzureIaasSink.init(Lorg/apache/hadoop/shaded/org/apache/commons/configuration2/SubsetConfiguration;)V at org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:209) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:535) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configureSinks(MetricsSystemImpl.java:507) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:483) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:188) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:163) at org.apache.hadoop.fs.azure.metrics.AzureFileSystemMetricsSystem.fileSystemStarted(AzureFileSystemMetricsSystem.java:41)

3 replies

JJanusGraph

•Created by Florian Hockmann on 5/24/2023 in #questions

Plans for the Hadoop integration

@florianhockmann : Created a separate post here https://discord.com/channels/981533699378135051/1113947036664995880, thanks 🙂

8 replies

JJanusGraph

•Created by Florian Hockmann on 5/24/2023 in #questions

Plans for the Hadoop integration

@HadoopMarc , @florianhockmann , @boxuanli : We are running into an issue while launching the gremlin.sh with hadoop (ref: https://tinkerpop.apache.org/docs/current/reference/#hadoop-gremlin) using janus. Would you folks have any tips for this issue? Error with Janus 0.6.3: 20:00:36 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Exception in thread "main" java.lang.RuntimeException: java.io.IOException: No FileSystem for scheme: wasbs I tried using the latest janus 1.0.0-rc2 as well, and that gave a different error Error with Janus 1.0.0-rc2: 21:32:16 WARN org.apache.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem$Secure not found

sshuser@hn0-spark:~/janusgraph/janusgraph-1.0.0-rc2$ bin/gremlin.sh

plugin activated: tinkerpop.tinkergraph
21:32:16 INFO  org.apache.tinkerpop.gremlin.hadoop.jsr223.HadoopGremlinPlugin.getCustomizers - HADOOP_GREMLIN_LIBS is set to: /home/sshuser/janusgraph/janusgraph-1.0.0-rc2/lib
21:32:16 WARN  org.apache.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem$Secure not found
        at org.apache.tinkerpop.gremlin.hadoop.jsr223.HadoopGremlinPlugin.lambda$static$0(HadoopGremlinPlugin.java:125)

        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2665)
        ... 67 more

sshuser@hn0-spark:~/janusgraph/janusgraph-1.0.0-rc2$ bin/gremlin.sh

plugin activated: tinkerpop.tinkergraph
21:32:16 INFO  org.apache.tinkerpop.gremlin.hadoop.jsr223.HadoopGremlinPlugin.getCustomizers - HADOOP_GREMLIN_LIBS is set to: /home/sshuser/janusgraph/janusgraph-1.0.0-rc2/lib
21:32:16 WARN  org.apache.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem$Secure not found
        at org.apache.tinkerpop.gremlin.hadoop.jsr223.HadoopGremlinPlugin.lambda$static$0(HadoopGremlinPlugin.java:125)

        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2665)
        ... 67 more

8 replies

Gaming

Programming