Isopropyl9
Isopropyl9
PostsComments
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
I have decided to just add a custom "id" property, as opposed to setting the actual ID controlled by janusgraph. I assumed providing custom IDs would be the most practical, but It seems that having a custom field works well enough for my purposes. Thank you for helping me with this, but I think there are too many intricate details for me to deal with and understand with this right now. As you suggested, the janusgraph.graph.set-vertex-id: true property in docker-compose.yml worked, but after trying to provide my own ID values it said that they were invalid. I have provided the full stack trace. Strangely, using an id of 0 returns a different error than other IDs, such as 1. 
> g.addV('person').property(T.id, 1).next()
gremlin_python.driver.protocol.GremlinServerError: 500: Not a valid vertex id: 1
> g.addV('person').property(T.id, 2).next()
gremlin_python.driver.protocol.GremlinServerError: 500: Not a valid vertex id: 2
> g.addV('person').property(T.id, 132331).next()
gremlin_python.driver.protocol.GremlinServerError: 500: Not a valid vertex id: 132331
> g.addV('person').property(T.id, 1).next()
gremlin_python.driver.protocol.GremlinServerError: 500: Not a valid vertex id: 1
> g.addV('person').property(T.id, 2).next()
gremlin_python.driver.protocol.GremlinServerError: 500: Not a valid vertex id: 2
> g.addV('person').property(T.id, 132331).next()
gremlin_python.driver.protocol.GremlinServerError: 500: Not a valid vertex id: 132331
Reading the documentation, it sounds like we're supposed to get a custom ID by using the ID manager like so: 
graph.getIDManager().fromVertexID(long)
graph.getIDManager().fromVertexID(long)
But I cannot figure out how to use this in gremlin-python. I believe it is janusgraph specific.
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
I also get the following messages to do with my "read only" volume at the beginning of my janusgraph logs: 
2023-11-10 18:16:07 cp: cannot create regular file '/etc/opt/janusgraph/janusgraph.properties': Read-only file system
2023-11-10 18:16:07 chown: changing ownership of '/etc/opt/janusgraph/janusgraph.properties': Read-only file system
2023-11-10 18:16:07 chmod: changing permissions of '/etc/opt/janusgraph/janusgraph.properties': Read-only file system
2023-11-10 18:16:07 sed: cannot rename /etc/opt/janusgraph/sedUO9BEQ: Device or resource busy
2023-11-10 18:16:07 sed: cannot rename /etc/opt/janusgraph/sedh8uf1G: Device or resource busy
2023-11-10 18:16:18 /etc/opt/janusgraph/janusgraph-server.yaml will be used to start JanusGraph Server in foreground
2023-11-10 18:16:07 cp: cannot create regular file '/etc/opt/janusgraph/janusgraph.properties': Read-only file system
2023-11-10 18:16:07 chown: changing ownership of '/etc/opt/janusgraph/janusgraph.properties': Read-only file system
2023-11-10 18:16:07 chmod: changing permissions of '/etc/opt/janusgraph/janusgraph.properties': Read-only file system
2023-11-10 18:16:07 sed: cannot rename /etc/opt/janusgraph/sedUO9BEQ: Device or resource busy
2023-11-10 18:16:07 sed: cannot rename /etc/opt/janusgraph/sedh8uf1G: Device or resource busy
2023-11-10 18:16:18 /etc/opt/janusgraph/janusgraph-server.yaml will be used to start JanusGraph Server in foreground
Could this be a problem? If I don't make it read-only, the config file gets overridden as soon as the container starts. Notice the "ro" at the end of my volume.
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
I would like to mention again that my docker config only contains the following: 
version: '3.8'
services:
  btc_janusgraph:
    # build: ./janusgraph
    image: janusgraph/janusgraph:latest
    container_name: btc_janusgraph
    environment:
      janusgraph.set-vertex-id: true
      set-vertex-id: true
      janusgraph.storage.backend: berkeleyje
      storage.backend: berkeleyje
    ports:
      - "${JANUSGRAPH_PORT:-8182}:${JANUSGRAPH_PORT:-8182}"
      - "8484:8184"
    networks:
      - btc-network
    volumes:
      - btc_janusgraph_data:/var/lib/janusgraph
      - "./janusgraph/janusgraph.properties:/etc/opt/janusgraph/janusgraph.properties:ro"
    healthcheck:
      test: ["CMD", "bin/gremlin.sh", "-e", "scripts/remote-connect.groovy"]
      interval: 10s
      timeout: 60s
      retries: 4
version: '3.8'
services:
  btc_janusgraph:
    # build: ./janusgraph
    image: janusgraph/janusgraph:latest
    container_name: btc_janusgraph
    environment:
      janusgraph.set-vertex-id: true
      set-vertex-id: true
      janusgraph.storage.backend: berkeleyje
      storage.backend: berkeleyje
    ports:
      - "${JANUSGRAPH_PORT:-8182}:${JANUSGRAPH_PORT:-8182}"
      - "8484:8184"
    networks:
      - btc-network
    volumes:
      - btc_janusgraph_data:/var/lib/janusgraph
      - "./janusgraph/janusgraph.properties:/etc/opt/janusgraph/janusgraph.properties:ro"
    healthcheck:
      test: ["CMD", "bin/gremlin.sh", "-e", "scripts/remote-connect.groovy"]
      interval: 10s
      timeout: 60s
      retries: 4
And my custom janusgraph.properties config file volume contains the following: 
gremlin.graph=org.janusgraph.core.JanusGraphFactory

storage.backend=berkeleyje
storage.directory=/var/lib/janusgraph/data
index.default.backend=lucene
index.default.directory=/var/lib/janusgraph/index
set-vertex-id=true
gremlin.graph=org.janusgraph.core.JanusGraphFactory

storage.backend=berkeleyje
storage.directory=/var/lib/janusgraph/data
index.default.backend=lucene
index.default.directory=/var/lib/janusgraph/index
set-vertex-id=true
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
I tried tearing down/rebuilding the docker container (and deleting volumes), and I got the same error to do with lock files. Then I tried deleting the lock file just to see what would happen, and I got the following. It seems that there is a global variable that is setting the indexing backend to elasticsearch. But I thought I was using lucene? I don't believe I need a fancy indexing backend for my simple project, so I thought lucene would make things more simple. And I am not sure how I could change these global values without being able to create a graph instance in the gremlin console.
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
And here is the stack trace when I run the same Python script:
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
Here is the stack trace for graph = JanusGraphFactory.open('conf/janusgraph-berkeleyje-server.properties')
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
I also tried the inmemory config file just in case, and I was able to successfully set the value, as you said: 
gremlin> graph = JanusGraphFactory.open('conf/janusgraph-inmemory-server.properties
')
04:41:33 INFO  org.janusgraph.diskstorage.configuration.builder.ReadConfigurationBuilder.setupTimestampProvider - Set default timestamp provider MICRO
04:41:33 INFO  org.janusgraph.graphdb.idmanagement.UniqueInstanceIdRetriever.getOrGenerateUniqueInstanceId - Generated unique-instance-id=c0a8e0025844-4b81751be49f1
04:41:33 INFO  org.janusgraph.diskstorage.configuration.ExecutorServiceBuilder.buildFixedExecutorService - Initiated fixed thread pool of size 24
04:41:33 INFO  org.janusgraph.diskstorage.Backend.initialize - Configuring total store cache size: 437259072
04:41:33 INFO  org.janusgraph.graphdb.database.StandardJanusGraph.<init> - Gremlin script evaluation is disabled
04:41:33 INFO  org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.initializeTimepoint - Loaded unidentified ReadMarker start time 2023-11-10T04:41:33.093469Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@72a0a60d
==>standardjanusgraph[inmemory:[127.0.0.1]]
gremlin> mgmt = graph.openManagement();
==>org.janusgraph.graphdb.database.management.ManagementSystem@2f4545c6
gremlin> mgmt.get('graph.set-vertex-id')
==>false
gremlin> mgmt.set("graph.set-vertex-id", true);
==>org.janusgraph.diskstorage.configuration.UserModifiableConfiguration@36f40d72
gremlin> mgmt.get('graph.set-vertex-id')
==>true
gremlin> graph = JanusGraphFactory.open('conf/janusgraph-inmemory-server.properties
')
04:41:33 INFO  org.janusgraph.diskstorage.configuration.builder.ReadConfigurationBuilder.setupTimestampProvider - Set default timestamp provider MICRO
04:41:33 INFO  org.janusgraph.graphdb.idmanagement.UniqueInstanceIdRetriever.getOrGenerateUniqueInstanceId - Generated unique-instance-id=c0a8e0025844-4b81751be49f1
04:41:33 INFO  org.janusgraph.diskstorage.configuration.ExecutorServiceBuilder.buildFixedExecutorService - Initiated fixed thread pool of size 24
04:41:33 INFO  org.janusgraph.diskstorage.Backend.initialize - Configuring total store cache size: 437259072
04:41:33 INFO  org.janusgraph.graphdb.database.StandardJanusGraph.<init> - Gremlin script evaluation is disabled
04:41:33 INFO  org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.initializeTimepoint - Loaded unidentified ReadMarker start time 2023-11-10T04:41:33.093469Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@72a0a60d
==>standardjanusgraph[inmemory:[127.0.0.1]]
gremlin> mgmt = graph.openManagement();
==>org.janusgraph.graphdb.database.management.ManagementSystem@2f4545c6
gremlin> mgmt.get('graph.set-vertex-id')
==>false
gremlin> mgmt.set("graph.set-vertex-id", true);
==>org.janusgraph.diskstorage.configuration.UserModifiableConfiguration@36f40d72
gremlin> mgmt.get('graph.set-vertex-id')
==>true
But my code still produced the same error.
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
Thank you very much for getting back to me. I tried connecting to the graph in gremlin console using the following: 
gremlin> graph = JanusGraphFactory.open('conf/janusgraph-berkeleyje-server.properties')
Could not instantiate implementation: org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager
Type ':help' or ':h' for help.
Display stack trace? [yN]
gremlin> graph = JanusGraphFactory.open('conf/janusgraph-berkeleyje-server.properties')
Could not instantiate implementation: org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager
Type ':help' or ':h' for help.
Display stack trace? [yN]
As you can see, it didn't work. So I think I must have bigger issues haha. I thought berkeley was used by default, but it doesn't seem to be working.
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
I've investigated the logs and found this message: 
btc_janusgraph  | 23:37:12 WARN  org.janusgraph.diskstorage.configuration.builder.ReadConfigurationBuilder.getOptionsWithDiscrepancies - Local setting graph.set-vertex-id=true (Type: GLOBAL_OFFLINE) is overridden by globally managed value (false).  Use the ManagementSystem interface instead of the local configuration to control this setting.
btc_janusgraph  | 23:37:12 WARN  org.janusgraph.diskstorage.configuration.builder.ReadConfigurationBuilder.getOptionsWithDiscrepancies - Local setting graph.set-vertex-id=true (Type: GLOBAL_OFFLINE) is overridden by globally managed value (false).  Use the ManagementSystem interface instead of the local configuration to control this setting.
So now my question becomes, how do I set the global value?
28 replies
ATApache TinkerPop
Created by Isopropyl9 on 11/9/2023 in #questions
Docker Janusgraph Custom ID Values
My ./janusgraph/janusgraph.properties contains the following: 
# Copyright 2023 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
...
# more default contents
...
# Directory to store index data locally
#
# Default:    (no default value)
# Data Type:  String
# Mutability: MASKABLE
index.search.directory = /var/lib/janusgraph/index

# ALLOW SETTING OF CUSTOM IDs
graph.set-vertex-id=true
# Copyright 2023 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
...
# more default contents
...
# Directory to store index data locally
#
# Default:    (no default value)
# Data Type:  String
# Mutability: MASKABLE
index.search.directory = /var/lib/janusgraph/index

# ALLOW SETTING OF CUSTOM IDs
graph.set-vertex-id=true
28 replies