Data Storage wit TTL
Hi Everyone, We have a requirement in which we have to store around ~80 million records daily in our graph storage and we should have a TTL of 90 (~7 Billion) days for this data but the issue we are having is that we can have TTL on static vertex only and we don't want to do that as that restrict us from further updates on that vertex (correct me if I'm wrong).
Please suggest some way in which we can store this data so that we can have TTL also.
We are using bigtable backend, so will it be possible for us if we directly can have gc policy(90 days) on Bigtable column families?
1 Reply
You cannot set TTL on BigTable directly because that will delete schema information too.
You may want to have a MapReduce or Spark job that runs and scans your entire graph DB everyday and actively purges outdated vertices.