League of Extraordinary FoundryVTT Developers•3y ago

Advice for compendium data in a custom system

Hello, i'm new here!

I was wondering if I could get a little direction for storing data about my games items/actors.
I looked at how it is done in pf2e and dnd5e. They both seem to have gone different directions.
DnD5e uses

.db

.db

files for their packs data https://github.com/foundryvtt/dnd5e/tree/master/packs
while PF2E uses

.json

.json

files https://github.com/foundryvtt/pf2e/tree/master/packs

Should I be using

.db

.db

files or

.json

.json

files for my packs?
and how should I prepare these files so Foundry can see the data in the packs?

One thing I tried to do was to:

create an empty world
set up a compendium pack by hand using my systems sheets
used the foundry cli tool to 'unpack' my worlds pack data
moved those json files into my systems /packs folder.
updated my manifest so it would point to the correct /packs, and restarted my other test world to see if it could use the packs provided by my system. The pack folders are there, but non of the items are inside.

In the future, I would like to be able to curate the data into a

.db

.db

file /

.json

.json

files in an automated way to avoid a bunch of manual data entry through my sheets.

Eranziel•11/21/23, 6:38 PM

My recommendation is to use

.json

.json

files, with a build script that uses the Foundry CLI to pack them into LevelDB databases for Foundry. Foundry won't read the

.json

.json

files directly, it needs them in the databases.

Eranziel•11/21/23, 6:39 PM

.db

.db

files are for NeDB, which is what Foundry used for its database in version 10 and earlier. v11+ uses LevelDB. If Foundry detects

.db

.db

files and no LevelDBs when launching a world, it will convert those

.db

.db

s to the LevelDB format and then ignore them from that point on.

Eranziel•11/21/23, 6:40 PM

Note that the CLI can use some other formats instead of

.json

.json

if you prefer. I know it supports YAML, for example.

EclipxsOP•11/21/23, 6:47 PM

Thanks!

So you have the

.json

.json

(or yml) files in /packs for src control, and then during release before zipping the system, you run the cli to pack them into LevelDB? then zip the system?

Eranziel•11/21/23, 6:48 PM

Yup, that's the general idea

EclipxsOP•11/21/23, 6:48 PM

Sweet! that makes sense now. tyvm

Eranziel•11/21/23, 6:50 PM

Examples from Lancer:

macros compendium content in yml: https://github.com/Eranziel/foundryvtt-lancer/tree/2.0-rc/src/packs/core_macros
gulpfile calls the CLI to pack those up: https://github.com/Eranziel/foundryvtt-lancer/blob/7473ea2f4491657755cf35cefb8682630d69b958/gulpfile.js#L46
gulpfile packages the build for distribution: https://github.com/Eranziel/foundryvtt-lancer/blob/7473ea2f4491657755cf35cefb8682630d69b958/gulpfile.js#L169

Eranziel•11/21/23, 6:51 PM

I wouldn't necessarily copy-paste our gulpfile, it's based on a rather old project template and been updated over time. If you're using one of the more modern project templates they probably have a bit better tooling in place, haha. But definitely take inspiration.

EclipxsOP•11/21/23, 6:52 PM

sounds good, thanks again. i'll take a look and adapt where needed

EclipxsOP•11/21/23, 10:12 PM

got this working with github actions if anyone is interested:

this was just a simple test with a single pack, though it wouldn't be hard to make this a little more generic and handle multiple packs.