Stadly Posts - Answer Overflow

Stadly

•Created by Stadly on 3/11/2025 in #questions

Validating Large JSON Schemas without high time/memory costs

I'm building a low-code tool for generating user interfaces. The UI structure is defined using JSON. The schema for the JSON structure is quite large. I have been using Zod for validation, but it is very slow—taking minutes to validate a single UI definition. I've tried switching to ArkType but it consumes too much memory, often causing the browser to crash. When it does not crash (and on the server where I can set a high memory limit) the call to type.module takes approximately 15 seconds, and then validation mere milliseconds. Given these performance issues, how can I efficiently validate large JSON structures against a schema without excessive time consumption or memory usage? Are there optimizations, alternative libraries, or hybrid approaches that could help? Would love to hear any insights or best practices! Here is a small section of the ArkType schema to give you an idea of how it works. As you can see, it is inherently cyclic (take a look at component).

const componentModule = type.module({
  "#accordionItem": {
    "title?": "component",
    "content?": "component",
  },
  accordion: {
    "...": componentBaseSchema,
    type: "'accordion'",
    items: "accordionItem[]",
  },

  button: {
    "...": componentBaseSchema.and(mouseEvents),
    type: "'button'",
    variant:
      "'default' | 'secondary' | 'destructive' | 'outline' | 'ghost' | 'link'",
    size: "'default' | 'sm' | 'lg' | 'icon' | 'compact'",
    "href?": "string",
    label: "component",
  },

  "#tabsItem": {
    id: "string",
    title: "component",
    content: "component",
  },
  tabs: {
    "...": componentBaseSchema,
    type: "'tabs'",
    "currentTabVariable?": "string",
    "onTabChange?": eventHandlersSchema,
    items: "tabsItem[]",
  },

  "#singleComponent":
    "string | accordion | button | tabs",
  component: "singleComponent | singleComponent[]",
});
const componentSchema = componentModule.component;

const componentModule = type.module({
  "#accordionItem": {
    "title?": "component",
    "content?": "component",
  },
  accordion: {
    "...": componentBaseSchema,
    type: "'accordion'",
    items: "accordionItem[]",
  },

  button: {
    "...": componentBaseSchema.and(mouseEvents),
    type: "'button'",
    variant:
      "'default' | 'secondary' | 'destructive' | 'outline' | 'ghost' | 'link'",
    size: "'default' | 'sm' | 'lg' | 'icon' | 'compact'",
    "href?": "string",
    label: "component",
  },

  "#tabsItem": {
    id: "string",
    title: "component",
    content: "component",
  },
  tabs: {
    "...": componentBaseSchema,
    type: "'tabs'",
    "currentTabVariable?": "string",
    "onTabChange?": eventHandlersSchema,
    items: "tabsItem[]",
  },

  "#singleComponent":
    "string | accordion | button | tabs",
  component: "singleComponent | singleComponent[]",
});
const componentSchema = componentModule.component;

7 replies

Aarktype

•Created by Stadly on 3/7/2025 in #questions

Schemas with cyclic references

In Zod, I can create circular schemas like this:

const typeABaseSchema = z.strictObject({
  type: z.literal("a"),
});
const typeASchema = typeABaseSchema.extend({
  nested: z.lazy(() => typeSchema).optional(),
});
type TypeADef = z.infer<typeof typeABaseSchema> & {
  nested?: TypeDef | undefined;
};

const typeBBaseSchema = z.strictObject({
  type: z.literal("b"),
});
const typeBSchema = typeBBaseSchema.extend({
  nested: z.lazy(() => typeSchema).optional(),
});
type TypeBDef = z.infer<typeof typeBBaseSchema> & {
  nested?: TypeDef | undefined;
};

const typeA: z.ZodType<TypeADef> = typeASchema;
const typeB: z.ZodType<TypeBDef> = typeBSchema;
const typeSchema = z.union([typeA, typeB]);
type TypeDef = z.infer<typeof typeSchema>;

typeSchema.parse({ type: "a", nested: { type: "b" } });
typeSchema.parse({ type: "a", nested: { type: "a", nested: { type: "b" } } });
typeSchema.parse({ type: "b", nested: { type: "b", nested: { type: "a" } } });

const typeABaseSchema = z.strictObject({
  type: z.literal("a"),
});
const typeASchema = typeABaseSchema.extend({
  nested: z.lazy(() => typeSchema).optional(),
});
type TypeADef = z.infer<typeof typeABaseSchema> & {
  nested?: TypeDef | undefined;
};

const typeBBaseSchema = z.strictObject({
  type: z.literal("b"),
});
const typeBSchema = typeBBaseSchema.extend({
  nested: z.lazy(() => typeSchema).optional(),
});
type TypeBDef = z.infer<typeof typeBBaseSchema> & {
  nested?: TypeDef | undefined;
};

const typeA: z.ZodType<TypeADef> = typeASchema;
const typeB: z.ZodType<TypeBDef> = typeBSchema;
const typeSchema = z.union([typeA, typeB]);
type TypeDef = z.infer<typeof typeSchema>;

typeSchema.parse({ type: "a", nested: { type: "b" } });
typeSchema.parse({ type: "a", nested: { type: "a", nested: { type: "b" } } });
typeSchema.parse({ type: "b", nested: { type: "b", nested: { type: "a" } } });

I've not been able to achieve the same thing using ArkType. Can anyone help? I need both typeASchema, typeBSchema and typeSchema, so cannot just create a single large scope.

4 replies

Gaming

Programming