Topics

arktype•4mo ago

Error internationalization (i18n)

Hello! Is it possible to translate the errors into other languages? In my previous setup I was using this for such funtionality: https://github.com/aiji42/zod-i18n I saw this issue opened for individual fields, which would really be great: https://github.com/arktypeio/arktype/issues/722 Is this the way for "general" errors? https://github.com/arktypeio/arktype/issues/404

GitHub

GitHub - aiji42/zod-i18n: Useful for translating zod error messages.

Useful for translating zod error messages. Contribute to aiji42/zod-i18n development by creating an account on GitHub.

GitHub

I18N error message customization · Issue #722 · arktypeio/arktype

Ideally any error message should be specifiable as a Record of strings representing locale IDs to the translations in that locale, e.g: const creditCard = type("number", {mustBe: "a ...

GitHub

I18n and error templates · Issue #404 · arktypeio/arktype

Description The current error messages or great at finding the failed validation returned via api, but this could also be harnessed in the frontend. So it would be really useful to be able to speci...

73 Replies

ssalbdivad•4mo ago

This hasn't been addressed internally yet but definitely something I'm interested in adding APIs for now that the rest of error customization is stable Some of the nuances of the sequential approach to building clear, composable errors in English that I created for ArkType may be harder to translate to other languages, but worst case scenario adding a top-level option to replace the whole message should definitely be doable The nice thing about the error message customization setup for English is that you can change individual parts of the message like expect, actual, or how they're composed together or integrated with other context like that path. That way, we can leverage that more granular level of description to write coherent messages even for unions

PIatOP•4mo ago

Thank you for the insight @ssalbdivad I believe that adding translations is very important for frontends, like in my use case, where the forms should be translated to the user's language of choice.

ssalbdivad•4mo ago

Yeah that makes sense

PIatOP•4mo ago

And your worst case scenario actually sounds like the best case scenario I would think of 😁

ssalbdivad•4mo ago

Realistically I do want to focus a bit on wrapping up docs and stabilizing the features that are there but this will likely be one of the first things I tackle after. I just want to make sure the API is good

PIatOP•4mo ago

That's what's done in the Zod library, give or take - instead of the message, a translation key + values are output, which are then passed into the i18n solution of choice Then I'm able to write own translation files for every language

"errors": {
    "invalid_type": "Expected {{expected}}, received {{received}}",
    "invalid_type_received_undefined": "Required",
    "invalid_type_received_null": "Required",
    "invalid_literal": "Invalid literal value, expected {{expected}}",
    "unrecognized_keys": "Unrecognized key(s) in object: {{- keys}}",
    "invalid_union": "Invalid input",
    "invalid_union_discriminator": "Invalid discriminator value. Expected {{- options}}",
    "invalid_enum_value": "Invalid enum value. Expected {{- options}}, received '{{received}}'",
    "invalid_arguments": "Invalid function arguments",
    "invalid_return_type": "Invalid function return type",
    "invalid_date": "Invalid date"
}

"errors": {
    "invalid_type": "Expected {{expected}}, received {{received}}",
    "invalid_type_received_undefined": "Required",
    "invalid_type_received_null": "Required",
    "invalid_literal": "Invalid literal value, expected {{expected}}",
    "unrecognized_keys": "Unrecognized key(s) in object: {{- keys}}",
    "invalid_union": "Invalid input",
    "invalid_union_discriminator": "Invalid discriminator value. Expected {{- options}}",
    "invalid_enum_value": "Invalid enum value. Expected {{- options}}, received '{{received}}'",
    "invalid_arguments": "Invalid function arguments",
    "invalid_return_type": "Invalid function return type",
    "invalid_date": "Invalid date"
}

ssalbdivad•4mo ago

"invalid_union": "Invalid input", This is the problem haha

PIatOP•4mo ago

Yes, this approach is very naive The whole file, if you're interested

ssalbdivad•4mo ago

I mean it's really complicated to write a good union error though I'm not saying all libraries should handle it I just need to think a bit about the best way to extend the existing approach to multiple languages

PIatOP•4mo ago

zod.json

ssalbdivad•4mo ago

Feel free to add your thoughts to the open issue so I can reference them when I work on it

PIatOP•4mo ago

If you are really going into it with i18n in mind, it will really be something else Okay!

ssalbdivad•4mo ago

This is the logic now for the default union error message:

No description

PIatOP•4mo ago

Many string literals will have to be refactored :/ Is this chainable?

ssalbdivad•4mo ago

Yeah, I'll have to think about how the composition part will work with other languages

ssalbdivad•4mo ago

But it leads to really clean errors like this

No description

ssalbdivad•4mo ago

I mean the string literals wouldn't be that hard to pull out as long as there's a 1:1 mapping of the structure of the sentence itself which is what I'm more worried about

PIatOP•4mo ago

My 2 cents I can give you is my knowledge of multiple languages, including Russian, Czech and German. I can always try the system and the grammar of each language to test how it would chain in the particular implementation, and which hurdles we might come across Yes, I see, it's atomic and modular in nature I can definitely imagine there would have to be names for the types too, as per the Zod implementation Because when you throw a user the error "expected a boolean but was missing", they have no clue But also what makes it hard is that currently the system returns one final string, but with an i18n system, it would most likely have to be an array of string codes and value objects, which would then have to be fed the i18n system, so that there really can be unlimited combinations So basically what you already have here, except it just wouldn't connect into a string, but redurn the "actual" data What do you think would be my best bet on translating the messages currently? Feeding them through a translator before sending?

ssalbdivad•4mo ago

All of that metadata already exists though The errors object you get on failed validation already includes stuff like code, data, expected, actual etc. so you could build it how you want

PIatOP•4mo ago

Oh really??? Should I try implementing it with i18next?

ssalbdivad•4mo ago

I think the easiest thing now would be a switch based on some context where it would have a different expected problem and message result depending ont he language (those are the two configs that piece together the parts of the error essentially) You can definitely give it a shot if it is possible with Zod it should be doable. At some point I just want to support it natively ideally with the same composability as the current errors

PIatOP•4mo ago

Having it natively would be awesome, but isn't the scope too big?

ssalbdivad•4mo ago

Also fair warning this part of the API isn't documented yet you'd have to use the types + unit tests: https://github.com/arktypeio/arktype/blob/8e3c9ec1fc4aaa269b2a36f3c32bdc16ab889c83/ark/type/__tests__/config.test.ts https://github.com/arktypeio/arktype/blob/8e3c9ec1fc4aaa269b2a36f3c32bdc16ab889c83/ark/type/__tests__/traverse.test.ts

GitHub

arktype/ark/type/tests/config.test.ts at 8e3c9ec1fc4aaa269b2a36...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

GitHub

arktype/ark/type/tests/traverse.test.ts at 8e3c9ec1fc4aaa269b2a...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

ssalbdivad•4mo ago

I wouldn't include the translations but I mean an API that would cleanly integrate them

PIatOP•4mo ago

i18n always has many problems and even more solutions, there would have to then be support for many i18n features, and it will be a whole project of its own So that the developers would bring in their i18next or typescript-i18n Sounds great I love reading through the tests 😁 I'm still mesmerized by what is possible

Dimava•4mo ago

🤨 appendUnique in foreach looks like O(n^2) Is it DDOS-safe? (i.e. n is comptime-dependent)

ssalbdivad•4mo ago

Most things related to unions are O(n^2) I promise they're optimized, it's just how de morgan's law works

Dimava•4mo ago

I mean, can you make a value such it has huge amount of errors.length there?

ssalbdivad•4mo ago

You're right that in this exact case I could probably optimize that redundant error message check But TBH the perf for behavior that only occurs when validation has already failed and we're building a descriptive error message is a lot less important

Dimava•4mo ago

I'd say unique = a => [...new Set(a)] may be better there

ssalbdivad•4mo ago

I think it will be very rare that people will have an undiscrimnated union with more than a dozen or so branches Remember that it will automatically try to discriminate every union and those errors will not be handled by this code

Dimava•4mo ago

I mean, if you can make a value that generates 1000 errors Can you make such a value for simple types?

ssalbdivad•4mo ago

No that's not what's going onhere It would only scale with the number of branches in an undiscriminated union, not how many errors there are for a given value Unions are always checked "fail fast" There will never be more than one error per branch

PIatOP•4mo ago

Where could I see the possible expected and actual values (like empty etc.) to translate them?

ssalbdivad•4mo ago

expected and actual are attached to each error ArkErrors is an array of ArkError with those props

PIatOP•4mo ago

@ssalbdivad What I mean is, if there's a list of all the possible types and sentences. So I can assign each to a key

ssalbdivad•4mo ago

Oh I see, no that would not happen until i18n has more builtin support. Honestly it would be one of the easier things to work on externally, to just pull out all the builtin strings in the defaults to a set of configurable variables

PIatOP•4mo ago

So far I've been experimenting and rewrote the end of this function to return a i18n-valid value. So I can handle the case of missing with no issues, using the string missing as a string of errors.value.missing for example. But I'd like to know if I could find all the possible responses somewhere

ssalbdivad•4mo ago

Look at each node type listed in ark/schema/kinds.ts. You will see something like this: I think that defaults section would be where the vast majority of the english would be

No description

ssalbdivad•4mo ago

I think the way it should be done is probably that all those config types should actually be i18n wrapped, so you could either have a record mapping locales to translations or if it's a single root value like it is now it would be assumed to be enUs or whatever So actually I guess by default that would mean you could leave those config values the way they are and that API would take care of externally accepting translations?

PIatOP•4mo ago

Perhaps, but in the end it doesn't matter as much whether you wrap with i18n here or in the very end, like I'm trying to do now. The first reason that comes to mind is that if people would want the different i18n libraries, each library has a different way of handling keys. Like i18next leans into separating levels with a ., while Fluent has -, since . has another function. Hence when people connect the i18n solutions, they'll probably want to change the translations in accordance to their current system. And to me it seems easiest to be able to build a string literal using the implementation.kind.required field, which could work with any nesting, namespacing etc a user might need

ssalbdivad•4mo ago

Okay yeah I buy that. You'll need to integrate it as a config value like i18n on ArkConfig which you could consume through ctx Then you can use that when writing all the defaults

PIatOP•4mo ago

Do you mean i18n would be a wrapper function, which would have kind as an argument and as a return it'd have to return the real string?

... 
return t(`errors.${kind}`)

... 
return t(`errors.${kind}`)

Writing code on a phone will be the end of me 😆

ssalbdivad•4mo ago

I think it should just be an object with all the translation names, right?

PIatOP•4mo ago

I'll write up a small POC to explain it with I also need to think where the concatenation should happen. My use case right now is: website, backend and a mobile app. Each loads translations differently and the backend uses different functions from the rest, which are also async

ssalbdivad•4mo ago

I definitely don't feel like anything on my end to do with translations should be async, that would have to be handled externally

PIatOP•4mo ago

Yes

ssalbdivad•4mo ago

I don't even have async validation yet

PIatOP•4mo ago

Okay, I'll see what I come up with and then will see how it could integrate with the current system Anything I'm thinking of with adding the translation handling directly into Arktype, just screams slowing it down unnecessarily

ssalbdivad•4mo ago

I don't think it would have to slow anything down because all that stuff is precomputed That's how configs work now When the node is created, it determines the expected values etc. By the time the validator runs, the config has already been parsed and applied to that node (or at least most of it, some things can't be prechecked like actual) But even for stuff like actual we would have already populated the ctx with the localized function that handles writing it

PIatOP•4mo ago

import { useTranslation } from 'react-i18next'

const formSchema = type({
    name: 'string>0',
    surname: 'string>4',
})

export default function Route() {
    const { t } = useTranslation()

    const errors = useMemo(() => {
        const formResult = formSchema({
            name: 'Pl',
            surname: 'at',
        })

        const res = []

        if (formResult instanceof type.errors) {
            for (const error of formResult) {
                res.push({
                    code: error.code,
                    path: error.path,
                    ...(error.input && {
                        data: {
                            [error.input.code]: error.input.rule,
                        },
                    }),
                })
            }
        }

        return res
    }, [])

    return (
        <div>
            {errors.map((e) => {
                return t(`errors.${e.code}`, {
                    replace: e.data,
                    ns: 'ark',
                })
            })}
        </div>
    )
}

import { useTranslation } from 'react-i18next'

const formSchema = type({
    name: 'string>0',
    surname: 'string>4',
})

export default function Route() {
    const { t } = useTranslation()

    const errors = useMemo(() => {
        const formResult = formSchema({
            name: 'Pl',
            surname: 'at',
        })

        const res = []

        if (formResult instanceof type.errors) {
            for (const error of formResult) {
                res.push({
                    code: error.code,
                    path: error.path,
                    ...(error.input && {
                        data: {
                            [error.input.code]: error.input.rule,
                        },
                    }),
                })
            }
        }

        return res
    }, [])

    return (
        <div>
            {errors.map((e) => {
                return t(`errors.${e.code}`, {
                    replace: e.data,
                    ns: 'ark',
                })
            })}
        </div>
    )
}

PIatOP•4mo ago

PIatOP•4mo ago

It does work in the most primitive cases. However, I couldn't find a way to externally get the names of value to fill in when needed

No description

PIatOP•4mo ago

ark.json

ssalbdivad•4mo ago

Where do these messages come from? If we're going to use them they have to match the ones that currently exist in ArkType

PIatOP•4mo ago

I made them up. The codebase is complicated for me, so I didn't look into finding all the instances for the POC yet. I'd add the translations when the code works, it's not that big of a deal

ssalbdivad•4mo ago

I'd have to look into it more, it seems like a lot of unnecesarry mapping I'd rather ArkErrors just already have the right shape ideally

PIatOP•4mo ago

Yes, this was an attempt of doing it completely externally To be fair, if values (string, function) weren't transformed into strings, or returned the code as well, even this would work just fine

ssalbdivad•4mo ago

But there is a .data prop already on ArkError that contains the relevant data no?

PIatOP•4mo ago

Let me see

import { type ArkError, type } from 'arktype'

export function toArkI18n(errors: ArkError) {
    const res = []

    if (errors instanceof type.errors) {
        for (const error of errors) {
            console.log(error)

            res.push({
                code: error.code,
                path: error.path,
                ...(error.input && {
                    data: {
                        [error.input.code]: error.input.rule,
                    },
                }),
            })
        }
    }

    return res
}

const formSchema = type({
    name: 'string>0',
    surname: 'string>4',
})

toArkI18n(
    formSchema({
        name: 1,
        surname: 'at',
    }),
),

import { type ArkError, type } from 'arktype'

export function toArkI18n(errors: ArkError) {
    const res = []

    if (errors instanceof type.errors) {
        for (const error of errors) {
            console.log(error)

            res.push({
                code: error.code,
                path: error.path,
                ...(error.input && {
                    data: {
                        [error.input.code]: error.input.rule,
                    },
                }),
            })
        }
    }

    return res
}

const formSchema = type({
    name: 'string>0',
    surname: 'string>4',
})

toArkI18n(
    formSchema({
        name: 1,
        surname: 'at',
    }),
),

{
  " arkKind": "error",
  "path": [
    "name"
  ],
  "data": 1,
  "nodeConfig": {},
  "input": {
    "code": "domain",
    "description": "a string",
    "domain": "string"
  },
  "code": "domain",
  "description": "a string",
  "domain": "string"
}

{
  " arkKind": "error",
  "path": [
    "name"
  ],
  "data": 1,
  "nodeConfig": {},
  "input": {
    "code": "domain",
    "description": "a string",
    "domain": "string"
  },
  "code": "domain",
  "description": "a string",
  "domain": "string"
}

{
  " arkKind": "error",
  "path": [
    "surname"
  ],
  "data": "at",
  "nodeConfig": {},
  "input": {
    "code": "minLength",
    "description": "at least length 5",
    "rule": 5
  },
  "code": "minLength",
  "description": "at least length 5",
  "rule": 5
}

{
  " arkKind": "error",
  "path": [
    "surname"
  ],
  "data": "at",
  "nodeConfig": {},
  "input": {
    "code": "minLength",
    "description": "at least length 5",
    "rule": 5
  },
  "code": "minLength",
  "description": "at least length 5",
  "rule": 5
}

ssalbdivad•4mo ago

I don't get this:

                ...(error.input && {
                    data: {
                        [error.input.code]: error.input.rule,
                    },
                }),

                ...(error.input && {
                    data: {
                        [error.input.code]: error.input.rule,
                    },
                }),

You can already just do error.data

PIatOP•4mo ago

In situations like this, data is populated with an already transformed string

ssalbdivad•4mo ago

That's not what it is it was the original data that caused the error actual takes data and transforms it to be a description of data In this case your data was literally "at"

PIatOP•4mo ago

Ah, sorry! data in this case is what will be added in for example {{minLength}} in the translation. So I need to know what the min legth should've been, not what it was

ssalbdivad•4mo ago

So that's kind of what rule is right?

PIatOP•4mo ago

String must have at least {{minLength}} character(s)

ssalbdivad•4mo ago

I don't think I want to use the template string approach. I'll have to revisit this

PIatOP•4mo ago

Yes, it could've been changed to 👍 👍 👍 👍

data: {
      [error.code]: error.rule,
}

data: {
      [error.code]: error.rule,
}

ssalbdivad•4mo ago

I know the docs around errors suck but I put a lot of thought into their structure. These transformations feel too much like arbitrarily restructuring things

PIatOP•4mo ago

Yes, I went the shortest way to make it work without changing Arktype's code and with little understanding of the system

ssalbdivad•4mo ago

Makes sense. I'll just have to think more about how I'd want to integrate it in practice

PIatOP•4mo ago

Definitely I see, domain errors would be used for the types keys Added the ?? rule to keep track of the error-specific data, like domain, proto, since in those cases a rule key isn't present

export function toArkI18n(errors: ArkError) {
    const res = []

    if (errors instanceof type.errors) {
        for (const error of errors) {
            res.push({
                code: error.code,
                path: error.path,
                data: {
                    [error.code]: error[error.code] ?? error.rule,
                },
            })
        }
    }

    return res
}

export function toArkI18n(errors: ArkError) {
    const res = []

    if (errors instanceof type.errors) {
        for (const error of errors) {
            res.push({
                code: error.code,
                path: error.path,
                data: {
                    [error.code]: error[error.code] ?? error.rule,
                },
            })
        }
    }

    return res
}

{
    "errors": {
        "domain": "Expected $t(ark:types.{{domain}}, { 'context': 'acc' })"
    },
    "types": {
        "string": "string",
        "string_acc": "string (accusative)"
    }
}

{
    "errors": {
        "domain": "Expected $t(ark:types.{{domain}}, { 'context': 'acc' })"
    },
    "types": {
        "string": "string",
        "string_acc": "string (accusative)"
    }
}

Complexities of grammar are handled by i18next

PIatOP•4mo ago

This is a friendly space centered around ArkType, TypeScript's 1:1 validator (https://github.com/arktypeio/arktype)

435Members

View on Discord

Want results from more Discord servers?

Add your server