Converting Zod to Arktype
Hello!
I'm really interested in using Conform instead of React Hook Form for better DX and server-side capabilities. However, it doesn't have Ark support yet, so I'll have to create my own simple patch.
Are there by any chance some examples or comparisons between Zod and Arktype architectures to go off of when translating these files, namely to extract the name of the constraints like
Array
in runtime?
expression
might be "key"? pun intended
https://github.com/edmundhung/conform/blob/main/packages/conform-zod/constraint.ts
https://github.com/edmundhung/conform/blob/main/packages/conform-zod/parse.ts
Also, is there an way to do something like superRefine
for custom run-time conditions?
https://zod.dev/?id=superrefine87 Replies
Sorry I missed this. It looks like you've figured out some of this.
You can iterate over the errors and check
.code
to check the kind of error it was. It has lots of additional introspectable information about the type of error.
superRefine
is just a long-winded narrow
😅Ah, thank you! I'm slowly starting to understand morphs and constraints
I've been looking through the tests, and got a bunch of questions answered
The possiibilities are endless 🤯
I appreciate your diligence! Unit tests definitely the best place to see comprehensive docs at the moment haha
Yes, I in turn appreciate your thoroughness in writing the tests, it's really helpful
It's also helpful for making sure everything works 😅
But I couldn't find an example on how to create/edit types on runtime 🥹
This seemed to get me far enough, but then when I acually use it, it thinks all the values are missing
If there is a specific test or discussion about it, I'd gladly read it, I just couldn't find the proper keywords to find it
This kind of internal type mapping stuff isn't really a big focus of the documentation yet. It mirrors a lot of operations in TS, but there's a ton of nuance to the way unions, structures etc. are handled (like in TS)
Here's a cleaner implementation though:
It will be nice when there is a mapped type abstraction built around this
Maybe I will add it now
So the first callback if for iterating the keys, and the second callback is for building the result object, which
distribute
returns?Yeah
Got ittttt.......
It works ;-;-;-;-;-;
That makes me so happy
I bashed my head on the wall for about 4 hours with this yesterday 😆
Haha yeah it helps to know how the type system works
Do you know why this acted like it did?
Your biggest problem is that you were treating
key
like a literal key but it was a node
So I guess it gets converted to a string using .expression
And the actual literal is
key.unit
Yeah
Well, the object always had the proper shape actually
age
and userName
was on it properlyI think you were just thinking it did because it looked similar but those strings were actualy quoted
Because that is how the literal "age" would be converted to an expression
"age"
So it would look for the key in quotesOhhhhhh
OHHHHH
;-;-;-;-;-;-;-;-;-;
I'm laughing so hard right now
Got it, thank you
Glad you're not crying 😅
I'm really thankful you showed me the proper solution, so I can move forward, nothing to cry about
It will be a lot easier once I add an API for mapped types
You shouldn't have to think about those structural nuances externally
How would I theoretically be able to know that the key was actually in unit?
Dimava the Monoreaper understands the library well, how come?
I don't know he's spent a lot of time answering questions I guess haha
You can see from the type
key
will not be a stringThere's a brief summary of the type node structure here: https://github.com/arktypeio/arktype/blob/8e3c9ec1fc4aaa269b2a36f3c32bdc16ab889c83/ark/schema/README.md
So you can see what each node does.
unit
represents a single value
You can use .hasKind
on an internal node to narrow it based on its kind
I guess I could have also just .assertHasKind("unit")
there
Oh that explanation kinda skips morphs haha
I guess that's why I said "the part that exists in TS"
Morphs are between unions and intersectionsSo every "type" has a different unit?
No only types that represent a literal value have a unit
Ahhhhh
So the key
"age"
represents exactly one value in JS
So it has a unit node
Or it is a unit node ratherRiggght
"age"|"name"
would be a union of two unit nodes
"age" | "name" | symbol
would be a union of two unit nodes and a domain nodeAnd
domain
is the keyword that comes out when the type was wrong
I know that from the errors 😁Domain is similar to
typeof
But adapted to the TS keywords, so object includes function but not null
It's basically all the lowercase primitive keywords TS provides
Except that in the type system, it only includes the non-enumerable domains, i.e. string
number
bigint
symbol
and object
The others are represented as unit nodes
So e.g. boolean
is actually a union of two unit nodes
But having it so that there is only one normalized representation for a given type means we can accurately compare themOkaaaaaaaaaay
So domain, proto and unit live on the same level
Then they can be joined into unions
Yeah because they will never coexist
This is so profound and cool
The easiest way to see how any of your types are structured is to use
.json
Thinking about this is so fun
Yes it's a really beautiful problem. Most of the work on the codebase was on the type system, I'm excited for more people to learn about it
Hehe, I got experience with that 😆
Me too
It's basically a pure, precise version of what TS approximates
That is also extended to include runtime constraints and morphs
Which is just wild
Like hoooooow
It's a string with quotation marks, how does it knoooowwww
Because it parses it first haha
But that's crazyyyy
It has to otherwise narrow would never work
Every time I use Arktype, I feel like it's not real
I'm surprised you're not using the extension it makes types like that a lot more readable
I only use dark theme when sending screenshots to you 😅
You don't need to use the theme though to benefit from the extension
It's called ArkDark but it also provides syntax highlighting
You can use it with whatever theme you want
Woooooooo
Wait, it doeeessssss
No waayyyy
I thought it was the theme that did it
;-; how
Also, did you think of the color palette for ArkDark yourself?
Thank you for the tip 🙏 🙏 🙏 🙏
Yeah it is kind of based on the palette I came up with for the website/logo though
I'm happy to announce the Conform parsing does work fine in my simple use-cases with this!
Looks promising! So are you publishing this as a package?
I would definitely like to, it covers the simplest use case
I even use it with the
shady-ark-i18n
to translate the messages
I already know a few teams that want to use it with conform, I'd honestly accept a PR for the main repo if you wanted to add it as an ecosystem package
From what I can see, Conform has all the packages listed in the repo like this:
https://github.com/edmundhung/conform/tree/main/packages
True that would be better
You should submit it as a PR there then instead I suppose
Hard to say
Well you should definitely submit it there, whether they accept it is their choice
It's still missing the constraint part to automatically add
min
, max
etc. to fields
But it could get the ball rollingYeah we'd want to make sure it's complete first
We could always add it to the arktype repo for now until it's complete and tested then submit a PR
Yes, let's do that
As I said, I never contributed like this, so you know better 😁
To be fair I still want it to be at least tested if not complete before adding it to the repo 😛
But if you start integrating it with the arktype repo you can use the same test patterns
Ooooh
My first unit tests 😅
Okay, will do 💪
@PIat Thanks for pointing me at this. I'm not using conform but this helps me with my problem.
One comment... I changed ? value.or('string.numeric.parse') to type('string.numeric.parse').pipe(value). Otherwise, if you have something like {age:'number<100'} and pass it a string like '101', it correcty parses to a number but the >100 constraint is lost.
Also... I'm a bit puzzled. I can't get the check for a boolean type to work. I have a boolean property in my type but it's not getting picked up by value.extends('boolean') or value.extends(type.boolean).
I assume it worked for you?
If I use value.overlaps('boolean'), it works ok. But I'm not sure if that's wise?
It's not lost, constraints apply to the input so
string.numeric.parse < 100
means a string less than 100 characters parsed into a number
There is actually an API I'm thinking about for withOut
that allows you specifically to chain constraints on your output, but it's really just sugar over pipe as you said
Are you sure your expectations align with how TS handles this for unions? What comparison specifically do you want to workBelow is the code I'm working with (and forgive me, I'm not exactly an experienced dev)
My points/questions:
1. In the examples above in this thread, the "detection" of a Boolean field in the type is done by checking value.extends('boolean'). For some reason I don't understand, that doesn't work here but value.overlaps('boolean') does. My assumption here is that I'm being an idiot, but I can't work out where.
2. In Plat's conform parser I see that value.or('string.numeric.parse') is used for a number field. This works but seems to mean that and numeric constraints that were on the original numeric field definition don't apply when it's parsed from a string. So I tried type('string.numeric.parse').pipe(value), which seems to keen any original numeric constraints.
3. The below works for numbers, but Boolean is causing me problems. The original type has the Boolean as an option (as an unchecked checkbox on a form won't provide a value in formData). But the function is converting the optional key back into required. So it works if a checkbox is checked and passes 'on' but doesn't work if it is unchecked.
The reason is the field is optional so it's actually
boolean|undefined
when you .get
it
You can always use stuff like console.log(value.expression)
if you want to see what your actual type looks like
There's a lot of introspectability
One option would be to do:
.expression
essentially converts any ArkType to an expanded TS syntax
The issue with optional and required is as bit trickier because you're just iterating over the literal keys. keyof
does not preserve associations with required or optionalIs there an easy(ish) way that I can force any boolean to be optional in the new type? (Because for this use case, it always will be, as I'll receive 'on' or nothing at all)
As a heuristic I'd say you could likely check as you're iterating over the keys:
I will add a wrapper API like
.map
around this. Even without native mapped type syntax, being able to map entries would be way easier externally if I do thisSounds good. And once again, thanks for the help!
I'm stuck again! (It's getting a bit embarrassing now, sorry David!).
This is a section of my current code.
I then pass this type through the function to get a modified type that is used to validate form input. If I then try to validate this data: I run into a problem with the default. The default for 'name' works ok. 'name' is a string input in both the original and modified types. But the defaults for age and is_male get lost, even though I have pipe the string through original with .pipe(value). That .pipe(value) is retaining the >18 number constraint on age but losing the =19. I've tried adding .default() into the function but it makes no difference. What am I missing? So what's happening is that if I don't have a 'is_male' (e.g. a checkbox is not checked) then the default is not being applied to an optional field so I end up with no 'is_male' value in the output, when I need false.
I then pass this type through the function to get a modified type that is used to validate form input. If I then try to validate this data: I run into a problem with the default. The default for 'name' works ok. 'name' is a string input in both the original and modified types. But the defaults for age and is_male get lost, even though I have pipe the string through original with .pipe(value). That .pipe(value) is retaining the >18 number constraint on age but losing the =19. I've tried adding .default() into the function but it makes no difference. What am I missing? So what's happening is that if I don't have a 'is_male' (e.g. a checkbox is not checked) then the default is not being applied to an optional field so I end up with no 'is_male' value in the output, when I need false.
See the discussion at https://github.com/arktypeio/arktype/issues/1089
There is a plan to optimize this internally to handle this case https://github.com/arktypeio/arktype/issues/1090
I think what you'd need to do here is map the default value internally as well. I should really just finish this
.map
API which would help a lot 😅
Going from just learning the library to internal type transforms like this is a bit trickyHey, you should look into camelCase. It'll be easier on the eyes for you, especially if you start integrating more libraries
I know it's controversial but I find snake_case easier to read.
I agree camelCase somehow looks better and is the js standard, but I find it easier to follow my code using snake_case.
It's sort of the opposite of 1089. I can apply the morph ok but the defaults on the initial type get ignored. Is that the same cause?
yeah not exactly the same but definitely related.
Your case is a bit more unique because it involes mapping the defaults themselves
If you think about it, to be able to handle that directly, we'd need to know how an arbitrary default provided by the user would translate to the default for your input value, so you'd have to write that logic
the default of a node is accessible if there is one as
meta.default
I'm still missing something.
The below is the simplest example of what I want to do. This works, but it requires me to generate 2 preprocessing types, one to make sure there is a string value present and the second to transform the string value to a Boolean.
I can't get my head around why that can't be combined into one preprocessing step.
It seems like adding the 'string="off"' default to the second preprocess type should work, but it just ignores it. And I can't put the logic in the morph, as the morph doesn't run if it doesn't get a string value. Maybe I just have to accept "that's just the way it is, deal with it"! Hi @ArkDavid , just to close this off, is the above example the way I need to go with this, or is there an approach that doesn't require 2 preprocess steps? I'm happy with either answer. And sorry for my ignorance, if that's what it is. You're doing a very impressive job with Arktype!
It seems like adding the 'string="off"' default to the second preprocess type should work, but it just ignores it. And I can't put the logic in the morph, as the morph doesn't run if it doesn't get a string value. Maybe I just have to accept "that's just the way it is, deal with it"! Hi @ArkDavid , just to close this off, is the above example the way I need to go with this, or is there an approach that doesn't require 2 preprocess steps? I'm happy with either answer. And sorry for my ignorance, if that's what it is. You're doing a very impressive job with Arktype!
Oh yeah sorry I forgot to follow up on this. I know this default mapping stuff is tricky to handle externally. Let me merge this new map API maybe that will help haha
Looking into this now
Honestly this behavior sucks I'm going to fix it
It is such a relief that it wasn't me being an idiot!
(and thanks)
Definitely not. On the one hand yes generally defaults play kind of a weird role when you also have transforms, but in this case it's not just awkward, the behavior is blatantly wrong. It asks for the output type as a default value but then rejects it at runtime.
Should be easy to fix though
All right there were a couple edge cases related to morphs that made it a bit more work than I anticipated but hopefully everything "just works" now.
The default is actually precomputed as soon as you instantiate your validator if you morph it, so you skip the transform logic altogether from
"off"
and go straight to false
at runtime 🎉
I've added both of these forms to unit tests, and hopefully the logic is developed in a way where however/whenver you add a default input, it should work out 🙏
(this is in 2.0.0-rc.10
)you skip the transform logic altogether fromI fail to understand how the transform logic can be skipped. If a string value ("on" or "off") is passed, then it does have to go though"off"
and go straight tofalse
at runtime
.pipe
first, right?That's great David, thanks!
I think it would be worth clearly mentioning in the docs that defaults refer to the input type, regardless of where you assign them. It's pretty obvious with 'string="off"', but if you add a .default() after the morph, I expect some people will assume that it applies to the morphed type.
Also, something like type('string|boolean=false') doesn't work. If there is no value passed then false as a default is not applied. If that's intentional (and I can see why it's a weird thing to do), then should it give a type error in the editor, rather than just ignore it?
Yeah I need to elaborate a bit on that, but yes it's intentional. Initially, defaults could only be added within an object.
I loosened that restriction to allow defaults to be specified as metadata so that if they were ever attached to an object, they'd then have the default value.
But if it just made the type accept
undefined
(which I've thought about) it would cause all sorts of inconsistencies in the type system
defaults and .optional
both basically attach metadata that is used when the type is referenced in an object, not change the type itself
There's a broader set of rules that essentialy any constraint you apply without .pipe
or .narrow
applies to the input. If you think about it it wouldn't really make sense any other way because usually if you just return something from a function we don't have any runtime representation to constrain
But also all the .default
types should enforce that whatever you pass is their input, so in most cases you'd just get an error if you did something like .default(false)
in that caseIt has to go through pipe, but it goes through pipe during compilation, so only once.
Whenever you actually use it to validate data, it just skips right to the computed result and attaches it to the data.
Ohhhhhh! 😵 Thanks