Post-morph constraints not checked if constraints not met on another property
If I have:
If I then try:
I will get an error for name, but not for age.
If I instead try
Then I now get an error about age being <18, now that the 'name' check has passed.
Is this by design?
117 Replies
Input shape is checked first before the morphs and narrows
In this case, the input shape is
interesting - I would expect that to be the case for a morph on the outer object type, but not for the members, which should be able to be processed independently
at least, that's how zod does it
I guess this is made for unions, where you first have to determine if the input is compatible before trying to morph
Not sure tho
sure, but in this case,
age
can be parsed independently of any other item in the object, no?
yeah, this is weird, and not how I would expect it to behaveIf it wouldn't be a morph it's work
here's my example:
It is weird that a completely independent type object's evaluation changes based on where and how it is included in a broader type
that is both surprising and misleading as a user
obviously a morph on the whole object only works if the input is valid - but why are individual object keys not fully evaluated the way they would be if you were to just call
ageType("2")
for example?I repeat
It chechs the "input shape" first
and then starts morphing
I get that. That's an implementation detail.
I'm asking why the morph happens at the object level, and not at the nested type level
this also leads to a pretty bad user experience, where e.g. if you have two fields, you can have an error on only one field, then as soon as you fix it, suddenly an error appears on the other, even though that field's validation is totally independent from the first field you modified
more importantly, as a user, I should not have to know that
type("string.numeric.parse").to("number>18")
and type(["string.numeric.parse", ":", (e, ctx) => +e > 18 || ctx.mustBe('greater then 18')])
behave differently at runtime as to under which conditions from an external object the constraints are evaluatedI agree actually
Consider making an issue on github
The second example doesn't build actually
yeah, you have to remove
.parse
and then you have to pipe it to a parse step at the end, to get a number out
but that's a separate problem from the "morphs should apply at the level at which they are defined" issue at hand hereGitHub
Morphs should apply at the level in which they are defined · Issue ...
Report a bug 🔎 Search Terms .to, nested object validation, .pipe 🧩 Context System: OS: macOS 15.0 CPU: (10) arm64 Apple M1 Pro Memory: 105.30 MB / 16.00 GB Shell: 5.9 - /bin/zsh Binaries: Node: 20....
this shit works
very slick. This is how it should work all the time 🙂
(or rather, I can't think of any cases where you wouldn't want this)
unions before you selected the correct one maybe
like a union of morphs, you mean?
Unions of
string.number.parse<18 | string.number.parse>=18
be like (╯°□°)╯︵ ┻━┻ lolwell, it depends on how you define it, I think?
consider this:
This works conceptually for me, the
typeC
morph only runs and potentially reports errors if the input type (union) matches
I haven't used these extensively so I don't even know if this is valid:
(typeA.to(typeC)).or(typeB.to(typeD))
that's trickier
actually, is it?that's what i'm talking about
it actually doesn't seem that bad
I dunno how unions work but I assume they evaluate branches, choose the first one that has no errors, or error if none of the branches have no errors
and in this case, a morphed type is just the same as a simple type - either it passes or fails, independently of anything else
They are expected to be discriminable otherwise that's type construct time error
yes! but that's ok, type construct time works fine
we're talking about runtime behavior
hang on, let me put together another (silly) example
@francis BTW I've made a patch that doesn't break on error, I can make a branch if you're interested to try
These work exactly the same in all the input tests I've used
let nativeParseNumber = type("string.numeric.parse") let morphedParseNumber = type("string").to("string.numeric.parse")It's the same Type
ah, ok
I was trying to find a silly example where I could compare native and morphed behavior, but apparently that isn't it
@francis if you read carefully I've used narrows
.numeric is a narrow, .parse is a morph
> 18 is a narrow after the morph
thanks for your insight - I need to step away from this problem, but I still think that ideally, all operations that can be performed for parsing/validation, should be, in order to generate as many actionable errors early in the process as possible
I think it should be a flag, same as
onUnknownKeys: ignore|reject|delete
I come back to the earlier question: is there any case in which you don't want a validation step defined only on a specific object key value to run until the rest of the object is correct?
I guess it's when you don't care about your errors as much as about performance
So, for API
....?
but it still checks. it just doesn't do the exhaustive check you are expecting
otherwise you would have an option to fail on first error and not report anything, basically, no?
It "fails fast"
Hmmmm
ok, there are two worlds:
we care about performance: in this case, why evaluate all object keys? if any are invalid, fail here, don't even check the rest
we don't care about performance: we should evaluate as much as possible, and return all errors
the current middle ground of "we evaluate all object keys but not thoroughly" solves neither of these
I think there is that thing but it's not used for the root object
I also really don't think checking that if the numeric string we have already checked to make sure it's numeric is >18 is going to kill our performance
if you are defining horridly expensive morphs that's kinda your own fault, because they will be evaluated eventually given the correct input
It doesn't check that before morph
It doesn't even try to morph before it ensures the shape
I understand what it's doing, but my point is WHY
if we don't care about performance, why not morph and get a possible error here?
if we do care about performance, why evaluate everything else after we've already found a failure?
I don't see why "the shape of the object is correct" is a necessary prerequisite for "do a morph on an individual key's value to see if it succeeds"
Okay IDK the implementation reasons (you should ask ArkDavid about them) but I can make a branch with no early exit and check how it behaves
I mean, we can say the user is wrong for assuming things, but ... don't we want to be user friendly?
in this case my fundamental assumption is "a type I define behaves the same regardless of which object I include it in as a value"
I've ran the tests, they say "we don't have a reliable way to know when we can apply moprhs, so whe stop applying morphs as soon as there are errors"
re: we don't have a reliable way to know when we can apply morphs?
how could that be true for morphs defined on a single constituent of a type? it's not possible to define a morph that depends on anything other than that input value
I mean you shouldn't even try to apply morphs on broken values
sorry, I am really struggling to explain this I think
tl;dr
say you have your { age: "2", ...... } - why exactly does the rest of the object being broken stop us from applying a morph that only operates on the value of age?
I've made some 💩 now all tests pass
it's not like "the input is broken" or "the input is ok", every level and element of the input can be broken or ok independently
okay I got why it's because nothing works
Lemme fix
Tests still pass but now it works lol
@francis would you like to test it?
I have been thoroughly nerd sniped and do not have time, sorry
@francis would you like to test it next week?
as in "play with it trying to break it"
Anyways
It's https://github.com/Dimava/arktype/tree/dimava/keep-digging ping me if you want me to package it
GitHub
GitHub - Dimava/arktype at dimava/keep-digging
TypeScript's 1:1 validator, optimized from editor to runtime - GitHub - Dimava/arktype at dimava/keep-digging
why not push a PR as a resolution to the issue I created?
GitHub
Morphs should apply at the level in which they are defined · Issue ...
Report a bug 🔎 Search Terms .to, nested object validation, .pipe 🧩 Context System: OS: macOS 15.0 CPU: (10) arm64 Apple M1 Pro Memory: 105.30 MB / 16.00 GB Shell: 5.9 - /bin/zsh Binaries: Node: 20....
It's a dirty draft that probably has some bugs
It's just a proof-of concept
Okay I'll link it here and ask Davin on his opinion on what should be done with this
This was definitely intentional when it was implemented and more important when morphs could mutate the original value.
Honestly I'm still fine with the original behavior as it scales better with how unions are checked etc.
The point is, it
darn I pressed enter again
It would be more reasonable now to apply the morphs even if it's doomed to fail
The point is, it's not required anywhere other then in unions
yep
My POC that it doesn't breaks tests is on https://github.com/arktypeio/arktype/compare/main...Dimava:arktype:dimava/keep-digging?expand=1#diff-c068e331eec7cec18f2a25a3c64461b74518e77fc2e803df2a6daf4f1cb1b876
GitHub
Comparing arktypeio:main...Dimava:dimava/keep-digging · arktypeio/a...
TypeScript's 1:1 validator, optimized from editor to runtime - Comparing arktypeio:main...Dimava:dimava/keep-digging · arktypeio/arktype
💩
quality tho, I've just came up with something that doesn't fail immediately
well... idk, much like input value mutation, it's not at all obvious to a user that type morphs which are defined on a single value won't be applied if that value is in an object and the object is invalid somewhere else
Sure I can see that. It's also not that crazy though that if all validation occurs before any morphs are applied, and that initial validation pass fails, that morphs wouldn't be applied.
to the user of the library, the "morph boundary" is not obvious
Well there will always be nuances of this sort of thing you have to build an intuition for. A type system that includes transforms? It's already something fundamentally new
I see
const ageType = type("string.numeric.parse").to('number>18')
that behaves in a logically consistent way - given an input, it either returns a number > 18, or it errorsI'm not disagreeing that it could be better in these cases to evaluate as much as we can safely
I guess what I'm saying is that if there is a behavior option that doesn't require new intuition building, it would be nice to match that existing intuition
It is logically consistent you'd just prefer it to work differently which I understand.
yeah
in my mind individual types should be able to be treated as black boxes and have the same behavior no matter whether they are included in other types or not
and that's the intuitive assumption
That doesn't work with unions though
If you consider that you have
type("string.numeric.parse").to('number>18')
as a property on an object, and that object is a branch of a union.
You can't just start applying morphs when you determine that prop is valid because if a neighboring prop is not valid, your object is now in a bad state if there is a future branch that is valid
That's why all validation has to come firstwell, the problem here is mutation, also
if it didn't mutate, you fail this branch, move on to the next, with the input unmodified
I strongly advocate for non-mutation of input because it makes stuff like this make intuitive sense
But then you have huge performance problems- tons of unnecessary object creation for branches that are already guaranteed to fail
Well that's not really my call how people implement their morphs
this comes back to the performance question from earlier. if a branch is already guaranteed to fail, why evaluate all the object keys as soon as one doesn't pass?
In a union I don't
ok, then why would it be a problem to skip morphs in a union case?
Nor does TypeScript for the same reason because you'd get huge unmanageable error messages
and follow them in a non-union case?
I get why this works the way it does, it just is incredibly surprising to people who don't understand the internals
type().to(...)
strongly implies that this is an operation on this type not collecting a morph to be applied laterWell not everything can work intuitively and ideally in every case
it's confusing enough that I would support removing
.to
entirelySometimes there are genuine trade offs
it's actively misleading
But I agree in a case like the one that's originally mentioned, we may be better off evaluating the morph
this isn't a correctness issue, it's a developer ergonomic issue. You'll never have a case where an object parses where it shouldn't, for example - it's just a weird case where as a dev you would expect the key's value to be evaluated independently, and it's not
You may be jumping to conclusions a bit here
idk if I'm explaining it well, but as a user, I think
type({ age: ageType, ... })
runs ageType(input.age)
and it doesn't
but that is the obvious (wrong) assumptionIt will always run input validation on that type
yes, but it doesn't run
ageType
, which is different
it runs part of it onlyThe only mental model you need to explain all of that is that if you transform your data, that won't happen unless all of it is valid
that's the part that is extremely surprising
@francis can you please provide more tests
currently we only have one and it's in OP post
that's the only one I can think of off the top of my head, is an example of an object where a key has a morph and the object is invalid in some other way
that's also the obvious use case for e.g. form field validation
ah, ok, now we're getting somewhere!
then my follow-up ask is: is there a way to do a narrowed validation without having it be considered a transformation? i.e. "this input string is a numeric string and it's greater than 18"
arktype gives a lot of very nice syntax to do that for just the input itself, but not if you use it in an object, because then you run into the input vs transformation problem
Yeah you'd just write the custom logic in the
.narrow
.could there be named narrowings that match the arktype types?
being able to use the arktype syntax is extremely nice
I am not sure you'd have to come up with a proposal but I'd be wary of adding a lot more new syntax. I want people to mostly benefit from TS syntax that is already familiar and supplement that with a few runtime constraints
I would be open to adding it on later post stable 2.0
ah, I mean something like
.narrow('number>18')
as opposed to .to('number>18')
though that's also confusing because what is the difference? there isn't one, at the time that this type is defined
the only difference appears if you use it somewhere else
I think that's what's making me most uneasy and confused
the behavior of a type constant changes based on whether it's in an object or not. I know that's not what's actually happening at runtime, but conceptually, that's what it's doing for a userActually there wouldn't be a difference I suppose
When I said
.narrow
, I meant it without a morph
That is the real difference- whether the type contains a morph or notah, right, of course, in this case
.parse
is the morph?Yeah
hmmmm. yeah this is real tricky
I dunno
And you can see that represented visually at a type-level any time you have something like
(In: x) => Out<y>
or (In: x) => To<y>
in your TypeI understand why it's happening but it's confusing as a user to have types not be independent of each other in terms of behavior
Well it's just a reality that they can't be if you want to handle unions well
well, as a user, that also makes sense ... I know it's a runtime performance hit but the behavior makes intuitive sense
So you'd just have to change your intuitions if you want to understand those nuances. But I would be willing to reconsider cases like the original one mentioned in this thread
I don't think you're considering how bad it would be to check every error on every branch of a union
oh, I get that it would be bad, but as you point out you already fail early on union branches
There's no world in which that would be a good decision on average
Right but failing early on union branches already breaks your pure composition model
well, it depends on how errors are reported
Because there's context in terms of which errors will be reported depending on whether the type is being referenced in a union or not
well in a union, I don't expect all errors to be reported for a very simple reason: if branch B matches, I don't care about all the errors from branch A
But what if no branches match
yeahhhh.... hm
I don't actually know how this works currently. Say you have two branches, each of which has morphs from the same input type, say one is a numeric parse and the other is a date parse, and then follow-up validation on those. What happens if you union those together?
That would fail when you created the type because a morph in a union has to be discriminable from every other branch of that union
This stuff is complicated though it does require some deep thinking about preserving the determinism and commutativity of these operators
Not every behavior can just be intuitively grasped as pure composition. I try to have as few edge cases as possible and I think the original issue represents an opportunity to give more helpful errors in general with little downside.
I think my underlying ask is that outside of unions, where we can match the intuitive composition behavior, it would be nice to do so
Chipping in as a fairly ignorant user (of ArkType) here... I'd imagine a lot of people wanting to use ArkType as part of a form validation process. With the current behaviour, you can't get full errors to your app user about what's wrong with an input form using the obvious approach. You end up with an error or two for the user to correct, they do so, and then another one pops up.
It seems the only way round that is to not use ArkType to do the data transformation from FormData strings into numeric. I don't think it's the behaviour most people would expect and is bound to cause a bunch of confusion.
If it stays the way it is, I think the docs should highlight this issue and maybe provide a suggested approach for form validation, as it's such a common use case.
I think the original issue represents an opportunity to give more helpful errors in general with little downside.@Stuart B @francis Yeah I agree this seems useful here
it's an interesting problem. I'm reminded of a conditional-logger I made recently, in which phase 1 of a process generates some diagnostics that become irrelevant if the thing had no problem
I guess it's maybe not very similar to the situation from the OP
@Stuart B @francis This will include the error from morph output as of
2.0.0-rc.13