Automatically applying parse.integer

Let's say I have a type I'm using to validate some API, e.g.
const personSchema = type({
name:'string',
age:'0<=number<=120'
})
const personSchema = type({
name:'string',
age:'0<=number<=120'
})
At some point, I also want to use this type to validate form input. As it's formData,the age will be a string not a number. I know I can do something like this:
const formDataParser=type('parse.formData')
const coerce=type({'age?':'parse.integer'})
const parsedFormInput = formDataParser(formInput).pipe(coerce).pipe(personSchema)
const formDataParser=type('parse.formData')
const coerce=type({'age?':'parse.integer'})
const parsedFormInput = formDataParser(formInput).pipe(coerce).pipe(personSchema)
However, is there a way to somehow automate the coerce step? I have lots of these types (some of them dynamically created) and I'd rather avoid having to manually create an extra type to pick out the numeric fields and parse.integer them. Maybe there's some way I can automatically pick out the number properties from a type and automatically build the coerce step so that parse.integer is only applied to any properties that are numeric in the final schema? I'm fairly new to this, so it could just be my lack of Typescript skill. Any pointers would be much appreciated.
108 Replies
ssalbdivad
ssalbdivad4mo ago
Sounds doable but definitely a bit tricky. There's no built-in "coerce" yet. I'd have to think about what the semantics of that would be relative to the existing parse keywords. Where does personSchema come from? Could that be defined in a way where the parsing happens there, and the output could be extracted separately? If you already have the personSchema type to compare with the form input, I guess you could iterate over the keys that way to see which should be converted to numbers
Stuart B
Stuart BOP4mo ago
I'd assumed 'parse.integer' would convert a string to an integer and leave an integer alone, but if fails if the input is already an integer. Would there be a problem with having parse.integer just pass through the existing value if it's already an integer? The personSchema is just a simple example. I have a whole bunch of data objects that can be written to a db, both via an API (where numerics will already be numbers) and forms (where the numerics will be strings). I've built a simple form validator, to which I pass the formData and the arktype type. The validator function uses the passed type to validate the input and passes back the original data and a success message or errors.
I can always deal with this by creating 2 types instead of 1. One is the "API" type and the other one is a parse.integer for the numeric fields and piped to the API type. It just felt a little redundant. Feels like there should be a more elegant way.
ssalbdivad
ssalbdivad4mo ago
I don't think I would ever make morphs accept their output types in general just because it's sometimes convenient But I might add some additional keywords to help with that e.g. there is a lift array that converts something to an array if it is not one already so it would be somewhat similar to that I'm sure the right abstractions could help but it's important to strike a balance and not start implementing magical transformations with lots of edge cases as built ins. Ideally they should primarily be simple I/O I think if I were going to add something like this though it would probably be like number.integer.coerced
Stuart B
Stuart BOP4mo ago
Thanks David. In the meantime, if I already have a type() defined, is there any way I can extract the keys of numeric properties from that type? I could then use them to build my parse.integer step.
ssalbdivad
ssalbdivad4mo ago
import { type, type Type } from "arktype"
import type { arkKeyOf, getArkKey } from "arktype/internal/keys.ts"

type keysWithNumberValue<o extends object> = {
[k in arkKeyOf<o>]: getArkKey<o, k> extends number ? k : never
}[arkKeyOf<o>] &
unknown

const number = type("number")

const keysWithNumberValue = <t extends Type<object>>(
t: t
): Type<keysWithNumberValue<t["t"]>> =>
t.keyof().internal.distribute(
key => (t.get(key as never).extends(number) ? key : null),
branches =>
t.$.internal.rootNode(branches.filter(branch => branch !== null))
) as never

const t = type({
foo: "number",
bar: "1",
baz: "string"
})

const keys = keysWithNumberValue(t)

console.log(keys.expression)
import { type, type Type } from "arktype"
import type { arkKeyOf, getArkKey } from "arktype/internal/keys.ts"

type keysWithNumberValue<o extends object> = {
[k in arkKeyOf<o>]: getArkKey<o, k> extends number ? k : never
}[arkKeyOf<o>] &
unknown

const number = type("number")

const keysWithNumberValue = <t extends Type<object>>(
t: t
): Type<keysWithNumberValue<t["t"]>> =>
t.keyof().internal.distribute(
key => (t.get(key as never).extends(number) ? key : null),
branches =>
t.$.internal.rootNode(branches.filter(branch => branch !== null))
) as never

const t = type({
foo: "number",
bar: "1",
baz: "string"
})

const keys = keysWithNumberValue(t)

console.log(keys.expression)
A bit tricky to do, especially if you don't know the runtime type system but this seems to wrok
Stuart B
Stuart BOP4mo ago
Thanks! And having read that, there's also the added bonus that I no longer feel dumb for not knowing how to do it myself 🤣
ssalbdivad
ssalbdivad4mo ago
Yeah like I said definitely not the most trivial thing either at a type-level or at runtime haha
PIat
PIat3mo ago
I have the same use case
PIat
PIat3mo ago
I would like to achieve such behavior
No description
ssalbdivad
ssalbdivad3mo ago
Can't you just write a morph?
PIat
PIat3mo ago
Ooooh, you updated docs!
ssalbdivad
ssalbdivad3mo ago
I added a few new sections. I also have some good abstractions for showing multiple definition types
PIat
PIat3mo ago
Oh righttttttt.... So this forum is about doing it inherently Sorry 🙈
ssalbdivad
ssalbdivad3mo ago
Yeah, youc ould always just write number.integer|string.parse.integer Which I think is a lot clearer anyways
PIat
PIat3mo ago
You could?????
ssalbdivad
ssalbdivad3mo ago
Why not?
PIat
PIat3mo ago
I never saw the parse.* being used in this way, it never even came across my mind Just assumed it wouldn't work
ssalbdivad
ssalbdivad3mo ago
Oh I guess I wrote it wrong haa It should have been string.integer.parse
PIat
PIat3mo ago
Because it seems so complicated to implement
ssalbdivad
ssalbdivad3mo ago
What part? It's all just composable rules in the type system
PIat
PIat3mo ago
Hehe... The DX is just too good to be true I love it
ssalbdivad
ssalbdivad3mo ago
A conform adapter would actually be really useful I know a couple teams that use conform that are interested in ArkType
PIat
PIat3mo ago
I am in the process of making one
ssalbdivad
ssalbdivad3mo ago
So I see, really excited to see that! Happy to do a review once it's ready
PIat
PIat3mo ago
It works with the strings, now just to add morphs I'm thinking whether creating an exported scope would be the best way... The scope would then be extended and have all the necessary keys
ssalbdivad
ssalbdivad3mo ago
Definitely an option. Most things you can do with or without a scope so it comes down to preference. Scopes let you use the string syntax more with your own keywords and there's a couple things they let you do that would otherwise not be possible: - Configuring sets of types at once - Representing cyclic types Outside that though it is down to preference really
PIat
PIat3mo ago
Because with Zod it's kind of simple to override the behaviour. Because it's not composable in the same way, the library authors can simply predict what the users might want But Arktype allows for many crazy things to happen
ssalbdivad
ssalbdivad3mo ago
You can just accept an option for preprocess or whatever Zod does right?
PIat
PIat3mo ago
Is it possible to create bookmarks on Discord? 😩
ssalbdivad
ssalbdivad3mo ago
Then make that the initial morph, and pipe it to whatever the validated types are Haha I don't think so.... I'll include all this in th edocs soon though:P
PIat
PIat3mo ago
Yay I'll bookmark that
ssalbdivad
ssalbdivad3mo ago
Right should be very convenient since the docs will be your home page anyways 😛
PIat
PIat3mo ago
If it'll have a theme toggle, I'd put it on the desktop
ssalbdivad
ssalbdivad3mo ago
Haha it used to but I don't have ArkLight so I got rid of it 😛
PIat
PIat3mo ago
So the same theme that would be used in VS code would go to the website?
ssalbdivad
ssalbdivad3mo ago
Yeah it literally imports ArkDark from the dir that publishes it
PIat
PIat3mo ago
I'd need to map the colors as you explained right?
ssalbdivad
ssalbdivad3mo ago
Yeah I would accept a PR for sure on that though If you added a parallel ArkLight theme + updated the css for the docs to include a light-mode palette
PIat
PIat3mo ago
I don't know how to do those :/ Never commited to a private repo
ssalbdivad
ssalbdivad3mo ago
It's not a private repo
PIat
PIat3mo ago
Got it public, public!
ssalbdivad
ssalbdivad3mo ago
Oh haha
PIat
PIat3mo ago
PRs, FR etc, no idea
ssalbdivad
ssalbdivad3mo ago
It's a little out of date possibly in some places but it is pretty straight forward
PIat
PIat3mo ago
:ooooo Alright, no excuses ;)
ssalbdivad
ssalbdivad3mo ago
It could be a bit tricky to get the styling customizations right for the code blocks, just try to use as many css variables as you can I guess To easily swap between palettes
PIat
PIat3mo ago
It's funny, because it already supports my @ark/i18n thing 🙈
ssalbdivad
ssalbdivad3mo ago
Also when you're editing ArkLight, by far the easiest way to test changes is going to Run and Debug in VSCode and clicking ArkDark Dev
No description
ssalbdivad
ssalbdivad3mo ago
It will launch a new VSCode window that reflects your changes
PIat
PIat3mo ago
Peeeeerfect Yes, I'll mirror the dark theme
ssalbdivad
ssalbdivad3mo ago
Could be great! ArkLight is a nice name too Kinda flows off the tongue
PIat
PIat3mo ago
But ArkDark rhymes 😁
ssalbdivad
ssalbdivad3mo ago
Yeah I mean it's not as good as that But ArkLight sounds like ArkWright I guess?
PIat
PIat3mo ago
Definitely, I'm a heavy light theme user I don't know when that happened, probably when I started coding outdoors You're ArkRight
ssalbdivad
ssalbdivad3mo ago
No description
ssalbdivad
ssalbdivad3mo ago
One of a small number of words that starts with Ark haha
PIat
PIat3mo ago
Oh my ArkWright will be the tool that generates ArkType schemas compatible with every programming language Or the tool that controls Playwright instances only using the composable type definitions, I'm still on the fence
ssalbdivad
ssalbdivad3mo ago
The really funny thing is the initial motivation for working on ArkType was actually a UI testing framework I was building on top of playwright so not far off
PIat
PIat3mo ago
This is how they do it in @conform/zod:
let schema: ZodTypeAny = type
const def = (type as ZodFirstPartySchemaTypes)._def

if (def.typeName === 'ZodNumber') {
schema = any()
.transform((value) =>
coerceString(value, (text) =>
text.trim() === '' ? Number.NaN : Number(text),
),
)
.pipe(type);
}
let schema: ZodTypeAny = type
const def = (type as ZodFirstPartySchemaTypes)._def

if (def.typeName === 'ZodNumber') {
schema = any()
.transform((value) =>
coerceString(value, (text) =>
text.trim() === '' ? Number.NaN : Number(text),
),
)
.pipe(type);
}
I guess the change will be _def to expression, like I did in shady-ark-i18n
ssalbdivad
ssalbdivad3mo ago
What are you looking to check, if the schema is a number?
PIat
PIat3mo ago
If the individual key was set up as a number
const typee = type({
userName: 'string',
age: 'number'
})
const typee = type({
userName: 'string',
age: 'number'
})
Age should be piped to number when an HTML field passes the stringified number
ssalbdivad
ssalbdivad3mo ago
typee.get("age").extends("number")
typee.get("age").extends("number")
PIat
PIat3mo ago
Oh, and it returns a boolean?
ssalbdivad
ssalbdivad3mo ago
Yeah
PIat
PIat3mo ago
Wow... I'm really complicating things in my head, aren't I
ssalbdivad
ssalbdivad3mo ago
This does seem a lot like the original question though
PIat
PIat3mo ago
Yeah...
ssalbdivad
ssalbdivad3mo ago
I would just do this then:
import { type, type Type } from "arktype"
import type { arkKeyOf, getArkKey } from "arktype/internal/keys.ts"

type keysWithNumberValue<o extends object> = {
[k in arkKeyOf<o>]: getArkKey<o, k> extends number ? k : never
}[arkKeyOf<o>] &
unknown

const keysWithNumberValue = <t extends Type<object>>(
t: t
): Type<keysWithNumberValue<t["t"]>> =>
t.keyof().internal.distribute(
key => (t.get(key as never).extends(type.number) ? key : null),
branches =>
t.$.internal.rootNode(branches.filter(branch => branch !== null))
) as never

const t = type({
foo: "number",
bar: "1",
baz: "string"
})

const keys = keysWithNumberValue(t)

console.log(keys.expression)
import { type, type Type } from "arktype"
import type { arkKeyOf, getArkKey } from "arktype/internal/keys.ts"

type keysWithNumberValue<o extends object> = {
[k in arkKeyOf<o>]: getArkKey<o, k> extends number ? k : never
}[arkKeyOf<o>] &
unknown

const keysWithNumberValue = <t extends Type<object>>(
t: t
): Type<keysWithNumberValue<t["t"]>> =>
t.keyof().internal.distribute(
key => (t.get(key as never).extends(type.number) ? key : null),
branches =>
t.$.internal.rootNode(branches.filter(branch => branch !== null))
) as never

const t = type({
foo: "number",
bar: "1",
baz: "string"
})

const keys = keysWithNumberValue(t)

console.log(keys.expression)
PIat
PIat3mo ago
This won't be enough, since there are other simple conditions
ssalbdivad
ssalbdivad3mo ago
But you can use the same patterns to transform the schemas
PIat
PIat3mo ago
Yes Seeing such code is so cool 😁 Peak TS
ssalbdivad
ssalbdivad3mo ago
Hah well ideally this kind of stuff is externally simple. I would definitely consider building in some abstraction for common form transformations
PIat
PIat3mo ago
Okay, I got it! Such patterns for each condition And in the end merge
ssalbdivad
ssalbdivad3mo ago
Actually I'd just write it as one mapping like that So you don't have to use merge Basically just iterate over each key the way I am there, but then directly create a new object with the updated types Would be a great use case for mapped types once those are implemented
PIat
PIat3mo ago
Ooooohhhhhhh Right, this was for filtering That's why it had the null Sorry, it's been a loooong day 😁
ssalbdivad
ssalbdivad3mo ago
Yeah in reality you'd check for the conditions then if it doesn't mean any of them just keep the value the way it is
PIat
PIat3mo ago
Yeah this is perfect
ssalbdivad
ssalbdivad3mo ago
No problem not your fault. It would be good to have builtins to make this easier
PIat
PIat3mo ago
Should this type return actual types or strings?
type coerceTypes<o extends Type['t']> = {
[k in arkKeyOf<o>]: getArkKey<o, k> extends number ? k : o[k]
}[arkKeyOf<o>] &
unknown
type coerceTypes<o extends Type['t']> = {
[k in arkKeyOf<o>]: getArkKey<o, k> extends number ? k : o[k]
}[arkKeyOf<o>] &
unknown
ssalbdivad
ssalbdivad3mo ago
o is t You don't want to pass a type instance Well Type["t"] is nothing so you can just remove that constraint
PIat
PIat3mo ago
I'm happy about that 😁
ssalbdivad
ssalbdivad3mo ago
k is the key getArkKey<o, k> is the value
PIat
PIat3mo ago
This type seems almost redundant?
ssalbdivad
ssalbdivad3mo ago
It's for filtering the keys
PIat
PIat3mo ago
Since we're not filtering anymore
ssalbdivad
ssalbdivad3mo ago
Just an easy way to distrubte over them
PIat
PIat3mo ago
Yes
ssalbdivad
ssalbdivad3mo ago
Yeah you'd probably want an actual mapped type So no index access at the end
PIat
PIat3mo ago
But since the result in the runtime will be what we put in the type Then there's no much reason to map over the type itself
ssalbdivad
ssalbdivad3mo ago
There might be because you will have to cast the mapped type potentially? I guess it depends if you need the exact types for this
PIat
PIat3mo ago
Almost like this would be enough?
const coerceTypes = <t extends Type<object>>(
t: t,
): t =>
const coerceTypes = <t extends Type<object>>(
t: t,
): t =>
PIat
PIat3mo ago
Oh no 😩 Updated to the latest version
No description
ssalbdivad
ssalbdivad3mo ago
string.url?
PIat
PIat3mo ago
No description
PIat
PIat3mo ago
Seems to also be missing I really like how it's under string.* now Will make everything more organized for sure
ssalbdivad
ssalbdivad3mo ago
Hmm really
PIat
PIat3mo ago
I'll just put in string for now
ssalbdivad
ssalbdivad3mo ago
string.url will work at runtime I just missed including the type It will be back in the next release
PIat
PIat3mo ago
Perfeeect I need the d.ts build in this particular package, because I couldn't figure any other way to share its Arktype definitions otherwise 😁 And one dts build takes 15-20 seconds, not sure why
ssalbdivad
ssalbdivad3mo ago
Hmm that sounds a bit sketchy but ok haha
PIat
PIat3mo ago
I'm happy it at least works. I update that package rarely anyways What is the proper way to ge the whole type in the end?
const keysWithNumberValue = <t extends Type<object>>(t: t): t =>
t.keyof().internal.distribute(
(key) => {
let tt = t.get(key as never)
if (tt.extends(type.number))
return { [key]: type('number|string').pipe((v) => Number(v)) }

return { [key]: key }
},
(branches) => type(branches.reduce((acc, cu) => ({ ...acc, ...cu }), {})),
) as never
const keysWithNumberValue = <t extends Type<object>>(t: t): t =>
t.keyof().internal.distribute(
(key) => {
let tt = t.get(key as never)
if (tt.extends(type.number))
return { [key]: type('number|string').pipe((v) => Number(v)) }

return { [key]: key }
},
(branches) => type(branches.reduce((acc, cu) => ({ ...acc, ...cu }), {})),
) as never
Dimava
Dimava3mo ago
You may make yourself a server and copy message link like https://discord.com/channels/957797212103016458/1276473191421837382/1281024651937714258
PIat
PIat3mo ago
Oooh, neat, thanks!
ssalbdivad
ssalbdivad3mo ago
@PIat string.url back in 2.0.0-rc.6!
PIat
PIat3mo ago
Yay, thanks!
Stuart B
Stuart BOP3mo ago
Did you every get this working?
Want results from more Discord servers?
Add your server