A
arktypeβ€’3mo ago
PIat

Accept number in string type

Hello! I'm parsing each value in a form with JSON.parse to convert string values like false to actual booleans. Unfortunately, this introduces the issue, that if someone inputs 123 into a text field, it gets parsed into a number type by JSON.parse. Then if the field is type string. An error is thrown, since number is not assignable to string. How should a type look like, if it should handle such a situation?
120 Replies
ssalbdivad
ssalbdivadβ€’3mo ago
Could you write a code snippet showing showing this?
PIat
PIatOPβ€’3mo ago
const formType = type({
intent: "'profile'",
name: 'string>0',
surname: 'string>4',
agree: 'boolean',
}).or({
intent: "'private'",
password: 'string>0',
})

const formData = await request.clone().formData()

// passing a value that would get returned
const convertedData =
{
intent: 'profile',
name: 123,
surname: 'PlatPlat',
agree: true,
} ?? generateFormData(formData)

const formResult = formData && formType(convertedData)
const formType = type({
intent: "'profile'",
name: 'string>0',
surname: 'string>4',
agree: 'boolean',
}).or({
intent: "'private'",
password: 'string>0',
})

const formData = await request.clone().formData()

// passing a value that would get returned
const convertedData =
{
intent: 'profile',
name: 123,
surname: 'PlatPlat',
agree: true,
} ?? generateFormData(formData)

const formResult = formData && formType(convertedData)
Result:
name must be a string (was number)
name must be a string (was number)
TizzySaurus
TizzySaurusβ€’3mo ago
What's this doing?
const convertedData =
{
intent: 'profile',
name: 123,
surname: 'PlatPlat',
agree: true,
} ?? generateFormData(formData)
const convertedData =
{
intent: 'profile',
name: 123,
surname: 'PlatPlat',
agree: true,
} ?? generateFormData(formData)
PIat
PIatOPβ€’3mo ago
generateFormData runs JSON.parse on every value. The left-hand site is what it would return given the request
{
intent: 'profile',
name: '123',
surname: 'PlatPlat',
agree: 'true',
}
{
intent: 'profile',
name: '123',
surname: 'PlatPlat',
agree: 'true',
}
TizzySaurus
TizzySaurusβ€’3mo ago
So.... just change generateFormData to not JSON.parse when the text field type is "string"? This doesn't seem like an AT issue
PIat
PIatOPβ€’3mo ago
It's not an AT issue, I am asking advice how it could be handled
TizzySaurus
TizzySaurusβ€’3mo ago
Right, then this is how ig ^
PIat
PIatOPβ€’3mo ago
I am converting it in such a way, since multipart/form-data has to send everything as strings
TizzySaurus
TizzySaurusβ€’3mo ago
You presumably have a way of knowing which input fields are going to be a text type
PIat
PIatOPβ€’3mo ago
Yes
TizzySaurus
TizzySaurusβ€’3mo ago
So just have your generateFormData function take that into account In a very verbose way, something like
const generateFormData(obj) => {
const newObj = {}

for (const [key, value] of Object.entries(obj)) {
if (stringTextFields.includes(key)) {
newObj[key] = value
else {
newObj[key] = JSON.parse(value)
}
}
return newObj
}
const generateFormData(obj) => {
const newObj = {}

for (const [key, value] of Object.entries(obj)) {
if (stringTextFields.includes(key)) {
newObj[key] = value
else {
newObj[key] = JSON.parse(value)
}
}
return newObj
}
Or as one line, something like
const generateFormData = (obj) => Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, stringTextFields.includes(k) ? v : JSON.parse(v)]))
const generateFormData = (obj) => Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, stringTextFields.includes(k) ? v : JSON.parse(v)]))
PIat
PIatOPβ€’3mo ago
I guess this could be use for the comparison
formType.get('name').equals('string')
formType.get('name').equals('string')
But will this also work when the type has contraints? (string>0)
TizzySaurus
TizzySaurusβ€’3mo ago
Possibly. I'm not familiar with the library you're using for forms Yeah; why wouldn't it? This is all before ArkType
PIat
PIatOPβ€’3mo ago
formType is an Arktype type
const formType = type({
intent: "'profile'",
name: 'string>0',
surname: 'string>4',
agree: 'boolean',
}).or({
intent: "'private'",
password: 'string>0',
})
const formType = type({
intent: "'profile'",
name: 'string>0',
surname: 'string>4',
agree: 'boolean',
}).or({
intent: "'private'",
password: 'string>0',
})
TizzySaurus
TizzySaurusβ€’3mo ago
It means that having a name of "123" is valid (which arguably shouldn't be a valid name, but I don't have the context etc.) but it will still apply the constraint of the length being >0 Oh I think I see what you mean now Hmm Let me get an ArkType environment open
PIat
PIatOPβ€’3mo ago
Another issue is I can't use formType.get('name'), since not all types in the union have that key:
ParseError: Key "name" does not exist on { intent: "private", password: string >= 1 }
ParseError: Key "name" does not exist on { intent: "private", password: string >= 1 }
So I'd need to know the intent in advance
TizzySaurus
TizzySaurusβ€’3mo ago
I meant the actual form things Not the keys within the ArkType type Stuff from the request So like if you had the html
<input id="firstName" type="text">Name</input>
<input id="lastName" type="text">Surname</input>
<input id="firstName" type="text">Name</input>
<input id="lastName" type="text">Surname</input>
then stringTextFields would be ["firstName", "lastName"]
PIat
PIatOPβ€’3mo ago
GitHub
remix-hook-form/src/utilities/index.ts at main Β· forge42dev/remix-h...
Open source wrapper for react-hook-form aimed at Remix.run - forge42dev/remix-hook-form
TizzySaurus
TizzySaurusβ€’3mo ago
Because (I assume) formData would be {firstName: "bob", lastName: "doe"}
PIat
PIatOPβ€’3mo ago
Yes
TizzySaurus
TizzySaurusβ€’3mo ago
Basically, stringTextFields is whatever the keys are in formData that you know should remain a string type
PIat
PIatOPβ€’3mo ago
But that's a little redundant, when I have already created an Arktype type which says it should be string
TizzySaurus
TizzySaurusβ€’3mo ago
It's not... because you're blindly doing parsing before AT So your parsing has to handle stuff that otherwise AT would do Or you don't blindly parse stuff and just leave everything to AT, which arguably makes more sense You've effectively got input -> parse to json -> ArkType ArkType is designed to have no middle step
PIat
PIatOPβ€’3mo ago
That would be perfect... Yes, it all feels wrong And unsafe
TizzySaurus
TizzySaurusβ€’3mo ago
One option is a morph, but there may be a better way I'm not up-to-date on the latest types and unfortunately the docs don't list them (yet) πŸ˜… And for some reason I can't get an ArkType environment working
ssalbdivad
ssalbdivadβ€’3mo ago
Are you using moduleResolution: "node"? It was broken in last release But I also released 2.0.0-rc.0 this morning
TizzySaurus
TizzySaurusβ€’3mo ago
I'm using the actual AT repo, so it's on whatever that uses
ssalbdivad
ssalbdivadβ€’3mo ago
What issue are you having in the repo/
TizzySaurus
TizzySaurusβ€’3mo ago
No description
PIat
PIatOPβ€’3mo ago
Yeah, I wouldn't know how to convert multipart/form-data inside of Arktype to handle booleans and arrays. Does parse.formData handle booleans?
TizzySaurus
TizzySaurusβ€’3mo ago
(I'm on my json schema branch fwiw, but have merged main into it)
ssalbdivad
ssalbdivadβ€’3mo ago
It's probably because I switched from paths to customConditions, how are you running the file
TizzySaurus
TizzySaurusβ€’3mo ago
tsx ark/jsonschema/del.ts tsx was previously working iirc
ssalbdivad
ssalbdivadβ€’3mo ago
Yeah if you look at the scripts in package.json you will see how you have to run it now You should just be able to use pnpm ts instead
TizzySaurus
TizzySaurusβ€’3mo ago
No description
TizzySaurus
TizzySaurusβ€’3mo ago
And pnpm tsc fails with a whole load of errors
ssalbdivad
ssalbdivadβ€’3mo ago
Maybe you have to build first I guess to get that You shouldn't have to build to not see errors though
TizzySaurus
TizzySaurusβ€’3mo ago
I think I may have just fked the branch :Shrug: Let me try recloning the repo
TizzySaurus
TizzySaurusβ€’3mo ago
Oof, big repo
No description
ssalbdivad
ssalbdivadβ€’3mo ago
It's really not big at all lol Your internet is just being slow A few months ago it was actually like 10x bigger I removed some GIFs from the history
TizzySaurus
TizzySaurusβ€’3mo ago
My internet is the same speed as normal (just over 100mbps) :Shrug:
ssalbdivad
ssalbdivadβ€’3mo ago
Lol That doesn't mean it's downloading it at 100mbps Maybe it's the server, my point is it's not the repo
TizzySaurus
TizzySaurusβ€’3mo ago
Yeah, fair enough
TizzySaurus
TizzySaurusβ€’3mo ago
Oh yeah
No description
ssalbdivad
ssalbdivadβ€’3mo ago
Yeah it helped a lot when I went through the history and removed all the files over a certain size
TizzySaurus
TizzySaurusβ€’3mo ago
How do I get it to import arktype? πŸ˜…
"dependencies": {
"arktype": "workspace:*",
"@arktype/schema": "workspace:*",
"@arktype/util": "workspace:*"
}
"dependencies": {
"arktype": "workspace:*",
"@arktype/schema": "workspace:*",
"@arktype/util": "workspace:*"
}
have got this but pnpm i isn't working and if I just run it says it can't find the arktype module
ssalbdivad
ssalbdivadβ€’3mo ago
It's probably a corepack thing? Ughh stupid corepack
TizzySaurus
TizzySaurusβ€’3mo ago
(This is the pnpm i error)
ssalbdivad
ssalbdivadβ€’3mo ago
You can add this to scripts: "ts": "node ./ark/repo/ts.js", You need to update pnpm
TizzySaurus
TizzySaurusβ€’3mo ago
Do I need to be using.node 22.6+?
ssalbdivad
ssalbdivadβ€’3mo ago
Only if you want to be able to run TS directly from node I have tehe scripts set up to work with 20 and 18 also Well it tried to fall back to tsx but that wasn't installed
TizzySaurus
TizzySaurusβ€’3mo ago
tsx is installed globally
ssalbdivad
ssalbdivadβ€’3mo ago
Pnpm doesn't respect that I guess you need to pnpm i
TizzySaurus
TizzySaurusβ€’3mo ago
But I can't pnpm i Oh yeah, update pnpm
ssalbdivad
ssalbdivadβ€’3mo ago
Damn
ssalbdivad
ssalbdivadβ€’3mo ago
Node actually had a solution to this BS but they just killed it https://www.youtube.com/watch?v=I7qMwaxNNOc
Theo - t3β€€gg
YouTube
Corepack is dead, and I'm scared
The removal of Corepack from Node is effectively a death sentence for the project, and I'm not pumped about it. SOURCE https://socket.dev/blog/node-js-takes-steps-towards-removing-corepack Check out my Twitch, Twitter, Discord more at https://t3.gg S/O Ph4se0n3 for the awesome edit πŸ™
TizzySaurus
TizzySaurusβ€’3mo ago
Yeah, I saw that video πŸ˜… I'm kinda lost πŸ˜…
ssalbdivad
ssalbdivadβ€’3mo ago
Like this would 100% solve your problem if it was enforced by default But you should just be able to pnpm upgrade && pnpm i Okay I might have lied about what upgrade did
TizzySaurus
TizzySaurusβ€’3mo ago
It failed anyway :KEKW:
ssalbdivad
ssalbdivadβ€’3mo ago
Maybe just try pnpm add -g pnpm I'm using corepack to manage it now but I guess I'll have to switch off pnpm add -g pnpm is what I always did in the past when I wanted to upgrade
TizzySaurus
TizzySaurusβ€’3mo ago
Doesn't look like it works :Shrug:
No description
PIat
PIatOPβ€’3mo ago
Wait what???? πŸ‘€πŸ‘€πŸ‘€πŸ‘€πŸ‘€ Can I ditch tsx now?
ssalbdivad
ssalbdivadβ€’3mo ago
Oh you're using homebrew and you still have an old version Lol I can't debug your env
TizzySaurus
TizzySaurusβ€’3mo ago
Theo - t3β€€gg
YouTube
Node FINALLY Supports TypeScript
Huge! Can't wait for the new Node release to enable type stripping with --experimental-strip-types SOURCES https://github.com/nodejs/node/pull/53725 https://github.com/tc39/proposal-type-annotations?tab=readme-ov-file Check out my Twitch, Twitter, Discord more at https://t3.gg S/O Ph4se0n3 for the awesome edit πŸ™
PIat
PIatOPβ€’3mo ago
Woooooooooooooaaaaaaaaahhh 🀩🀩🀩🀩🀩
ssalbdivad
ssalbdivadβ€’3mo ago
I would keep it around a bit longer but I was able to get AT's unit tests running in ~30 minutes
PIat
PIatOPβ€’3mo ago
I am tired of registering ts-node and how workers run when they are written in TS
ssalbdivad
ssalbdivadβ€’3mo ago
I mean tsx is pretty great as far as an external solution goes But yeah very cool to be able to run it natively It's so weird in the debugger it literally just displays the file with all the type annotations replaced with whitespace lol
TizzySaurus
TizzySaurusβ€’3mo ago
Thanks; I really should pay more attention to terminal output πŸ˜… Looks like it worked
PIat
PIatOPβ€’3mo ago
Why is Theo respected?
TizzySaurus
TizzySaurusβ€’3mo ago
(Sorry @PIat, I totally took over this channel πŸ˜…)
ssalbdivad
ssalbdivadβ€’3mo ago
From what I've seen he has pretty nuanced takes on a lot of topics, I like him. He even did a video on attest
PIat
PIatOPβ€’3mo ago
I'm sorry you guys have to go through all this because of me 😬
ssalbdivad
ssalbdivadβ€’3mo ago
I don't think that's what happening I think was inevitable based on Tizzy's env clusterfuck
PIat
PIatOPβ€’3mo ago
So he's purely a YouTuber?
ssalbdivad
ssalbdivadβ€’3mo ago
Not exactly But I mean yeah he's generally a content creator, but I think if done well it serves a really valuable role in the ecosystem
TizzySaurus
TizzySaurusβ€’3mo ago
Yeah, it would've happened anyway when I continued working on https://github.com/arktypeio/arktype/issues/729 πŸ˜…
GitHub
Support JSON-schema as an input format Β· Issue #729 Β· arktypeio/ark...
This would be a large feature allowing JSON schemas as an alternative definition format, that would be statically validated and inferred, similarly to the primary TS-based syntax. Some additional i...
ssalbdivad
ssalbdivadβ€’3mo ago
Yeah will be interesting to revisit that now that I have some of the infra setup for the output types
PIat
PIatOPβ€’3mo ago
I see, thank you
TizzySaurus
TizzySaurusβ€’3mo ago
I'm just waiting on https://github.com/arktypeio/arktype/issues/1033 (friendly nudge πŸ˜‰)
GitHub
Incorrect discriminated union processing Β· Issue #1033 Β· arktypeio/...
Report a bug πŸ”Ž Search Terms boolean unioned with a recursive discriminated union, recursion, recursive discriminated union 🧩 Context ArkType version: 2.0-dev.26 TypeScript version (5.1+): 5.5 Other...
ssalbdivad
ssalbdivadβ€’3mo ago
Just find a workaround haa If you want to work on it There are other ways
TizzySaurus
TizzySaurusβ€’3mo ago
Yeah, my time is better spent elsewhere atm anyway tbf As in, I'm too busy to work on it atm anyway
ssalbdivad
ssalbdivadβ€’3mo ago
More like worse spent :kappa:
TizzySaurus
TizzySaurusβ€’3mo ago
@PIat To finally answer your question, this seems to work:
import { type } from "arktype"

const formType = type({
intent: "'profile'",
name: "string>0",
surname: "string>4",
agree: "boolean"
}).or({
intent: "'private'",
password: "string>0"
})

const keepAsString = ["intent", "name", "surname", "password"]

const parseNonStringFieldsToJson = type("object", "=>", obj =>
Object.fromEntries(
Object.entries(obj).map(([k, v]) => [
k,
keepAsString.includes(k) ? v : JSON.parse(v)
])
)
)

const parseFormData = parseNonStringFieldsToJson.pipe(formType)
const input = {
intent: "profile",
name: "John",
surname: "Doe",
agree: "true"
}

const result = parseFormData.assert(input)
console.log(result)
import { type } from "arktype"

const formType = type({
intent: "'profile'",
name: "string>0",
surname: "string>4",
agree: "boolean"
}).or({
intent: "'private'",
password: "string>0"
})

const keepAsString = ["intent", "name", "surname", "password"]

const parseNonStringFieldsToJson = type("object", "=>", obj =>
Object.fromEntries(
Object.entries(obj).map(([k, v]) => [
k,
keepAsString.includes(k) ? v : JSON.parse(v)
])
)
)

const parseFormData = parseNonStringFieldsToJson.pipe(formType)
const input = {
intent: "profile",
name: "John",
surname: "Doe",
agree: "true"
}

const result = parseFormData.assert(input)
console.log(result)
You just need a way of determining what keepAsString should be, which from the sounds of it you know so isn't an issue
PIat
PIatOPβ€’3mo ago
Would it somehow be possible to extract keepAsString from the formType? I've been looking into that for a while now
TizzySaurus
TizzySaurusβ€’3mo ago
Sort of. You could look at the internal schema structure
TizzySaurus
TizzySaurusβ€’3mo ago
See where the domain is "string", or it's a unit of type string
No description
TizzySaurus
TizzySaurusβ€’3mo ago
(That json being formType.json) @PIat
PIat
PIatOPβ€’3mo ago
Thank you! This is what I ended up with:
function checkDomain(
type: Type,
keyToFind: string,
domainToCheck = 'string',
): boolean {
const obj = type.json
// Normalize into an array
const objArray = Array.isArray(obj) ? obj : [obj]

for (const schema of objArray) {
for (const field of schema.required) {
if (field.key === keyToFind) {
// Check if value is an array or an object
if (Array.isArray(field.value)) {
// Loop through each value in the array
for (const value of field.value) {
if (value.domain === domainToCheck) {
return true
}
}
} else if (field.value.domain === domainToCheck) {
return true
}
}
}
}
return false
}
function checkDomain(
type: Type,
keyToFind: string,
domainToCheck = 'string',
): boolean {
const obj = type.json
// Normalize into an array
const objArray = Array.isArray(obj) ? obj : [obj]

for (const schema of objArray) {
for (const field of schema.required) {
if (field.key === keyToFind) {
// Check if value is an array or an object
if (Array.isArray(field.value)) {
// Loop through each value in the array
for (const value of field.value) {
if (value.domain === domainToCheck) {
return true
}
}
} else if (field.value.domain === domainToCheck) {
return true
}
}
}
}
return false
}
const data = checkDomain(type, key)
? value
: tryParseJSON(value.toString())
const data = checkDomain(type, key)
? value
: tryParseJSON(value.toString())
Will see what kinds of issues this will yield in the future 😬
ssalbdivad
ssalbdivadβ€’3mo ago
This is so sketchy haha I wouldn't rely on this. You can use node.internal to get access to a bunch of APIs that are useful for this kind of manipulation including .distribute which maps over the branches of a type
PIat
PIatOPβ€’3mo ago
Yeah I tried to understand those methods but failed miserably
ssalbdivad
ssalbdivadβ€’3mo ago
A lot of them just mirror the TS APIs
PIat
PIatOPβ€’3mo ago
Oh, I didn't think of it that way
ssalbdivad
ssalbdivadβ€’3mo ago
keyof pick omit required and partial all of which are available on object types are just like TS And then extract and exclude are on every type, which is very useful for filtering
PIat
PIatOPβ€’3mo ago
What is BaseRoot?
No description
ssalbdivad
ssalbdivadβ€’3mo ago
BaseRoot is what type is internally It's kind of like type without baked-in inference and with a bunch more powerful transformation methods, compilation etc.
PIat
PIatOPβ€’3mo ago
I see! But why is it returned from keyof?
ssalbdivad
ssalbdivadβ€’3mo ago
Because that's what type.internal is Once you're in internal, you're not going back to type I'm adding .distribute to Type in the next release though so you won't have to lose inference to use it
PIat
PIatOPβ€’3mo ago
Oh wow Does using node.children make sense with distribute?
ssalbdivad
ssalbdivadβ€’3mo ago
Not really, distribute is for iterating over branches ofa union If it's not a union, the root node itself would be the branch
PIat
PIatOPβ€’3mo ago
And each branch is a BaseType
ssalbdivad
ssalbdivadβ€’3mo ago
Yeah each branch is more specifically any BaseRoot that is not a union
PIat
PIatOPβ€’3mo ago
This is so cool
ssalbdivad
ssalbdivadβ€’3mo ago
If you want to see a summary of each node you're looking at .expression will be useful
PIat
PIatOPβ€’3mo ago
I'm actually trying to understand typescript for the first type, and it feels useful now, since I'm using the same knowledge in the runtime as well
ssalbdivad
ssalbdivadβ€’3mo ago
Yeah that is a really unique new capability now that ArkType exists
PIat
PIatOPβ€’3mo ago
It's insane A real bridge between static and runtime
ssalbdivad
ssalbdivadβ€’3mo ago
That's what the whole branding on the homepage is about! An ark that saves your types from the flood of compilation haha
PIat
PIatOPβ€’3mo ago
I do see it give such a value, but have trouble finding out how to check if password is string. I don't see the way to check the keys
{ intent: "private", password: string >= 1 }
{ intent: "private", password: string >= 1 }
I can't wait till I understand it fully ☺️
ssalbdivad
ssalbdivadβ€’3mo ago
.get is like index access
PIat
PIatOPβ€’3mo ago
It'll be wroom wroom Even now it's the backbone of my whole job messaging system
let isMatch = false
formType.internal.distribute((node) => {
try {
const child = node.get('name')

if (child.domain.domain === 'string') {
isMatch = true
}
} catch {}
})
let isMatch = false
formType.internal.distribute((node) => {
try {
const child = node.get('name')

if (child.domain.domain === 'string') {
isMatch = true
}
} catch {}
})
ssalbdivad
ssalbdivadβ€’3mo ago
Well you should definitely never need to use try catch haha
PIat
PIatOPβ€’3mo ago
I don't know how much worse this is than the json solution πŸ˜† It's to prevent
ParseError: Key "name" does not exist on { intent: "private", password: string >= 1 }
ParseError: Key "name" does not exist on { intent: "private", password: string >= 1 }
ssalbdivad
ssalbdivadβ€’3mo ago
What about this:
declare const t: Type<{ name: string } | {}>



const isMatch = t.extract({ name: "string" }).get("name")
declare const t: Type<{ name: string } | {}>



const isMatch = t.extract({ name: "string" }).get("name")
PIat
PIatOPβ€’3mo ago
Rigggghhhhtt 🫠
function isKeyDomain(type: Type, key: string, domain = 'string') {
return type.extract({ [key]: domain }).get(key).expression !== 'never'
}
console.log(isKeyDomain(formType, 'name'))
function isKeyDomain(type: Type, key: string, domain = 'string') {
return type.extract({ [key]: domain }).get(key).expression !== 'never'
}
console.log(isKeyDomain(formType, 'name'))
It makes sense
ssalbdivad
ssalbdivadβ€’3mo ago
Careful though a lot of expressions will throw by default if they create an unsatisfiable value so you don't accidentally create validators that will never pass
PIat
PIatOPβ€’3mo ago
Thank you! I still have a lot to learn
ssalbdivad
ssalbdivadβ€’3mo ago
Well I appreciate you delving into the type system aspect of things. I'm still really the only one with much context on that stuff! Luckily working on docs now ✏️
PIat
PIatOPβ€’3mo ago
Already? 😡 I'm living in the best timeline 😁
ssalbdivad
ssalbdivadβ€’3mo ago
Once they're published that will be true haha
Want results from more Discord servers?
Add your server