codecs
with morphs, I am thinking of arktype types as sort of input/output functions
for
const t = type(...)
t.allows(input)
tells if you if input is the right shape
t(input)
takes input and turns it into output (or errors)
is there a way to extract a def/type of the input type (pre-morph) vs output type (post-morph)?171 Replies
I saw
t.in
and t.out
but I don't think they give me what I want, or at least their types don't
if a type has no morphs, then in/out would be ~equivalentI am prob misunderstanding arktype internals but my assumption was that the 6th assertion would be false
might better describe the behavior
If you're just trying to test the validation, just use
params({ limit: "1" })
? Or is this more for the sake of discovery?
At I guess I would say perhaps you aren't meant to use allows/travers on in/out. But i'm not at all certain
there are tIn
tOut
which have the in and out types, and tValidatedOut
which is { limit: unknown }
for your code.
in
is related to tIn
, but weirdly out
is related to tValidatedOut
I actually made this improvement recently to help show the discrepancy bewteen the type information that is available at runtime and what we can statically determine.
@frolic I'm guessing in your case,
.out
is typed as unknown
. That's because if you just apply an arbitrary transformation, we can infer the return statically, but at runtime there's no way to know what you expect your function to return, so .out
is just unknown
.
If you want a validated output type, you can .pipe
to another type or .to(def)
, then it will infer the validated output and make it available at runtimeI am rethinking my API around this because I don't think I can get what I want with just a single type with morphs
Well yeah if you need the validated output you'd need a type for it just like the input
oh I see what you're saying now
would it make sense to allow describing the output type of morphs?
wondering if its a small enough API change that enables a runtime validator for output
You're misunderstanding, this is already possible you just have to provide one
Pipe to a type
You can pipe from an input, to a morph, then to a type
Then it will be available and inferred
at a property level?
Types are composable, so yeah. You could do it with nested tuple expressions although like I said this is the sort of territory where chaining is usually more natural
like
string.integer
for example, string
is the input type and it morphs to an integer, but does it also pipe to number
?The builtin keyword doesn't right now because I don't want to perform redundant validation and it's relatively rare people need to extract
in/out
if they already have the inferred type.
That said I do want to add a feature that allows you to essentially "cast" the output type of a morph so you can introspect it without adding validation stepsokay another question/approach: can I validate that a type def has no morphs?
as in no chance of anything inside resolving to
unknown
Yeah there is a
.internal.includesMorphs()
what about at a type level?
Yeah there is an
includesMorphs
at a type-level alsoperfect, will play with this
You'd have to do an internal import from
arktype/internal/ast.ts
my use case is defining the shape of JSON-serializable output of a function that I want to use both to enforce the output of the function follows the type def and that I can parse JSON from the server with that same type
so I can prob just avoid morphs here
although it might be nice to have some way to encode/decode types that aren't JSON-serializable like dates or bigints or whatever, but that would require a separate input and output type each with morphs (if I understand what you're saying above)
Is it possible then to reuse the input def from the other side instead?
Another thing to consider is when we were first talking about the API, I mentioned
input, output, query
, were kind of like type(input).pipe(query).to(output)
If it was defined that way, you would just have all of those things alreadyso ignore the input/query part of the flow for now
I have a server that returns JSON
I would like to be able to define a type that expresses what the JSON shape is so I can 1) ensure I am returning the right data and 2) client can fetch the data and validate it with the type (i.e. can trust the server response)
this works fine with a single type if there are no morphs, but once you add morphs (say for converting Dates), you'd need the type to be "bidirectional"?
I think my mental model for this has some gaps so I need to do a bit more sketching
You just need a type that represents what the client should be parsing right? It seems like you could define that in a common location and reuse it as the output validator for the server function if you want
The JSON serialization issue is different though, there's not a builtin solution for that yet
yep, this works if there are no morphs
but maybe it also works with morphs? because the endpoint can just enforce that the data matches the type input?
and then the client does validating+parsing that morphs it?
Right yeah, even if there are morphs, extracting
.in
is always much more reliable because generally if you had an input you have a type for itgot it
this sounds promising
will tinker with it tonight after work :)
thanks
actually, the other end might be the thing that needs to be bidirectional
I define a type that represents query params where they're often morphed because in the URL or form data they are strings
the function that defines the endpoint takes the
infer
as input and returns JSON
inferIn
would be the "raw" (string) params
if I want to go the other way and fetch this JSON from the server but allow infer
as args to that function (to e.g. create filters via URL params), I need a way to get it back to inferIn
type (serialize?)
(just thinking aloud, will play more with this tonight and have a better understanding of needs)
okay I am back with more info!
if I have a type with morphs, it's effectively a one-way operation with current arktype
so if I want it to be bidirectional, I would need to have a type that represents one way (input -> output) and another type that represents the other way (output -> input)
if I have deep/complex type, I am wondering how I might define each direction's type without having to repeat everythingI am interested in some-builtin support for graphed relationships at some point. It's really more about figuring out the right API than anything:
https://github.com/arktypeio/arktype/issues/909
GitHub
Support two-way serialization/deserialization · Issue #909 · arktyp...
🤷 Motivation We have types like JS Date or an internal LocalDate (only calendar dates, no times) that we want to encode/decode to JSON (or a string, like in a query string). Libraries like Zod/Arkt...
oh cool, same use case hahah
For now though I don't think it should be hard to avoid repeating any of your validation types?
not sure I follow
What is actually being duplicated when you write the transform in both directions?
this is a simple example, that
from
might be a property deep in some object like an array of users or whatever
I guess what I am asking for is the hard part you haven't solved for yet: what should the API be for a bidirectional typeWell I'm not 100% on that, but to me the more important genreal question is what is the API for establishing relationships between any group of types
And the bidirectional case would just be sugar over that
I think I understand
I guess what you're sort of asking for is to be able to infer a bidirectional morph from a top-level type that includes some combination of unmorphed types and bidirectional morphs at paths. Would definitely be a nice API convenience, would be interested to see what you come up with
curious about an operator that specifies types on both sides and conversions to/from
I actually did just add a
declaredIn
and declaredOut
that helps solve the problem of being able to introspect returns you don't actually want to validate.
Yeah maybe something like this for the bidirectional caseI was considering the operator being something like
"<>"
but then its unclear the "default direction" of the morphThe real trick is you'd need a top-level API for basically validating that all types in a scope or something implement all morphs this way so you could freely reference between them
That's why in general thinking about it as a graph is better because none of the vertices have special properties it just connects types with morphs
Two of those vertices might happen to be named
in
and out
, but there could be others connected to one or more of them
If you built the underlying representation for bidirectionality, you'd be screwed if there were ever 3 variantsanother route
define a morph from one type to another and requires the existence of one or both
*.to.*
directions?What if you define a scope including only unmorphed types, then you pass a second parameter that connects that implements the directed transformations you want to support
got a quick pseudocode snippet to demonstrate?
that could work too
Actually this would be better if TS can infer it:
what would the usage look like?
Maybe then you automatically get a keyword like
string.to.Date
I think what I want to be able to extract is the automatic reverse of the morphed data
It could autocomplete whatever transformations you have implemented
That's impossible unless they've been manually implemened in both directions either internally or externally
right, that'd be requirement to have a "complete" inverse
otherwise you get gaps with
unknown
or whateverWell with relationships like this defined it would be possible. I think I mentioned in the issue that would be a cool optimization that if you define how certain transforms work, it could basically infer how to transform larger structures
That functionality would be super powerful
.in
or .out
can never have morphs in them
I guess it would be like encode/decode or somethingyeah I didn't mean to overload your existing names
just pseudocode to demonstrate
It is an important part of how
`.in
and .out
work though that they recursively extract one side of the transform so they can always guarantee it's just a validator at that point
Definitely one of the cooler parts of the type systemI guess if you had a complete enough graph for your a type, you could technically recusively extract/resolve a type with morphs into its pure input (before morph) and output (after morph)?
Yeah that already works as long as you define output validators for your morphs
wondering if I can get what I am looking for with generics
Yeah generics are super powerful as soon as I publish the next release there will be an API I think will be very helpful that doesn't involve HKT hshs
I don't know if this would actually work, but here's what I had in mind:
There's no native syntax for index access yet unfortunately
But the scoped generic syntax is better
is there a way to do it without native index access syntax?
With an HKT yeah
You can see what that looks like here: https://github.com/arktypeio/arktype/blob/8ea6a6ad8cdd5a50b7fc882ad5c7d708dd029621/ark/type/__tests__/generic.test.ts#L502
I should wrap up this release though so you can use the new standalone generics
GitHub
arktype/ark/type/tests/generic.test.ts at 8ea6a6ad8cdd5a50b7fc8...
TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype
is there a "type" or "def" type in arktype? haha
trying to constrain a type def by another type def
Unfortunately generics don't support that yet https://github.com/arktypeio/arktype/issues/1053
GitHub
Allow generic constraints referencing other generic parameters · Is...
This would allow constraints like: type("<t, k extends keyof t>", ["t", "k"]); As is, this would fail to parse as t is not considered in scope when parsing a con...
You'd have to write your own generic logic for that
love that you already have issues for everything I come up with
Haha yeah always thinking of things i want to build 😬
easy enough, already doing this for enforcing json-serializable shapes
morphs have some recursive nature to them right?
Not inherently
Depends what you mean I guess
now that I type this out, I don't know if/how this would actually work
Yeah these APIs look tricky I would look at what I have for generic inference w/ Hkts and standard definitions +how they work in scopes, but this would be complex for sure
maybe this would give me what I want
getting closer 👀
Did you see how in the scope example I sent generic params are on the scope key itself?
The syntax is definitely nicer than a tuple
oh yeah, didn't update
there
the tuple syntax is available now? and the generic syntax is coming soon?
No the only way you can create generics in a scope is the better syntax I mentioned
oh I see, I was interpreting unit tests as if I could use it as a tuple too
and that syntax isn't released yet, right?
No it is it's always been that way
oh great
I think this'll hold me over for a bit!
Excited to see what you come up with!
oh wait hrmm, maybe not hehe
that would resolve
from
to a type, but not call it with a value I pass inIn that case, best to just define it as a standalone generic
any way to define this inline to get the right output type of the morph?
I assumed it would have inferred from the morph function
(it not working might be related to the arg type also not working)
Looking at that part of the inference doesn't necessarily tell you what's inferred as output, although it could be a problem.
The inferred output of the entire type is also unknown?
Generally you shouldn't have to ever explicitly type morph input or output, but I'd definitely expect in the cases TS struggles to infer them otherwise (which do exist), that explicit typing should fix it
damn, thought I had something with this
but
.infer
shows from: string | Date
I assume because the from
type is resolved at definition time, which explains the union, but kinda hoped it would have been narrowed by the genericI think the problem is just that you have two generics with the same name I should add a validation error for that
haha that makes more sense
I have a working prototype using a combo of a few strategies, just seeing how close I can get to doing this purely in arktype
Hkts would make it easy but not quite as clean. Not terrible though
can you pass an object to a generic?
actually iirc some tests for this so I think the answer is yes, just gotta find the right syntax for doing it in a scope
that's the great thing about scopes is you can always serialized anything by making it a keyword
If you just want it as an intermediate without exposing it you can make it private
#privateAlias
I wanted to inline it rather than new definitions per generic value
Generally if you're defining objects, it's good to give them a name anyways IMO.
Definitely not for primitives e.g.
'encode' | 'decode'
or whatever but an object literal with keys? It's somewhat rare you wouldn't want to reuse that in some way, or at least have access to it
An anti-pattern I see a lot is people define complex structures then extract back out the parts they need which is expensive both in terms of runtime and type-level perfIt was more to constrain the type entering the generic
Even so, having a
BaseConstraint
type or whatever seems kinda good? Regardless there is no way to do that inline in a scope without defining it as an aliasis there an example of using a scope generic like this but uses
Hkt
class?Here's the builtin TS generics in a scope:
I guess you'd want to call
generic
instead of genericNode
but it's basically the same thing
It's valid as a root scope defah I guess what I was wondering was something like
Yeah that doesn't exist would be a cool API though
TBH it's already such a niche use case
for sure
if I wanna dynamically make a scope but I wanna attach some preexisting stuff, seems like
validateScope<scope & otherStuff>
isn't quite what I am looking for
ends up being that the scope
input is wanting definitions for the keys on otherStuff
You're just attaching the props internally? Then it shouldn't affect
validateScope
, just how it's inferred
ignore the return value for now
If you look at how scopes are validated, you'll notice I basically never use constraints because they break inference quite often
Especially not
k: string
so
codecs
is finevalidateCodecs
should be able to handle providing the type-level errors and autocompleteas is
validateCodecs
for autocomplete etc
what I want is to extend scope
with codecs
when I run it through validateScope
so I don't get "codec.date is unresolvable"Well, just bear in mind because it works with some definitions doesn't mean it will work with all. Would recommend avoiding it if possible
but if I do
validateScope<scope & codecs>
Hmm, I used to directly support parent/child scopes but I removed those in favor of the export/import syntax which I thought was clearer
thinking maybe what I want is one scope that has a constraint that enforces
codecs
object with a certain shapeThere's also a way to change what resolves globally, so that would be another option
oh I missed import
To do it directly you'd basically have to duplicate
validateScope
and add a type parameter
for the parent scope which is how it was in alphaGitHub
arktype/ark/type/tests/imports.test.ts at 8876a4bb493d93e1ce4ca...
TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype
This is kind ofthe tldr
scope creep hahaha perfect
exports + spread worked well, thank you
Ahh coool I thought you werew trying to create a scope users could use to create scopes
sort of am
just prototyping some stuff, then lifting it out into functions
In that case you would have to do one of the other things I mentioned
so this works nicely to constrain the codecs shape, but when I use
defineCodecs
, I lose some propagation of types or something?(mentioned this above but at least this is more contained now to demonstrate it)
Inferreing function inputs is really hard when you start to customize this stuff best thing I can recommend is looking at how it's implemented in AT
mmk
in
inferTypeRoot<t, $>
what should $
be? a Scope
? inferScope
? a scope def
?
looks like inferScope
for $
in inferTypeRoot
but still unclear if it should be inferScope
or scope def
for $
in validateTypeRoot
Use inferAmbient if you don't need the scope
Hm
Check scope.type impl
I do need the scope, that's why I'm asking about it
have dug around the source and haven't found any hints as to what
$
should be in validate
typesscope.t
can you point to a line in the source to describe what you're referring to?
Where do you pick the scope from?
GitHub
arktype/ark/type/scope.ts at 8876a4bb493d93e1ce4ca72f1bca0a691f1f87...
TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype
GitHub
arktype/ark/type/scope.ts at 8876a4bb493d93e1ce4ca72f1bca0a691f1f87...
TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype
my function takes in a scope def, I use it in
validateTypeRoot
and inferTypeRoot
, and just want to know what kind of scope to use in validateTypeRoot
GitHub
arktype/ark/type/type.ts at 8876a4bb493d93e1ce4ca72f1bca0a691f1f875...
TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype
I appreciate you trying to help but this isn't answering my question
Can you provide the function definition
It looks like you gonna need 4 generics or smth
not exactly this but just demonstrating:
wondering which of these for
validateTypeRoot
Why don't you use a regular compiled scope here?
As arg
it can accept both
which makes it easier to use the function externally for simple cases without having to import arktype directly
Hmm
What's your use case?
If you make it chainable it will be literal AT
so you don't even need your own fns
this thread has all the context if you're curious
well, maybe not all, this was a carry-over from DMs
GitHub
typed-http/src/arktype/action.test.ts at main · holic/typed-http
Contribute to holic/typed-http development by creating an account on GitHub.
Now I see
that's what I have currently, just wanted to confirm this is correct but couldn't find any hints in arktype source to give me confience
circling back, things are looking good with the lib: https://github.com/holic/typed-http/blob/main/src/arktype/http.test.ts
should have some full end to end tests soon
GitHub
typed-http/src/arktype/http.test.ts at main · holic/typed-http
Contribute to holic/typed-http development by creating an account on GitHub.
still curious about this
also could greatly simplify this lib if there was some notion of bidirectional morphs native to arktype, wouldn't need the whole codec thing
Or if you want to accept an already instantiated scope you'd do
scope: Scope<$>, def: validateTypeRoot<def, $>
Generally there's not a lot of APIs like this because you'd usually either create the type as an alias in your scope or use the .type
instance attached to the scope to create one latergreat, that's basically what I am doing
But I can see how you might need some custom APIs if you're wanting to define relationships etc.
Definitely something I want to tackle with that graph angle in mind, but I think there's enough possibilities in terms of the API that I likely won't work on it until 2.0 has been stable for a while
@frolic Btw you should check out the generic API in
2.0.0-rc.2
, feels really clean and might help with some of the stuff you were looking at:
I tried fiddling with generics for quite a while and couldn't figure out a way to help simplify things
without index access
curious if you had a specific use case in mind
I was just thinking about some of the original cases where you wanted to accept a constrained type definition as a function input
You mean syntactically? There is
.get
. The fact that it already exists internally means supporting index access would really just be adding it to the parseryeah there's a transformation step I'm doing to turn individual codecs with encode/decode functions into an object for all encoders and all decoders
so would let me do that transformation with pure arktype generics I think
unclear if strong types carry through though
Definitely tougher when you're wanting to define relationships like that. HKTs can theoretically do anything though
I couldn't figure those out 🙈
iirc I kept wanting to access outer scope, not just the immediate value or whatever
but maybe thinking about it wrong
Can the parser have subscopes?
Dunno what you mean
Can you
..
Whatever that is
It has to be a module to reference it like that
So yeah if you add
.export
to the end of scope
or use type.module
@frolic consider trying making a type.module thingy then
Then you may define reverse operation on every thing I guess
Or maybe make the root just e.g.
Map
and add [,][] => Map
and Map => [,][]
on topanything new I should be aware of for the codec use case?
currently bumping to latest arktype RC
I don't think so it's mostly been small bug fixes and cleanup. Actually reintegrating
this
nownot sure what
this
you're referring toIt's an alias you can reference in types to recurse directly without a scope
(kind of funny JS joke too if you think about it)
Yeah whenever I refer to it it sounds ridiculous almost too easy haha
not a bad diff :) feels more readable after those
type.*
changes: https://github.com/holic/typed-http/commit/44a5c9ecebece61b72e238e53d26bf99707a2c8aimport { type type }
is funny to meYeah I guess that can happen now that I'm using it as a namespace like that h aha
I think organizing those generics as well as the builtin keywords by namespace should help with discoverability for authors wanting to integrate and users respectively