arktype•4mo ago

codecs

with morphs, I am thinking of arktype types as sort of input/output functions for const t = type(...) t.allows(input) tells if you if input is the right shape t(input) takes input and turns it into output (or errors) is there a way to extract a def/type of the input type (pre-morph) vs output type (post-morph)?

171 Replies

frolicOP•4mo ago

I saw t.in and t.out but I don't think they give me what I want, or at least their types don't if a type has no morphs, then in/out would be ~equivalent

frolicOP•4mo ago

I am prob misunderstanding arktype internals but my assumption was that the 6th assertion would be false

frolicOP•4mo ago

might better describe the behavior

Unknown User•4mo ago

Message Not Public

ssalbdivad•4mo ago

I actually made this improvement recently to help show the discrepancy bewteen the type information that is available at runtime and what we can statically determine. @frolic I'm guessing in your case, .out is typed as unknown. That's because if you just apply an arbitrary transformation, we can infer the return statically, but at runtime there's no way to know what you expect your function to return, so .out is just unknown. If you want a validated output type, you can .pipe to another type or .to(def), then it will infer the validated output and make it available at runtime

frolicOP•4mo ago

I am rethinking my API around this because I don't think I can get what I want with just a single type with morphs

ssalbdivad•4mo ago

Well yeah if you need the validated output you'd need a type for it just like the input

frolicOP•4mo ago

oh I see what you're saying now would it make sense to allow describing the output type of morphs? wondering if its a small enough API change that enables a runtime validator for output

ssalbdivad•4mo ago

You're misunderstanding, this is already possible you just have to provide one Pipe to a type You can pipe from an input, to a morph, then to a type Then it will be available and inferred

frolicOP•4mo ago

at a property level?

ssalbdivad•4mo ago

Types are composable, so yeah. You could do it with nested tuple expressions although like I said this is the sort of territory where chaining is usually more natural

frolicOP•4mo ago

like string.integer for example, string is the input type and it morphs to an integer, but does it also pipe to number?

ssalbdivad•4mo ago

The builtin keyword doesn't right now because I don't want to perform redundant validation and it's relatively rare people need to extract in/out if they already have the inferred type. That said I do want to add a feature that allows you to essentially "cast" the output type of a morph so you can introspect it without adding validation steps

frolicOP•4mo ago

okay another question/approach: can I validate that a type def has no morphs? as in no chance of anything inside resolving to unknown

ssalbdivad•4mo ago

Yeah there is a .internal.includesMorphs()

frolicOP•4mo ago

what about at a type level?

ssalbdivad•4mo ago

Yeah there is an includesMorphs at a type-level also

frolicOP•4mo ago

perfect, will play with this

ssalbdivad•4mo ago

You'd have to do an internal import from arktype/internal/ast.ts

frolicOP•4mo ago

my use case is defining the shape of JSON-serializable output of a function that I want to use both to enforce the output of the function follows the type def and that I can parse JSON from the server with that same type so I can prob just avoid morphs here although it might be nice to have some way to encode/decode types that aren't JSON-serializable like dates or bigints or whatever, but that would require a separate input and output type each with morphs (if I understand what you're saying above)

ssalbdivad•4mo ago

Is it possible then to reuse the input def from the other side instead? Another thing to consider is when we were first talking about the API, I mentioned input, output, query, were kind of like type(input).pipe(query).to(output) If it was defined that way, you would just have all of those things already

frolicOP•4mo ago

so ignore the input/query part of the flow for now I have a server that returns JSON I would like to be able to define a type that expresses what the JSON shape is so I can 1) ensure I am returning the right data and 2) client can fetch the data and validate it with the type (i.e. can trust the server response) this works fine with a single type if there are no morphs, but once you add morphs (say for converting Dates), you'd need the type to be "bidirectional"? I think my mental model for this has some gaps so I need to do a bit more sketching

ssalbdivad•4mo ago

You just need a type that represents what the client should be parsing right? It seems like you could define that in a common location and reuse it as the output validator for the server function if you want The JSON serialization issue is different though, there's not a builtin solution for that yet

frolicOP•4mo ago

yep, this works if there are no morphs but maybe it also works with morphs? because the endpoint can just enforce that the data matches the type input? and then the client does validating+parsing that morphs it?

ssalbdivad•4mo ago

Right yeah, even if there are morphs, extracting .in is always much more reliable because generally if you had an input you have a type for it

frolicOP•4mo ago

got it this sounds promising will tinker with it tonight after work :) thanks actually, the other end might be the thing that needs to be bidirectional I define a type that represents query params where they're often morphed because in the URL or form data they are strings the function that defines the endpoint takes the infer as input and returns JSON inferIn would be the "raw" (string) params if I want to go the other way and fetch this JSON from the server but allow infer as args to that function (to e.g. create filters via URL params), I need a way to get it back to inferIn type (serialize?) (just thinking aloud, will play more with this tonight and have a better understanding of needs) okay I am back with more info! if I have a type with morphs, it's effectively a one-way operation with current arktype so if I want it to be bidirectional, I would need to have a type that represents one way (input -> output) and another type that represents the other way (output -> input) if I have deep/complex type, I am wondering how I might define each direction's type without having to repeat everything

ssalbdivad•4mo ago

I am interested in some-builtin support for graphed relationships at some point. It's really more about figuring out the right API than anything: https://github.com/arktypeio/arktype/issues/909

GitHub

Support two-way serialization/deserialization · Issue #909 · arktyp...

🤷 Motivation We have types like JS Date or an internal LocalDate (only calendar dates, no times) that we want to encode/decode to JSON (or a string, like in a query string). Libraries like Zod/Arkt...

frolicOP•4mo ago

const input = type({
  from: "parse.date"
});

const output = type({
  from: ["Date", "=>", (d) => d.toISOString()]
});

const input = type({
  from: "parse.date"
});

const output = type({
  from: ["Date", "=>", (d) => d.toISOString()]
});

oh cool, same use case hahah

ssalbdivad•4mo ago

For now though I don't think it should be hard to avoid repeating any of your validation types?

frolicOP•4mo ago

not sure I follow

ssalbdivad•4mo ago

What is actually being duplicated when you write the transform in both directions?

frolicOP•4mo ago

this is a simple example, that from might be a property deep in some object like an array of users or whatever I guess what I am asking for is the hard part you haven't solved for yet: what should the API be for a bidirectional type

ssalbdivad•4mo ago

Well I'm not 100% on that, but to me the more important genreal question is what is the API for establishing relationships between any group of types And the bidirectional case would just be sugar over that

frolicOP•4mo ago

I think I understand

ssalbdivad•4mo ago

I guess what you're sort of asking for is to be able to infer a bidirectional morph from a top-level type that includes some combination of unmorphed types and bidirectional morphs at paths. Would definitely be a nice API convenience, would be interested to see what you come up with

frolicOP•4mo ago

curious about an operator that specifies types on both sides and conversions to/from

type({
  "from": ["string", "=>", "Date", (s) => Date.parse(s), (d) => d.toISOString()]
});

type({
  "from": ["string", "=>", "Date", (s) => Date.parse(s), (d) => d.toISOString()]
});

ssalbdivad•4mo ago

I actually did just add a declaredIn and declaredOut that helps solve the problem of being able to introspect returns you don't actually want to validate. Yeah maybe something like this for the bidirectional case

frolicOP•4mo ago

I was considering the operator being something like "<>" but then its unclear the "default direction" of the morph

ssalbdivad•4mo ago

The real trick is you'd need a top-level API for basically validating that all types in a scope or something implement all morphs this way so you could freely reference between them That's why in general thinking about it as a graph is better because none of the vertices have special properties it just connects types with morphs Two of those vertices might happen to be named in and out, but there could be others connected to one or more of them If you built the underlying representation for bidirectionality, you'd be screwed if there were ever 3 variants

frolicOP•4mo ago

another route

scope({
  "string.to.Date": ["string", "=>", (s) => Date.parse(s)]
  "Date.to.string": ["Date", "=>", (d) => d.toISOString()],

  "query": {
    "from": ["string", "=>", "Date"]
  },
});

scope({
  "string.to.Date": ["string", "=>", (s) => Date.parse(s)]
  "Date.to.string": ["Date", "=>", (d) => d.toISOString()],

  "query": {
    "from": ["string", "=>", "Date"]
  },
});

define a morph from one type to another and requires the existence of one or both *.to.* directions?

ssalbdivad•4mo ago

What if you define a scope including only unmorphed types, then you pass a second parameter that connects that implements the directed transformations you want to support

frolicOP•4mo ago

got a quick pseudocode snippet to demonstrate?

ssalbdivad•4mo ago

scope({
    Date: ["instanceof", Date],
    string: "string"
}).relate({
    Date: {
        string: d => d.toISOString()
    },
    string: {
        Date: s => new Date(s)
    }
})

scope({
    Date: ["instanceof", Date],
    string: "string"
}).relate({
    Date: {
        string: d => d.toISOString()
    },
    string: {
        Date: s => new Date(s)
    }
})

frolicOP•4mo ago

that could work too

ssalbdivad•4mo ago

Actually this would be better if TS can infer it:

scope({
    Date: [
        ["instanceof", Date],
        "=>",
        {
            string: d => d.toISOString()
        }
    ],
    string: [
        "string",
        "=>",
        {
            Date: s => new Date(s)
        }
    ]
})

scope({
    Date: [
        ["instanceof", Date],
        "=>",
        {
            string: d => d.toISOString()
        }
    ],
    string: [
        "string",
        "=>",
        {
            Date: s => new Date(s)
        }
    ]
})

frolicOP•4mo ago

what would the usage look like?

ssalbdivad•4mo ago

Maybe then you automatically get a keyword like string.to.Date

frolicOP•4mo ago

I think what I want to be able to extract is the automatic reverse of the morphed data

ssalbdivad•4mo ago

It could autocomplete whatever transformations you have implemented That's impossible unless they've been manually implemened in both directions either internally or externally

frolicOP•4mo ago

right, that'd be requirement to have a "complete" inverse otherwise you get gaps with unknown or whatever

ssalbdivad•4mo ago

Well with relationships like this defined it would be possible. I think I mentioned in the issue that would be a cool optimization that if you define how certain transforms work, it could basically infer how to transform larger structures That functionality would be super powerful

frolicOP•4mo ago

const types = scope({
  string: ...
  Date: ...

  postsFilter: {
    from: "string.to.Date"
  }
});

types.postsFilter.in // { from: "string.to.Date" }
types.postsFilter.out // { from: "Date.to.string" }
// or if not found in graph
types.postsFilter.out // { from: unknown }

const types = scope({
  string: ...
  Date: ...

  postsFilter: {
    from: "string.to.Date"
  }
});

types.postsFilter.in // { from: "string.to.Date" }
types.postsFilter.out // { from: "Date.to.string" }
// or if not found in graph
types.postsFilter.out // { from: unknown }

ssalbdivad•4mo ago

.in or .out can never have morphs in them I guess it would be like encode/decode or something

frolicOP•4mo ago

yeah I didn't mean to overload your existing names just pseudocode to demonstrate

ssalbdivad•4mo ago

It is an important part of how `.in and .out work though that they recursively extract one side of the transform so they can always guarantee it's just a validator at that point Definitely one of the cooler parts of the type system

frolicOP•4mo ago

I guess if you had a complete enough graph for your a type, you could technically recusively extract/resolve a type with morphs into its pure input (before morph) and output (after morph)?

ssalbdivad•4mo ago

Yeah that already works as long as you define output validators for your morphs

frolicOP•4mo ago

wondering if I can get what I am looking for with generics

ssalbdivad•4mo ago

Yeah generics are super powerful as soon as I publish the next release there will be an API I think will be very helpful that doesn't involve HKT hshs

frolicOP•4mo ago

I don't know if this would actually work, but here's what I had in mind:

const types = scope({
  DateCodec: {
    encode: ["Date", "=>", (d) => d.toISOString()],
    decode: ["string", "=>", (s) => Date.parse(s)],
  },

  postsFilter: ["<op extends 'encode'|'decode'>", {
    "from": "DateCodec[op]",
  ]}
}).export();

types.postsFilter('encode')({ from: new Date() });
types.postsFilter('decode')({ from: "2024-08-22" });

const types = scope({
  DateCodec: {
    encode: ["Date", "=>", (d) => d.toISOString()],
    decode: ["string", "=>", (s) => Date.parse(s)],
  },

  postsFilter: ["<op extends 'encode'|'decode'>", {
    "from": "DateCodec[op]",
  ]}
}).export();

types.postsFilter('encode')({ from: new Date() });
types.postsFilter('decode')({ from: "2024-08-22" });

ssalbdivad•4mo ago

There's no native syntax for index access yet unfortunately But the scoped generic syntax is better

            const $ = scope({
                "entry<k extends key, v>": ["k", "v"],
                key: "string | symbol"
            })

            const $ = scope({
                "entry<k extends key, v>": ["k", "v"],
                key: "string | symbol"
            })

frolicOP•4mo ago

is there a way to do it without native index access syntax?

ssalbdivad•4mo ago

With an HKT yeah

ssalbdivad•4mo ago

You can see what that looks like here: https://github.com/arktypeio/arktype/blob/8ea6a6ad8cdd5a50b7fc882ad5c7d708dd029621/ark/type/__tests__/generic.test.ts#L502 I should wrap up this release though so you can use the new standalone generics

GitHub

arktype/ark/type/tests/generic.test.ts at 8ea6a6ad8cdd5a50b7fc8...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

frolicOP•4mo ago

is there a "type" or "def" type in arktype? haha

const codec({ encode: "def", decode: "def" });
scope({
  DateCodec: codec({
    encode: ["Date", "=>", (d) => d.toISOString()],
    decode: ["string", "=>", (s) => Date.parse(s)],
  }),
});

const codec({ encode: "def", decode: "def" });
scope({
  DateCodec: codec({
    encode: ["Date", "=>", (d) => d.toISOString()],
    decode: ["string", "=>", (s) => Date.parse(s)],
  }),
});

trying to constrain a type def by another type def

ssalbdivad•4mo ago

Unfortunately generics don't support that yet https://github.com/arktypeio/arktype/issues/1053

GitHub

Allow generic constraints referencing other generic parameters · Is...

This would allow constraints like: type("<t, k extends keyof t>", ["t", "k"]); As is, this would fail to parse as t is not considered in scope when parsing a con...

ssalbdivad•4mo ago

You'd have to write your own generic logic for that

frolicOP•4mo ago

love that you already have issues for everything I come up with

ssalbdivad•4mo ago

Haha yeah always thinking of things i want to build 😬

frolicOP•4mo ago

easy enough, already doing this for enforcing json-serializable shapes morphs have some recursive nature to them right?

ssalbdivad•4mo ago

Not inherently Depends what you mean I guess

frolicOP•4mo ago

now that I type this out, I don't know if/how this would actually work

const codecs = scope({
  Date: {
    encode: ["Date", "=>", (d) => d.toISOString()],
    decode: ["string", "=>", (s) => Date.parse(s)],
  },
}).export();

const types = scope({
  codec: "'encode' | 'decode'",
  postsFilter: ["<op extends codec>", {
    "from": ["op", "=>", (op) => codecs.Date[op]], // not sure how to get to the inner type here
  ]}
}).export();

types.postsFilter('encode')({ from: new Date() });
types.postsFilter('decode')({ from: "2024-08-22" });

const codecs = scope({
  Date: {
    encode: ["Date", "=>", (d) => d.toISOString()],
    decode: ["string", "=>", (s) => Date.parse(s)],
  },
}).export();

const types = scope({
  codec: "'encode' | 'decode'",
  postsFilter: ["<op extends codec>", {
    "from": ["op", "=>", (op) => codecs.Date[op]], // not sure how to get to the inner type here
  ]}
}).export();

types.postsFilter('encode')({ from: new Date() });
types.postsFilter('decode')({ from: "2024-08-22" });

ssalbdivad•4mo ago

Yeah these APIs look tricky I would look at what I have for generic inference w/ Hkts and standard definitions +how they work in scopes, but this would be complex for sure

frolicOP•4mo ago

maybe this would give me what I want

const codecs = {
  Date: {
    encode: type(["Date", "=>", (d) => d.toISOString()]),
    decode: type(["string", "=>", (s) => Date.parse(s)]),
  },
};

const types = scope({
  codec: "'encode' | 'decode'",
  "postsFilter<op extends codec>": {
    "from": ["op", "=>", (op) => codecs.Date[op]],
  }
}).export();

types.postsFilter('encode')({ from: new Date() });
types.postsFilter('decode')({ from: "2024-08-22" });

const codecs = {
  Date: {
    encode: type(["Date", "=>", (d) => d.toISOString()]),
    decode: type(["string", "=>", (s) => Date.parse(s)]),
  },
};

const types = scope({
  codec: "'encode' | 'decode'",
  "postsFilter<op extends codec>": {
    "from": ["op", "=>", (op) => codecs.Date[op]],
  }
}).export();

types.postsFilter('encode')({ from: new Date() });
types.postsFilter('decode')({ from: "2024-08-22" });

getting closer 👀

ssalbdivad•4mo ago

Did you see how in the scope example I sent generic params are on the scope key itself? The syntax is definitely nicer than a tuple

frolicOP•4mo ago

oh yeah, didn't update there the tuple syntax is available now? and the generic syntax is coming soon?

ssalbdivad•4mo ago

No the only way you can create generics in a scope is the better syntax I mentioned

frolicOP•4mo ago

oh I see, I was interpreting unit tests as if I could use it as a tuple too and that syntax isn't released yet, right?

ssalbdivad•4mo ago

No it is it's always been that way

frolicOP•4mo ago

oh great I think this'll hold me over for a bit!

ssalbdivad•4mo ago

Excited to see what you come up with!

frolicOP•4mo ago

oh wait hrmm, maybe not hehe that would resolve from to a type, but not call it with a value I pass in

ssalbdivad•4mo ago

In that case, best to just define it as a standalone generic

frolicOP•4mo ago

any way to define this inline to get the right output type of the morph?

frolicOP•4mo ago

I assumed it would have inferred from the morph function (it not working might be related to the arg type also not working)

ssalbdivad•4mo ago

Looking at that part of the inference doesn't necessarily tell you what's inferred as output, although it could be a problem. The inferred output of the entire type is also unknown? Generally you shouldn't have to ever explicitly type morph input or output, but I'd definitely expect in the cases TS struggles to infer them otherwise (which do exist), that explicit typing should fix it

frolicOP•4mo ago

damn, thought I had something with this

const $ = scope({
  "codec<'date', 'encode'>": ["Date", "=>", (d) => d.toISOString()],
  "codec<'date', 'decode'>": ["string", "=>", (s) => new Date(s)],
  "types<op extends 'encode' | 'decode'>": {
    postsFilter: {
      from: "codec<'date', op>",
    },
  },
});

$.type("types<'encode'>").get("postsFilter").infer;

const $ = scope({
  "codec<'date', 'encode'>": ["Date", "=>", (d) => d.toISOString()],
  "codec<'date', 'decode'>": ["string", "=>", (s) => new Date(s)],
  "types<op extends 'encode' | 'decode'>": {
    postsFilter: {
      from: "codec<'date', op>",
    },
  },
});

$.type("types<'encode'>").get("postsFilter").infer;

but .infer shows from: string | Date I assume because the from type is resolved at definition time, which explains the union, but kinda hoped it would have been narrowed by the generic

ssalbdivad•4mo ago

I think the problem is just that you have two generics with the same name I should add a validation error for that

frolicOP•4mo ago

haha that makes more sense I have a working prototype using a combo of a few strategies, just seeing how close I can get to doing this purely in arktype

ssalbdivad•4mo ago

Hkts would make it easy but not quite as clean. Not terrible though

frolicOP•4mo ago

can you pass an object to a generic? actually iirc some tests for this so I think the answer is yes, just gotta find the right syntax for doing it in a scope

ssalbdivad•4mo ago

that's the great thing about scopes is you can always serialized anything by making it a keyword If you just want it as an intermediate without exposing it you can make it private #privateAlias

frolicOP•4mo ago

I wanted to inline it rather than new definitions per generic value

ssalbdivad•4mo ago

Generally if you're defining objects, it's good to give them a name anyways IMO. Definitely not for primitives e.g. 'encode' | 'decode' or whatever but an object literal with keys? It's somewhat rare you wouldn't want to reuse that in some way, or at least have access to it An anti-pattern I see a lot is people define complex structures then extract back out the parts they need which is expensive both in terms of runtime and type-level perf

frolicOP•4mo ago

It was more to constrain the type entering the generic

ssalbdivad•4mo ago

Even so, having a BaseConstraint type or whatever seems kinda good? Regardless there is no way to do that inline in a scope without defining it as an alias

frolicOP•3mo ago

is there an example of using a scope generic like this but uses Hkt class?

ssalbdivad•3mo ago

Here's the builtin TS generics in a scope:

class RecordHkt extends Hkt<[Key, unknown]> {
    declare body: Record<this[0], this[1]>
}

const Record = genericNode(["K", intrinsic.key], "V")(
    args => ({
        domain: "object",
        index: {
            signature: args.K,
            value: args.V
        }
    }),
    RecordHkt
)

class PickHkt extends Hkt<[object, Key]> {
    declare body: pick<this[0], this[1] & keyof this[0]>
}

const Pick = genericNode(["T", intrinsic.object], ["K", intrinsic.key])(
    args => args.T.pick(args.K as never),
    PickHkt
)

class OmitHkt extends Hkt<[object, Key]> {
    declare body: omit<this[0], this[1] & keyof this[0]>
}

const Omit = genericNode(["T", intrinsic.object], ["K", intrinsic.key])(
    args => args.T.omit(args.K as never),
    OmitHkt
)

class PartialHkt extends Hkt<[object]> {
    declare body: show<Partial<this[0]>>
}

const Partial = genericNode(["T", intrinsic.object])(
    args => args.T.partial(),
    PartialHkt
)

class RequiredHkt extends Hkt<[object]> {
    declare body: show<Required<this[0]>>
}

const Required = genericNode(["T", intrinsic.object])(
    args => args.T.required(),
    RequiredHkt
)

class ExcludeHkt extends Hkt<[unknown, unknown]> {
    declare body: Exclude<this[0], this[1]>
}

const Exclude = genericNode("T", "U")(
    args => args.T.exclude(args.U),
    ExcludeHkt
)

class ExtractHkt extends Hkt<[unknown, unknown]> {
    declare body: Extract<this[0], this[1]>
}

const Extract = genericNode("T", "U")(
    args => args.T.extract(args.U),
    ExtractHkt
)

export const arkTsGenerics: Module<arkTsGenerics> = submodule({
    Record,
    Pick,
    Omit,
    Exclude,
    Extract,
    Partial,
    Required
})

class RecordHkt extends Hkt<[Key, unknown]> {
    declare body: Record<this[0], this[1]>
}

const Record = genericNode(["K", intrinsic.key], "V")(
    args => ({
        domain: "object",
        index: {
            signature: args.K,
            value: args.V
        }
    }),
    RecordHkt
)

class PickHkt extends Hkt<[object, Key]> {
    declare body: pick<this[0], this[1] & keyof this[0]>
}

const Pick = genericNode(["T", intrinsic.object], ["K", intrinsic.key])(
    args => args.T.pick(args.K as never),
    PickHkt
)

class OmitHkt extends Hkt<[object, Key]> {
    declare body: omit<this[0], this[1] & keyof this[0]>
}

const Omit = genericNode(["T", intrinsic.object], ["K", intrinsic.key])(
    args => args.T.omit(args.K as never),
    OmitHkt
)

class PartialHkt extends Hkt<[object]> {
    declare body: show<Partial<this[0]>>
}

const Partial = genericNode(["T", intrinsic.object])(
    args => args.T.partial(),
    PartialHkt
)

class RequiredHkt extends Hkt<[object]> {
    declare body: show<Required<this[0]>>
}

const Required = genericNode(["T", intrinsic.object])(
    args => args.T.required(),
    RequiredHkt
)

class ExcludeHkt extends Hkt<[unknown, unknown]> {
    declare body: Exclude<this[0], this[1]>
}

const Exclude = genericNode("T", "U")(
    args => args.T.exclude(args.U),
    ExcludeHkt
)

class ExtractHkt extends Hkt<[unknown, unknown]> {
    declare body: Extract<this[0], this[1]>
}

const Extract = genericNode("T", "U")(
    args => args.T.extract(args.U),
    ExtractHkt
)

export const arkTsGenerics: Module<arkTsGenerics> = submodule({
    Record,
    Pick,
    Omit,
    Exclude,
    Extract,
    Partial,
    Required
})

I guess you'd want to call generic instead of genericNode but it's basically the same thing It's valid as a root scope def

frolicOP•3mo ago

ah I guess what I was wondering was something like

scope({
  "Partial<T extends object>": [
    args => args.T.partial(),
    class PartialHkt extends Hkt<[object]> {
      declare body: show<Partial<this[0]>>
    }
  ]
});

scope({
  "Partial<T extends object>": [
    args => args.T.partial(),
    class PartialHkt extends Hkt<[object]> {
      declare body: show<Partial<this[0]>>
    }
  ]
});

ssalbdivad•3mo ago

Yeah that doesn't exist would be a cool API though TBH it's already such a niche use case

frolicOP•3mo ago

for sure if I wanna dynamically make a scope but I wanna attach some preexisting stuff, seems like validateScope<scope & otherStuff> isn't quite what I am looking for ends up being that the scope input is wanting definitions for the keys on otherStuff

ssalbdivad•3mo ago

You're just attaching the props internally? Then it shouldn't affect validateScope, just how it's inferred

frolicOP•3mo ago

function withCodecs<
  const codecs extends { [k: string]: { encode: any; decode: any } },
  const scope,
>(codecs: validateCodecs<codecs>, types: validateScope<scope>): codecs {
  return codecs as codecs;
}

function withCodecs<
  const codecs extends { [k: string]: { encode: any; decode: any } },
  const scope,
>(codecs: validateCodecs<codecs>, types: validateScope<scope>): codecs {
  return codecs as codecs;
}

ignore the return value for now

frolicOP•3mo ago

ssalbdivad•3mo ago

If you look at how scopes are validated, you'll notice I basically never use constraints because they break inference quite often Especially not k: string

frolicOP•3mo ago

so codecs is fine

ssalbdivad•3mo ago

validateCodecs should be able to handle providing the type-level errors and autocomplete

frolicOP•3mo ago

as is validateCodecs for autocomplete etc what I want is to extend scope with codecs when I run it through validateScope so I don't get "codec.date is unresolvable"

ssalbdivad•3mo ago

Well, just bear in mind because it works with some definitions doesn't mean it will work with all. Would recommend avoiding it if possible

frolicOP•3mo ago

but if I do validateScope<scope & codecs>

ssalbdivad•3mo ago

Hmm, I used to directly support parent/child scopes but I removed those in favor of the export/import syntax which I thought was clearer

frolicOP•3mo ago

thinking maybe what I want is one scope that has a constraint that enforces codecs object with a certain shape

ssalbdivad•3mo ago

There's also a way to change what resolves globally, so that would be another option

frolicOP•3mo ago

oh I missed import

ssalbdivad•3mo ago

To do it directly you'd basically have to duplicate validateScope and add a type parameter for the parent scope which is how it was in alpha

ssalbdivad•3mo ago

https://github.com/arktypeio/arktype/blob/8876a4bb493d93e1ce4ca72f1bca0a691f1f875e/ark/type/__tests__/imports.test.ts

GitHub

arktype/ark/type/tests/imports.test.ts at 8876a4bb493d93e1ce4ca...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

ssalbdivad•3mo ago

This is kind ofthe tldr

ssalbdivad•3mo ago

const threeSixtyNoScope = scope({
    three: "3",
    sixty: "60",
    no: "'no'"
})

const scopeCreep = scope({
    hasCrept: "true"
})

const types = scope({
    ...threeSixtyNoScope.import("three", "no"),
    ...scopeCreep.export(),
    public: "hasCrept|three|no|private",
    "#private": "string.uuid"
}).export()

const threeSixtyNoScope = scope({
    three: "3",
    sixty: "60",
    no: "'no'"
})

const scopeCreep = scope({
    hasCrept: "true"
})

const types = scope({
    ...threeSixtyNoScope.import("three", "no"),
    ...scopeCreep.export(),
    public: "hasCrept|three|no|private",
    "#private": "string.uuid"
}).export()

frolicOP•3mo ago

scope creep hahaha perfect

frolicOP•3mo ago

exports + spread worked well, thank you

ssalbdivad•3mo ago

Ahh coool I thought you werew trying to create a scope users could use to create scopes

frolicOP•3mo ago

sort of am just prototyping some stuff, then lifting it out into functions

ssalbdivad•3mo ago

In that case you would have to do one of the other things I mentioned

frolicOP•3mo ago

type expectedCodec = { encode: any; decode: any };
type validateCodec<codec> = "encode" | "decode" extends keyof codec
  ? Omit<keyof codec, "encode" | "decode"> extends never
    ? validateAmbient<codec>
    : expectedCodec
  : expectedCodec;

type validateCodecs<codecs> = {
  [k in keyof codecs]: validateCodec<codecs[k]>;
};

function defineCodecs<const codecs>(codecs: validateCodecs<codecs>): codecs {
  return codecs as never;
}

const codecs = scope(
  defineCodecs({
    "codec.date": {
      encode: ["Date", "=>", (v) => v.toISOString()],
      decode: ["string", "=>", (s: string) => new Date(s)],
    },
  })
);

type expectedCodec = { encode: any; decode: any };
type validateCodec<codec> = "encode" | "decode" extends keyof codec
  ? Omit<keyof codec, "encode" | "decode"> extends never
    ? validateAmbient<codec>
    : expectedCodec
  : expectedCodec;

type validateCodecs<codecs> = {
  [k in keyof codecs]: validateCodec<codecs[k]>;
};

function defineCodecs<const codecs>(codecs: validateCodecs<codecs>): codecs {
  return codecs as never;
}

const codecs = scope(
  defineCodecs({
    "codec.date": {
      encode: ["Date", "=>", (v) => v.toISOString()],
      decode: ["string", "=>", (s: string) => new Date(s)],
    },
  })
);

so this works nicely to constrain the codecs shape, but when I use defineCodecs, I lose some propagation of types or something?

frolicOP•3mo ago

(mentioned this above but at least this is more contained now to demonstrate it)

ssalbdivad•3mo ago

Inferreing function inputs is really hard when you start to customize this stuff best thing I can recommend is looking at how it's implemented in AT

frolicOP•3mo ago

mmk in inferTypeRoot<t, $> what should $ be? a Scope? inferScope? a scope def? looks like inferScope for $ in inferTypeRoot but still unclear if it should be inferScope or scope def for $ in validateTypeRoot

Dimava•3mo ago

Use inferAmbient if you don't need the scope Hm Check scope.type impl

frolicOP•3mo ago

I do need the scope, that's why I'm asking about it have dug around the source and haven't found any hints as to what $ should be in validate types

Dimava•3mo ago

scope.t

frolicOP•3mo ago

can you point to a line in the source to describe what you're referring to?

Dimava•3mo ago

Where do you pick the scope from?

Dimava•3mo ago

https://github.com/arktypeio/arktype/blob/8876a4bb493d93e1ce4ca72f1bca0a691f1f875e/ark/type/scope.ts#L337

GitHub

arktype/ark/type/scope.ts at 8876a4bb493d93e1ce4ca72f1bca0a691f1f87...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

Dimava•3mo ago

https://github.com/arktypeio/arktype/blob/8876a4bb493d93e1ce4ca72f1bca0a691f1f875e/ark/type/scope.ts#L359

GitHub

arktype/ark/type/scope.ts at 8876a4bb493d93e1ce4ca72f1bca0a691f1f87...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

frolicOP•3mo ago

my function takes in a scope def, I use it in validateTypeRoot and inferTypeRoot, and just want to know what kind of scope to use in validateTypeRoot

Dimava•3mo ago

https://github.com/arktypeio/arktype/blob/8876a4bb493d93e1ce4ca72f1bca0a691f1f875e/ark/type/type.ts#L47

GitHub

arktype/ark/type/type.ts at 8876a4bb493d93e1ce4ca72f1bca0a691f1f875...

TypeScript's 1:1 validator, optimized from editor to runtime - arktypeio/arktype

frolicOP•3mo ago

I appreciate you trying to help but this isn't answering my question

Dimava•3mo ago

Can you provide the function definition It looks like you gonna need 4 generics or smth

frolicOP•3mo ago

not exactly this but just demonstrating:

function <const def, const scopeDef>(
  scope: validateScope<scopeDef>,
  def: validateTypeRoot<def, scopeDef>
): inferTypeRoot<def, inferScope<scopeDef>>

function <const def, const scopeDef>(
  scope: validateScope<scopeDef>,
  def: validateTypeRoot<def, scopeDef>
): inferTypeRoot<def, inferScope<scopeDef>>

wondering which of these for validateTypeRoot

validateTypeRoot<def, scopeDef>
validateTypeRoot<def, inferScope<scopeDef>>
validateTypeRoot<def, Scope<inferScope<scopeDef>>

validateTypeRoot<def, scopeDef>
validateTypeRoot<def, inferScope<scopeDef>>
validateTypeRoot<def, Scope<inferScope<scopeDef>>

Dimava•3mo ago

Why don't you use a regular compiled scope here? As arg

frolicOP•3mo ago

it can accept both which makes it easier to use the function externally for simple cases without having to import arktype directly

Dimava•3mo ago

Hmm What's your use case? If you make it chainable it will be literal AT

scope(scopeDef).type(def)

scope(scopeDef).type(def)

so you don't even need your own fns

frolicOP•3mo ago

this thread has all the context if you're curious well, maybe not all, this was a carry-over from DMs

frolicOP•3mo ago

https://github.com/holic/typed-http/blob/main/src/arktype/action.test.ts

GitHub

typed-http/src/arktype/action.test.ts at main · holic/typed-http

Contribute to holic/typed-http development by creating an account on GitHub.

Dimava•3mo ago

Now I see

function <const def, const scopeDef>(
  scope: validateScope<scopeDef>,
   def: validateTypeRoot<def, inferScope<scopeDef>>
): inferTypeRoot<def, inferScope<scopeDef>>

function <const def, const scopeDef>(
  scope: validateScope<scopeDef>,
   def: validateTypeRoot<def, inferScope<scopeDef>>
): inferTypeRoot<def, inferScope<scopeDef>>

frolicOP•3mo ago

that's what I have currently, just wanted to confirm this is correct but couldn't find any hints in arktype source to give me confience

frolicOP•3mo ago

circling back, things are looking good with the lib: https://github.com/holic/typed-http/blob/main/src/arktype/http.test.ts should have some full end to end tests soon

GitHub

typed-http/src/arktype/http.test.ts at main · holic/typed-http

Contribute to holic/typed-http development by creating an account on GitHub.

frolicOP•3mo ago

still curious about this also could greatly simplify this lib if there was some notion of bidirectional morphs native to arktype, wouldn't need the whole codec thing

ssalbdivad•3mo ago

validateTypeRoot<def, inferScope<scopeDef>>

validateTypeRoot<def, inferScope<scopeDef>>

Or if you want to accept an already instantiated scope you'd do scope: Scope<$>, def: validateTypeRoot<def, $> Generally there's not a lot of APIs like this because you'd usually either create the type as an alias in your scope or use the .type instance attached to the scope to create one later

frolicOP•3mo ago

great, that's basically what I am doing

ssalbdivad•3mo ago

But I can see how you might need some custom APIs if you're wanting to define relationships etc. Definitely something I want to tackle with that graph angle in mind, but I think there's enough possibilities in terms of the API that I likely won't work on it until 2.0 has been stable for a while @frolic Btw you should check out the generic API in 2.0.0-rc.2, feels really clean and might help with some of the stuff you were looking at:

const boxOf = generic([
    "t",
    {
        foo: "number"
    }
])({ boxOf: "t" })

const t = boxOf({
    foo: "1"
})

const boxOf = generic([
    "t",
    {
        foo: "number"
    }
])({ boxOf: "t" })

const t = boxOf({
    foo: "1"
})

frolicOP•3mo ago

I tried fiddling with generics for quite a while and couldn't figure out a way to help simplify things without index access curious if you had a specific use case in mind

ssalbdivad•3mo ago

I was just thinking about some of the original cases where you wanted to accept a constrained type definition as a function input You mean syntactically? There is .get. The fact that it already exists internally means supporting index access would really just be adding it to the parser

frolicOP•3mo ago

yeah there's a transformation step I'm doing to turn individual codecs with encode/decode functions into an object for all encoders and all decoders so would let me do that transformation with pure arktype generics I think unclear if strong types carry through though

ssalbdivad•3mo ago

Definitely tougher when you're wanting to define relationships like that. HKTs can theoretically do anything though

frolicOP•3mo ago

I couldn't figure those out 🙈 iirc I kept wanting to access outer scope, not just the immediate value or whatever but maybe thinking about it wrong

Dimava•3mo ago

Can the parser have subscopes?

ssalbdivad•3mo ago

Dunno what you mean

Dimava•3mo ago

Can you

Scope({
  myNumber: scope({
    $root: 'string.numeric.fromNumber`
    rev: 'number.toString`
  })
})

Scope({
  myNumber: scope({
    $root: 'string.numeric.fromNumber`
    rev: 'number.toString`
  })
})

.. Whatever that is

ssalbdivad•3mo ago

It has to be a module to reference it like that So yeah if you add .export to the end of scope or use type.module

Dimava•3mo ago

@frolic consider trying making a type.module thingy then Then you may define reverse operation on every thing I guess Or maybe make the root just e.g. Map and add [,][] => Map and Map => [,][] on top

frolicOP•3mo ago

anything new I should be aware of for the codec use case? currently bumping to latest arktype RC

ssalbdivad•3mo ago

I don't think so it's mostly been small bug fixes and cleanup. Actually reintegrating this now

frolicOP•3mo ago

not sure what this you're referring to

ssalbdivad•3mo ago

It's an alias you can reference in types to recurse directly without a scope

frolicOP•3mo ago

(kind of funny JS joke too if you think about it)

ssalbdivad•3mo ago

Yeah whenever I refer to it it sounds ridiculous almost too easy haha

frolicOP•3mo ago

not a bad diff :) feels more readable after those type.* changes: https://github.com/holic/typed-http/commit/44a5c9ecebece61b72e238e53d26bf99707a2c8a

GitHub

bump arktype · holic/typed-http@44a5c9e

frolicOP•3mo ago

import { type type } is funny to me

ssalbdivad•3mo ago

Yeah I guess that can happen now that I'm using it as a namespace like that h aha I think organizing those generics as well as the builtin keywords by namespace should help with discoverability for authors wanting to integrate and users respectively

Gaming

Programming

codecs