A
arktypeā€¢3mo ago
PIat

Tried to initialize an $ark registry in a monorepo

Hello! In a pnpm workspace, I have 2 packages depending on arktype. This means that each package depends on arktype separately. However, when I import them into a central vite project, I get the error:
Tried to initialize an $ark registry but one already existed. This probably means you are either depending on multiple versions of an arktype package, or importing the same package from both ESM and CJS.
Review package.json versions across your repo to ensure consistency.
Tried to initialize an $ark registry but one already existed. This probably means you are either depending on multiple versions of an arktype package, or importing the same package from both ESM and CJS.
Review package.json versions across your repo to ensure consistency.
Both packages are compiled to ESM, and the arktype version is identical, so the issue must be in them clashing inside of the vite project. What would be a good way to avoid this error?
126 Replies
Dimava
Dimavaā€¢3mo ago
Is AT a peer dep? Is lockfile AT dep the same?
PIat
PIatOPā€¢3mo ago
It is a peer dep, since the package is installed as a dependency of an app, and arktype is the dependency of the package The issue was in me bundling all dependencies into the build by accident. So each package was running globalThis.$ark = registry; independently Also it was important to add public-hoist-pattern[]=*arktype* to .npmrc
ssalbdivad
ssalbdivadā€¢3mo ago
Which version of ArkType is this? Does it crash or is it this just logged as a warning. Based on my discussions with @Andarist, I'm not sure there's anything I can do internally to avoid this, hopefully getting the error as you did was helpful in this case? I do think having an option to disable the warning in situations where it's inevitable but you know it is safe would be good
PIat
PIatOPā€¢3mo ago
Hello @ssalbdivad! It's the version 2.0.0-dev.26. It crashed the app completely. I don't think it's a bad thing, the error message was very useful, and I understood in a bit what I needed to do. It's actually a similar issue you run into with for example Prisma when using pnpm workspaces. You need to put public-hoist-pattern[]=*arktype* into .npmrc to make sure that the package isn't installed and ran from in package or its dependents separately, but is only installed in a single location - at the root of the workspace.
Dimava
Dimavaā€¢3mo ago
šŸ¤” Does error message contain a link to arktype.io which contains a link to github discussion about how to fix this problem?
ssalbdivad
ssalbdivadā€¢3mo ago
Hmm, but ideally even if it's not hoisted pnpm should create symlinks that resolve to the same files. Does it load twice in that case? No, but the error especially in more recent versions is extremely specific there is nothing else generalizable I could add
PIat
PIatOPā€¢3mo ago
Since without hoisting it it gets installed into the node_modules folders separately, I'd expect arktype to load twice if the app depends on two modules, which both depend on arktype
ssalbdivad
ssalbdivadā€¢3mo ago
Yeah in general the idea of pnpm is that it dedupes dependencies, but I guess I'm only aware of how that works transitively. I'm not 100% on how it works with symlinks directly in each package's node_modules, although I'd expect it would also be the same?
PIat
PIatOPā€¢3mo ago
I also have a very rough idea of it, I don't want to confuse you with disinformation However, each time I need to make sure a package is instantiated only once, I hoist it
ssalbdivad
ssalbdivadā€¢3mo ago
Interested to know what behavior you get when you update since I made some changes around this. I'd guess if you turned off hoisting you'd just get a console warning
PIat
PIatOPā€¢3mo ago
Let's try! This was thrown:
throw new ctor(message);
ParseError: 'parse.number' is unresolvable
at throwError (.../node_modules/.pnpm/@[email protected]/node_modules/@ark/util/out/errors.js:5:11)
at throwParseError
throw new ctor(message);
ParseError: 'parse.number' is unresolvable
at throwError (.../node_modules/.pnpm/@[email protected]/node_modules/@ark/util/out/errors.js:5:11)
at throwParseError
ssalbdivad
ssalbdivadā€¢3mo ago
šŸ˜­ How would it get worse
PIat
PIatOPā€¢3mo ago
Version 2.0.0-beta.5
ssalbdivad
ssalbdivadā€¢3mo ago
Maybe I was right to just try and throw šŸ¤£ It does seem related to the registry issue You don't get a warning before that though?
PIat
PIatOPā€¢3mo ago
Sorry it took so long, I accidentally did i -r instead of up -r and got it installed in 21 modules šŸ˜† I didn't I'll try with hoisting now
ssalbdivad
ssalbdivadā€¢3mo ago
I' d assume it would work with hoisting This is the new logic:
let _registryName = "$ark"
let suffix = 2

while (_registryName in globalThis) _registryName = `$ark${suffix++}`

export const registryName = _registryName
;(globalThis as any)[registryName] = registry

export const $ark: ArkSchemaRegistry = registry as never

if (suffix !== 2) {
const g: any = globalThis
const registries: InitialRegistryContents[] = [g.$ark]
for (let i = 2; i < suffix; i++)
if (g[`$ark${i}`]) registries.push(g[`$ark${i}`])

console.warn(
`Multiple @ark registries detected. This can lead to unexpected behavior.`
)
const byPath = groupBy(registries, "filename")

const paths = Object.keys(byPath)

for (const path of paths) {
if (byPath[path]!.length > 1) {
console.warn(
`File ${path} was initialized multiple times, likely due to being imported from both CJS and ESM contexts.`
)
}
}

if (paths.length > 1) {
console.warn(
`Registries were initialized at the following paths:` +
paths
.map(
path => ` ${path} (@ark/util version ${byPath[path]![0].version})`
)
.join("\n")
)
}
}
let _registryName = "$ark"
let suffix = 2

while (_registryName in globalThis) _registryName = `$ark${suffix++}`

export const registryName = _registryName
;(globalThis as any)[registryName] = registry

export const $ark: ArkSchemaRegistry = registry as never

if (suffix !== 2) {
const g: any = globalThis
const registries: InitialRegistryContents[] = [g.$ark]
for (let i = 2; i < suffix; i++)
if (g[`$ark${i}`]) registries.push(g[`$ark${i}`])

console.warn(
`Multiple @ark registries detected. This can lead to unexpected behavior.`
)
const byPath = groupBy(registries, "filename")

const paths = Object.keys(byPath)

for (const path of paths) {
if (byPath[path]!.length > 1) {
console.warn(
`File ${path} was initialized multiple times, likely due to being imported from both CJS and ESM contexts.`
)
}
}

if (paths.length > 1) {
console.warn(
`Registries were initialized at the following paths:` +
paths
.map(
path => ` ${path} (@ark/util version ${byPath[path]![0].version})`
)
.join("\n")
)
}
}
So I'm surprised if you're getting that error but not the warning first
PIat
PIatOPā€¢3mo ago
Actually the error gets thrown even with hoisting
ssalbdivad
ssalbdivadā€¢3mo ago
Hmm well I guess that helps explain why you don't see the warning first I wonder if it has to do with the way import resolutions change during bundling I'd be interested if you have a chance to catch the error in a debugger or log globalThis.$ark and globalThis.$ark2
PIat
PIatOPā€¢3mo ago
Let's see Unfortunately I can't catch the error It doesn't catch as a caught nor uncaught exception. Just kills the process, there I see the error in the terminal, but not the debug console I added logging globalThis.$ark to the throwError function, but it' too large to show in the terminal. Should I send some specific part of it?
ssalbdivad
ssalbdivadā€¢3mo ago
Are you using VSCode?
PIat
PIatOPā€¢3mo ago
Yes
ssalbdivad
ssalbdivadā€¢3mo ago
Do you know how to open a debug terminal? If you ctrl+shift+P by default to open command palette then type open debug terminal you should be able to find it
PIat
PIatOPā€¢3mo ago
Yes, I got it opened
ssalbdivad
ssalbdivadā€¢3mo ago
Then you just run whatever you were running that led to the error from that terminal, but before you do, check the Breakpoints=>Caught Exceptions box:
No description
PIat
PIatOPā€¢3mo ago
But as I say, terminal shows the error, but debug console doesn't
No description
No description
ssalbdivad
ssalbdivadā€¢3mo ago
Hopefully the strategy I mentioned works, I find it super useful for diagnosing crashes
PIat
PIatOPā€¢3mo ago
Yes, found got the place of the error
PIat
PIatOPā€¢3mo ago
No description
ssalbdivad
ssalbdivadā€¢3mo ago
Okay so then if you open the debug console, you should be able to look at globalThis and see what $ark objects are attached Most importantly- are there multiple objects starting with $ark on globalThis or just one? What are the keys of those objects
PIat
PIatOPā€¢3mo ago
Yeah I don't usually use it, since every time I launch anything, it first goes through about 20 pnpm errors
ssalbdivad
ssalbdivadā€¢3mo ago
Yeah this is actually the bane of my existence I have never been able to figure out how to avoid that
PIat
PIatOPā€¢3mo ago
You also click through all of it?
ssalbdivad
ssalbdivadā€¢3mo ago
Well, at least not by default No, the best workaround I've found is you add a breakpoint at the top of your file or something, then once you hit that, enable break on Caught Exceptions instead of before running the process
PIat
PIatOPā€¢3mo ago
I don't seem to be able to find it
No description
ssalbdivad
ssalbdivadā€¢3mo ago
Is this at the top of the imports or when the error occurs
PIat
PIatOPā€¢3mo ago
When it error occurs But when I logged it from errors.js, it gave me the output in the console
ssalbdivad
ssalbdivadā€¢3mo ago
So when you look at globalThis in the debugger when the exception is thrown from the parse error, you don't see $ark at all? The only way I can imagine that would differ from the console log is if it is not being caught in the same process
PIat
PIatOPā€¢3mo ago
It really looks like it I can't see globalThis
ssalbdivad
ssalbdivadā€¢3mo ago
I guess it doesn't really matter as long as you can tell me when the error is actually thrown which $ark objects are attached and what there keys are, console.log should be fine
PIat
PIatOPā€¢3mo ago
Just this global pane
ssalbdivad
ssalbdivadā€¢3mo ago
Maybe @Dimava can save you he's better at debugging than me anyways In exchange I will implement toJsonSchema()
Dimava
Dimavaā€¢3mo ago
hi @ssalbdivad do you store import.meta.URL in $ark ? @PIat what's the problem again
ssalbdivad
ssalbdivadā€¢3mo ago
Yeah
PIat
PIatOPā€¢3mo ago
On the newest version an error is thrown And we're trying to find out what's the cause My terminal and debug console aren't matching up And I don't see $ark in global object @ssalbdivad On 2.0.0-dev.26 it runs fine Actually I think the problem is my code
ssalbdivad
ssalbdivadā€¢3mo ago
I made a lot of changes to how the registry works to try and make it more flexible so I assume it relates to that Ideally it should never crash in that way because of your code Maybe if your bundler does something weird though
PIat
PIatOPā€¢3mo ago
Yeahhhh, the build doesn't go through Yeahhh, sorry
ssalbdivad
ssalbdivadā€¢3mo ago
It's not your fault what is the bundler you are using?
PIat
PIatOPā€¢3mo ago
tsup
ssalbdivad
ssalbdivadā€¢3mo ago
I mean I definitely want to know if/how it's not compatible with common tools like that My most recent changes to the registry were designed to make compatibility easier so if it's breaking stuff that is bad
PIat
PIatOPā€¢3mo ago
The issue is that something changed in inferTypeRoot, so there's a type error
ssalbdivad
ssalbdivadā€¢3mo ago
A type error shouldn't relate to that runtime crash you were getting though I'm not super surprised about type errors building with TSUp due to transitive type resolutions, that might require some finessing But runtime is different
PIat
PIatOPā€¢3mo ago
I guess it was running the old bundled version but using new dependencies?
inferTypeRoot<JobType['output']>
inferTypeRoot<JobType['output']>
How should I use it?
ssalbdivad
ssalbdivadā€¢3mo ago
What exactly are you trying to build that's failing? What's the TS up error? / is it just that type error or is there a runtime issue as well after updating?
PIat
PIatOPā€¢3mo ago
No description
ssalbdivad
ssalbdivadā€¢3mo ago
Yeah this looks like multiple versions somehow
PIat
PIatOPā€¢3mo ago
It's hard to tell at this point. I bundle the package and then the bundled version gets picked up by another tsup app
Dimava
Dimavaā€¢3mo ago
@PIat rm -rf **/node_modules
ssalbdivad
ssalbdivadā€¢3mo ago
I remember changing that recently where the second param used to be optional, but I made it required and there's a new inferAmbient that doesn't take a scope This is actually one of the reasons hoisting is problematic, normally pnpm is better about avoiding this kind of thing
PIat
PIatOPā€¢3mo ago
But how can I change it so this code would be valid?
No description
PIat
PIatOPā€¢3mo ago
I'm extracting the type from the type object
ssalbdivad
ssalbdivadā€¢3mo ago
Ahh your code? You want inferAmbient if you weren't passing a param before
PIat
PIatOPā€¢3mo ago
Thank you! Let's see how it goes
ssalbdivad
ssalbdivadā€¢3mo ago
That part at least I'm confident will work haha
PIat
PIatOPā€¢3mo ago
Great, it built fine! Buuuut, there's still the same error šŸ˜© šŸ˜© šŸ˜© šŸ˜© šŸ˜© šŸ˜©
ssalbdivad
ssalbdivadā€¢3mo ago
The runtime issue? yeah it wouldn't be related
PIat
PIatOPā€¢3mo ago
No description
PIat
PIatOPā€¢3mo ago
ParseError: 'parse.number' is unresolvable
ParseError: 'parse.number' is unresolvable
ssalbdivad
ssalbdivadā€¢3mo ago
Wait why is it PORT lol Oh I see
PIat
PIatOPā€¢3mo ago
Changing it to string makes the error go away
ssalbdivad
ssalbdivadā€¢3mo ago
It's just a key of an object
PIat
PIatOPā€¢3mo ago
Yes, sorry
ssalbdivad
ssalbdivadā€¢3mo ago
No problem I was worried it was an env var somehow
PIat
PIatOPā€¢3mo ago
I try to remove as much noise as possible from examples
Dimava
Dimavaā€¢3mo ago
if you type('parse.number') before that does it crash
PIat
PIatOPā€¢3mo ago
It doesn't crash
ssalbdivad
ssalbdivadā€¢3mo ago
My initial intuition is this would happen because there are multiple instances and arkKind used to be a symbol
No description
PIat
PIatOPā€¢3mo ago
Back to just 'parse,number' and it crashes
ssalbdivad
ssalbdivadā€¢3mo ago
But the weird thing is that I recently changed it to be a string literal to try and avoid exactly this situation
Dimava
Dimavaā€¢3mo ago
if you PORT: type('parse.number') does it crash
PIat
PIatOPā€¢3mo ago
It doesn't
Dimava
Dimavaā€¢3mo ago
show var foo = type/scope remove all non-PORT lines and try
PIat
PIatOPā€¢3mo ago
What does it mean?
Dimava
Dimavaā€¢3mo ago
this +- 10 lines
ssalbdivad
ssalbdivadā€¢3mo ago
I'd guess it would be any subscope access like parse.number or format.trim would fail
PIat
PIatOPā€¢3mo ago
Ahhh Oh, could this be the issue?
const parser = type(
envVars.get('shared').get('env'),
'&',
envVars.get(envFile.PUBLIC_APP_MODE).get('env'),
)

const parsed = parser.assert(envFile)
const parser = type(
envVars.get('shared').get('env'),
'&',
envVars.get(envFile.PUBLIC_APP_MODE).get('env'),
)

const parsed = parser.assert(envFile)
const envScope = scope({
nonEmptyString: 'string>0',
})

export const envVars = envScope.type({
btc: {
env: {},
},
tg: {
env: {
TG_BOT_USERNAME: 'nonEmptyString',
TG_API_ID: ['string', '=>', (s: string) => Number.parseInt(s, 10)],
TG_API_HASH: 'nonEmptyString',
TG_BOT_TOKEN: 'nonEmptyString',
},
},
shared: {
routes: [],
env: {
PUBLIC_APP_MODE: "'btc' | 'tg' | 'shared'",
NODE_ENV: ["'development' | 'production'", '=', 'development'],
DOMAIN: 'nonEmptyString',
PORT: 'parse.number',
},
},
})
const envScope = scope({
nonEmptyString: 'string>0',
})

export const envVars = envScope.type({
btc: {
env: {},
},
tg: {
env: {
TG_BOT_USERNAME: 'nonEmptyString',
TG_API_ID: ['string', '=>', (s: string) => Number.parseInt(s, 10)],
TG_API_HASH: 'nonEmptyString',
TG_BOT_TOKEN: 'nonEmptyString',
},
},
shared: {
routes: [],
env: {
PUBLIC_APP_MODE: "'btc' | 'tg' | 'shared'",
NODE_ENV: ["'development' | 'production'", '=', 'development'],
DOMAIN: 'nonEmptyString',
PORT: 'parse.number',
},
},
})
ssalbdivad
ssalbdivadā€¢3mo ago
I assume if you replace parse.number with this it works?
type("string").pipe(s => Number.parseFloat(s))
type("string").pipe(s => Number.parseFloat(s))
PIat
PIatOPā€¢3mo ago
Let's try
ssalbdivad
ssalbdivadā€¢3mo ago
It should, it's simlar to what you're already doing with TG_API_ID It's definitely a specific issue with submodule resolutions. I'd guess maybe there are still old versions involved somehow? Hard to see how else it would happen based on the current implementation
PIat
PIatOPā€¢3mo ago
Yeah, tooling issues ;-; Yes, it works
ssalbdivad
ssalbdivadā€¢3mo ago
If you ever see $ark.version with a value other than 0.2.1 that would be a dead giveaway something is going wrong
ssalbdivad
ssalbdivadā€¢3mo ago
No description
PIat
PIatOPā€¢3mo ago
No description
PIat
PIatOPā€¢3mo ago
No description
PIat
PIatOPā€¢3mo ago
Seems fine?
ssalbdivad
ssalbdivadā€¢3mo ago
Looks right to me, I can't see how submodule resolutions are getting screwed up The function you'd want to debug is maybeResolveSubalias in ark/schema/scope.js Is there any chance you're still importing from some bundled version of the dep that hasn't been updated even if the one in node_modules has? If not you can try adding adding console log statements or using the debugger in that funciton to see what is going wrong
PIat
PIatOPā€¢3mo ago
Let's try I don't think there can be another version I went back and forth and deleted the node_modules multiple times Nothing is installed globally
ssalbdivad
ssalbdivadā€¢3mo ago
Try adding those console.logs/breakpoints then in that file, should help
PIat
PIatOPā€¢3mo ago
I don't seem to have that subfolder
PIat
PIatOPā€¢3mo ago
No description
PIat
PIatOPā€¢3mo ago
Also I just noticed arktype was not hoisted the whole last hour, it was working without it Hoisting doesn't change the situation
ssalbdivad
ssalbdivadā€¢3mo ago
It's not a folder it's that scope.js at the root I guess I was referring to it relative to my repo
PIat
PIatOPā€¢3mo ago
It doesn't have the function in question
ssalbdivad
ssalbdivadā€¢3mo ago
Can you just paste the entire contents?
ssalbdivad
ssalbdivadā€¢3mo ago
That function has been around for a while
No description
PIat
PIatOPā€¢3mo ago
ssalbdivad
ssalbdivadā€¢3mo ago
This is in arktype we need the one in @ark/schema
PIat
PIatOPā€¢3mo ago
Ohhhhhhhhhhhhhhhhh šŸ¤¦ā€ā™‚ļø
ssalbdivad
ssalbdivadā€¢3mo ago
There's a lot of similar concepts across both so it can be confusing haha
PIat
PIatOPā€¢3mo ago
If you say it without the @, I assume it's in arktype šŸ˜ It never gets to it when it crashes because of 'parse.number'
ssalbdivad
ssalbdivadā€¢3mo ago
What is the stack trace? Guess I should have checked that earlier haha
PIat
PIatOPā€¢3mo ago
No description
PIat
PIatOPā€¢3mo ago
How's it better to send? With text or picture?
ssalbdivad
ssalbdivadā€¢3mo ago
Ahh I think I found it
No description
ssalbdivad
ssalbdivadā€¢3mo ago
It's because I'm using instanceof here which would break if there are a multiple versions of the package If you change that line to hasArkKind(resolution, "root") instance of resolution instanceof BaseRoot I bet it would work Obviously that kind of thing is why it's ideal to not have multiple versions of the package anyways But I will change it in the next release
PIat
PIatOPā€¢3mo ago
I'll try it in a bit šŸ™
ssalbdivad
ssalbdivadā€¢3mo ago
Hmm Well that was my initial guess anyways, but honestly I'm still kinda confused Because yeah that should help with avoiding duplicate issues if there are mutliple versions but if we're returning undefined there, that check shouldn't matter anyways
PIat
PIatOPā€¢3mo ago
Unfortunately it still throws the error Yesss, but there's no other way apart from completely reexporting it from a package in the monorepo
ssalbdivad
ssalbdivadā€¢3mo ago
There must be something unique going on in your setup though. I'd guess most people using arktype use pnpm in monorepos- it's ubiquitous in modern TS. TSUp and republishing types I could see going wrong but if there's ever a bundling step involved I'd suspect that was to blame for whatever else is going on. If you can post a repro somewhere and log a bug I can take a look at some point
PIat
PIatOPā€¢3mo ago
Definitely something with my setup But how come it works on the previous version?
ssalbdivad
ssalbdivadā€¢3mo ago
Well I made a bunch of changes so I guess one of them was incompatible with your setup haha That doesn't mean it was a regression or bug, but I'd like to try and avoid it happening whenever possible obviously
PIat
PIatOPā€¢3mo ago
For now I'll use this
ssalbdivad
ssalbdivadā€¢3mo ago
Yeah seems like you should be safe if you don't use any of those subaliases
PIat
PIatOPā€¢3mo ago
In my experience minimal repros don't lead far and for the whole setup there's a LOT of small manual changes to do to start up So in any case I'm not sure it'd help pinpoint the issue, but would take hours
ssalbdivad
ssalbdivadā€¢3mo ago
I don't need a minimal repro I just need any repro haha
PIat
PIatOPā€¢3mo ago
How come I haven't updated in a week and Turbo completely revamped their whole CLI GUI It's so slick now I created a minimal repro - two artype packages, similar structure But it works flawlessly Without hoisting too So I'll be digging around my project to see what might cause the issue Thank you for your patience šŸ™
ssalbdivad
ssalbdivadā€¢3mo ago
No problem, keep me posted!
Want results from more Discord servers?
Add your server