Martin Vuyk Comments - Answer Overflow

Topics

Martin Vuyk

•Created by taalhaataahir01022001 on 11/14/2024 in #questions

Parallelize Issue

And you're not freeing the pointer after using it, you'll have a memory leak if this is called by another function etc. with this example the memory gets freed once the program ends, but in a larger codebase it will lead to problems

4 replies

•Created by taalhaataahir01022001 on 11/14/2024 in #questions

Parallelize Issue

This seems like a problem we are still having with closures not extending the lifetime of variables. You'll have to manually discard the variables after the call to parallelize

parallelize[...](...)
_ = A^
_ = B^

parallelize[...](...)
_ = A^
_ = B^

4 replies

•Created by Juan Casta on 11/17/2024 in #questions

Exploring Mojo for Real-Time AI Voice Call Solutions

Yes Mojo would be a very good fit for that use case. However, it is still in its very early stages. I would not recommend building any company or product around it just yet. We don't have networking nor very good multi threading, and stdlib is still rapidly changing and evolving (breaking changes are very normal currently)

4 replies

•Created by omarelb on 10/29/2024 in #questions

What pointer lifetime to use when iterating through a tree?

Not sure if you are using nightly or stable. But I'll say origin instead of lifetime since it became the new term (in current stable it's still the __lifetime_of() operator). You are trying to append a child with a different origin than the parent and you declared the list as being a list of pointers to node roots. Unsafe pointer lets you bypass that. Currently Pointer is very limited. You might want to read some of /utils/span.mojo code in the stdlib to see some parametrized origin use. If your child nodes can have the origin of the parent you might avoid the issue. You could also try to force a rebind[Pointer[BKTNode, __origin_of(self.root.value())]](Pointer.address_of(child[])) Play around with those concepts, but don't worry it's mainly issues with the language not being fully fledged yet. And this is a solid use case for Unsafe pointer, where the language doesn't let you go on safe mode, go raw. (UnsafePointer will also be able to carry an origin in the future to be safer, I'm working on a PR for that)

3 replies

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

We'll see where it goes, might get changed for something else 🤷‍♂️ . The whole stdlib is still WIP

38 replies

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

I've opened a proposal to change the way we do __getitem__(self, slice: Slice) to return an Iterator instead of a new instance

38 replies

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

maybe in the future, but currently it doesn't

38 replies

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

so overall using StringSlice as much as possible and Span[Byte] as well are the best ways to go since they are non-owning types that just offer a view into the data

38 replies

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

yep since it returns a String instance which owns its data. Also it should be noted that it currently does not work by unicode codepoints and it will in the future, which will also add overhead

38 replies

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

Hi @bgreni cool library. Some comments on your approach: I have a PR open which will make split more efficient and make StringSlice.split() return a List[StringSlice] so there is no allocation beyond building that list. My next feature in line is doing something similar for splitlines so that you will have the option to have your Reader struct already have everything semi-tokenized very cheaply, because splitlines splits by every character except " " (which you dont want to split since your fields might have strings with whitespace). You can also implement your own version since we follow Python that also takes some newline separators into account that the JSON spec doesn't (AFAIK). If you want to keep the peek approach, you can make it faster by going over a byte Span or using UnsafePointer since string slicing is expensive because it checks bounds and allocates one each time. You can look at the code in the split PR to take inspiration. Mojo is very cool and you can make your const types be very readable:

alias `"` = Byte(ord('"'))
alias `t` = Byte(ord("t"))
alias `f` = Byte(ord("f"))
alias `n` = Byte(ord("n"))
alias `{` = Byte(ord("{"))
alias `}` = Byte(ord("}"))
alias `[` = Byte(ord("["))
alias `]` = Byte(ord("]"))
alias `:` = Byte(ord(":"))
alias `,` = Byte(ord(","))

alias `"` = Byte(ord('"'))
alias `t` = Byte(ord("t"))
alias `f` = Byte(ord("f"))
alias `n` = Byte(ord("n"))
alias `{` = Byte(ord("{"))
alias `}` = Byte(ord("}"))
alias `[` = Byte(ord("["))
alias `]` = Byte(ord("]"))
alias `:` = Byte(ord(":"))
alias `,` = Byte(ord(","))

Anyway GLHF! looking forward to a PR/review of one by you on this 🙂

38 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

or we could do it with a pointer idk

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

This is what I've currently got

    async fn recv_fds(self, maxfds: UInt) -> Optional[List[FileDescriptor]]:
        """Receive up to maxfds file descriptors.

        Args:
            maxfds: The maximum amount of file descriptors.

        Returns:
            The file descriptors.
        """

        @parameter
        if sock_platform is SockPlatform.LINUX:
            return (
                await self._impl.unsafe_get[Self._linux_s]()[].recv_fds(maxfds)
            )^
        else:
            constrained[False, "Platform not supported yet."]()
            return None

    async fn recv_fds(self, maxfds: UInt) -> Optional[List[FileDescriptor]]:
        """Receive up to maxfds file descriptors.

        Args:
            maxfds: The maximum amount of file descriptors.

        Returns:
            The file descriptors.
        """

        @parameter
        if sock_platform is SockPlatform.LINUX:
            return (
                await self._impl.unsafe_get[Self._linux_s]()[].recv_fds(maxfds)
            )^
        else:
            constrained[False, "Platform not supported yet."]()
            return None

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

if hasattr(_socket.socket, "recvmsg"):
    import array

    def recv_fds(sock, bufsize, maxfds, flags=0):
        """ recv_fds(sock, bufsize, maxfds[, flags]) -> (data, list of file
        descriptors, msg_flags, address)

        Receive up to maxfds file descriptors returning the message
        data and a list containing the descriptors.
        """
        # Array of ints
        fds = array.array("i")
        msg, ancdata, flags, addr = sock.recvmsg(bufsize,
            _socket.CMSG_LEN(maxfds * fds.itemsize))
        for cmsg_level, cmsg_type, cmsg_data in ancdata:
            if (cmsg_level == _socket.SOL_SOCKET and cmsg_type == _socket.SCM_RIGHTS):
                fds.frombytes(cmsg_data[:
                        len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])

        return msg, list(fds), flags, addr
    __all__.append("recv_fds")

if hasattr(_socket.socket, "recvmsg"):
    import array

    def recv_fds(sock, bufsize, maxfds, flags=0):
        """ recv_fds(sock, bufsize, maxfds[, flags]) -> (data, list of file
        descriptors, msg_flags, address)

        Receive up to maxfds file descriptors returning the message
        data and a list containing the descriptors.
        """
        # Array of ints
        fds = array.array("i")
        msg, ancdata, flags, addr = sock.recvmsg(bufsize,
            _socket.CMSG_LEN(maxfds * fds.itemsize))
        for cmsg_level, cmsg_type, cmsg_data in ancdata:
            if (cmsg_level == _socket.SOL_SOCKET and cmsg_type == _socket.SCM_RIGHTS):
                fds.frombytes(cmsg_data[:
                        len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])

        return msg, list(fds), flags, addr
    __all__.append("recv_fds")

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

I think I saw that function name in the Python code as well.

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

ok, but as I said we can provide a "use this unified API for ez mode, get into customization to get maximum performance. Here are the APIs"

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

recv makes no sense for UDP,

why wouldn't an async recv that yields the byte buffers as they arrive be useful ?

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

no idea if they're eager or lazy

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

you can see the valid address types I'm building https://github.com/martinvuyk/forge-tools/blob/main/src/forge_tools/socket/address.mojo

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

I'm still not sure how async in Mojo works (monday is the presentation yay), but I think If you return a Coroutine and await it then it doesn't create an async loop (?) if the function where you await it is not async (??)

145 replies

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

I'm mostly thinking of building overloads with conditional conformance for the self and anything to do with SockAddr in a way in that it doesn't drive anyone crazy (hopefully lol)

145 replies