Martin Vuyk Comments - Answer Overflow

so overall using StringSlice as much as possible and Span[Byte] as well are the best ways to go since they are non-owning types that just offer a view into the data

24 replies

MModular

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

yep since it returns a String instance which owns its data. Also it should be noted that it currently does not work by unicode codepoints and it will in the future, which will also add overhead

24 replies

MModular

•Created by eggsquad on 10/16/2024 in #community-showcase

EmberJson: High level JSON library

Hi @bgreni cool library. Some comments on your approach: I have a PR open which will make split more efficient and make StringSlice.split() return a List[StringSlice] so there is no allocation beyond building that list. My next feature in line is doing something similar for splitlines so that you will have the option to have your Reader struct already have everything semi-tokenized very cheaply, because splitlines splits by every character except " " (which you dont want to split since your fields might have strings with whitespace). You can also implement your own version since we follow Python that also takes some newline separators into account that the JSON spec doesn't (AFAIK). If you want to keep the peek approach, you can make it faster by going over a byte Span or using UnsafePointer since string slicing is expensive because it checks bounds and allocates one each time. You can look at the code in the split PR to take inspiration. Mojo is very cool and you can make your const types be very readable:

alias `"` = Byte(ord('"'))
alias `t` = Byte(ord("t"))
alias `f` = Byte(ord("f"))
alias `n` = Byte(ord("n"))
alias `{` = Byte(ord("{"))
alias `}` = Byte(ord("}"))
alias `[` = Byte(ord("["))
alias `]` = Byte(ord("]"))
alias `:` = Byte(ord(":"))
alias `,` = Byte(ord(","))

alias `"` = Byte(ord('"'))
alias `t` = Byte(ord("t"))
alias `f` = Byte(ord("f"))
alias `n` = Byte(ord("n"))
alias `{` = Byte(ord("{"))
alias `}` = Byte(ord("}"))
alias `[` = Byte(ord("["))
alias `]` = Byte(ord("]"))
alias `:` = Byte(ord(":"))
alias `,` = Byte(ord(","))

Anyway GLHF! looking forward to a PR/review of one by you on this 🙂

24 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

or we could do it with a pointer idk

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

This is what I've currently got

    async fn recv_fds(self, maxfds: UInt) -> Optional[List[FileDescriptor]]:
        """Receive up to maxfds file descriptors.

        Args:
            maxfds: The maximum amount of file descriptors.

        Returns:
            The file descriptors.
        """

        @parameter
        if sock_platform is SockPlatform.LINUX:
            return (
                await self._impl.unsafe_get[Self._linux_s]()[].recv_fds(maxfds)
            )^
        else:
            constrained[False, "Platform not supported yet."]()
            return None

    async fn recv_fds(self, maxfds: UInt) -> Optional[List[FileDescriptor]]:
        """Receive up to maxfds file descriptors.

        Args:
            maxfds: The maximum amount of file descriptors.

        Returns:
            The file descriptors.
        """

        @parameter
        if sock_platform is SockPlatform.LINUX:
            return (
                await self._impl.unsafe_get[Self._linux_s]()[].recv_fds(maxfds)
            )^
        else:
            constrained[False, "Platform not supported yet."]()
            return None

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

if hasattr(_socket.socket, "recvmsg"):
    import array

    def recv_fds(sock, bufsize, maxfds, flags=0):
        """ recv_fds(sock, bufsize, maxfds[, flags]) -> (data, list of file
        descriptors, msg_flags, address)

        Receive up to maxfds file descriptors returning the message
        data and a list containing the descriptors.
        """
        # Array of ints
        fds = array.array("i")
        msg, ancdata, flags, addr = sock.recvmsg(bufsize,
            _socket.CMSG_LEN(maxfds * fds.itemsize))
        for cmsg_level, cmsg_type, cmsg_data in ancdata:
            if (cmsg_level == _socket.SOL_SOCKET and cmsg_type == _socket.SCM_RIGHTS):
                fds.frombytes(cmsg_data[:
                        len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])

        return msg, list(fds), flags, addr
    __all__.append("recv_fds")

if hasattr(_socket.socket, "recvmsg"):
    import array

    def recv_fds(sock, bufsize, maxfds, flags=0):
        """ recv_fds(sock, bufsize, maxfds[, flags]) -> (data, list of file
        descriptors, msg_flags, address)

        Receive up to maxfds file descriptors returning the message
        data and a list containing the descriptors.
        """
        # Array of ints
        fds = array.array("i")
        msg, ancdata, flags, addr = sock.recvmsg(bufsize,
            _socket.CMSG_LEN(maxfds * fds.itemsize))
        for cmsg_level, cmsg_type, cmsg_data in ancdata:
            if (cmsg_level == _socket.SOL_SOCKET and cmsg_type == _socket.SCM_RIGHTS):
                fds.frombytes(cmsg_data[:
                        len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])

        return msg, list(fds), flags, addr
    __all__.append("recv_fds")

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

I think I saw that function name in the Python code as well.

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

ok, but as I said we can provide a "use this unified API for ez mode, get into customization to get maximum performance. Here are the APIs"

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

recv makes no sense for UDP,

why wouldn't an async recv that yields the byte buffers as they arrive be useful ?

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

no idea if they're eager or lazy

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

you can see the valid address types I'm building https://github.com/martinvuyk/forge-tools/blob/main/src/forge_tools/socket/address.mojo

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

I'm still not sure how async in Mojo works (monday is the presentation yay), but I think If you return a Coroutine and await it then it doesn't create an async loop (?) if the function where you await it is not async (??)

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

I'm mostly thinking of building overloads with conditional conformance for the self and anything to do with SockAddr in a way in that it doesn't drive anyone crazy (hopefully lol)

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

btw. the code I'm looking at https://doc.rust-lang.org/src/std/net/tcp.rs.html is quite literally the API I mean that a socket implementation could have, we just build socket one level above that

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

Mojo has compile time parameters for a reason, and theoretically you get the compilation to include only what is used

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

and all OS offer that functionality?

145 replies

MModular

•Created by Dmitry Salin on 7/14/2024 in #community-showcase

io_uring

ooohhhh, nice

145 replies

Gaming

Programming