M
Modularβ€’9mo ago
Jack Clayton

Lightbug HTTP: Mojo web framework

GitHub
GitHub - saviorand/lightbug_http: Simple and fast HTTP framework fo...
Simple and fast HTTP framework for Mojo! πŸ”₯. Contribute to saviorand/lightbug_http development by creating an account on GitHub.
92 Replies
Jack Clayton
Jack ClaytonOPβ€’9mo ago
by @a2svior
alain
alainβ€’9mo ago
hi, I have been experimenting with Lightbug as a serving interface for https://github.com/alainrollejr/mocodes
GitHub
GitHub - alainrollejr/mocodes: Error Correction (De)Coding with Mojo
Error Correction (De)Coding with Mojo. Contribute to alainrollejr/mocodes development by creating an account on GitHub.
alain
alainβ€’9mo ago
A couple of questions which I don't know are worth filing change requests for. First one is I actually needed HttpService's func to be allowed to change self (essentially to keep state over incoming requests). For now solved that through pointers but would be easier if I could just change some self.counter value from func. Second one, my test client sends POST messages of a certain length L (smaller than 4096 bytes). Is there a way for the serving handler only to be called when the full L bytes have been received on the socket ? Mostly that is the case but random occurences happen whereby func() is called on body with unexpected length way smaller than L (requiring me to keep even more local state). Last, my application actually does numerical work on Int8 bytes. So I think if there would be a way that Bytes() could actually be a DTypePointer into Int8 rather than a List of Int8, might be beneficial for my application even though List has this scary "steal_data" functionality
Jack Clayton
Jack ClaytonOPβ€’9mo ago
@a2svior fyi ^
a2svior
a2sviorβ€’9mo ago
@Jack Clayton Thanks a lot for tagging! @alain great questions, I think these are definitely worth making issues for. I'll create some issues today and link here. Then let's continue the discussion on GitHub if that works πŸ‘
benny
bennyβ€’9mo ago
@a2svior you may both be interested in the Bytes implantation in Basalt, in the utils folder
a2svior
a2sviorβ€’9mo ago
Ah perfect, thanks a lot for the rec!! @alain made these three issues, tried to rephrase them based on my understanding. Let me know if I made mistakes somewhere 1. HTTPService's Self should be mutable to maintain state over incoming requests 2. Request handler is called on a body with unexpected length 3. Bytes should be a DTypePointer instead of a List @alain how urgent/critical are these? Are you still able to achieve your goals with Lightbug without them? Thinking which ones I should prioritize
alain
alainβ€’9mo ago
that's awesome @a2svior. Descriptions are capturing the ideas well. (2) is the most critical and urgent one as it actually stops me from being able to use Mocodes in my application and I don't have enough HTTP/sockets knowledge myself to go fix it. Having said that, I took a stab at (1) myself by simply changing the HTTPService trait func(self, ...) to func(inout self,...) but then ran into issues with rValue and mutability in server.mojo which I did not know how to circumvent.
a2svior
a2sviorβ€’9mo ago
Got it. Can you maybe attach a code snippet inside the issue #2 with the service code/request payload you're using so I can try and reproduce the problem on my end? #1 should be fixable, I'll look into it
alain
alainβ€’9mo ago
Done !
a2svior
a2sviorβ€’9mo ago
Thank you, will see if I can fix this today. I'll keep you posted @alain couldn't reproduce yet, asked a couple questions in the issue. If you can let me know the OS you're using that would also help πŸ™ @alain (2) should be good now as long as a Content-Length header is set (which is the case in your code). Will try (1) next
alain
alainβ€’9mo ago
hi @a2svior I confirm (2) is fixed also on my system with those fixes you made. Many thanks !! Hello @a2svior , me again πŸ˜‰ So I have finally found some time to do speed benchmarking with the simplest possible lightbug server that mimicks the behaviour I ultimately need from my lightbug version of the decoder This simple lightbug server just sends back the received packet wrapped in an OK() message. I have a test client that sends packets in a loop to this server, and it measures the elapsed time. What I found is as I increase the size of the packet, with a lightbug server the packet rate decreases rapidly whereas with a flask or fastapi python server that does the same job, the packet rate decreases much less. Again I don't want to just go ahead and create a GIT issue for this as I realise that large binary HTTP packets (up to several 100kByte) were probably not what you had in mind for lightbug. Please advise.
a2svior
a2sviorβ€’9mo ago
@alain in general, performance definitely needs improvement. I suspect that in your particular case this has to do with the fact that there is currently conversion to strings and back going on in various places. I'm actually removing these conversions by default, which should also make it easier to serve binary files like images, see this PR https://github.com/saviorand/lightbug_http/pull/43 . I haven't benchmarked it yet though, maybe we can try it out with your test code? In this (general) performance improvement issue someone also posted nice flame graphs from perf for Lightbug, those were running an older Lightbug version though https://github.com/saviorand/lightbug_http/issues/6 . Definitely interested to try and improve performance for your case. I'm a newbie in the performance world though, would appreciate any tips/suggestions
a2svior
a2sviorβ€’9mo ago
Another issue is the practical absence of async (since there's no runtime for async functions) in Mojo, this is also a drag on performance. But in your case I think as long as we remove redundant conversions and re-assignments of variables and boil it down to a minimum Mojo layer + external C calls to socket apis we should be able to get close to flask (or better) in terms of performance
alain
alainβ€’9mo ago
sounds good. I am happy to post my test client code eg in a github issue that exists or one that you create for it, let me know
a2svior
a2sviorβ€’9mo ago
Created an issue here, can you share the code? So I can try on my end. Thanks a lot, really appreciate this info
a2svior
a2sviorβ€’9mo ago
GitHub
Significant slowdown with large payloads Β· Issue #45 Β· saviorand/li...
Issue raised by the author of https://github.com/alainrollejr/mocodes Currently, the performance rapidly deteriorates with the increase in packet size and request/response payloads. Compared to Pyt...
a2svior
a2sviorβ€’9mo ago
@alain I also saw in your repo gRPC support is something that could be interesting? I might create an issue for that as well
alain
alainβ€’8mo ago
I would LOVE grpc support over plain HTTP every day, if that is within your roadmap ideas that would be fantastic. The way I see it the biggest impediment to Mojo adoption is that it is not taking care of IO. Wonderful if you can demonstrate a fast algorithm on a file or data in RAM. Now how do we get that into a microservices oriented ecosystem because no, we don't all want to go rewrite all of the existing code in C, C++ etc in pure mojo. Real programs act on real data coming from the external world and send the result back to the real world, preferably at the same whopping speed of the algorithm itself.
a2svior
a2sviorβ€’8mo ago
Yes, I'm definitely interested in gRPC support as well. Haven't thought through the logistics yet. Maybe a separate library makes more sense, like lightbug_grpc . To keep things separate
NobinKhan
NobinKhanβ€’8mo ago
@a2svior Instead of directly building it as a gRPC-specific library, It will be better if we structure it as a general RPC (Remote Procedure Call) library. The gRPC implementation can then be built on top of this general RPC framework. Also another like Cap’n Proto can be built on top of general rpc.
alain
alainβ€’8mo ago
@a2svior I updated the ticket with test client and server code and corresponding results obtained on my machine. Happy hunting !
a2svior
a2sviorβ€’8mo ago
@alain thanks! Will give it a go and keep you posted if I can improve something @NobinKhan good point, let me know if you wanna collaborate on this, we can coordinate over DMs
NobinKhan
NobinKhanβ€’8mo ago
Thank you so much. I am interested. It will be a great opportunity for me to learning new things and gain some skills.
alain
alainβ€’8mo ago
hi @a2svior I was wondering whether you have made any progress on the speedup exercise for large message sizes ? Are you close to a solution ? If not, I may have to fork and hack/slash myself in order to meet my project deadline
a2svior
a2sviorβ€’8mo ago
We managed to achieve better performance on this PR https://github.com/saviorand/lightbug_http/pull/50, @toasty helped a lot with that. But now I'm getting some issues with running your test, I think I need to refactor request processing logic. If you can try pulling this and have any ideas on how to debug the errors we're getting would really help! The only thing is today's hightly of Mojo broke some things again, if you can maybe use yesterday's nightly version Let me know if you encounter any issues running this It's still slow on the largest payload in your test , but I have an idea on how to fix that. Will try today
alain
alainβ€’8mo ago
Hi @a2svior I myself stay on stable builds rather than nightly to avoid unexpected time lost any given day. Eg now I am on v24.3.0. Are you saying your fixes/improvements won't be compatible with that Mojo version ?
a2svior
a2sviorβ€’8mo ago
Haven't tested it on a stable build yet, I'm developing on nightly @alain looks like a release is dropping today so the changes from nightly should be incorporated into the stable build
a2svior
a2sviorβ€’8mo ago
Tested this with 24.4, should work. But I'm still debugging to make your test work, right now breaks after some point. If you have any idea on how to fix let me know https://github.com/saviorand/lightbug_http/pull/50
alain
alainβ€’8mo ago
okay @a2svior as soon as I have upgraded to 24.4 I will give it a go !
ModularBot
ModularBotβ€’8mo ago
Congrats @alain, you just advanced to level 5!
alain
alainβ€’8mo ago
I've just given it a try but it seems to crash real quick with my test client indeed. One thing I do notice is that while I am sending "Content-Type': 'application/octet-stream'", the lightbug server reports: "Content-Type: text/plain" and also while I send 1000 byte packet body lightbug server reports "Content-Length: 998". Maybe this is stuff I should comment in the pull request ticket
a2svior
a2sviorβ€’8mo ago
Yup, thanks. Let's continue there!
a2svior
a2sviorβ€’7mo ago
Lightbug has reached 420 stars on Gitihub 😏 🚬 πŸ”₯
No description
rd4com
rd4comβ€’6mo ago
Hello, SSE(server sent event) can be an easy to use alternative to websockets until it gets implemented with async await: https://www.pubnub.com/guides/server-sent-events/ It basically upgrade an http get request into a single-way stream. The server first set the header to "text/event-stream.", then the socket is stored into an array. Whenever the server need to send data, it just write to the socket πŸ‘ On the browser-side, a simple js callback is set for whenever an event is received. This is particularly useful for 2D games in the browser, but also for your AI ui's ! The advantage is that it has way less complexity than websockets. If there are 10 visitors, it is as easy as looping an array and sending data (usually json). Because the socket stays open, it can be used as an session too. (ping to @a2svior)
PubNub
What are Server-Sent Events (SSE)?
Read the Real-time communication API Blog now.
a2svior
a2sviorβ€’6mo ago
This is great, thanks a lot! Actually this might be a good idea to implement first before Websockets
rd4com
rd4comβ€’6mo ago
There is a great test suite to test websocket implementations: https://github.com/crossbario/autobahn-testsuite/ The list of projects and users that used it is impressive πŸ‘ (ping to @a2svior)
GitHub
GitHub - crossbario/autobahn-testsuite: Autobahn WebSocket protocol...
Autobahn WebSocket protocol testsuite. Contribute to crossbario/autobahn-testsuite development by creating an account on GitHub.
a2svior
a2sviorβ€’6mo ago
Looks cool, will check it out. By the way, if you're interested in contributing to a SSE/websocket implementation let me know, happy to discuss!
rd4com
rd4comβ€’6mo ago
Working on it πŸ”₯ ! already have the connection upgrade to websocket and the ability to receive messages of all sizes ! It is quite difficult to implement all the features (fragments) and many things could raise. Let's focus on an example that can do receive and send. We might need to get on an audio conference and adapt it to lightbug πŸ‘ Documentation: https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#the_websocket_handshake
Would you mind if i PR a websocket.mojo to lightbug in a work_in_progress folder? (That way you can scavage the example and integrate to lightbug)
a2svior
a2sviorβ€’6mo ago
Definitely!! Also happy to get on an audio conference. Maybe let's do the PR first, I'll try my best to integrate, and then we can coordinate/talk to resolve any remaining questions
rd4com
rd4comβ€’6mo ago
πŸ₯³ here is the commit: https://github.com/saviorand/lightbug_http/pull/54 (send receive upgrade)
a2svior
a2sviorβ€’4mo ago
Lightbug 0.1.3 Release just dropped! Featuring performance and stability improvements and a new installation workflow -- you can now add Lightbug as a dependency to your mojoproject.toml and import Lightbug with Magic to use in your projects. Check out the README for details on how to try it out: https://github.com/saviorand/lightbug_http Lightbug 0.1.4 Release just dropped! Headers are much more ergonomic in Lightbug 0.1.4 thanks to @bgreni 's contribution! There are now three options for specifying the headers that are accepted as input to HTTPRequest or HTTPResponse: 1. Assigning to headers directly:
var res = HTTPResponse()
res.headers["Connection"] = "keep-alive"
res.headers["Content-Type"] = "application/json"
var res = HTTPResponse()
res.headers["Connection"] = "keep-alive"
res.headers["Content-Type"] = "application/json"
2. Passing one or more instances of the Header struct to Headers:
var header = Headers(Header("Connection", "keep-alive"), Header("Content-Type", "application/json")
var res = HTTPResponse(headers)
var header = Headers(Header("Connection", "keep-alive"), Header("Content-Type", "application/json")
var res = HTTPResponse(headers)
3. Using the parse_raw method on Headers:
var headers_str = bytes(
"""GET /index.html HTTP/1.1\r\nContent-Type: application/json\r\nContent-Length: 1234\r\nConnection: keep-alive\r\n\r\n"""
)
var header = Headers()
var b = Bytes(headers_str)
var reader = ByteReader(b^)
var method: String
var protocol: String
var uri: String
method, uri, protocol = header.parse_raw(reader)
var headers_str = bytes(
"""GET /index.html HTTP/1.1\r\nContent-Type: application/json\r\nContent-Length: 1234\r\nConnection: keep-alive\r\n\r\n"""
)
var header = Headers()
var b = Bytes(headers_str)
var reader = ByteReader(b^)
var method: String
var protocol: String
var uri: String
method, uri, protocol = header.parse_raw(reader)
The headers can then be accessed as header["Content-Type"], "text/html" The codebase is also much more Pythonic now with refactors from @bgreni , with more use of dunder methods and direct string operations.
ModularBot
ModularBotβ€’4mo ago
Congrats @a2svior, you just advanced to level 7!
Peter Homola
Peter Homolaβ€’4mo ago
Do you have some benchmarks?
a2svior
a2sviorβ€’4mo ago
@Peter Homola yup, some of the latest ones were posted by @bgreni here https://github.com/saviorand/lightbug_http/pull/61#issuecomment-2362104634
GitHub
Refactor header parsing and data structure by bgreni Β· Pull Request...
@saviorand I figured I might post a draft PR for this so I can see what you think about my approach before I get too much deeper into applying the changes and doing more thorough unit testing Main ...
eggsquad
eggsquadβ€’4mo ago
FYI I was running it on an M3 chip
Peter Homola
Peter Homolaβ€’4mo ago
Do you plan on having some sort of templates? I wrote a PoC, can be seen here: https://arax.ee/mojo I’d be interested in having something similar to Go.
a2svior
a2sviorβ€’4mo ago
Do you mean for HTML specifically or something general-purpose like https://pkg.go.dev/text/template ? For HTML I have some future plans but this will be in a separate library called lightbug_webthat will build on lightbug_http
Peter Homola
Peter Homolaβ€’4mo ago
HTML given the context.
Manuel Saelices
Manuel Saelicesβ€’4mo ago
I'm starting to implement a mojo-websockets package. Totally WIP but my intention is to roughly conform the python-websockets one, first starting with the sync version, and later the async one
a2svior
a2sviorβ€’4mo ago
@msaelices we also have an open PR with @rd4com to add websockets to lightbug, in case you'd like to take a look: https://github.com/saviorand/lightbug_http/pull/57
Manuel Saelices
Manuel Saelicesβ€’4mo ago
wow! I did not know that PR! Will take a look. Thanks!
a2svior
a2sviorβ€’4mo ago
Lightbug 0.1.5 Release just dropped! The most important contribution is by @bgreni - the HTTP client now follows redirects. We've also reorganized the code quite a bit and removed the client and server implementations that were calling into Python for socket interactions. It's all Mojo now :mojo: https://github.com/saviorand/lightbug_http/releases/tag/v0.1.5
GitHub
Release v0.1.5 Β· saviorand/lightbug_http
What's Changed Fix import statement in docs by @cosenal in #63 Follow redirect responses by @bgreni in #60 Use Mojo test runner by @bgreni in #64 Split http module into multiple files by @bgre...
toasty
toastyβ€’4mo ago
Great job with the speed improvements! @a2svior @bgreni
alain
alainβ€’2mo ago
hi @a2svior , I am curious how you guys will deal with mojo 24.6 and the symbol clash on "write", I think that should hit your fn write() from lightbug's libc.mojo ...
a2svior
a2sviorβ€’2mo ago
Hi, hmm no idea, didn't look into it yet, going to do that soon probably, along with other 24.6 updates. If you solve this somehow let me know!
alain
alainβ€’2mo ago
hi, fyi this is how I solved it in 24.6 making use of mojo FileDescriptor and Span, after a life saving hint from @Owen Hilyard and then some searching in mojo stdlib open source code to find examples on how to work with Span: fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int: var fildes_mojo = FileDescriptor(int(fildes)) var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)
fildes_mojo.write_bytes(byte_span) return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written As per the comment on last line, I'm not super happy with this as I lost the functionality of returning the amount of bytes actually written. But I verified the functionality is otherwise identical.
Darkmatter
Darkmatterβ€’2mo ago
The ``` needs to be on its own line.
alain
alainβ€’2mo ago
God I am so bad at these editing tricks πŸ™‚ πŸ™‚
Darkmatter
Darkmatterβ€’2mo ago
fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int:

var fildes_mojo = FileDescriptor(int(fildes))
var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)

fildes_mojo.write_bytes(byte_span)
return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written
fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int:

var fildes_mojo = FileDescriptor(int(fildes))
var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)

fildes_mojo.write_bytes(byte_span)
return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written
https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline
a2svior
a2sviorβ€’4w ago
Lightbug 0.1.6 Release just dropped! Featuring many great improvements from @eggsquad and cookie support by @robin-schoch We also support Mojo 24.6 now πŸ˜„ The full changelog, same as on Github: - Keep persistent connections in client by @bgreni in #69 - add script for server benchmarking by @bgreni in #71 - Add support for chunked transfer encoding by @bgreni in #70 - Feature/47 cookie support by @robin-schoch in #74 - add integration test by @bgreni in #72 - Allow mutation in server-side service implementation by @bgreni in #76 - Catch exceptions from service handler and return internal error by @bgreni in #77 - bump to 24 6 by @saviorand in #78 https://github.com/saviorand/lightbug_http/releases/tag/v0.1.6
GitHub
Release v0.1.6 Β· saviorand/lightbug_http
What's Changed Keep persistent connections in client by @bgreni in #69 add script for server benchmarking by @bgreni in #71 Add support for chunked transfer encoding by @bgreni in #70 Feature/...
a2svior
a2sviorβ€’3w ago
There were a couple issues with v0.1.6 so we shipped a hotfix release 0.1.7. Don't forget to update Lightbug's version in the dependencies section of your mojoproject.toml ! https://github.com/saviorand/lightbug_http/releases/tag/v0.1.7
GitHub
Release v0.1.7 Β· saviorand/lightbug_http
This is a hotfix release What's Changed fix recv hanging on chunked transfer by @bgreni in #79 fix small_time version in the recipe.yaml Full Changelog: v0.1.6...v0.1.7
a2svior
a2sviorβ€’5d ago
Lightbug 0.1.9 Release just dropped! This time with some bangers -- including a refactor of how the socket works (it's a struct now) + basic UDP support, both by @toasty ! Full changelog: - Introduce Socket and refactor connection caching by @thatstoasty in #86 - UDP Socket support by @thatstoasty in #87 https://github.com/Lightbug-HQ/lightbug_http/releases/tag/v0.1.9
GitHub
Release v0.1.9 Β· Lightbug-HQ/lightbug_http
What's Changed Introduce Socket and refactor connection caching by @thatstoasty in #86 UDP Socket support by @thatstoasty in #87 Full Changelog: v0.1.8...v0.1.9
carlcaulkett
carlcaulkettβ€’4d ago
I've just tried to install a freshly init'd project, with the following mojoproject.toml...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/modular-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/modular-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
But, when I run mojo install, I get the error message: Cannot solve the request because of: No candidates were found for lightbug_http >=0.1.9.. Any idea what the problem is? UPDATE: I should also add that even if I change the [dependencies] to lightbug_http = ">= 0.1.8" or even `lightbug_http = ">= 0.1.7", I get similar errors...
toasty
toastyβ€’4d ago
I don’t think it’s in the modular-community channel yet. But it is in the mojo-community one
a2svior
a2sviorβ€’4d ago
Yup, the PR to add Lightbug to modular-community is pending for now
carlcaulkett
carlcaulkettβ€’4d ago
Good catch! I changed my mojoproject.toml to...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
But I get the same error! @a2svior If I change the dependency to lightbug_http = ">=0.1.8", it installs. Does this mean that the 0.1.9 release is not yet in the mojo-community, as @toasty suggested?
a2svior
a2sviorβ€’4d ago
Hmm when I check here it looks like 0.1.9 wasn't published πŸ€” https://prefix.dev/channels/mojo-community/packages/lightbug_http Looks like there was an error in the publishing CI job, but the job didn't fail for some reason, so I assumed everything was fine https://github.com/Lightbug-HQ/lightbug_http/actions/runs/12835368948/job/35794658728#step:3:1 Will fix that first thing tomorrow. Thanks for the catch! But hopefully you can test with 0.1.8 for now, unless you need UDP πŸ™‚
carlcaulkett
carlcaulkettβ€’3d ago
Cool! At least everyone know what the score is πŸ˜ƒ I'll begin testing in earnest tomorrow morning. I can live without UDP for the moment (~ pauses to look up what UDP stands for ~) πŸ˜‰ Just to confirm, the project installs with 0.1.9 now. Thanks πŸ™πŸ½
a2svior
a2sviorβ€’3d ago
Yup, just fixed the issue 20 minutes ago! Looks like a bug in rattler-build. That's what we get for using their binary from the latest release, haha. Pinned it to a version from a couple weeks ago, now it works. On the upside, was able to report the problem to maintainers so hopefully they can resolve it before it affects more people: https://github.com/prefix-dev/rattler-build/issues/1354
carlcaulkett
carlcaulkettβ€’3d ago
While you're here, I'm just looking at the README.md and I'm getting confused as to first steps to get something up and running. I have an empty test project with just the magic.lock and the mojoproject.toml files. The README talks about "Add your handler in lightbug.πŸ”₯"... does this mean adding to the lightbug.:fire: that's in the lightbug_http folder, or to a copy in my own project folder? The you mentions "For example, to make a Printer service that prints some details about the request to console:" and give some mojo code for a printer service, followed by, in step 6, some more mojo code for a server. Do these pieces of code go in separate .mojo units and are they run separately? Sorry if I seem dumb here! There's probably just one vital thing that I'm missing here, and I'm sure that once I've got a test example working, I'll be able to proceed...
carlcaulkett
carlcaulkettβ€’3d ago
Panic over! with the help of this https://hackernoon.com/lightbug-the-first-mojo-http-framework, I got the default page showing...
No description
carlcaulkett
carlcaulkettβ€’3d ago
Presumable the idea is for my own project, to copy the lightbug.πŸ”₯ file to my own folder and then build an app from there...
a2svior
a2sviorβ€’3d ago
Glad you got it working! Actually you can use Lightbug's primitives, like Server, HTTPRequest, HTTPResponse, and the HTTPService trait, from anywhere in your code. lightbug.πŸ”₯ is just an example. So as long as you define some services with needed functionality somewhere in your code and start a server, e.g. with server.listen_and_serve(), it can be in any file in your repo
carlcaulkett
carlcaulkettβ€’3d ago
tbh, your instructions in the hackernoon article were a little easier than the github README.md to follow, the only problem is that some of the info, I'm guessing, is outdated in this brave new world of Magic we are in πŸ˜‰
a2svior
a2sviorβ€’3d ago
Ah, true. I'll check out the article once more, will maybe borrow something for the README. It's been a while now πŸ˜…
carlcaulkett
carlcaulkettβ€’3d ago
I've just created this test client.mojo based on your example code...
from lightbug_http import *

@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)

var body = req.body_raw
print("Request Body: ", to_string(body))

return OK(body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
from lightbug_http import *

@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)

var body = req.body_raw
print("Request Body: ", to_string(body))

return OK(body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
and I ran it with magic run mojo client.mojo, but in the browser it comes up with Failed to process request which means that the server code is running but presumably has failed in the above code, somewhere. Is there an easy way of debugging this code? Doh! That's what the print statements are for πŸ˜‰
For each run, I'm seeing...
Request URI: /
Request protocol: HTTP/1.1
Request method: GET
Request URI: /
Request protocol: HTTP/1.1
Request method: GET
which suggests it's failing on the...
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
line...
toasty
toastyβ€’3d ago
Can you try printing the request object? HTTPRequest implements Writable so you should be able to print it and see what headers you received Alternatively, Headers is also writable, so you can print req.headers as well
carlcaulkett
carlcaulkettβ€’3d ago
This is the contents of req...
Request: GET / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
Request: GET / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
and req.headers...
Request Headers: host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
Request Headers: host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
The code is failing on...
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
because headers does not contain a dict entry of `"content-type"!
a2svior
a2sviorβ€’3d ago
Oh wow, does it work if you comment this out?
carlcaulkett
carlcaulkettβ€’3d ago
Yes it does!
a2svior
a2sviorβ€’3d ago
Okay, so here the Printer service raises an exception at that point when a non-existent key is being accessed, and this "Failed to process request" is how we log these kinds of service errors for now. We might improve that in the future e.g by making the HTTPService return either a response or an error. For now I would suggest to put things that could throw into a try/except block so you could catch an process the errors in your handler when they happen
carlcaulkett
carlcaulkettβ€’3d ago
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.body("white", "", "", "", "", "", "black")
html.title("Test")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.end_body()
html.end_html()
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.body("white", "", "", "", "", "", "black")
html.title("Test")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.end_body()
html.end_html()
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
which gives...
No description
carlcaulkett
carlcaulkettβ€’3d ago
. I found some 25 year old Delphi code of mine for dynamically writing HTML, and I've been converting it πŸ˜‰
a2svior
a2sviorβ€’3d ago
Crazy! 😁
carlcaulkett
carlcaulkettβ€’2d ago
Can you think of a way that I can get my test app to dynamically generate images? atm my code has html.image("/images/earlyspring.png") which is generating the HTML code... <img src="/images/earlyspring.png" align="left" border="0"> but the browser is not loading the image fie because, I guess, it has no way of knowing where the file is, because there is no static root directory for the site.
No description
a2svior
a2sviorβ€’2d ago
Did you try out this example from the README? It should be possible to create e.g a static folder of your own, then use Mojo's filesystem functions, e.g read_bytes() to serve a file from that folder on a path. And then you could just refer to that file in your HTML. That's basically how the welcome example works https://github.com/Lightbug-HQ/lightbug_http?tab=readme-ov-file#serving-static-files
carlcaulkett
carlcaulkettβ€’2d ago
I did see that, but it seemed to be slightly different from my case. I'm building the HTML as a long string and then passing it to the result of the Printer func...
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
# html.image("/images/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html() # html contains the <img src="/images/earlyspring.png" align="left" border="0"> tag
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
# html.image("/images/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html() # html contains the <img src="/images/earlyspring.png" align="left" border="0"> tag
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
UPDATE: I think I get it now. The script is called separately for the main body of the HTML and for the image. Is this how my code should look?
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.image("/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html()
return OK(str(html), "text/html")
if uri.path == "/earlyspring.png":
with open("static/earlyspring.png", "r") as f:
image = f.read_bytes()
return OK(image, "image/png")
return NotFound(uri.path)
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.image("/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html()
return OK(str(html), "text/html")
if uri.path == "/earlyspring.png":
with open("static/earlyspring.png", "r") as f:
image = f.read_bytes()
return OK(image, "image/png")
return NotFound(uri.path)
carlcaulkett
carlcaulkettβ€’2d ago
UPDATE 2: Yes! It works! Thank you so much πŸ™πŸ½
No description
a2svior
a2sviorβ€’2d ago
yes, perfect! Sorry, just saw this. Let me know if you have any other questions later πŸ™‚
carlcaulkett
carlcaulkettβ€’2d ago
Thanks! I think that after my initial confusion, my understanding has reached a kind of critical mass, so hopefully I'll be able to work out any questions I have, but, yes, if I get really stuck, I'll give you a shout πŸ“£
a2svior
a2sviorβ€’2d ago
Deal 🀝

Did you find this page helpful?