Lightbug HTTP: Mojo web framework

GitHub
GitHub - saviorand/lightbug_http: Simple and fast HTTP framework fo...
Simple and fast HTTP framework for Mojo! 🔥. Contribute to saviorand/lightbug_http development by creating an account on GitHub.
258 Replies
Jack Clayton
Jack ClaytonOP10mo ago
by @a2svior
alain
alain10mo ago
hi, I have been experimenting with Lightbug as a serving interface for https://github.com/alainrollejr/mocodes
GitHub
GitHub - alainrollejr/mocodes: Error Correction (De)Coding with Mojo
Error Correction (De)Coding with Mojo. Contribute to alainrollejr/mocodes development by creating an account on GitHub.
alain
alain10mo ago
A couple of questions which I don't know are worth filing change requests for. First one is I actually needed HttpService's func to be allowed to change self (essentially to keep state over incoming requests). For now solved that through pointers but would be easier if I could just change some self.counter value from func. Second one, my test client sends POST messages of a certain length L (smaller than 4096 bytes). Is there a way for the serving handler only to be called when the full L bytes have been received on the socket ? Mostly that is the case but random occurences happen whereby func() is called on body with unexpected length way smaller than L (requiring me to keep even more local state). Last, my application actually does numerical work on Int8 bytes. So I think if there would be a way that Bytes() could actually be a DTypePointer into Int8 rather than a List of Int8, might be beneficial for my application even though List has this scary "steal_data" functionality
Jack Clayton
Jack ClaytonOP10mo ago
@a2svior fyi ^
a2svior
a2svior10mo ago
@Jack Clayton Thanks a lot for tagging! @alain great questions, I think these are definitely worth making issues for. I'll create some issues today and link here. Then let's continue the discussion on GitHub if that works 👍
benny
benny10mo ago
@a2svior you may both be interested in the Bytes implantation in Basalt, in the utils folder
a2svior
a2svior10mo ago
Ah perfect, thanks a lot for the rec!! @alain made these three issues, tried to rephrase them based on my understanding. Let me know if I made mistakes somewhere 1. HTTPService's Self should be mutable to maintain state over incoming requests 2. Request handler is called on a body with unexpected length 3. Bytes should be a DTypePointer instead of a List @alain how urgent/critical are these? Are you still able to achieve your goals with Lightbug without them? Thinking which ones I should prioritize
alain
alain10mo ago
that's awesome @a2svior. Descriptions are capturing the ideas well. (2) is the most critical and urgent one as it actually stops me from being able to use Mocodes in my application and I don't have enough HTTP/sockets knowledge myself to go fix it. Having said that, I took a stab at (1) myself by simply changing the HTTPService trait func(self, ...) to func(inout self,...) but then ran into issues with rValue and mutability in server.mojo which I did not know how to circumvent.
a2svior
a2svior10mo ago
Got it. Can you maybe attach a code snippet inside the issue #2 with the service code/request payload you're using so I can try and reproduce the problem on my end? #1 should be fixable, I'll look into it
alain
alain10mo ago
Done !
a2svior
a2svior10mo ago
Thank you, will see if I can fix this today. I'll keep you posted @alain couldn't reproduce yet, asked a couple questions in the issue. If you can let me know the OS you're using that would also help 🙏 @alain (2) should be good now as long as a Content-Length header is set (which is the case in your code). Will try (1) next
alain
alain10mo ago
hi @a2svior I confirm (2) is fixed also on my system with those fixes you made. Many thanks !! Hello @a2svior , me again 😉 So I have finally found some time to do speed benchmarking with the simplest possible lightbug server that mimicks the behaviour I ultimately need from my lightbug version of the decoder This simple lightbug server just sends back the received packet wrapped in an OK() message. I have a test client that sends packets in a loop to this server, and it measures the elapsed time. What I found is as I increase the size of the packet, with a lightbug server the packet rate decreases rapidly whereas with a flask or fastapi python server that does the same job, the packet rate decreases much less. Again I don't want to just go ahead and create a GIT issue for this as I realise that large binary HTTP packets (up to several 100kByte) were probably not what you had in mind for lightbug. Please advise.
a2svior
a2svior10mo ago
@alain in general, performance definitely needs improvement. I suspect that in your particular case this has to do with the fact that there is currently conversion to strings and back going on in various places. I'm actually removing these conversions by default, which should also make it easier to serve binary files like images, see this PR https://github.com/saviorand/lightbug_http/pull/43 . I haven't benchmarked it yet though, maybe we can try it out with your test code? In this (general) performance improvement issue someone also posted nice flame graphs from perf for Lightbug, those were running an older Lightbug version though https://github.com/saviorand/lightbug_http/issues/6 . Definitely interested to try and improve performance for your case. I'm a newbie in the performance world though, would appreciate any tips/suggestions
a2svior
a2svior10mo ago
Another issue is the practical absence of async (since there's no runtime for async functions) in Mojo, this is also a drag on performance. But in your case I think as long as we remove redundant conversions and re-assignments of variables and boil it down to a minimum Mojo layer + external C calls to socket apis we should be able to get close to flask (or better) in terms of performance
alain
alain10mo ago
sounds good. I am happy to post my test client code eg in a github issue that exists or one that you create for it, let me know
a2svior
a2svior10mo ago
Created an issue here, can you share the code? So I can try on my end. Thanks a lot, really appreciate this info
a2svior
a2svior10mo ago
GitHub
Significant slowdown with large payloads · Issue #45 · saviorand/li...
Issue raised by the author of https://github.com/alainrollejr/mocodes Currently, the performance rapidly deteriorates with the increase in packet size and request/response payloads. Compared to Pyt...
a2svior
a2svior10mo ago
@alain I also saw in your repo gRPC support is something that could be interesting? I might create an issue for that as well
alain
alain10mo ago
I would LOVE grpc support over plain HTTP every day, if that is within your roadmap ideas that would be fantastic. The way I see it the biggest impediment to Mojo adoption is that it is not taking care of IO. Wonderful if you can demonstrate a fast algorithm on a file or data in RAM. Now how do we get that into a microservices oriented ecosystem because no, we don't all want to go rewrite all of the existing code in C, C++ etc in pure mojo. Real programs act on real data coming from the external world and send the result back to the real world, preferably at the same whopping speed of the algorithm itself.
a2svior
a2svior10mo ago
Yes, I'm definitely interested in gRPC support as well. Haven't thought through the logistics yet. Maybe a separate library makes more sense, like lightbug_grpc . To keep things separate
NobinKhan
NobinKhan10mo ago
@a2svior Instead of directly building it as a gRPC-specific library, It will be better if we structure it as a general RPC (Remote Procedure Call) library. The gRPC implementation can then be built on top of this general RPC framework. Also another like Cap’n Proto can be built on top of general rpc.
alain
alain10mo ago
@a2svior I updated the ticket with test client and server code and corresponding results obtained on my machine. Happy hunting !
a2svior
a2svior10mo ago
@alain thanks! Will give it a go and keep you posted if I can improve something @NobinKhan good point, let me know if you wanna collaborate on this, we can coordinate over DMs
NobinKhan
NobinKhan10mo ago
Thank you so much. I am interested. It will be a great opportunity for me to learning new things and gain some skills.
alain
alain9mo ago
hi @a2svior I was wondering whether you have made any progress on the speedup exercise for large message sizes ? Are you close to a solution ? If not, I may have to fork and hack/slash myself in order to meet my project deadline
a2svior
a2svior9mo ago
We managed to achieve better performance on this PR https://github.com/saviorand/lightbug_http/pull/50, @toasty helped a lot with that. But now I'm getting some issues with running your test, I think I need to refactor request processing logic. If you can try pulling this and have any ideas on how to debug the errors we're getting would really help! The only thing is today's hightly of Mojo broke some things again, if you can maybe use yesterday's nightly version Let me know if you encounter any issues running this It's still slow on the largest payload in your test , but I have an idea on how to fix that. Will try today
alain
alain9mo ago
Hi @a2svior I myself stay on stable builds rather than nightly to avoid unexpected time lost any given day. Eg now I am on v24.3.0. Are you saying your fixes/improvements won't be compatible with that Mojo version ?
a2svior
a2svior9mo ago
Haven't tested it on a stable build yet, I'm developing on nightly @alain looks like a release is dropping today so the changes from nightly should be incorporated into the stable build
a2svior
a2svior9mo ago
Tested this with 24.4, should work. But I'm still debugging to make your test work, right now breaks after some point. If you have any idea on how to fix let me know https://github.com/saviorand/lightbug_http/pull/50
alain
alain9mo ago
okay @a2svior as soon as I have upgraded to 24.4 I will give it a go !
ModularBot
ModularBot9mo ago
Congrats @alain, you just advanced to level 5!
alain
alain9mo ago
I've just given it a try but it seems to crash real quick with my test client indeed. One thing I do notice is that while I am sending "Content-Type': 'application/octet-stream'", the lightbug server reports: "Content-Type: text/plain" and also while I send 1000 byte packet body lightbug server reports "Content-Length: 998". Maybe this is stuff I should comment in the pull request ticket
a2svior
a2svior9mo ago
Yup, thanks. Let's continue there!
a2svior
a2svior8mo ago
Lightbug has reached 420 stars on Gitihub 😏 🚬 🔥
No description
rd4com
rd4com7mo ago
Hello, SSE(server sent event) can be an easy to use alternative to websockets until it gets implemented with async await: https://www.pubnub.com/guides/server-sent-events/ It basically upgrade an http get request into a single-way stream. The server first set the header to "text/event-stream.", then the socket is stored into an array. Whenever the server need to send data, it just write to the socket 👍 On the browser-side, a simple js callback is set for whenever an event is received. This is particularly useful for 2D games in the browser, but also for your AI ui's ! The advantage is that it has way less complexity than websockets. If there are 10 visitors, it is as easy as looping an array and sending data (usually json). Because the socket stays open, it can be used as an session too. (ping to @a2svior)
PubNub
What are Server-Sent Events (SSE)?
Read the Real-time communication API Blog now.
a2svior
a2svior7mo ago
This is great, thanks a lot! Actually this might be a good idea to implement first before Websockets
rd4com
rd4com7mo ago
There is a great test suite to test websocket implementations: https://github.com/crossbario/autobahn-testsuite/ The list of projects and users that used it is impressive 👍 (ping to @a2svior)
GitHub
GitHub - crossbario/autobahn-testsuite: Autobahn WebSocket protocol...
Autobahn WebSocket protocol testsuite. Contribute to crossbario/autobahn-testsuite development by creating an account on GitHub.
a2svior
a2svior7mo ago
Looks cool, will check it out. By the way, if you're interested in contributing to a SSE/websocket implementation let me know, happy to discuss!
rd4com
rd4com7mo ago
Working on it 🔥 ! already have the connection upgrade to websocket and the ability to receive messages of all sizes ! It is quite difficult to implement all the features (fragments) and many things could raise. Let's focus on an example that can do receive and send. We might need to get on an audio conference and adapt it to lightbug 👍 Documentation: https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#the_websocket_handshake
Would you mind if i PR a websocket.mojo to lightbug in a work_in_progress folder? (That way you can scavage the example and integrate to lightbug)
a2svior
a2svior7mo ago
Definitely!! Also happy to get on an audio conference. Maybe let's do the PR first, I'll try my best to integrate, and then we can coordinate/talk to resolve any remaining questions
rd4com
rd4com7mo ago
🥳 here is the commit: https://github.com/saviorand/lightbug_http/pull/54 (send receive upgrade)
a2svior
a2svior6mo ago
Lightbug 0.1.3 Release just dropped! Featuring performance and stability improvements and a new installation workflow -- you can now add Lightbug as a dependency to your mojoproject.toml and import Lightbug with Magic to use in your projects. Check out the README for details on how to try it out: https://github.com/saviorand/lightbug_http Lightbug 0.1.4 Release just dropped! Headers are much more ergonomic in Lightbug 0.1.4 thanks to @bgreni 's contribution! There are now three options for specifying the headers that are accepted as input to HTTPRequest or HTTPResponse: 1. Assigning to headers directly:
var res = HTTPResponse()
res.headers["Connection"] = "keep-alive"
res.headers["Content-Type"] = "application/json"
var res = HTTPResponse()
res.headers["Connection"] = "keep-alive"
res.headers["Content-Type"] = "application/json"
2. Passing one or more instances of the Header struct to Headers:
var header = Headers(Header("Connection", "keep-alive"), Header("Content-Type", "application/json")
var res = HTTPResponse(headers)
var header = Headers(Header("Connection", "keep-alive"), Header("Content-Type", "application/json")
var res = HTTPResponse(headers)
3. Using the parse_raw method on Headers:
var headers_str = bytes(
"""GET /index.html HTTP/1.1\r\nContent-Type: application/json\r\nContent-Length: 1234\r\nConnection: keep-alive\r\n\r\n"""
)
var header = Headers()
var b = Bytes(headers_str)
var reader = ByteReader(b^)
var method: String
var protocol: String
var uri: String
method, uri, protocol = header.parse_raw(reader)
var headers_str = bytes(
"""GET /index.html HTTP/1.1\r\nContent-Type: application/json\r\nContent-Length: 1234\r\nConnection: keep-alive\r\n\r\n"""
)
var header = Headers()
var b = Bytes(headers_str)
var reader = ByteReader(b^)
var method: String
var protocol: String
var uri: String
method, uri, protocol = header.parse_raw(reader)
The headers can then be accessed as header["Content-Type"], "text/html" The codebase is also much more Pythonic now with refactors from @bgreni , with more use of dunder methods and direct string operations.
ModularBot
ModularBot6mo ago
Congrats @a2svior, you just advanced to level 7!
Peter Homola
Peter Homola5mo ago
Do you have some benchmarks?
a2svior
a2svior5mo ago
@Peter Homola yup, some of the latest ones were posted by @bgreni here https://github.com/saviorand/lightbug_http/pull/61#issuecomment-2362104634
GitHub
Refactor header parsing and data structure by bgreni · Pull Request...
@saviorand I figured I might post a draft PR for this so I can see what you think about my approach before I get too much deeper into applying the changes and doing more thorough unit testing Main ...
eggsquad
eggsquad5mo ago
FYI I was running it on an M3 chip
Peter Homola
Peter Homola5mo ago
Do you plan on having some sort of templates? I wrote a PoC, can be seen here: https://arax.ee/mojo I’d be interested in having something similar to Go.
a2svior
a2svior5mo ago
Do you mean for HTML specifically or something general-purpose like https://pkg.go.dev/text/template ? For HTML I have some future plans but this will be in a separate library called lightbug_webthat will build on lightbug_http
Peter Homola
Peter Homola5mo ago
HTML given the context.
Manuel Saelices
Manuel Saelices5mo ago
I'm starting to implement a mojo-websockets package. Totally WIP but my intention is to roughly conform the python-websockets one, first starting with the sync version, and later the async one
a2svior
a2svior5mo ago
@msaelices we also have an open PR with @rd4com to add websockets to lightbug, in case you'd like to take a look: https://github.com/saviorand/lightbug_http/pull/57
Manuel Saelices
Manuel Saelices5mo ago
wow! I did not know that PR! Will take a look. Thanks!
a2svior
a2svior5mo ago
Lightbug 0.1.5 Release just dropped! The most important contribution is by @bgreni - the HTTP client now follows redirects. We've also reorganized the code quite a bit and removed the client and server implementations that were calling into Python for socket interactions. It's all Mojo now :mojo: https://github.com/saviorand/lightbug_http/releases/tag/v0.1.5
GitHub
Release v0.1.5 · saviorand/lightbug_http
What's Changed Fix import statement in docs by @cosenal in #63 Follow redirect responses by @bgreni in #60 Use Mojo test runner by @bgreni in #64 Split http module into multiple files by @bgre...
toasty
toasty5mo ago
Great job with the speed improvements! @a2svior @bgreni
alain
alain3mo ago
hi @a2svior , I am curious how you guys will deal with mojo 24.6 and the symbol clash on "write", I think that should hit your fn write() from lightbug's libc.mojo ...
a2svior
a2svior3mo ago
Hi, hmm no idea, didn't look into it yet, going to do that soon probably, along with other 24.6 updates. If you solve this somehow let me know!
alain
alain3mo ago
hi, fyi this is how I solved it in 24.6 making use of mojo FileDescriptor and Span, after a life saving hint from @Owen Hilyard and then some searching in mojo stdlib open source code to find examples on how to work with Span: fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int: var fildes_mojo = FileDescriptor(int(fildes)) var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)
fildes_mojo.write_bytes(byte_span) return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written As per the comment on last line, I'm not super happy with this as I lost the functionality of returning the amount of bytes actually written. But I verified the functionality is otherwise identical.
Darkmatter
Darkmatter3mo ago
The ``` needs to be on its own line.
alain
alain3mo ago
God I am so bad at these editing tricks 🙂 🙂
Darkmatter
Darkmatter3mo ago
fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int:

var fildes_mojo = FileDescriptor(int(fildes))
var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)

fildes_mojo.write_bytes(byte_span)
return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written
fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int:

var fildes_mojo = FileDescriptor(int(fildes))
var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)

fildes_mojo.write_bytes(byte_span)
return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written
https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline
a2svior
a2svior2mo ago
Lightbug 0.1.6 Release just dropped! Featuring many great improvements from @eggsquad and cookie support by @robin-schoch We also support Mojo 24.6 now 😄 The full changelog, same as on Github: - Keep persistent connections in client by @bgreni in #69 - add script for server benchmarking by @bgreni in #71 - Add support for chunked transfer encoding by @bgreni in #70 - Feature/47 cookie support by @robin-schoch in #74 - add integration test by @bgreni in #72 - Allow mutation in server-side service implementation by @bgreni in #76 - Catch exceptions from service handler and return internal error by @bgreni in #77 - bump to 24 6 by @saviorand in #78 https://github.com/saviorand/lightbug_http/releases/tag/v0.1.6
GitHub
Release v0.1.6 · saviorand/lightbug_http
What's Changed Keep persistent connections in client by @bgreni in #69 add script for server benchmarking by @bgreni in #71 Add support for chunked transfer encoding by @bgreni in #70 Feature/...
a2svior
a2svior2mo ago
There were a couple issues with v0.1.6 so we shipped a hotfix release 0.1.7. Don't forget to update Lightbug's version in the dependencies section of your mojoproject.toml ! https://github.com/saviorand/lightbug_http/releases/tag/v0.1.7
GitHub
Release v0.1.7 · saviorand/lightbug_http
This is a hotfix release What's Changed fix recv hanging on chunked transfer by @bgreni in #79 fix small_time version in the recipe.yaml Full Changelog: v0.1.6...v0.1.7
a2svior
a2svior2mo ago
Lightbug 0.1.9 Release just dropped! This time with some bangers -- including a refactor of how the socket works (it's a struct now) + basic UDP support, both by @toasty ! Full changelog: - Introduce Socket and refactor connection caching by @thatstoasty in #86 - UDP Socket support by @thatstoasty in #87 https://github.com/Lightbug-HQ/lightbug_http/releases/tag/v0.1.9
GitHub
Release v0.1.9 · Lightbug-HQ/lightbug_http
What's Changed Introduce Socket and refactor connection caching by @thatstoasty in #86 UDP Socket support by @thatstoasty in #87 Full Changelog: v0.1.8...v0.1.9
carlcaulkett
carlcaulkett2mo ago
I've just tried to install a freshly init'd project, with the following mojoproject.toml...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/modular-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/modular-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
But, when I run mojo install, I get the error message: Cannot solve the request because of: No candidates were found for lightbug_http >=0.1.9.. Any idea what the problem is? UPDATE: I should also add that even if I change the [dependencies] to lightbug_http = ">= 0.1.8" or even `lightbug_http = ">= 0.1.7", I get similar errors...
toasty
toasty2mo ago
I don’t think it’s in the modular-community channel yet. But it is in the mojo-community one
a2svior
a2svior2mo ago
Yup, the PR to add Lightbug to modular-community is pending for now
carlcaulkett
carlcaulkett2mo ago
Good catch! I changed my mojoproject.toml to...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"
But I get the same error! @a2svior If I change the dependency to lightbug_http = ">=0.1.8", it installs. Does this mean that the 0.1.9 release is not yet in the mojo-community, as @toasty suggested?
a2svior
a2svior2mo ago
Hmm when I check here it looks like 0.1.9 wasn't published 🤔 https://prefix.dev/channels/mojo-community/packages/lightbug_http Looks like there was an error in the publishing CI job, but the job didn't fail for some reason, so I assumed everything was fine https://github.com/Lightbug-HQ/lightbug_http/actions/runs/12835368948/job/35794658728#step:3:1 Will fix that first thing tomorrow. Thanks for the catch! But hopefully you can test with 0.1.8 for now, unless you need UDP 🙂
carlcaulkett
carlcaulkett2mo ago
Cool! At least everyone know what the score is 😃 I'll begin testing in earnest tomorrow morning. I can live without UDP for the moment (~ pauses to look up what UDP stands for ~) 😉 Just to confirm, the project installs with 0.1.9 now. Thanks 🙏🏽
a2svior
a2svior2mo ago
Yup, just fixed the issue 20 minutes ago! Looks like a bug in rattler-build. That's what we get for using their binary from the latest release, haha. Pinned it to a version from a couple weeks ago, now it works. On the upside, was able to report the problem to maintainers so hopefully they can resolve it before it affects more people: https://github.com/prefix-dev/rattler-build/issues/1354
carlcaulkett
carlcaulkett2mo ago
While you're here, I'm just looking at the README.md and I'm getting confused as to first steps to get something up and running. I have an empty test project with just the magic.lock and the mojoproject.toml files. The README talks about "Add your handler in lightbug.🔥"... does this mean adding to the lightbug.:fire: that's in the lightbug_http folder, or to a copy in my own project folder? The you mentions "For example, to make a Printer service that prints some details about the request to console:" and give some mojo code for a printer service, followed by, in step 6, some more mojo code for a server. Do these pieces of code go in separate .mojo units and are they run separately? Sorry if I seem dumb here! There's probably just one vital thing that I'm missing here, and I'm sure that once I've got a test example working, I'll be able to proceed...
carlcaulkett
carlcaulkett2mo ago
Panic over! with the help of this https://hackernoon.com/lightbug-the-first-mojo-http-framework, I got the default page showing...
No description
carlcaulkett
carlcaulkett2mo ago
Presumable the idea is for my own project, to copy the lightbug.🔥 file to my own folder and then build an app from there...
a2svior
a2svior2mo ago
Glad you got it working! Actually you can use Lightbug's primitives, like Server, HTTPRequest, HTTPResponse, and the HTTPService trait, from anywhere in your code. lightbug.🔥 is just an example. So as long as you define some services with needed functionality somewhere in your code and start a server, e.g. with server.listen_and_serve(), it can be in any file in your repo
carlcaulkett
carlcaulkett2mo ago
tbh, your instructions in the hackernoon article were a little easier than the github README.md to follow, the only problem is that some of the info, I'm guessing, is outdated in this brave new world of Magic we are in 😉
a2svior
a2svior2mo ago
Ah, true. I'll check out the article once more, will maybe borrow something for the README. It's been a while now 😅
carlcaulkett
carlcaulkett2mo ago
I've just created this test client.mojo based on your example code...
from lightbug_http import *

@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)

var body = req.body_raw
print("Request Body: ", to_string(body))

return OK(body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
from lightbug_http import *

@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)

var body = req.body_raw
print("Request Body: ", to_string(body))

return OK(body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
and I ran it with magic run mojo client.mojo, but in the browser it comes up with Failed to process request which means that the server code is running but presumably has failed in the above code, somewhere. Is there an easy way of debugging this code? Doh! That's what the print statements are for 😉
For each run, I'm seeing...
Request URI: /
Request protocol: HTTP/1.1
Request method: GET
Request URI: /
Request protocol: HTTP/1.1
Request method: GET
which suggests it's failing on the...
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
line...
toasty
toasty2mo ago
Can you try printing the request object? HTTPRequest implements Writable so you should be able to print it and see what headers you received Alternatively, Headers is also writable, so you can print req.headers as well
carlcaulkett
carlcaulkett2mo ago
This is the contents of req...
Request: GET / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
Request: GET / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
and req.headers...
Request Headers: host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
Request Headers: host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0
The code is failing on...
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
print(
"Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
)
because headers does not contain a dict entry of `"content-type"!
a2svior
a2svior2mo ago
Oh wow, does it work if you comment this out?
carlcaulkett
carlcaulkett2mo ago
Yes it does!
a2svior
a2svior2mo ago
Okay, so here the Printer service raises an exception at that point when a non-existent key is being accessed, and this "Failed to process request" is how we log these kinds of service errors for now. We might improve that in the future e.g by making the HTTPService return either a response or an error. For now I would suggest to put things that could throw into a try/except block so you could catch an process the errors in your handler when they happen
carlcaulkett
carlcaulkett2mo ago
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.body("white", "", "", "", "", "", "black")
html.title("Test")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.end_body()
html.end_html()
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.body("white", "", "", "", "", "", "black")
html.title("Test")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.end_body()
html.end_html()
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
which gives...
No description
carlcaulkett
carlcaulkett2mo ago
. I found some 25 year old Delphi code of mine for dynamically writing HTML, and I've been converting it 😉
a2svior
a2svior2mo ago
Crazy! 😁
carlcaulkett
carlcaulkett2mo ago
Can you think of a way that I can get my test app to dynamically generate images? atm my code has html.image("/images/earlyspring.png") which is generating the HTML code... <img src="/images/earlyspring.png" align="left" border="0"> but the browser is not loading the image fie because, I guess, it has no way of knowing where the file is, because there is no static root directory for the site.
No description
a2svior
a2svior2mo ago
Did you try out this example from the README? It should be possible to create e.g a static folder of your own, then use Mojo's filesystem functions, e.g read_bytes() to serve a file from that folder on a path. And then you could just refer to that file in your HTML. That's basically how the welcome example works https://github.com/Lightbug-HQ/lightbug_http?tab=readme-ov-file#serving-static-files
carlcaulkett
carlcaulkett2mo ago
I did see that, but it seemed to be slightly different from my case. I'm building the HTML as a long string and then passing it to the result of the Printer func...
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
# html.image("/images/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html() # html contains the <img src="/images/earlyspring.png" align="left" border="0"> tag
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
print("Request URI: ", to_string(uri.request_uri))

var header = req.headers
print("Request protocol: ", req.protocol)
print("Request method: ", req.method)

try:
print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
except e:
print("Request Content-Length: Not found")

var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
# html.image("/images/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html() # html contains the <img src="/images/earlyspring.png" align="left" border="0"> tag
return OK(str(html), "text/html")
# (body)

fn main() raises:
var server = Server()
var handler = Printer()
server.listen_and_serve("0.0.0.0:8080", handler)
UPDATE: I think I get it now. The script is called separately for the main body of the HTML and for the image. Is this how my code should look?
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.image("/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html()
return OK(str(html), "text/html")
if uri.path == "/earlyspring.png":
with open("static/earlyspring.png", "r") as f:
image = f.read_bytes()
return OK(image, "image/png")
return NotFound(uri.path)
@value
struct Printer(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
var body = req.body_raw
print("Request Body: ", to_string(body))
var html = Html()
html.html()
html.head()
html.title("Test")
html.end_head()
html.body("white", "", "", "", "", "", "black")
html.h1("Test Heading 1")
html.h2("Test Heading 2")
html.h3("Test Heading 3")
html.h4("Test Heading 4")
html.image("/earlyspring.png")
html.para(html.lorem())
html.para(html.post_modern())
html.end_body()
html.end_html()
return OK(str(html), "text/html")
if uri.path == "/earlyspring.png":
with open("static/earlyspring.png", "r") as f:
image = f.read_bytes()
return OK(image, "image/png")
return NotFound(uri.path)
carlcaulkett
carlcaulkett2mo ago
UPDATE 2: Yes! It works! Thank you so much 🙏🏽
No description
a2svior
a2svior2mo ago
yes, perfect! Sorry, just saw this. Let me know if you have any other questions later 🙂
carlcaulkett
carlcaulkett2mo ago
Thanks! I think that after my initial confusion, my understanding has reached a kind of critical mass, so hopefully I'll be able to work out any questions I have, but, yes, if I get really stuck, I'll give you a shout 📣
a2svior
a2svior2mo ago
Deal 🤝
carlcaulkett
carlcaulkett2mo ago
Hello! Is there any reason why, if I point mymojoproject.toml at `"https://conda.modular.com/max", my project builds, but if I point it at "https://conda.modular.com/max", my project fails to build with the folllowing errors...
~/code/mojo/ca_web magic run default
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:5:16: error: 'Printer' has no 'HTTPService' member
struct Printer(HTTPService):
^~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:45:16: error: use of unknown declaration 'Server'
var server = Server()
^~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[1 - ERROR][Thu Jan 23 2025 6:54pm (GMT+0000)] (main) [*] took 4s
~/code/mojo/ca_web magic run default
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:5:16: error: 'Printer' has no 'HTTPService' member
struct Printer(HTTPService):
^~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:45:16: error: use of unknown declaration 'Server'
var server = Server()
^~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[1 - ERROR][Thu Jan 23 2025 6:54pm (GMT+0000)] (main) [*] took 4s
a2svior
a2svior2mo ago
Not sure what's the difference between these two Max conda URLs 🤔 They look the same?
carlcaulkett
carlcaulkett2mo ago
Sorry! I forgot to amend one of them 😮 All I know is that if I have my mojoproject.toml set to...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"
but, if I set it to...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max-nightly", # THE ONLY CHANGE
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
"https://conda.modular.com/max-nightly", # THE ONLY CHANGE
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"
It fails to build with those errors posted above...
toasty
toasty2mo ago
Mojo versions become incompatible fairly quickly In my experience it was like every 1-3 nightly versions. So, trying to maintain a nightly version of any lib is quite time consuming.
ModularBot
ModularBot2mo ago
Congrats @toasty, you just advanced to level 9!
Darkmatter
Darkmatter2mo ago
Yup, that's why none of mine are open right now, I can't actually support them. But I need stuff from nightly.
a2svior
a2svior2mo ago
We have a nightly version, but I haven't updated it for a while. Might ship a new one this week, will let you know
toasty
toasty2mo ago
@Owen Hilyard If you're open to sending out invites, I'd love to see what you're working on!
Darkmatter
Darkmatter2mo ago
Do you have 5 servers? Most of my work is distributed DBs, and networking between them. No portability yet, so a CX6, CX7, CX8, or Bluefield is currently min spec NIC for this one. The other libs I have are also probably not super useful, since they assume you've decided that Linux's network stack is too bloated for your app. And I'm not sure two of them would run on anything aside from my version of fedora on a zen 4 CPU.
ModularBot
ModularBot2mo ago
carlcaulkett has been warned
Reason: Zalgo usage
carlcaulkett
carlcaulkett2mo ago
It seems that this is a known bug or inconsistency with nightly⁠... https://discord.com/channels/1087530497313357884/1119100298456215572/1332085767866023987
toasty
toasty2mo ago
At most I have 3 servers I use for my k8s cluster haha. Definitely not in the distributed DB realm outside of work stuff
Darkmatter
Darkmatter2mo ago
And I'm guessing "I own the whole server" is not an attitude you would want. DBs are one of the last applications allowed to do that, so the DB actually tunes the server and shoves everything else onto a single CPU core.
a2svior
a2svior2mo ago
I mean in general it makes sense that some errors pop up, main branch of Lightbug / latest release tag only works on the latest stable version of MAX/Mojo.
carlcaulkett
carlcaulkett2mo ago
I guess I'm going to have to stick with stable until it gets sorted. This didn't help 🤣
No description
carlcaulkett
carlcaulkett2mo ago
@a2svior I'm making a start on CSS...
fn get_page_html(mut self) -> String:
var page = Html()
var style = Style()

style.p().color("purple")
style.h1().color("red")
style.h2().color("yellow")
style.h3().color("green")

page.html_head("lightspeed_http and ca_web test", style)
page.body("grey", "", "", "", "", "", "")
page.set_font("Arial")
page.para("", "datetime")
page.script('let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;')
page.h1("Test Heading 1").h2("Test Heading 2").h3("Test Heading 3").h4("Test Heading 4").h5("Test Heading 5").h6("Test Heading 6")
page.end_font()
page.image("/earlyspring.png")
page.set_font("Arial", 5, "Teal")
page.para(page.lorem())
page.para(page.post_modern())
page.input_text("password", "1234go", 23, 23)
page.input_text("password", "1234go", 23, 23, True)
page.end_font()
page.end_body()
page.end_html()
page.prettify()
return str(page)
fn get_page_html(mut self) -> String:
var page = Html()
var style = Style()

style.p().color("purple")
style.h1().color("red")
style.h2().color("yellow")
style.h3().color("green")

page.html_head("lightspeed_http and ca_web test", style)
page.body("grey", "", "", "", "", "", "")
page.set_font("Arial")
page.para("", "datetime")
page.script('let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;')
page.h1("Test Heading 1").h2("Test Heading 2").h3("Test Heading 3").h4("Test Heading 4").h5("Test Heading 5").h6("Test Heading 6")
page.end_font()
page.image("/earlyspring.png")
page.set_font("Arial", 5, "Teal")
page.para(page.lorem())
page.para(page.post_modern())
page.input_text("password", "1234go", 23, 23)
page.input_text("password", "1234go", 23, 23, True)
page.end_font()
page.end_body()
page.end_html()
page.prettify()
return str(page)
this generates...
No description
a2svior
a2svior5w ago
Beautiful 🤌
carlcaulkett
carlcaulkett5w ago
I'm not claiming that the visuals are anything to write home about, but the fluent nature of the API is coming on quite nicely 😉
a2svior
a2svior5w ago
I'm not a design specialist — nice that it works!
carlcaulkett
carlcaulkett4w ago
Google Fonts FTW 😉
fn get_page_html(mut self) raises -> String:
var page = Html()
var style = Style()

_ = style.p().
font_family("arial").
color(Colors.blueviolet).
background_color(Colors.yellow)
_ = style.h1().
font_family(GoogleFonts.Audiowide).
color(Colors.red).
background_color(Colors.lightblue)
_ = style.h2().
font_family(GoogleFonts.Sofia).
color(Colors.goldenrod).
background_color(Colors.lightgreen)
_ = style.h3().
font_family(GoogleFonts.Trirong).
color(Colors.green).
background_color(Colors.lightcoral)
_ = style.body()
.background_color(Colors.azure)
.font_size(16, FontUnit.PX)
.font_family("Arial, sans-serif")
.color(Colors.darkblue)

_ = page.html_head("lightspeed_http and ca_web test", style)
_ = page.para("", "datetime")
_ = page.script('let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;')
_ = page.h1("Test Heading 1").h2("Test Heading 2").h3("Test Heading 3").h4("Test Heading 4").h5("Test Heading 5").h6("Test Heading 6")
_ = page.image("/earlyspring.png")
_ = page.para(page.lorem())
_ = page.para(page.post_modern())
_ = page.input_text("password", "1234go", 23, 23)
_ = page.input_text("password", "1234go", 23, 23, True)
_ = page.end_html()
_ = page.prettify()
return str(page)
fn get_page_html(mut self) raises -> String:
var page = Html()
var style = Style()

_ = style.p().
font_family("arial").
color(Colors.blueviolet).
background_color(Colors.yellow)
_ = style.h1().
font_family(GoogleFonts.Audiowide).
color(Colors.red).
background_color(Colors.lightblue)
_ = style.h2().
font_family(GoogleFonts.Sofia).
color(Colors.goldenrod).
background_color(Colors.lightgreen)
_ = style.h3().
font_family(GoogleFonts.Trirong).
color(Colors.green).
background_color(Colors.lightcoral)
_ = style.body()
.background_color(Colors.azure)
.font_size(16, FontUnit.PX)
.font_family("Arial, sans-serif")
.color(Colors.darkblue)

_ = page.html_head("lightspeed_http and ca_web test", style)
_ = page.para("", "datetime")
_ = page.script('let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;')
_ = page.h1("Test Heading 1").h2("Test Heading 2").h3("Test Heading 3").h4("Test Heading 4").h5("Test Heading 5").h6("Test Heading 6")
_ = page.image("/earlyspring.png")
_ = page.para(page.lorem())
_ = page.para(page.post_modern())
_ = page.input_text("password", "1234go", 23, 23)
_ = page.input_text("password", "1234go", 23, 23, True)
_ = page.end_html()
_ = page.prettify()
return str(page)
No description
carlcaulkett
carlcaulkett4w ago
Here's the HTML source...
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset='utf-8'>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Audiowide|Sofia|Trirong">
<style>
p {
font-family: arial;
color: blueviolet;
background-color: yellow;
}
h1 {
font-family: Audiowide;
color: red;
background-color: lightblue;
}
h2 {
font-family: Sofia;
color: goldenrod;
background-color: lightgreen;
}
h3 {
font-family: Trirong;
color: green;
background-color: lightcoral;
}
body {
background-color: azure;
font-size: 16.0px;
font-family: Arial, sans-serif;
color: darkblue;
}
</style>
<title>lightspeed_http and ca_web test</title>
</head>
<body>
<p id='datetime'/>
<script>
let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;
</script>
<h1>Test Heading 1</h1>
<h2>Test Heading 2</h2>
<h3>Test Heading 3</h3>
<h4>Test Heading 4</h4>
<h5>Test Heading 5</h5>
<h6>Test Heading 6</h6>
<img src="/earlyspring.png" align="left" border="0">
<p>lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diem nonummy nibh euismod tincidunt ut lacreet dolore magna aliguam erat volutpat. ut wisis enim ad minim veniam, quis nostrud exerci tution ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. duis te feugifacilisi. duis autem dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit au gue duis dolore te feugat nulla facilisi. ut wisi enim ad minim veniam, quis nostrud exerci taion ullamcorper suscipit lobortis nisl ut aliquip ex en commodo consequat.</p>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset='utf-8'>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Audiowide|Sofia|Trirong">
<style>
p {
font-family: arial;
color: blueviolet;
background-color: yellow;
}
h1 {
font-family: Audiowide;
color: red;
background-color: lightblue;
}
h2 {
font-family: Sofia;
color: goldenrod;
background-color: lightgreen;
}
h3 {
font-family: Trirong;
color: green;
background-color: lightcoral;
}
body {
background-color: azure;
font-size: 16.0px;
font-family: Arial, sans-serif;
color: darkblue;
}
</style>
<title>lightspeed_http and ca_web test</title>
</head>
<body>
<p id='datetime'/>
<script>
let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;
</script>
<h1>Test Heading 1</h1>
<h2>Test Heading 2</h2>
<h3>Test Heading 3</h3>
<h4>Test Heading 4</h4>
<h5>Test Heading 5</h5>
<h6>Test Heading 6</h6>
<img src="/earlyspring.png" align="left" border="0">
<p>lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diem nonummy nibh euismod tincidunt ut lacreet dolore magna aliguam erat volutpat. ut wisis enim ad minim veniam, quis nostrud exerci tution ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. duis te feugifacilisi. duis autem dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit au gue duis dolore te feugat nulla facilisi. ut wisi enim ad minim veniam, quis nostrud exerci taion ullamcorper suscipit lobortis nisl ut aliquip ex en commodo consequat.</p>
<p>Latin-looking words arranged in nonsensical sentences, set in columns to give the appearance of text on a page. Dummy text is used by the designer to help approximate the look of a page at a stage in the design process before the written text has been received. This way the designer is able, in a very real sense, to shape the material presence of words before they are written and before their meaning is known. Conventionally, due to constraints of time, ability, budget, and habit, the designer is not the author. So conventionally, the student of typography is not encouraged to write (or even read prepared copy) which would waste valuable time spent getting to grips with the mechanics of text layout. Such established working/teaching methods increase the danger of the typographer becoming detached from the meaning of texts. The treatment of copy in purely formal terms, reduced to blocks of texture on a page, has lead to the typographer's obsession with craft and disregard of meaning. Dummy text is text that is not meant to be read, but only looked at; a shape. The choice of Latin is crucial to this function in that it is a dead language. Working with dummy text, the designer only brings into play part of his/her array of tools/skills to convey meaning.</p>
<input type=text name="password" value="1234go" size="23" maxlength="23" >
<input type=password name="password" value="1234go" size="23" maxlength="23" >
</body>
</html>
<p>Latin-looking words arranged in nonsensical sentences, set in columns to give the appearance of text on a page. Dummy text is used by the designer to help approximate the look of a page at a stage in the design process before the written text has been received. This way the designer is able, in a very real sense, to shape the material presence of words before they are written and before their meaning is known. Conventionally, due to constraints of time, ability, budget, and habit, the designer is not the author. So conventionally, the student of typography is not encouraged to write (or even read prepared copy) which would waste valuable time spent getting to grips with the mechanics of text layout. Such established working/teaching methods increase the danger of the typographer becoming detached from the meaning of texts. The treatment of copy in purely formal terms, reduced to blocks of texture on a page, has lead to the typographer's obsession with craft and disregard of meaning. Dummy text is text that is not meant to be read, but only looked at; a shape. The choice of Latin is crucial to this function in that it is a dead language. Working with dummy text, the designer only brings into play part of his/her array of tools/skills to convey meaning.</p>
<input type=text name="password" value="1234go" size="23" maxlength="23" >
<input type=password name="password" value="1234go" size="23" maxlength="23" >
</body>
</html>
a2svior
a2svior4w ago
We're getting close to being able to publish a blog written in Mojo!
carlcaulkett
carlcaulkett4w ago
I'm moving more of the visuals away from the HTML into CSS...
No description
carlcaulkett
carlcaulkett4w ago
Now including standard fonts and all 1799 Google fonts 😉
fn get_page_html(mut self) raises -> String:
var page = Html()
var style = Style()

_ = style.
image_style(Class.round_image).
width(150).
height(150).
border(20, "solid", Colors.darkblue).
border_radius(75)

_ = style.
input_style(Class.fancy_input).
padding(10).
margin(5).
border(2, "dotted", Colors.blue).
border_radius(5)

_ = style.p().
font_family("arial").
color(Colors.blueviolet).
background_color(Colors.yellow)

_ = style.set_h_scale_factor(2)
fn get_page_html(mut self) raises -> String:
var page = Html()
var style = Style()

_ = style.
image_style(Class.round_image).
width(150).
height(150).
border(20, "solid", Colors.darkblue).
border_radius(75)

_ = style.
input_style(Class.fancy_input).
padding(10).
margin(5).
border(2, "dotted", Colors.blue).
border_radius(5)

_ = style.p().
font_family("arial").
color(Colors.blueviolet).
background_color(Colors.yellow)

_ = style.set_h_scale_factor(2)
_ = style.
h1().
font_family(GoogleFonts.Audiowide).
color(Colors.red).
background_color(Colors.lightblue)

_ = style.
h2().
font_family(GoogleFonts.Sofia).
color(Colors.goldenrod).
background_color(Colors.lightgreen)

_ = style.
h3().
font_family(GoogleFonts.Trirong).
color(Colors.green).
background_color(Colors.lightcoral)

_ = style.
h4().
font_family(GoogleFonts.Aclonica).
color(Colors.blueviolet).
background_color(Colors.lightblue)

_ = style.
h5().
font_family(GoogleFonts.Bilbo).
color(Colors.crimson).
background_color(Colors.lightgreen)

_ = style.
h6().
font_family(GoogleFonts.Salsa).
color(Colors.black).
background_color(Colors.lightcoral)

_ = style.
body()
.background_color(Colors.azure).
font_size(16, FontUnit.PX).
font_family("Arial, sans-serif").
color(Colors.darkblue)

_ = style.
id(id.lorem).
font_family("Times New Roman, serif", ).
color(Colors.chartreuse).
background_color(Colors.darkblue).
margin_top(0).margin_bottom(0).
padding(10).
font_size(110, FontUnit.PERCENT)

_ = style.
id(id.post_modern).
font_family("Futura, sans-serif").
color(Colors.gainsboro).
background_color(Colors.darkblue).
margin(0).
padding(20)

_ = style.
id(id.datetime).
color(Colors.darkblue).
background_color(Colors.lightblue).
margin(0).
padding(20).
font_size(20, FontUnit.PX)
_ = style.
h1().
font_family(GoogleFonts.Audiowide).
color(Colors.red).
background_color(Colors.lightblue)

_ = style.
h2().
font_family(GoogleFonts.Sofia).
color(Colors.goldenrod).
background_color(Colors.lightgreen)

_ = style.
h3().
font_family(GoogleFonts.Trirong).
color(Colors.green).
background_color(Colors.lightcoral)

_ = style.
h4().
font_family(GoogleFonts.Aclonica).
color(Colors.blueviolet).
background_color(Colors.lightblue)

_ = style.
h5().
font_family(GoogleFonts.Bilbo).
color(Colors.crimson).
background_color(Colors.lightgreen)

_ = style.
h6().
font_family(GoogleFonts.Salsa).
color(Colors.black).
background_color(Colors.lightcoral)

_ = style.
body()
.background_color(Colors.azure).
font_size(16, FontUnit.PX).
font_family("Arial, sans-serif").
color(Colors.darkblue)

_ = style.
id(id.lorem).
font_family("Times New Roman, serif", ).
color(Colors.chartreuse).
background_color(Colors.darkblue).
margin_top(0).margin_bottom(0).
padding(10).
font_size(110, FontUnit.PERCENT)

_ = style.
id(id.post_modern).
font_family("Futura, sans-serif").
color(Colors.gainsboro).
background_color(Colors.darkblue).
margin(0).
padding(20)

_ = style.
id(id.datetime).
color(Colors.darkblue).
background_color(Colors.lightblue).
margin(0).
padding(20).
font_size(20, FontUnit.PX)
_ = page.html_head("lightspeed_http and ca_web test", style)
_ = page.para("", id.datetime)
_ = page.script(id.datetime, 'new Date(); document.getElementById("datetime").innerHTML = datetime;')
_ = page.
h1(GoogleFonts.Audiowide).
h2(GoogleFonts.Sofia).
h3(GoogleFonts.Trirong).
h4(GoogleFonts.Aclonica).
h5(GoogleFonts.Bilbo).
h6(GoogleFonts.Salsa)
_ = page.image("/earlyspring.png", Class.round_image)
_ = page.para(page.lorem(), id.lorem)
_ = page.para(page.post_modern(), id.post_modern)
_ = page.input_text(id.username, "carl", Class.fancy_input, 23, 23, False)
_ = page.input_text(id.password, "1234go", Class.fancy_input, 23, 23, True)
_ = page.end_html()
page.prettify()
return str(page)
_ = page.html_head("lightspeed_http and ca_web test", style)
_ = page.para("", id.datetime)
_ = page.script(id.datetime, 'new Date(); document.getElementById("datetime").innerHTML = datetime;')
_ = page.
h1(GoogleFonts.Audiowide).
h2(GoogleFonts.Sofia).
h3(GoogleFonts.Trirong).
h4(GoogleFonts.Aclonica).
h5(GoogleFonts.Bilbo).
h6(GoogleFonts.Salsa)
_ = page.image("/earlyspring.png", Class.round_image)
_ = page.para(page.lorem(), id.lorem)
_ = page.para(page.post_modern(), id.post_modern)
_ = page.input_text(id.username, "carl", Class.fancy_input, 23, 23, False)
_ = page.input_text(id.password, "1234go", Class.fancy_input, 23, 23, True)
_ = page.end_html()
page.prettify()
return str(page)
@a2svior I'm starting to investigate how to handle subitted forms. My understanding from 20 years ago is that the form will be submitted using a "POST" method, as opposed to a "GET" method, but the question then is how can read the submitted for, in particular the input box with the id "username". My code editor's AI is suggesting this amendment to the func methods...
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
var username_option = req.form_data.get("username")
if username_option:
return OK(self.get_page_html(username_option.value()), "text/html")
else:
return OK(self.get_page_html(""), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
var username_option = req.form_data.get("username")
if username_option:
return OK(self.get_page_html(username_option.value()), "text/html")
else:
return OK(self.get_page_html(""), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
but it seems to be hallucinating HTTPRequest.form_data 😮 Is there a way I can use HTTPRequest.get_body() and parse the raw data? I've just tried it...
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
var raw_date = String(req.get_body())
print(raw_date)
return OK(self.get_page_html(), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
var raw_date = String(req.get_body())
print(raw_date)
return OK(self.get_page_html(), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
and look what we get... 😃
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...

username=carl&password=1234
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...

username=carl&password=1234
I think i can work with that 😉
a2svior
a2svior4w ago
MDN Web Docs
Sending form data - Learn web development | MDN
Once the form data has been validated on the client-side, it is okay to submit the form. And, since we covered validation in the previous article, we're ready to submit! This article looks at what happens when a user submits a form — where does the data go, and how do we handle it when it gets there? We also look at some of the security concerns...
carlcaulkett
carlcaulkett4w ago
Hi @a2svior! Are you around?
a2svior
a2svior4w ago
Hi, yup I'm here
carlcaulkett
carlcaulkett4w ago
I think there may be a problem with HTTPRequest.get_body() in request.mojo. It seems to be deleting the last 2 characters of what ever is in the page. You know that thing I posted yesterday...
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...

username=carl&password=1234
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...

username=carl&password=1234
Well, the page it came from had the content...
<form name=form action=/ method=post>
<input type=text name="username" value="carl" class="fancy_input" size="23" maxlength="23" >
<input type=password name="password" value="1234go" class="fancy_input" size="23" maxlength="23" >
<input type="submit" value="Submit">
</form>
<form name=form action=/ method=post>
<input type=text name="username" value="carl" class="fancy_input" size="23" maxlength="23" >
<input type=password name="password" value="1234go" class="fancy_input" size="23" maxlength="23" >
<input type="submit" value="Submit">
</form>
In other words, the password value of "1234go" is being trancated to "1234". I've checked my code and I've examined your code as far as the body_raw property, but I cannot find the missing characters! Any ideas?
a2svior
a2svior4w ago
Ah yes, could be the way we deal with null terminators. There are some inconsistencies with the way null terminators work when dealing with strings/bytes in Mojo, I thought we have addressed that, but there was a lot of refactoring recently, could be that there's a regression and it's broken again. Is this version 0.1.9 ? I'll create an issue and investigate
carlcaulkett
carlcaulkett4w ago
Yes, it is 0.1.9. I did notice that the request body had a coupleof extra bytes at the start which I'm getting rid of with this code...
from collections.dict import Dict

@value
struct PostResponse():
var _raw_data: String

fn __init__(out self):
self._raw_data = String()

fn __init__(out self, raw_data: String):
self._raw_data = raw_data

fn build_dict(self) raises -> Dict[String, String]:
var dict = Dict[String, String]()
var raw_data = self._raw_data
if raw_data == "":
return dict
if "&" not in raw_data:
return dict
if raw_data.startswith("\r\n"):
raw_data = self._raw_data[2:]
var lines = raw_data.split("&")
for line in lines:
if "=" not in line[]:
continue
var parts = line[].split("=")
if len(parts) == 2:
dict[parts[0]] = parts[1]
return dict

fn get(self, key: String) raises -> String:
try:
var dict = self.build_dict()
if key in dict:
return dict[key]
return ""
except:
return ""

fn dict(self) raises -> Dict[String, String]:
return self.build_dict()
from collections.dict import Dict

@value
struct PostResponse():
var _raw_data: String

fn __init__(out self):
self._raw_data = String()

fn __init__(out self, raw_data: String):
self._raw_data = raw_data

fn build_dict(self) raises -> Dict[String, String]:
var dict = Dict[String, String]()
var raw_data = self._raw_data
if raw_data == "":
return dict
if "&" not in raw_data:
return dict
if raw_data.startswith("\r\n"):
raw_data = self._raw_data[2:]
var lines = raw_data.split("&")
for line in lines:
if "=" not in line[]:
continue
var parts = line[].split("=")
if len(parts) == 2:
dict[parts[0]] = parts[1]
return dict

fn get(self, key: String) raises -> String:
try:
var dict = self.build_dict()
if key in dict:
return dict[key]
return ""
except:
return ""

fn dict(self) raises -> Dict[String, String]:
return self.build_dict()
At first i though that the problem was due to my fix, largely because if also dealt with a 2 character difference!
a2svior
a2svior4w ago
Are you using some conversion methods in your code elsewhere, like as_bytes() ?
carlcaulkett
carlcaulkett4w ago
No, as_bytes is not is not in my code at all. Anything else I should be looking for?
a2svior
a2svior4w ago
Is this the handler? When you print raw_date , is it already cut off?
carlcaulkett
carlcaulkett4w ago
With this code...
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
print(req)
var post_response = PostResponse(String(req.get_body()))
return OK(self.get_page_html(post_response), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
print(req)
var post_response = PostResponse(String(req.get_body()))
return OK(self.get_page_html(post_response), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
I get...
POST / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
referer: http://0.0.0.0:8080/
content-type: application/x-www-form-urlencoded
content-length: 29
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=4


username=carl&password=1234
POST / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
referer: http://0.0.0.0:8080/
content-type: application/x-www-form-urlencoded
content-length: 29
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=4


username=carl&password=1234
The two spaces before username=carl&password=1234 are, I guess, because of the \r\n being added...
rd4com
rd4com4w ago
Hello, simple checkbox with emoji's if needed:
var tmp_autoplay = "✅ AutoPlay" if PlaySoundsSequence else "⬜ AutoPlay"
var tmp_autoplay = "✅ AutoPlay" if PlaySoundsSequence else "⬜ AutoPlay"
(ping to @carlcaulkett)
a2svior
a2svior4w ago
Do I understand correctly, this is happening on print(req)? Or are the logs from someplace else
carlcaulkett
carlcaulkett4w ago
Yes! That log two messages ago was from the print(req)! What was that checkbox stuff all about?
a2svior
a2svior4w ago
So it must be this line in our code then , will try to reproduce on my end. If you print req.get_body() or req.body_raw it's the same issue?
carlcaulkett
carlcaulkett4w ago
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
print("req: HTTPRequest: " + str(req))
print("req.get_body(): " + str(req.get_body()))
print("req.body_raw: " + String(req.body_raw))
var post_response = PostResponse(String(req.get_body()))
return OK(self.get_page_html(post_response), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
var uri = req.uri
if uri.path == "/":
if req.method == "GET":
return OK(self.get_page_html(), "text/html")
elif req.method == "POST":
print("req: HTTPRequest: " + str(req))
print("req.get_body(): " + str(req.get_body()))
print("req.body_raw: " + String(req.body_raw))
var post_response = PostResponse(String(req.get_body()))
return OK(self.get_page_html(post_response), "text/html")
if uri.path.endswith(".png"):
return OK(self.get_image(uri.path), "image/png")
return NotFound(uri.path)
req: HTTPRequest:

POST / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
content-type: application/x-www-form-urlencoded
content-length: 29
origin: http://0.0.0.0:8080
sec-gpc: 1
connection: keep-alive
referer: http://0.0.0.0:8080/
upgrade-insecure-requests: 1
priority: u=0, i


username=carl&password=1234

req.get_body():
username=carl&password=1234

req.body_raw:
username=carl&password=123
req: HTTPRequest:

POST / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
content-type: application/x-www-form-urlencoded
content-length: 29
origin: http://0.0.0.0:8080
sec-gpc: 1
connection: keep-alive
referer: http://0.0.0.0:8080/
upgrade-insecure-requests: 1
priority: u=0, i


username=carl&password=1234

req.get_body():
username=carl&password=1234

req.body_raw:
username=carl&password=123
The last one is even weirder, and is not a typo on my part 😉 If you want to try the code, it's at https://github.com/carlca/ca_web
toasty
toasty4w ago
The last one is truncated because body_raw is a list of bytes that are not null terminated. So you lose the last character Usually if I want to print bytes I’ll make it a stringslice If you repr the request, are there 3 \r\n before the request body?
carlcaulkett
carlcaulkett4w ago
I'm trying to make it a stringslice...
print("req.body_raw: " + StringSlice.from_utf8(req.body_raw))
print("req.body_raw: " + StringSlice.from_utf8(req.body_raw))
but I get the error...
mojo: 'StringSlice[?]' value has no attribute 'from_utf8'
mojo: 'StringSlice[?]' value has no attribute 'from_utf8'
My code was based on a suggestion from kapa ai... What do you mean by repr the request?
a2svior
a2svior4w ago
Maybe this https://docs.modular.com/mojo/stdlib/builtin/repr/repr/ can also try this? StringSlice(unsafe_from_utf8=req.body_raw)
toasty
toasty4w ago
I was mistaken, HTTPrequest does not implement repr at the moment, but I'm trying to run print(repr(req.get_body())) to check on the newlines present. I ran it with no body and it looks fine, I see 2 CRLF (\r\n\r\n) which is correct, but I need to test it with a request that has a body. I don't think we have an unit tests for request parsing that includes a body yet. I think I fixed the bug, I will open a PR for it We should check if content-length > 0 before trying to read the body, and we should also skip any leading \r\n before reading the body. There was a leading \r\n that consumed 2 bytes that was expected in the body itself, so that's why you saw go truncated off the end.
carlcaulkett
carlcaulkett4w ago
Still getting username=carl&password=1234 - missing the final "go" characters. Aha! I had noticed the leading \r\n in the body earlier this afternoon, and took steps to remove them...
from collections.dict import Dict

@value
struct PostResponse():
var _raw_data: String

fn __init__(out self):
self._raw_data = String()

fn __init__(out self, raw_data: String):
self._raw_data = raw_data

fn build_dict(self) raises -> Dict[String, String]:
var dict = Dict[String, String]()
var raw_data = self._raw_data
if raw_data == "":
return dict
if "&" not in raw_data:
return dict
if raw_data.startswith("\r\n"): # <== FIX HERE
raw_data = self._raw_data[2:]
var lines = raw_data.split("&")
for line in lines:
if "=" not in line[]:
continue
var parts = line[].split("=")
if len(parts) == 2:
dict[parts[0]] = parts[1]
return dict

fn get(self, key: String) raises -> String:
try:
var dict = self.build_dict()
if key in dict:
return dict[key]
return ""
except:
return ""

fn dict(self) raises -> Dict[String, String]:
return self.build_dict()
from collections.dict import Dict

@value
struct PostResponse():
var _raw_data: String

fn __init__(out self):
self._raw_data = String()

fn __init__(out self, raw_data: String):
self._raw_data = raw_data

fn build_dict(self) raises -> Dict[String, String]:
var dict = Dict[String, String]()
var raw_data = self._raw_data
if raw_data == "":
return dict
if "&" not in raw_data:
return dict
if raw_data.startswith("\r\n"): # <== FIX HERE
raw_data = self._raw_data[2:]
var lines = raw_data.split("&")
for line in lines:
if "=" not in line[]:
continue
var parts = line[].split("=")
if len(parts) == 2:
dict[parts[0]] = parts[1]
return dict

fn get(self, key: String) raises -> String:
try:
var dict = self.build_dict()
if key in dict:
return dict[key]
return ""
except:
return ""

fn dict(self) raises -> Dict[String, String]:
return self.build_dict()
but I didn't think to compensate at the other end. It did seem like a bit too much of a coincidence, though 😉 Do I have to wait for @a2svior to accept the PR or is there a fix I can apply in the meantime. I still fear the intricacies of Git - I try to avoid things like PRs, branches and checkouts 😮 Anyhow, thanks for your help on this, both of you 🙏🏽. I'll be waiting for the nod to update 😉
rd4com
rd4com4w ago
There is github desktop and sublime merge for easy gui git works if needed
a2svior
a2svior4w ago
I'll ship a new release today, there are some other updates we wanted to get in anyway
carlcaulkett
carlcaulkett3w ago
Cool! Ready when you are 😉 Thanks! I'm actually using Gitkraken on the Mac, and it's getting easier, slowly, as I tentatively investigate more options 😉 @a2svior! A quick question, if I may... If I want to use an external style sheet, let's call it style.css andI have it stored in my ./static folder alongside my EarlySpring.png image file, how should I refer to it in my HTML which is being served by lightbug_http?
a2svior
a2svior3w ago
Sure, same approach!
@value
struct Welcome(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
if req.uri.path == "/":
with open("static/lightbug_welcome.html", "r") as f:
return OK(f.read_bytes(), "text/html; charset=utf-8")

if req.uri.path == "/logo.png":
with open("static/logo.png", "r") as f:
return OK(f.read_bytes(), "image/png")

if req.uri.path == "/styles.css":
try:
with open("static/styles.css", "r") as f:
content = f.read_bytes()
print("CSS file loaded successfully") # Debug print
return OK(content, "text/css")
except:
print("Error loading CSS file") # Debug print

return NotFound(req.uri.path)
@value
struct Welcome(HTTPService):
fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
if req.uri.path == "/":
with open("static/lightbug_welcome.html", "r") as f:
return OK(f.read_bytes(), "text/html; charset=utf-8")

if req.uri.path == "/logo.png":
with open("static/logo.png", "r") as f:
return OK(f.read_bytes(), "image/png")

if req.uri.path == "/styles.css":
try:
with open("static/styles.css", "r") as f:
content = f.read_bytes()
print("CSS file loaded successfully") # Debug print
return OK(content, "text/css")
except:
print("Error loading CSS file") # Debug print

return NotFound(req.uri.path)
Just need to include it in your HTML like so:
<link rel="stylesheet" href="/styles.css">
<link rel="stylesheet" href="/styles.css">
Lightbug 0.1.10 Release just dropped! With amazing first contributions from @Hristo Gueorguiev and lots of refactoring/stability improvements! Full changelog: - Parse URI queries in to key value pairs by @izo0x90 in #237 - Adds url escape sequence decoding by @izo0x90 in #238 - Fix socket by @thatstoasty in #235 - Fix reader not skipping over all CRLF by @thatstoasty in #241 - Process localhost in parse_address by @saviorand in #239 https://github.com/Lightbug-HQ/lightbug_http/releases/tag/v0.1.10
carlcaulkett
carlcaulkett3w ago
I'm happy to confirm that my missing two characters have returned to me 😃. Thanks you @a2svior and @toasty 🙏🏽
a2svior
a2svior3w ago
Yay, the Return of the Bytes
carlcaulkett
carlcaulkett3w ago
Thanks for that code. It was the missing part of the puzzle that has been eluding me all day. It's now enabled me to switch between embedding the style css within the page or referencing it as a separate file...
var use_static_css = False

if use_static_css:
_ = style.save_to_file("static/style.css")
_ = style.clear()
_ = page.html_head("lightspeed_http and ca_web test", "/style.css", style)
else:
_ = page.html_head("lightspeed_http and ca_web test", "", style)
var use_static_css = False

if use_static_css:
_ = style.save_to_file("static/style.css")
_ = style.clear()
_ = page.html_head("lightspeed_http and ca_web test", "/style.css", style)
else:
_ = page.html_head("lightspeed_http and ca_web test", "", style)
I've been badgering Google Gemini 2.0 Flash AI a lot today. I think I came close to giving it a nervous breakdown. What didn't help was that my browser Zen decided that today would be a good day to not display the up-to-date source code on "View Page Source". Things improved when I cleared the cache 😉
rd4com
rd4com3w ago
I think it is possible to add an HTTP header that specify that the cache should expire after a certain amount of time 👍
carlcaulkett
carlcaulkett3w ago
👍
carlcaulkett
carlcaulkett3w ago
Hi @a2svior! Am I right in thinking that any project using lightbug_http still needs to be build with Max/Mojo Stable as opposed to nightly?
a2svior
a2svior3w ago
Yup, haven't updated the nightly branch yet. I might get to it today though
carlcaulkett
carlcaulkett3w ago
No worries! In your own time. As long as it's a known issue 😉
a2svior
a2svior3w ago
@carlcaulkett actually looks like I can't for now, the nightly branch breaks quite frequently + Lightbug is dependent on other libraries from other authors. So even if I update it now, keeping nightly up-to-date would require all the libraries to commit to updating their codebases every day, which we don't have the collective capacity for right now. Maybe later. I think I'll just close the nightly branch for now.
carlcaulkett
carlcaulkett3w ago
No worries! I guess I'm more concerned about making sure that MY code is fully nightly compatible, whichI guess I can do by temporarily switching to "https://conda.modular.com/max-nightly" tweaking the errors to do with my code only, then switch back to stable. Famous last words 😮 I'm gonna have to stick with stable syntax for now...
a2svior
a2svior3w ago
Yeah, it's complicated 😅
a2svior
a2svior3w ago
Nice, looks like you're adding JS?
carlcaulkett
carlcaulkett3w ago
Indeed! This is my get_page_html method now. Featuring a fluent api, static or dynamic CSS, scripting with JS, standard or Google fonts...
fn get_page_html(mut self, post_response: PostResponse = PostResponse()) raises -> String:
var page = Html()
var style = Style()

_ = style.
image_style(Class.round_image).
width(150).height(150).
border(20, "solid", Colors.darkblue).border_radius(75)

_ = style.
input_style(Class.fancy_input).
padding(10).margin(5).
border(2, "dotted", Colors.blue).border_radius(5)

_ = style.p().
font_family("arial").color(Colors.blueviolet).background_color(Colors.yellow)

_ = style.set_h_scale_factor(2)

_ = style.h1().
font_family(GoogleFonts.Audiowide).color(Colors.red).background_color(Colors.lightblue)

_ = style.h2().
font_family(GoogleFonts.Sofia).color(Colors.goldenrod).background_color(Colors.lightgreen)

_ = style.h3().
font_family(GoogleFonts.Trirong).color(Colors.green).background_color(Colors.lightcoral)

_ = style.h4().
font_family(GoogleFonts.Aclonica).color(Colors.blueviolet).background_color(Colors.lightblue)

_ = style.h5().
font_family(GoogleFonts.Bilbo).color(Colors.crimson).background_color(Colors.lightgreen)

_ = style.h6().
font_family(GoogleFonts.Salsa).color(Colors.black).background_color(Colors.lightcoral)
fn get_page_html(mut self, post_response: PostResponse = PostResponse()) raises -> String:
var page = Html()
var style = Style()

_ = style.
image_style(Class.round_image).
width(150).height(150).
border(20, "solid", Colors.darkblue).border_radius(75)

_ = style.
input_style(Class.fancy_input).
padding(10).margin(5).
border(2, "dotted", Colors.blue).border_radius(5)

_ = style.p().
font_family("arial").color(Colors.blueviolet).background_color(Colors.yellow)

_ = style.set_h_scale_factor(2)

_ = style.h1().
font_family(GoogleFonts.Audiowide).color(Colors.red).background_color(Colors.lightblue)

_ = style.h2().
font_family(GoogleFonts.Sofia).color(Colors.goldenrod).background_color(Colors.lightgreen)

_ = style.h3().
font_family(GoogleFonts.Trirong).color(Colors.green).background_color(Colors.lightcoral)

_ = style.h4().
font_family(GoogleFonts.Aclonica).color(Colors.blueviolet).background_color(Colors.lightblue)

_ = style.h5().
font_family(GoogleFonts.Bilbo).color(Colors.crimson).background_color(Colors.lightgreen)

_ = style.h6().
font_family(GoogleFonts.Salsa).color(Colors.black).background_color(Colors.lightcoral)
_ = style.body().
color(Colors.darkblue).background_color(Colors.azure).
font_size(16, FontUnit.PX).font_family("Arial, sans-serif")

_ = style.id(id.lorem).
font_family("Times New Roman, serif", ).font_size(110, FontUnit.PERCENT).
color(Colors.chartreuse).background_color(Colors.darkblue).
margin_top(0).margin_bottom(0).padding_top(10).padding_bottom(2).padding_left(10).padding_right(10)

_ = style.id(id.post_modern).
font_family("Futura, sans-serif").
color(Colors.gainsboro).background_color(Colors.darkblue).
margin(0).padding_top(2).padding_bottom(10).padding_left(10).padding_right(10)

_ = style.id(id.datetime).
color(Colors.darkblue).background_color(Colors.lightblue).
margin(0).padding(0).
font_size(20, FontUnit.PX)

var use_static_css = True

if use_static_css:
_ = style.save_to_file("static/style.css")
_ = style.clear()
_ = page.html_head("lightspeed_http and ca_web test", "/style.css", style)
else:
_ = page.html_head("lightspeed_http and ca_web test", "", style)

_ = page.para("", id.datetime)
_ = page.script(Script(id.datetime).update_time())
_ = style.body().
color(Colors.darkblue).background_color(Colors.azure).
font_size(16, FontUnit.PX).font_family("Arial, sans-serif")

_ = style.id(id.lorem).
font_family("Times New Roman, serif", ).font_size(110, FontUnit.PERCENT).
color(Colors.chartreuse).background_color(Colors.darkblue).
margin_top(0).margin_bottom(0).padding_top(10).padding_bottom(2).padding_left(10).padding_right(10)

_ = style.id(id.post_modern).
font_family("Futura, sans-serif").
color(Colors.gainsboro).background_color(Colors.darkblue).
margin(0).padding_top(2).padding_bottom(10).padding_left(10).padding_right(10)

_ = style.id(id.datetime).
color(Colors.darkblue).background_color(Colors.lightblue).
margin(0).padding(0).
font_size(20, FontUnit.PX)

var use_static_css = True

if use_static_css:
_ = style.save_to_file("static/style.css")
_ = style.clear()
_ = page.html_head("lightspeed_http and ca_web test", "/style.css", style)
else:
_ = page.html_head("lightspeed_http and ca_web test", "", style)

_ = page.para("", id.datetime)
_ = page.script(Script(id.datetime).update_time())
_ = page.
h1(GoogleFonts.Audiowide).h2(GoogleFonts.Sofia).h3(GoogleFonts.Trirong).h4(GoogleFonts.Aclonica).h5(GoogleFonts.Bilbo).h6(GoogleFonts.Salsa)
_ = page.image("/earlyspring.png", Class.round_image)
_ = page.para(page.lorem(), id.lorem)
_ = page.para(page.post_modern(), id.post_modern)

_ = page.form()
_ = page.input_text(id.username, "carl", Class.fancy_input, 23, 23, False)
_ = page.input_text(id.password, "1234go", Class.fancy_input, 23, 23, True)
_ = page.submit()
_ = page.end_form()

var post_data = PostData(post_response.get("username"), post_response.get("password"))
_ = page.para("Username (POST): " + post_data.username, id.username_post)
_ = page.para("Password (POST): " + post_data.password, id.password_post)

_ = page.para("Username (DOM): ", id.username)
_ = page.para("Password (DOM): ", id.password)

_ = page.button("Update Outputs", id.update_dom)
_ = page.script(Script(id.update_dom).update_dom(
(id.username, post_data.username, True),
(id.password, post_data.password, True)))

_ = page.end_html()
page.prettify()
return str(page)
_ = page.
h1(GoogleFonts.Audiowide).h2(GoogleFonts.Sofia).h3(GoogleFonts.Trirong).h4(GoogleFonts.Aclonica).h5(GoogleFonts.Bilbo).h6(GoogleFonts.Salsa)
_ = page.image("/earlyspring.png", Class.round_image)
_ = page.para(page.lorem(), id.lorem)
_ = page.para(page.post_modern(), id.post_modern)

_ = page.form()
_ = page.input_text(id.username, "carl", Class.fancy_input, 23, 23, False)
_ = page.input_text(id.password, "1234go", Class.fancy_input, 23, 23, True)
_ = page.submit()
_ = page.end_form()

var post_data = PostData(post_response.get("username"), post_response.get("password"))
_ = page.para("Username (POST): " + post_data.username, id.username_post)
_ = page.para("Password (POST): " + post_data.password, id.password_post)

_ = page.para("Username (DOM): ", id.username)
_ = page.para("Password (DOM): ", id.password)

_ = page.button("Update Outputs", id.update_dom)
_ = page.script(Script(id.update_dom).update_dom(
(id.username, post_data.username, True),
(id.password, post_data.password, True)))

_ = page.end_html()
page.prettify()
return str(page)
You are welcome to try it out! It's at https://github.com/carlca/ca_web.git. Just run magic run default. If you want inline CSS, just change this line (127) in client.mojo from True to False...
var use_static_css = True
var use_static_css = True
a2svior
a2svior3w ago
Cool, I'll try this out soon! It's getting pretty feature-rich now 👍
carlcaulkett
carlcaulkett3w ago
You mentioned a while back about hosting with Docker. How complicated is that?
a2svior
a2svior3w ago
I'm actually currenrly trying to update the dockerfile for Lightbug to work with this: https://github.com/modular/magic-docker/tree/main . Will let you know how it goes
ModularBot
ModularBot3w ago
Congrats @a2svior, you just advanced to level 9!
carlcaulkett
carlcaulkett3w ago
In the meantime, I've been experimenting with Netlify, and I've made some progress! Here is the latest build log...
3:40:49 PM: Netlify Build
3:40:49 PM: ────────────────────────────────────────────────────────────────
3:40:49 PM: ​
3:40:49 PM: ❯ Version
3:40:49 PM: @netlify/build 29.58.9
3:40:49 PM: ​
3:40:49 PM: ❯ Flags
3:40:49 PM: accountId: 5a26827ba6188f059225dccd
3:40:49 PM: baseRelDir: true
3:40:49 PM: buildId: 67a4d7def8ab342da742bcf4
3:40:49 PM: deployId: 67a4d7def8ab342da742bcf6
3:40:49 PM: ​
3:40:49 PM: ❯ Current directory
3:40:49 PM: /opt/build/repo
3:40:49 PM: ​
3:40:49 PM: ❯ Config file
3:40:49 PM: No config file was defined: using default values.
3:40:49 PM: ​
3:40:49 PM: ❯ Context
3:40:49 PM: production
3:40:49 PM: ​
3:40:49 PM: Netlify Build
3:40:49 PM: ────────────────────────────────────────────────────────────────
3:40:49 PM: ​
3:40:49 PM: ❯ Version
3:40:49 PM: @netlify/build 29.58.9
3:40:49 PM: ​
3:40:49 PM: ❯ Flags
3:40:49 PM: accountId: 5a26827ba6188f059225dccd
3:40:49 PM: baseRelDir: true
3:40:49 PM: buildId: 67a4d7def8ab342da742bcf4
3:40:49 PM: deployId: 67a4d7def8ab342da742bcf6
3:40:49 PM: ​
3:40:49 PM: ❯ Current directory
3:40:49 PM: /opt/build/repo
3:40:49 PM: ​
3:40:49 PM: ❯ Config file
3:40:49 PM: No config file was defined: using default values.
3:40:49 PM: ​
3:40:49 PM: ❯ Context
3:40:49 PM: production
3:40:49 PM: ​
3:40:49 PM: Build command from Netlify app
3:40:49 PM: ────────────────────────────────────────────────────────────────
3:40:49 PM: ​
3:40:49 PM: $ curl -ssL https://magic.modular.com/deb11594-7cdd-4b2d-968c-386eb908dd12 | bash && export PATH="$PATH:/opt/buildhome/.modular/bin" && magic project platform add linux-64 && magic run default
3:40:50 PM: Installing the latest version of Magic...
3:40:52 PM: Done. The 'magic' binary is in '/opt/buildhome/.modular/bin'
3:40:52 PM: Could not update shell: nologin
3:40:52 PM: Please permanently add '/opt/buildhome/.modular/bin' to your /opt/build/repo/node_modules/.bin:/opt/build/node_modules/.bin:/opt/node_modules/.bin:/node_modules/.bin:/opt/buildhome/.nvm/versions/node/v18.20.6/bin:/opt/buildhome/.nvm/versions/node/v18.20.6/bin:/home/linuxbrew/.linuxbrew/bin:/opt/buildhome/.swiftenv/bin:/opt/buildhome/.swiftenv/shims:/opt/buildhome/.php:/opt/buildhome/.binrc/bin:/opt/buildhome/.deno/bin:/opt/buildhome/.local/bin:/opt/buildhome/.local/share/mise/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/buildhome/.cask/bin:/opt/buildhome/.dotnet/tools:/opt/buildhome/.dotnet:/opt/buildhome/.cargo/bin to enable the 'magic' command.
3:41:24 PM: ✔ Added linux-64
3:41:32 PM: :fire::bee: Lightbug is listening on http://0.0.0.0:8080
3:41:32 PM: Ready to accept connections...
3:40:49 PM: Build command from Netlify app
3:40:49 PM: ────────────────────────────────────────────────────────────────
3:40:49 PM: ​
3:40:49 PM: $ curl -ssL https://magic.modular.com/deb11594-7cdd-4b2d-968c-386eb908dd12 | bash && export PATH="$PATH:/opt/buildhome/.modular/bin" && magic project platform add linux-64 && magic run default
3:40:50 PM: Installing the latest version of Magic...
3:40:52 PM: Done. The 'magic' binary is in '/opt/buildhome/.modular/bin'
3:40:52 PM: Could not update shell: nologin
3:40:52 PM: Please permanently add '/opt/buildhome/.modular/bin' to your /opt/build/repo/node_modules/.bin:/opt/build/node_modules/.bin:/opt/node_modules/.bin:/node_modules/.bin:/opt/buildhome/.nvm/versions/node/v18.20.6/bin:/opt/buildhome/.nvm/versions/node/v18.20.6/bin:/home/linuxbrew/.linuxbrew/bin:/opt/buildhome/.swiftenv/bin:/opt/buildhome/.swiftenv/shims:/opt/buildhome/.php:/opt/buildhome/.binrc/bin:/opt/buildhome/.deno/bin:/opt/buildhome/.local/bin:/opt/buildhome/.local/share/mise/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/buildhome/.cask/bin:/opt/buildhome/.dotnet/tools:/opt/buildhome/.dotnet:/opt/buildhome/.cargo/bin to enable the 'magic' command.
3:41:24 PM: ✔ Added linux-64
3:41:32 PM: :fire::bee: Lightbug is listening on http://0.0.0.0:8080
3:41:32 PM: Ready to accept connections...
Look what we have at the bottom! It shows that lightbug_http is running on Netlify. The only problem, I think, is that it needs to be pursuaded that it should serve on https://cawebtest.netlify.app/ rather than http://0.0.0.0:8080. Is this something that can be added in a config?
a2svior
a2svior3w ago
just need a reverse proxy on top of it, like NGINX. If you can share the config / how you do it I can also try it out?
carlcaulkett
carlcaulkett3w ago
If you want to try it on Netlify yourself, the critical settings are Base directory and Publish directory which are both set to / and the Build command which is curl -ssL https://magic.modular.com/deb11594-7cdd-4b2d-968c-386eb908dd12 | bash && export PATH="$PATH:/opt/buildhome/.modular/bin" && magic project platform add linux-64 && magic run default. Good luck 🤞😉
a2svior
a2svior3w ago
trying out now Hmm, apparently Netlify only supports static sites and edge functions, since it's designed for JAMStack. Lightbug's server is not a static site so I'm not sure it's even possible to run it like that 🤔 See here: https://answers.netlify.com/t/support-guide-can-i-run-a-web-server-http-listener-and-or-database-at-netlify/3078 + looks like their Functions only support typescript/javascript/go code and not custom binaries or Docker, see here: https://answers.netlify.com/t/execute-binary-or-run-docker-container/5926 maybe it's easier to just use a cloud provider like https://www.digitalocean.com/ in this case e.g I know Oracle Cloud has an always-free tier, that's probably useful for testing things out
Darkmatter
Darkmatter3w ago
There's an asterisk on that, you need to rate limit your network bandwidth. I can hand over the command for that if you go that route.
carlcaulkett
carlcaulkett3w ago
Interesting! I know that Hugo websites are supported, and Hugo is a framework that produced dynamic content written in Go. Here is my site there https://relaxed-williams-d8a311.netlify.app/. Though I hasten to add that I am speaking from a position of barely contained ignorance on all of the underlying net technologies. What I do know about sockets and all of that jazz stopped being useful in about 2003 😉
Darkmatter
Darkmatter3w ago
Could there be an option for "respond to everything, no matter what address"? That would be nice for local dev, since it would let me hand over a v6 address running on my laptop for coworkers to look at if I'm in an office.
carlcaulkett
carlcaulkett3w ago
I'd be interested in any help with getting on Oracle Cloud. I had an encounter with Digital Ocean in 2018 and that experience rather put me off. The fact that when I went back there just now, the first thing that they seemed to care about was securing my payment details and charging me $5 for nothing,
Darkmatter
Darkmatter3w ago
Everything is fixed-price except for bandwidth. In order to keep bandwidth at the correct amount, you can use tc and the token bucket filter to force a rate limit that will let you use bandwidth in bursts, but will never exceed the free tier. You can use the "burst" argument to ensure you don't have issues with monthly bandwidth resetting.
a2svior
a2svior3w ago
to make sure I understand, do you mean 0.0.0.0/::, or responding at any path like /, test, etc? or are you talking about NGINX rules?
Darkmatter
Darkmatter3w ago
Accept any IP address, but also ignore the Host header. Anything which resolves to the right IP address would work.
carlcaulkett
carlcaulkett3w ago
Am I right in thinking that tc etc is a Linux only solution? I'm on macOS!
Darkmatter
Darkmatter3w ago
But the Oracle VM will not be on MacOS.
carlcaulkett
carlcaulkett3w ago
True!! To keep my options open, I went ahead with the Digital Oceans sign up. Despite it quite clearly stating that there would be a $5 charge, to verify my payment capacity, the bastards have just charged me £9.61. Quite how $% in US currency equates to £9.61 sterling escapes me 😡
ModularBot
ModularBot3w ago
Congrats @carlcaulkett, you just advanced to level 16!
Darkmatter
Darkmatter3w ago
You are probably paying the currency exchange fee.
carlcaulkett
carlcaulkett3w ago
@Darkmatter I'm nipping at your heals 😉
Darkmatter
Darkmatter3w ago
I think your next target is Jack. I wonder what I get for level 50. I might have a feature request 😉
a2svior
a2svior3w ago
Alright, I'll see what I can do. Will keep you posted
Hristo Gueorguiev
So (based on this https://docs.netlify.com/frameworks/hugo/) seems like Hugo is a static site gen framework so all the artifacts get created during the build process and then hosted on Netlify ... much like the rest of the frameworks that it (netlify) is compatible with. @carlcaulkett from what you shared about your framework it appears that you could def. have it build static sites in the same way that it would produce the markup and scripts for a dynamic site with just the limitation that any values are frozen at build time ... could be a cool feature to have like nextjs etc. ... hosting static sites on netlify build with your own framework would also be pretty sweet.
carlcaulkett
carlcaulkett3w ago
@a2svior I've managed to get the Netlify hosted site working https://cawebtest.netlify.app/, but, as you and @Hristo Gueorguiev say, this is a static site in that the content is all generated in the build stage, a concept that I hadn't fully grasped yesterday 😉 The latest code is at https://github.com/carlca/ca_web.git. It looks as if Netlify won't be able to support my ultimate aim of having a truly dynamic Mojo driven website where the page reacts to user input and generates updated content from the Mojo backend code and updates the screen by manipulating the DOM.
a2svior
a2svior3w ago
ah cool, and I was able to put the default Lightbug welcome handler behind HTTPS on DigitalOcean via NGINX. Updating the Dockerfile in the repo now to make it easier to reproduce
carlcaulkett
carlcaulkett3w ago
Is that working as a dynamic app or a static app on DigitalOcean?
a2svior
a2svior3w ago
Dynamic
carlcaulkett
carlcaulkett3w ago
Also, I've found a free web hostings service which, apparently supports dynamic and static web sites. It doesn't have built in support for Mojo, unsurprisingly, so I guess Docker is the way to go here. The service is called Render and can be found at https://render.com/, I found it in this list of "10 Free Web Hosting Solutions for Static and Dynamic Sites" - at https://dev.to/anticoder03/10-free-web-hosting-solutions-for-static-and-dynamic-sites-48g1 (it's number 9). Ooh! Now I'm interested! How much does DO cost, again?
a2svior
a2svior3w ago
I went with a 1 vCPU / 1GB / 25GB Disk droplet ($6/mo), but that's apparently not enough memory for Docker with Magic, so I also added a 2GB swap file , then it works Render is pretty well-known, yup
carlcaulkett
carlcaulkett3w ago
@a2svior Hi! Did you have any joy investigating Docker with lightpad? I ask because I'm trying to build a barebones Dockerfile...
FROM ghcr.io/modular/magic:latest AS build

COPY ca_web /app
WORKDIR /app

RUN magic run build # <--- Focus on this failing command
FROM ghcr.io/modular/magic:latest AS build

COPY ca_web /app
WORKDIR /app

RUN magic run build # <--- Focus on this failing command
and a mojoproject.toml of...
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
# Having to use Stable for now as the nightly build is giving issues... https://discord.com/channels/1087530497313357884/1119100298456215572/1332085767866023987
# "https://conda.modular.com/max-nightly",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
# platforms = ["osx-arm64"]
platforms = ["linux-aarch64"] # <== Note the change here for Docker/Render
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"
[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
"conda-forge",
# Having to use Stable for now as the nightly build is giving issues... https://discord.com/channels/1087530497313357884/1119100298456215572/1332085767866023987
# "https://conda.modular.com/max-nightly",
"https://conda.modular.com/max",
"https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
# platforms = ["osx-arm64"]
platforms = ["linux-aarch64"] # <== Note the change here for Docker/Render
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"
and it's failing because of...
=> ERROR [4/4] RUN magic run build # <--- Focus on this failing command 19.5s
------
> [4/4] RUN magic run build # <--- Focus on this failing command:
19.36
19.36 × failed to solve the conda requirements of 'default' 'linux-aarch64'
19.36 ╰─▶ Cannot solve the request because of: No candidates were found for
19.36 lightbug_http >=0.1.9.
19.36
19.36
------
Dockerfile:6
--------------------
4 | WORKDIR /app
5 |
6 | >>> RUN magic run build # <--- Focus on this failing command
7 |
--------------------
ERROR: failed to solve: process "/bin/sh -c magic run build # <--- Focus on this failing command" did not complete successfully: exit code: 1
=> ERROR [4/4] RUN magic run build # <--- Focus on this failing command 19.5s
------
> [4/4] RUN magic run build # <--- Focus on this failing command:
19.36
19.36 × failed to solve the conda requirements of 'default' 'linux-aarch64'
19.36 ╰─▶ Cannot solve the request because of: No candidates were found for
19.36 lightbug_http >=0.1.9.
19.36
19.36
------
Dockerfile:6
--------------------
4 | WORKDIR /app
5 |
6 | >>> RUN magic run build # <--- Focus on this failing command
7 |
--------------------
ERROR: failed to solve: process "/bin/sh -c magic run build # <--- Focus on this failing command" did not complete successfully: exit code: 1
Any ideas?
a2svior
a2svior3w ago
There's only "linux-64" and osx available I think, not "linux-aarch64". Also, I have a working Dockerfile for the Lightbug itself now under /docker in the repo, feel free to check it out!
carlcaulkett
carlcaulkett2w ago
At one point yesterday,I got the message...
failed to parse project from /app/mojoproject.toml: Expected one of
0.246 │ 'linux-32', 'linux-64', 'linux-aarch64', 'linux-armv6l', 'linux-armv7l',
0.246 │ 'linux-ppc64le', 'linux-ppc64', 'linux-s390x', 'linux-riscv32', 'linux-
0.246 │ riscv64', 'osx-64', 'osx-arm64', 'win-32', 'win-64', 'win-arm64',
0.246 │ 'emscripten-wasm32', 'wasi-wasm32', 'zos-z'
0.246
failed to parse project from /app/mojoproject.toml: Expected one of
0.246 │ 'linux-32', 'linux-64', 'linux-aarch64', 'linux-armv6l', 'linux-armv7l',
0.246 │ 'linux-ppc64le', 'linux-ppc64', 'linux-s390x', 'linux-riscv32', 'linux-
0.246 │ riscv64', 'osx-64', 'osx-arm64', 'win-32', 'win-64', 'win-arm64',
0.246 │ 'emscripten-wasm32', 'wasi-wasm32', 'zos-z'
0.246
I'll check out the Dockerfile on your website. Thanks for the heads up. btw, what do you think of these comments about lightbug? Fair, justified, or completely wide of the mark? https://discord.com/channels/1087530497313357884/1151418092052815884/1337943679201181759
a2svior
a2svior2w ago
It's all true, Lightbug is not ready for production use yet, and we will indeed have a big "rewrite everything" day once async is added to Mojo. But hopefully by the time that's done we're also finished with RFC compliance and features like TLS/HTTP2 support
Darkmatter
Darkmatter2w ago
Remember that you'll also want HTTP/3 support, which involves an entire network protocol.
eggsquad
eggsquad2w ago
Yeah I was kinda hoping to wait for a QUIC impl to come around so we didn’t have to do that ourselves lol. Plenty of work to be done in 1.1 and 2.0 before we start worrying about that
Darkmatter
Darkmatter2w ago
We might be able to use bindings to Cloudflare's quiche, since it has a C API. But, Mojo native would be better.
carlcaulkett
carlcaulkett2w ago
@a2svior Me again! This code works...
var server = Server()
var handler = PageHandler()
server.listen_and_serve("0.0.0.0:10000", handler)
var server = Server()
var handler = PageHandler()
server.listen_and_serve("0.0.0.0:10000", handler)
but this code doesn't...
var port = getenv("PORT")
var address = String("0.0.0.0")
if port.strip() == "":
address += ":10000"
else:
address += ":" + port
var server = Server()
var handler = PageHandler()
server.listen_and_serve(address, handler)
var port = getenv("PORT")
var address = String("0.0.0.0")
if port.strip() == "":
address += ":10000"
else:
address += ":" + port
var server = Server()
var handler = PageHandler()
server.listen_and_serve(address, handler)
listen_and_serve takes a String type, no? Why is the compiler telling me...
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:184:28: error: invalid call to 'listen_and_serve': method argument #0 cannot be converted from 'String' to 'StringLiteral'
server.listen_and_serve(address, handler)
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: note: function declared here
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:184:28: error: invalid call to 'listen_and_serve': method argument #0 cannot be converted from 'String' to 'StringLiteral'
server.listen_and_serve(address, handler)
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: note: function declared here
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module
Even this doesn't work...
var server = Server()
var handler = PageHandler()
var address = String("0.0.0.0:10000")
server.listen_and_serve(address, handler)
var server = Server()
var handler = PageHandler()
var address = String("0.0.0.0:10000")
server.listen_and_serve(address, handler)
yet listen_and_serve is defined as...
fn listen_and_serve[T: HTTPService](mut self, address: String, mut handler: T) raises:
fn listen_and_serve[T: HTTPService](mut self, address: String, mut handler: T) raises:
The error is...
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:187:28: error: invalid call to 'listen_and_serve': method argument #0 cannot be converted from 'String' to 'StringLiteral'
server.listen_and_serve(address, handler)
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: note: function declared here
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:187:28: error: invalid call to 'listen_and_serve': method argument #0 cannot be converted from 'String' to 'StringLiteral'
server.listen_and_serve(address, handler)
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: note: function declared here
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module
What am I missing?
a2svior
a2svior2w ago
@carlcaulkett actually I was dumb enough to switch this to a StringLiteral in the latest release, it's already reverted back to String in main, I'll ship a release today so you can use the newest version 😄
carlcaulkett
carlcaulkett2w ago
Just let me know when! If it's within the next 5 hours, great, otherwise I'll look at it tomorrow. In any case, thanks for your efforts 🙏🏽
a2svior
a2svior2w ago
0.1.11 just in, can you try it out?
carlcaulkett
carlcaulkett2w ago
I'm going in 😉 Looking good! This compiles...
fn main() raises:
if use_lightbug_http:
var port = getenv("PORT")
if port.strip() == "":
port = "10000"
var server = Server()
var handler = PageHandler()
var address: String = String("0.0.0.0:{}".format(port)) # Explicitly type address as String
server.listen_and_serve(address, handler)
fn main() raises:
if use_lightbug_http:
var port = getenv("PORT")
if port.strip() == "":
port = "10000"
var server = Server()
var handler = PageHandler()
var address: String = String("0.0.0.0:{}".format(port)) # Explicitly type address as String
server.listen_and_serve(address, handler)
@a2svior While you're around, do you know much about how Dockerfiles work?
a2svior
a2svior2w ago
I know a thing or two, why? Did you have a question about that
carlcaulkett
carlcaulkett2w ago
Here's my Dockerfile...
FROM ghcr.io/modular/magic:latest

RUN apt-get update && apt-get install -y git

RUN git clone https://github.com/Lightbug-HQ/lightbug_http

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app
# WORKDIR /lightbug_http

ARG DEFAULT_SERVER_PORT=8080
ARG DEFAULT_SERVER_HOST=localhost

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=lightbug.🔥

CMD ["magic", "run", "default"]
FROM ghcr.io/modular/magic:latest

RUN apt-get update && apt-get install -y git

RUN git clone https://github.com/Lightbug-HQ/lightbug_http

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app
# WORKDIR /lightbug_http

ARG DEFAULT_SERVER_PORT=8080
ARG DEFAULT_SERVER_HOST=localhost

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=lightbug.🔥

CMD ["magic", "run", "default"]
You'll notice that I'm pulling in lighybug_http using a git clone. Although I'm setting WORKDIR to /app which contains ca_web, my code, I'm not letting Docker know that lightbug_http is present, as far as I can tell I did try having WORKDIR /lightbug_http at one point, but that caused all kinds of errors. As it is, I'm getting these errors...
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
I've just tried changing DEFAULT_SERVER_PORT to 10000, since thta seems to be the preferred port for Render to serve through, but that gives the same errors about no ports detected. Any ideas? Could it be the use of localhost. Should I set DEFAULT_SERVER_HOST to 0.0.0.0 instead?
Darkmatter
Darkmatter2w ago
Do that. Docker makes a fake network, so it will never see traffic from the primary namespace as localhost traffic.
carlcaulkett
carlcaulkett2w ago
Thanks! Trying it as I type 😉
a2svior
a2svior2w ago
Might make sense to build and run this Dockerfile locally first, to make sure it works, and then use with the render platform But yes 0.0.0.0 is a good idea
carlcaulkett
carlcaulkett2w ago
That sounds good. How do I run the Dockerfile locally?
a2svior
a2svior2w ago
Something like this 1. build the Docker image docker build -t lightbug-app 2. Run the container docker run -p 8080:8080 lightbug-app Where instead of 8080:8080 it's the port e.g. 10000:10000
carlcaulkett
carlcaulkett2w ago
~/Code/Mojo docker build -t lightbug-app ERROR: "docker buildx build" requires exactly 1 argument. See 'docker buildx build --help'. Okay, this is building... docker build -t lightbug-app -f magic-docker/Dockerfile . When I ran docker run -p 10000:10000 lightbug-app, nothing visible happened on screen, and nothing was server on http://0.0.0.0:10000/. Did you see what I was asking about the setting of WORKDIR in my Dockerfile? I've rationalised my Dockerfile down to...
FROM ghcr.io/modular/magic:latest

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=lightbug.🔥

CMD ["magic", "run", "default"]
FROM ghcr.io/modular/magic:latest

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=lightbug.🔥

CMD ["magic", "run", "default"]
On the basis that the ca_web application, via its mojoproject.toml file, should take care of any dependency on lightpad_http making the RUN git clone https://github.com/Lightbug-HQ/lightbug_http line I had before, superfluous. But still the Render deploy is failing with...
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
I'm setting the environent variable PORT=10000 in the Render settings. I'll re-read https://render.com/docs/web-services#port-binding and see if anything jumps out at me 🤔 The full set of errors is...
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Out of memory (used over 512Mi)
==> Common ways to troubleshoot your deploy: https://render.com/docs/troubleshooting-deploys
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Out of memory (used over 512Mi)
==> Common ways to troubleshoot your deploy: https://render.com/docs/troubleshooting-deploys
I also tweaked the Dockerfile to...
FROM ghcr.io/modular/magic:latest

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=client.mojo

CMD ["magic", "run", "default"]
FROM ghcr.io/modular/magic:latest

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=client.mojo

CMD ["magic", "run", "default"]
setting the APP_ENTRYPOINT to client.mojo, the main file of my app, rather than lightbug.🔥! Still the same errors... I've posted a message on the Render forum, but I have the sneaking suspicion that this type of error comes with only having a VM with 512 MB and that you have to shell out $25 per month for a 2 gb VM in order to find out if the thing will work at all. In the meantime I'm going to investigate Oracle Cloud... tomorrow 😉
rd4com
rd4com2w ago
Hello, good job everybody, there is a small thing that can help a lot for HTML performance: when web browser render a page, it then request the icon for the tab, if the server don't give one, it will request it all the time for all pages, (so all requests from interaction results in multiple ones) in mojo ui html, solved it by creating an inlined default one: example for inlined favicon: https://stackoverflow.com/questions/66935329/use-inline-svg-as-favicon
a2svior
a2svior2w ago
Maybe try adding some swap space to the machine on render, might need to SSH in there. Here an instruction, although from DigitalOcean https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-20-04 About the workdir, I don't think you need to clone lightbug's repo if Lightbug is a dependency in your project. What you would want to do instead is clone your repo and open e.g the app directory as a workdir, without the COPY directive
carlcaulkett
carlcaulkett2w ago
So...
FROM ghcr.io/modular/magic:latest

RUN apt-get update && apt-get install -y git

RUN git clone https://github.com/carlca/ca_web

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

WORKDIR /ca_web

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /ca_web/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=client.mojo

CMD ["magic", "run", "default"]
FROM ghcr.io/modular/magic:latest

RUN apt-get update && apt-get install -y git

RUN git clone https://github.com/carlca/ca_web

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

WORKDIR /ca_web

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /ca_web/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=client.mojo

CMD ["magic", "run", "default"]
? I'll give it a try 😉 UPDATED!
a2svior
a2svior2w ago
Yup, something like that, although if client.mojo does what I think it does it will not start a server listening on a port. For that you'll probably need a file with your main function where you call listen_and_serve()
carlcaulkett
carlcaulkett2w ago
client.mojo ends with...
fn main() raises:
if use_lightbug_http:
var port = getenv("PORT")
if port.strip() == "":
port = "10000"
var server = Server()
var handler = PageHandler()
var address: String = String("0.0.0.0:{}".format(port)) # Explicitly type address as String
server.listen_and_serve(address, handler)
fn main() raises:
if use_lightbug_http:
var port = getenv("PORT")
if port.strip() == "":
port = "10000"
var server = Server()
var handler = PageHandler()
var address: String = String("0.0.0.0:{}".format(port)) # Explicitly type address as String
server.listen_and_serve(address, handler)
Same old story...
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Out of memory (used over 512Mi)
==> Common ways to troubleshoot your deploy: https://render.com/docs/troubleshooting-deploys
==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Out of memory (used over 512Mi)
==> Common ways to troubleshoot your deploy: https://render.com/docs/troubleshooting-deploys
😦
Darkmatter
Darkmatter2w ago
Mojo pulls in a lot of stuff, I'm not sure 512MB is a good amount, considering that TCP buffers count against you on most platforms, and those need to be fairly large for modern line rates.
a2svior
a2svior2w ago
Did you add a swap file?
carlcaulkett
carlcaulkett2w ago
I haven't yet. I'm hoping that Oracle Cloud with its 1GB allowance for its free tier will be a bit more forgiving. Besides, from what I can gather, it's very difficult to add a swap file to a Render VM with not much benefits. I'm ready to be persuaded otherwise, of course 😉
a2svior
a2svior2w ago
I have a 1GB droplet on DigitalOcean and that's not enough either 🤷‍♂️
Darkmatter
Darkmatter2w ago
Oracle allows 24 GB, not 1 GB.
carlcaulkett
carlcaulkett2w ago
Are you talking about disk storage or the VM RAM size?
Darkmatter
Darkmatter2w ago
24GB of memory, and 2 100GB virtual disks.
carlcaulkett
carlcaulkett2w ago
On the free tier? Wow!
Darkmatter
Darkmatter2w ago
4 ARM cores as well. inbound is free, 10 TB per month outbound is free as well. If you rate limit the network to 30 Mbps, you'll never see issues. If you get a bit more fancy, you can probably burst up to gigabit. Also cert management is free up to a point. You can technically also have 2 1GB 1/8 core AMD VMs, if you really want x86. Go to https://www.oracle.com/cloud/free/#free-cloud-trial and check the "always free" tier type. There's a lot more, because Oracle is desperate to get new customers.
carlcaulkett
carlcaulkett2w ago
Interesting! But I am confused... this is the VM I created yesterday, with only a paltry 1 GB RAM available to my VM. How can I access these other Oracle offers? Seems I can't upload screenshots to a thread. Here's a link... https://discord.com/channels/1087530497313357884/1151418092052815884/1339332526283554827 Okay, I've looked at https://www.oracle.com/uk/cloud/free/ and it looks as if I have created a VM in the AMD based Compute VMs with 1/8 OCPU and 1 GB memory each category. You, presumable are talking about the Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs category next to that.
Darkmatter
Darkmatter2w ago
I am talking about the ARM VM.
carlcaulkett
carlcaulkett2w ago
@a2svior @Owen Hilyard Almost there 😃 http://132.145.47.167:10000/ Just a missing image file 😮 Oracle Cloud + ARM, for the win 🏆 Thanks for the suggestion @Owen Hilyard 🙏🏽 The missing image file was caused by that old chestnut, file name case differences between macOS and Linux! Not to mention when I changed the case alone and pushed it to Github, the change did not register; I had to change the file name spelling and then change it back again for Github to take the revised file name 😮
a2svior
a2svior2w ago
Nice! I don't see anything when I open the page, could be related to an issue I saw myself when testing the Docker setup. I'm currently investigating this
carlcaulkett
carlcaulkett2w ago
@a2svior I've just discovered a problem with the latest version of Mojo Stable, 25.1. I get an error... https://discord.com/channels/1087530497313357884/1151418092052815884/1339678092783910995 If I set the version in mojoproject.toml with...
[dependencies]
max = "=24.6"
lightbug_http = ">=0.1.9"
[dependencies]
max = "=24.6"
lightbug_http = ">=0.1.9"
it build and runs, but if I have...
[dependencies]
max = "=25.1"
lightbug_http = ">=0.1.9"
[dependencies]
max = "=25.1"
lightbug_http = ">=0.1.9"
then that error, and lots of others suddenly appear 😮
eggsquad
eggsquad2w ago
oh didn't even realize 25.1 was released yet
Darkmatter
Darkmatter2w ago
You probably need to update lightbug, I imagine that libraries aren't stable between versions.
a2svior
a2svior2w ago
Looks like it was released today 😄 I just learned that it was released, need to update the library first, might take a little bit of time. Will post an update here when done
carlcaulkett
carlcaulkett2w ago
No worries! I can stick with 24.6 for the moment 🙂 In the latest version, the CSS is generated by the Mojo code but is saved as a static resource, keeping the HTML nice and clean 🙂 http://132.145.47.167:8080/
Hristo Gueorguiev
Let me know if I can help with any of that, not sure if a few ppl on it will make it go faster or slower 🤣
carlcaulkett
carlcaulkett2w ago
A Zen student went to a temple and asked how long it would take him to gain enlightenment if he joined the temple. "10 years," said the Zen master. "Well, how about if I really work hard and double my effort?" "In that case, 20 years." 😉 @Owen Hilyard I've got a problem with Oracle Cloud and I wonder if you might know the answer. My instance is running and Docker is running and hosting my Mojo app just fine. If I ssh inyo the Oracle Cloud and issue a curl http://localhost:8080 the correct HTML comes back proving that the Mojo app is running. Where I'm having problems is accessing the page via http://132.145.47.167:8080/. It seems to be okay right at the second, but every time I do an update and push a revised Docker, it seems to take ages before I can gain access to it again. I wondered if it was down to the firewall provisions. I've added an Ingress rule to allow access on port 8080, and, for a while I was workiing on the assumption that I had to delete the existing Ingress rule and reenter it exactly as before, in order to give the firewall a nudge, as it were, But i've abandoned this theory. Instead, it just seems to take ages before I can access the site again. It's really frustrating 😮
Darkmatter
Darkmatter2w ago
Make sure you have a public address attached. If you have IPv6 at home, that is usually cheaper.
carlcaulkett
carlcaulkett2w ago
That's what http://132.145.47.167:8080/ is, isn't it. To give you an example of the hassle I'm having, that address worked 5 minutes ago, but is not working now!
Darkmatter
Darkmatter2w ago
Huh I never head these issues.
a2svior
a2svior2w ago
Yes I think it's a bug in Lightbug ( 😅 ) that I discovered last week. Looking into it currently
Darkmatter
Darkmatter2w ago
I've used that free VM to host a virtual tabletop for DnD for years. For once, don't blame Oracle I guess.
a2svior
a2svior2w ago
The server dies after a request due to some issues with processing logic it seems. Interestingly, this neither happens locally, nor in our integration tests. Seems like it happens only when running it in a Docker container
Darkmatter
Darkmatter2w ago
Oh fun!
a2svior
a2svior2w ago
The error I was getting is
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/lightbug_http/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/lightbug_http/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
This is when just sending a GET from the browser to the running default handler Must be something straightforward, just haven't had the time to look into it properly yet
carlcaulkett
carlcaulkett2w ago
It's interesting that for a lot of the time that I couldn't access the web page via the address I've just mentioned, I could still ssh into the Oracle Cloud CLI and issue curl http://localhost:8080 and get the full HTML, proving that my app and lightbug were both working as expected. It was just that access to the public web address was being blocked by something, and my first thoughts were that it was firewall related.
rd4com
rd4com2w ago
Could it be a cache thing ?
carlcaulkett
carlcaulkett2w ago
Now it's working again. Go figure 🙃 Now it's not working, I've got visions of some vindictive person radomly flicking a switch on and off to block or allow me access 😮
Darkmatter
Darkmatter2w ago
The problem is that you aren't using OracleDB to store things /s
carlcaulkett
carlcaulkett2w ago
Here is my Oracle Cloud CLI...
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8c821b727d2 ghcr.io/carlca/magic-image:latest "magic run default" 8 seconds ago Up 7 seconds 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp ca-web-container
f2944de77a11 hello-world "/hello" 42 hours ago Exited (0) 42 hours ago objective_yonath
[opc@ca-web-arm ~]$
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8c821b727d2 ghcr.io/carlca/magic-image:latest "magic run default" 8 seconds ago Up 7 seconds 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp ca-web-container
f2944de77a11 hello-world "/hello" 42 hours ago Exited (0) 42 hours ago objective_yonath
[opc@ca-web-arm ~]$
Darkmatter
Darkmatter2w ago
Is the container going down?
carlcaulkett
carlcaulkett2w ago
and here is the HTML from my app..
[opc@ca-web-arm ~]$ curl http://localhost:8080
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset='utf-8'>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Audiowide|Sofia|Trirong|Aclonica|Bilbo|Salsa">
<link rel="stylesheet" type="text/css" href="style.css">
<title>lightspeed_http and ca_web test</title>
</head>
<body>
<p id='datetime'></p>
<script>
let updateTime = function() {
var datetime = new Date();
document.getElementById('datetime').innerHTML = datetime
}
setInterval(updateTime, 1000);
</script>
<h1>Audiowide</h1>
<h2>Sofia</h2>
<h3>Trirong</h3>
<h4>Aclonica</h4>
<h5>Bilbo</h5>
<h6>Salsa</h6>
<form name=form action=/ method=post>
<input type=text name="username" value="carl" class="fancy_input" size="23" maxlength="23" >
<input type=password name="password" value="1234go" class="fancy_input" size="23" maxlength="23" >
<input type=submit value=Submit>
</form>
<p id='username_post'>Username (POST): </p>
<p id='password_post'>Password (POST): </p>
<p id='username'>Username (DOM): </p>
<p id='password'>Password (DOM): </p>
<button type="button" onclick="updateDom()" >Update Outputs</button>
<script>
let updateDom = function() {
document.getElementById('username').innerHTML += '';
document.getElementById('password').innerHTML += '';
}
</script>
</body>
</html>
[opc@ca-web-arm ~]$ curl http://localhost:8080
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset='utf-8'>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Audiowide|Sofia|Trirong|Aclonica|Bilbo|Salsa">
<link rel="stylesheet" type="text/css" href="style.css">
<title>lightspeed_http and ca_web test</title>
</head>
<body>
<p id='datetime'></p>
<script>
let updateTime = function() {
var datetime = new Date();
document.getElementById('datetime').innerHTML = datetime
}
setInterval(updateTime, 1000);
</script>
<h1>Audiowide</h1>
<h2>Sofia</h2>
<h3>Trirong</h3>
<h4>Aclonica</h4>
<h5>Bilbo</h5>
<h6>Salsa</h6>
<form name=form action=/ method=post>
<input type=text name="username" value="carl" class="fancy_input" size="23" maxlength="23" >
<input type=password name="password" value="1234go" class="fancy_input" size="23" maxlength="23" >
<input type=submit value=Submit>
</form>
<p id='username_post'>Username (POST): </p>
<p id='password_post'>Password (POST): </p>
<p id='username'>Username (DOM): </p>
<p id='password'>Password (DOM): </p>
<button type="button" onclick="updateDom()" >Update Outputs</button>
<script>
let updateDom = function() {
document.getElementById('username').innerHTML += '';
document.getElementById('password').innerHTML += '';
}
</script>
</body>
</html>
No! That's the confusing thing... the container doesn't seem to be going down, just access to it from outside of Oracle, ie. through http://132.145.47.167:8080, seems to be intermittent...
Darkmatter
Darkmatter2w ago
Hmmm Not sure what the problem is.
a2svior
a2svior2w ago
do you see the logs in the container? I bet it's the same issue I was having
carlcaulkett
carlcaulkett2w ago
Where do I find them?
a2svior
a2svior2w ago
If you have SSH access or direct access over from Render to the VM you can do docker ps -a to show all running Docker containers with IDs, and docker logs container_id with the container ID of this container
carlcaulkett
carlcaulkett2w ago
I'm using Oracle Cloud now but yeah, even when I couldn't access my page by the web, docker ps -a was showing...
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8c821b727d2 ghcr.io/carlca/magic-image:latest "magic run default" 8 seconds ago Up 7 seconds 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp ca-web-container
f2944de77a11 hello-world "/hello" 42 hours ago Exited (0) 42 hours ago objective_yonath
[opc@ca-web-arm ~]$
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8c821b727d2 ghcr.io/carlca/magic-image:latest "magic run default" 8 seconds ago Up 7 seconds 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp ca-web-container
f2944de77a11 hello-world "/hello" 42 hours ago Exited (0) 42 hours ago objective_yonath
[opc@ca-web-arm ~]$
I didn't know about docker logs container-id though... I'lll try that now 😉 Is this what you had?
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
I see it is the same 😃 It seems strange they way the page is sometimes served but not others. Maybe it's some kind of cumulative problem whereby something has to reach a critical level before an exception is thrown... 🤔
a2svior
a2svior2w ago
Yup, it's a bug. I'll make sure this is addressed, probably on the weekend, once we switch to the new stable release Thanks for sharing the details
carlcaulkett
carlcaulkett2w ago
np. Good catch 🫴⚾️ @a2svior Interesting that sometimes it seems to work okay...
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6bf9730aa18a ghcr.io/carlca/magic-image:latest "magic run default" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp ca-web-container
f2944de77a11 hello-world "/hello" 2 days ago Exited (0) 2 days ago objective_yonath
[opc@ca-web-arm ~]$ docker logs 6bf9730aa18a


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$ docker logs 6bf9730aa18a


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6bf9730aa18a ghcr.io/carlca/magic-image:latest "magic run default" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp ca-web-container
f2944de77a11 hello-world "/hello" 2 days ago Exited (0) 2 days ago objective_yonath
[opc@ca-web-arm ~]$ docker logs 6bf9730aa18a


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$ docker logs 6bf9730aa18a


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$
Amazingly, it has been up and working for an hour or more, but I just cannot work out what action will cause it to fail... 🤔 @a2svior Just woken up, and after leaving it in a working state last night, I tried it again, and found that it was failing, this time with a slightly different message...
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to read request
ERROR - ReceiveError: An error occurred while attempting to receive data from the socket. Error code: 104
ERROR - Socket.receive: Failed to read data from connection.
ERROR - TCPConnection.read: Failed to read data from connection.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to read request
ERROR - ReceiveError: An error occurred while attempting to receive data from the socket. Error code: 104
ERROR - Socket.receive: Failed to read data from connection.
ERROR - TCPConnection.read: Failed to read data from connection.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
a2svior
a2svior2w ago
104 is "connection reset by peer" I think.. Thanks for sharing, will try to reproduce
carlcaulkett
carlcaulkett5d ago
Hi @a2svior! I see that Lightbug is up to v0.1.12 as of 2 days ago. Does this mean that you've sorted out the crashing problem? I guess not...
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
[opc@ca-web-arm ~]$
🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
[opc@ca-web-arm ~]$
a2svior
a2svior5d ago
Not yet, this release was for bumping to Mojo 25.1 . But it's not on Modular official channel yet, that's why I haven't announced it so far. The crash is still a work in progress, will post here when it's fixed!
carlcaulkett
carlcaulkett5d ago
No pressure 😉 Good luck 🤞
a2svior
a2svior17h ago
@carlcaulkett we got a fix! Going to release it tomorrow. Still not on the modular-community channel though, but I'll try to get it published on the mojo-community one
carlcaulkett
carlcaulkett13h ago
Ooh! How exciting 😃 I look froward to seeing it. Good work, that man 😉

Did you find this page helpful?