FastAPI - wkhtmltopdf

Hi there - I have a FastAPI server that requires wkhtmltopdf to run. I added an Aptfile to my repo but it doesn't seem to be getting picked up Railway. Any tips? Thanks!!
33 Replies
Percy
Percyā€¢14mo ago
Project ID: a460e728-9647-40f6-811c-fe5d99abb663
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
a460e728-9647-40f6-811c-fe5d99abb663 I've added it as an environment variable: NIXPACKS_APT_PKGS yet it still errors when running (it is there but does not run properly). I have also tried to do it in the railway.json Here is the actual error that shows up when endpoint is pinged: /bin/wkhtmltopdf: /nix/store/wprxx5zkkk13hpj6k1v6qadjylh3vq9m-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libQt5WebKit.so.5) INFO: 192.168.0.2:43880 - "GET / HTTP/1.1" 200 OK
Brody
Brodyā€¢14mo ago
could you undo everything you have tried and then give me the error again? you may have made things worse
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
haha oops! Let's do it again Okay, so when I don't do anything with the Aptfile, etc it of course shows no wkhtmltopdf exists (OSError: No wkhtmltopdf executable found: "b''") When I add an environment variable for NIXPACKS_APT_PKGS with value wkhtmltopdf, I see that it is being installed in the build logs which is a good sign. This is the error I get starting from where it's relevant pdf_bytes = pdfkit.from_string(html, False) File "/opt/venv/lib/python3.8/site-packages/pdfkit/api.py", line 75, in from_string return r.to_pdf(output_path) File "/opt/venv/lib/python3.8/site-packages/pdfkit/pdfkit.py", line 201, in to_pdf self.handle_error(exit_code, stderr) File "/opt/venv/lib/python3.8/site-packages/pdfkit/pdfkit.py", line 158, in handle_error raise IOError("wkhtmltopdf exited with non-zero code {0}. error:\n{1}".format(exit_code, error_msg)) OSError: wkhtmltopdf exited with non-zero code 1. error: /bin/wkhtmltopdf: /nix/store/wprxx5zkkk13hpj6k1v6qadjylh3vq9m-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libQt5WebKit.so.5)
Brody
Brodyā€¢14mo ago
please remove any attempts you are making at installing wkhtmltopdf yourself, and then show me the error
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
Sorry, I'm not quite sure what you mean? When I don't have it anywhere in my code, the error is as shown above: OSError: No wkhtmltopdf executable found: "b''"
Brody
Brodyā€¢14mo ago
remove any aptfiles and NIXPACKS_APT_PKGS
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
Yep, all done
Brody
Brodyā€¢14mo ago
in this you mention NIXPACKS_APT_PKGS with value wkhtmltopdf, if you remove this variable do you still get the same error im looking for the error before you started trying to fix this yourself
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
Sorry, let me clarify...it builds properly yes but when I actually test my app it just says wkhtmltopdf doesn't exist If you look at the most recent error in the deploy logs you can see it
Brody
Brodyā€¢14mo ago
maybe there is some confusion here, i cant access your project at all thats why im asking for you to provide me with the error before you started trying to fix it yourself
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
pdf_bytes = pdfkit.from_string(html, False) File "/opt/venv/lib/python3.8/site-packages/pdfkit/api.py", line 72, in from_string r = PDFKit(input, 'string', options=options, toc=toc, cover=cover, css=css, File "/opt/venv/lib/python3.8/site-packages/pdfkit/pdfkit.py", line 45, in init self.configuration = (Configuration() if configuration is None File "/opt/venv/lib/python3.8/site-packages/pdfkit/configuration.py", line 38, in init raise IOError('No wkhtmltopdf executable found: "%s"\n' OSError: No wkhtmltopdf executable found: "b''" If this file exists please check that this process can read it or you can pass path to it manually in method call, check README. Otherwise please install wkhtmltopdf - https://github.com/JazzCore/python-pdfkit/wiki/Installing-wkhtmltopdf
GitHub
Installing wkhtmltopdf
Wkhtmltopdf python wrapper to convert html to pdf. Contribute to JazzCore/python-pdfkit development by creating an account on GitHub.
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
So this is the error when the package doesn't exist at all This is what the app was at before I tried fixing it myself
Brody
Brodyā€¢14mo ago
there we go, thats what im asking for
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
Sorry about that!
Brody
Brodyā€¢14mo ago
add this as a nixpacks.toml file to your project
[phases.setup]
nixPkgs = ["...", "wkhtmltopdf-bin"]
[phases.setup]
nixPkgs = ["...", "wkhtmltopdf-bin"]
untested of course, so i am not fully sure this would work
arandomfellanamedmike
arandomfellanamedmikeā€¢14mo ago
Let's give it a shot! There we go! Works beautifully! You are the best, thanks!
Brody
Brodyā€¢14mo ago
no problem, glad i could help šŸ™‚
patmw
patmwā€¢8mo ago
Hey there, Hoping someone can help me out - the problem doesn't seem to be the same but it's related to the same package (pdfkit / wkhtmltopdf) I'm running a FastAPI application on Railway and I'm using PDFKit to convert templated HTML files to pds, which are attached to emails that I send out to my users when they purchase a ticket I'm having a strange issue, and can't really debug any further because the package is not throwing any errors On localhost, the application works as expected. On the production environment, the pdf is created but the formatting is all wrong -----
patmw
patmwā€¢8mo ago
1. Template HTML File
patmw
patmwā€¢8mo ago
2. Output on localhost:
patmw
patmwā€¢8mo ago
3. Output from production:
patmw
patmwā€¢8mo ago
---- Relevant code:
def generate_ticket_pdf_from_template(
qr_code_urls: List[str],
event_name: str,
customer_name: str,
order_id: str,
purchase_date: str,
event_image_url: str,
):
env = Environment(loader=FileSystemLoader("."))
template = env.get_template("./static/templates/ticket_page.html")

html_pages = []

log.info(f"got template: {template}")
for i, url in enumerate(qr_code_urls):
log.info(f"rendering html template with qr code: {url}")
# Render the template with the variables
html = template.render(
order_id=order_id,
event_name=event_name,
customer_name=customer_name,
current_ticket_index=str(i + 1),
total_ticket_index=len(qr_code_urls),
purchase_date=purchase_date,
qr_code_url=url,
event_image_url=event_image_url,
)
# log.info(f"rendered template: {html}")
html_pages.append(html)

log.info(f"generated html pages: {len(html_pages)} pages total")
email_content = "<div style='page-break-after: always;'></div>".join(html_pages)

log.info(f"generated html content")

# Convert the HTML to PDF

log.info(f"attempting to convert html to pdf")
path = f"/temp/FarragoTickets_{order_id}.pdf"
log.info(f"attempting to save pdf to: {path}")
try:
pdfkit.from_string(email_content, path, options={"enable-local-file-access": ""})
except Exception as e:
log.error(f"*** Error converting html to pdf*** {e}", exc_info=True)
raise Exception(f"{e}")

log.info(f"generated pdf: {path}")
return path
def generate_ticket_pdf_from_template(
qr_code_urls: List[str],
event_name: str,
customer_name: str,
order_id: str,
purchase_date: str,
event_image_url: str,
):
env = Environment(loader=FileSystemLoader("."))
template = env.get_template("./static/templates/ticket_page.html")

html_pages = []

log.info(f"got template: {template}")
for i, url in enumerate(qr_code_urls):
log.info(f"rendering html template with qr code: {url}")
# Render the template with the variables
html = template.render(
order_id=order_id,
event_name=event_name,
customer_name=customer_name,
current_ticket_index=str(i + 1),
total_ticket_index=len(qr_code_urls),
purchase_date=purchase_date,
qr_code_url=url,
event_image_url=event_image_url,
)
# log.info(f"rendered template: {html}")
html_pages.append(html)

log.info(f"generated html pages: {len(html_pages)} pages total")
email_content = "<div style='page-break-after: always;'></div>".join(html_pages)

log.info(f"generated html content")

# Convert the HTML to PDF

log.info(f"attempting to convert html to pdf")
path = f"/temp/FarragoTickets_{order_id}.pdf"
log.info(f"attempting to save pdf to: {path}")
try:
pdfkit.from_string(email_content, path, options={"enable-local-file-access": ""})
except Exception as e:
log.error(f"*** Error converting html to pdf*** {e}", exc_info=True)
raise Exception(f"{e}")

log.info(f"generated pdf: {path}")
return path
Railway logs:
INFO generating pdf with qr codes: ['https://admin.farrago.club/_/api/qr/[542604,879083142]', 'https://admin.farrago.club/_/api/qr/[542604,879083143]', 'https://admin.farrago.club/_/api/qr/[542604,879083145]']

/app/app/domain/ticketing/__init__.py:60

INFO got template: <Template './static/templates/ticket_page.html'>

/app/app/domain/pdf/__init__.py:20

INFO rendering html template with qr code: https://admin.farrago.club/_/api/qr/[542604,879083142]

/app/app/domain/pdf/__init__.py:22

INFO rendering html template with qr code: https://admin.farrago.club/_/api/qr/[542604,879083143]

/app/app/domain/pdf/__init__.py:22

INFO rendering html template with qr code: https://admin.farrago.club/_/api/qr/[542604,879083145]

/app/app/domain/pdf/__init__.py:22

INFO generated html pages: 3 pages total

/app/app/domain/pdf/__init__.py:37

INFO generated html content

/app/app/domain/pdf/__init__.py:40

INFO attempting to convert html to pdf

/app/app/domain/pdf/__init__.py:44

INFO attempting to save pdf to: /temp/FarragoTickets_3x5-76r.pdf

/app/app/domain/pdf/__init__.py:46

INFO generated pdf: /temp/FarragoTickets_3x5-76r.pdf

/app/app/domain/pdf/__init__.py:53

INFO send_email to: pmassowalsh@gmail.com

/app/app/domain/emails/sendgrid.py:34

INFO substituting content for template: /app/static/templates/email_generic.html

/app/app/domain/emails/sendgrid.py:36

INFO substituting variables in email content

/app/app/domain/emails/sendgrid.py:42

INFO Email sent successfully. Status code: 202

/app/app/domain/emails/sendgrid.py:83
INFO generating pdf with qr codes: ['https://admin.farrago.club/_/api/qr/[542604,879083142]', 'https://admin.farrago.club/_/api/qr/[542604,879083143]', 'https://admin.farrago.club/_/api/qr/[542604,879083145]']

/app/app/domain/ticketing/__init__.py:60

INFO got template: <Template './static/templates/ticket_page.html'>

/app/app/domain/pdf/__init__.py:20

INFO rendering html template with qr code: https://admin.farrago.club/_/api/qr/[542604,879083142]

/app/app/domain/pdf/__init__.py:22

INFO rendering html template with qr code: https://admin.farrago.club/_/api/qr/[542604,879083143]

/app/app/domain/pdf/__init__.py:22

INFO rendering html template with qr code: https://admin.farrago.club/_/api/qr/[542604,879083145]

/app/app/domain/pdf/__init__.py:22

INFO generated html pages: 3 pages total

/app/app/domain/pdf/__init__.py:37

INFO generated html content

/app/app/domain/pdf/__init__.py:40

INFO attempting to convert html to pdf

/app/app/domain/pdf/__init__.py:44

INFO attempting to save pdf to: /temp/FarragoTickets_3x5-76r.pdf

/app/app/domain/pdf/__init__.py:46

INFO generated pdf: /temp/FarragoTickets_3x5-76r.pdf

/app/app/domain/pdf/__init__.py:53

INFO send_email to: pmassowalsh@gmail.com

/app/app/domain/emails/sendgrid.py:34

INFO substituting content for template: /app/static/templates/email_generic.html

/app/app/domain/emails/sendgrid.py:36

INFO substituting variables in email content

/app/app/domain/emails/sendgrid.py:42

INFO Email sent successfully. Status code: 202

/app/app/domain/emails/sendgrid.py:83
--- As you can see, there are no errors thrown, and the app continues to send the email with the badly formatted pdf Not sure if this is related or not, but previously I was getting the following error log on production:
ERROR Error generating pdf: wkhtmltopdf exited with non-zero code 1. error:

Fontconfig error: Cannot load default config file: No such file: (null)

QPainter::begin(): Returned false

Exit with code 1, due to unknown error.

/app/app/domain/ticketing/__init__.py:70
ERROR Error generating pdf: wkhtmltopdf exited with non-zero code 1. error:

Fontconfig error: Cannot load default config file: No such file: (null)

QPainter::begin(): Returned false

Exit with code 1, due to unknown error.

/app/app/domain/ticketing/__init__.py:70
This seems to actually be two separate issues, but the issue went away when I added a file volume to my production deployment within Railway dashboard. (I searched for QPainter::begin(): Returned false and someone suggested that it could be due to a writing to disk issue) The file volume has made these errors go away - but I can't help but feel like theres something weird going on with the config for the underlying binaries I'm way in over my head with this - my devops experience is very limited (hence why I am using Railway). I hope someone can help! Thank you ā¤ļø @Brody tagging you because you seemed to be very on it with the previous thread
Brody
Brodyā€¢8mo ago
well first off, your html template file has 37 validation errors according to https://validator.w3.org/#validate_by_upload did you use the nixpacks.toml file i sent above?
patmw
patmwā€¢8mo ago
speedy response, appreciated. The html template is a slightly modified exported version from SendGrid's designer (https://docs.sendgrid.com/ui/sending-email/editor) I'll check the validation issues but it's strange that it worked just fine on my local - and the html file also compiles fine if you run it in a live server from VSCode / open it in chrome
Design & Code Editor
Twilio SendGrid Marketing Campaigns' editing gives you complete control over your emails. Use a flexible drag-and-drop Design editor or a robust HTML code editor.
patmw
patmwā€¢8mo ago
Yup, first thing I did was add the nixpacks.toml file - before I even tried merging to master
Brody
Brodyā€¢8mo ago
maybe adding fontconfig will help?
[phases.setup]
nixPkgs = ['...', 'wkhtmltopdf-bin', 'fontconfig']
[phases.setup]
nixPkgs = ['...', 'wkhtmltopdf-bin', 'fontconfig']
patmw
patmwā€¢8mo ago
makes sense, let me give that a go
patmw
patmwā€¢8mo ago
didn't do the trick im afraid. I added fontconfig but the file is the same with the weird left-formatting, no images, etc. (no changes to the fonts either)
patmw
patmwā€¢8mo ago
What should I look out for in the build logs to make sure that these binaries were installed properly?
Brody
Brodyā€¢8mo ago
check the build table at the top of the build logs
patmw
patmwā€¢8mo ago
ā•”ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā• Nixpacks v1.19.0 ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•—

ā•‘ setup ā”‚ python310, gcc, wkhtmltopdf-bin, fontconfig ā•‘

ā•‘ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā•‘

ā•‘ install ā”‚ python -m venv --copies /opt/venv && . /opt/venv/bin/activate ā•‘

ā•‘ ā”‚ && pip install -r requirements.txt ā•‘

ā•‘ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā•‘

ā•‘ start ā”‚ hypercorn main:app --bind "[::]:$PORT" ā•‘

ā•šā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•
ā•”ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā• Nixpacks v1.19.0 ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•—

ā•‘ setup ā”‚ python310, gcc, wkhtmltopdf-bin, fontconfig ā•‘

ā•‘ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā•‘

ā•‘ install ā”‚ python -m venv --copies /opt/venv && . /opt/venv/bin/activate ā•‘

ā•‘ ā”‚ && pip install -r requirements.txt ā•‘

ā•‘ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā•‘

ā•‘ start ā”‚ hypercorn main:app --bind "[::]:$PORT" ā•‘

ā•šā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•ā•
looks like it did work (im suspecting that the issue is in fact with the html file so I'll try again tomorrow morning with a fresh template) Will report back, nonetheless I massively appreciate your speedy support ā¤ļø
Brody
Brodyā€¢8mo ago
no problem!