Cloudflare Developers•13mo ago

email worker webhook not working

I am currently trying to relay emails sent to a worker to my webhook. it works fine when i send a test on the cloudflare site but when a real email is sent it doesnt send anything to my webhook. in the overview tab it shows that the email was received (although it says it was dropped ive read in other threads that that's just a default response) nothing is sent to the webhook. I am in control of the server and have put as many things as i can think of to log or find out what the issue is but ive come to the conclusion that it just isnt sending the request. anyone have any idea how to fix it?

61 Replies

Chaika•13mo ago

The most helpful tool you have for debugging live workers is Tailing them, either via wrangler tail or the dashboard, in the Worker under Logs -> Begin Log Stream A few things I see: Why is that http and not https? Are you specifying a port? Custom Ports would get dropped I doubt you can just stringify the message, most likely you would at least lose the body Yea JSON doesn't know how to serialize the message object, it just returns {}. You can just pass the message body itself as it's a readable stream, like this:

Gute NachtOP•13mo ago

im using http because its not a domain its my vps ip yes i am specifcying a port

Chaika•13mo ago

You can't fetch raw IPs in Workers, have to be domains

Gute NachtOP•13mo ago

well thats the issue then

Chaika•13mo ago

and yea ports just get dropped unless the Worker is executing on the same zone as the fetch target (ex. a Worker running on worker.example.com can fetch origin.example.com with a custom port (with some restrictions), but Email Workers execute on your workers.dev so it'll just always drop it

Chaika•13mo ago

zone = website in Cloudflare, ex. anything under the "Websites" tab (Just including that flow chart for completeness, like I said none of that really matters to you other then the fact ports just aren't supported in email worker fetches since it'll never be same-zone) As for the worker code itself, you can use something like this:

export default {
  async email(message, env, ctx) {
    await fetch("https://yourwebserver.example.com", {
      body: message.raw,
      method: "POST",
      headers: {
        "email-to": message.to,
        "email-from": message.from,
        "apikeyorsomeidentifideryouuse": env.secret,
      }
    });  
  }
}

export default {
  async email(message, env, ctx) {
    await fetch("https://yourwebserver.example.com", {
      body: message.raw,
      method: "POST",
      headers: {
        "email-to": message.to,
        "email-from": message.from,
        "apikeyorsomeidentifideryouuse": env.secret,
      }
    });  
  }
}

That would forward the email, and the from/to properties, to a target. You could then just read the request body to get the raw email

Gute NachtOP•13mo ago

tjaml ypu chjaika

Chaika•13mo ago

no worries. Email Workers are super cool but don't have the most examples and such around them right now, can be a bit hard to get started. If your intent is to use your VPS for actual production use, Cloudflare offers Origin Certificates under your website -> SSL/TLS -> Origin Server, they work with proxy enabled. You can create a DNS A Record to point at your server, create and install an origin ca certificate on it, and then use that to pass the email event to it. If you run into any issues, remember that Tail is your best bet when debugging deployed Workers. npx wrangler tail if you're using Wrangler or go to the Worker in the dashboard, Logs -> Real-time Logs -> Begin Log Stream, and wait a few seconds and then send the email. You should see the email event after a bit and any logs/exceptions that occured.

Gute NachtOP•13mo ago

@Chaika any ideas?

Chaika•13mo ago

That doesn't send JSON, that sends just the raw email Looks like your using Express with body-parser which thinks its JSON, probably because you're setting Content-type json or something else

Gute NachtOP•13mo ago

mesage.raw does the same thing

Chaika•13mo ago

message.raw isn't JSON either

Gute NachtOP•13mo ago

is there a way for me to use like axios for this or a different library ive never used fetch before 😭

Chaika•13mo ago

no but it wouldn't help I would drop the Content-Type: 'application/json' everything else though is on Express's end

Chaika•13mo ago

message.raw is the raw rfc822 formatted email, it's not valid JSON. Your backend (the express app, not the worker) needs to not treat it as JSON either. Right now it is, hence the error. I'm not too familiar with Express or body-parser, but the first thing I see off is you setting the Content-type in the worker fetch. Your backend can use a library like https://www.npmjs.com/package/mailparser to parse that email

npm

mailparser

Parse e-mails. Latest version: 3.6.5, last published: 6 months ago. Start using mailparser in your project by running npm i mailparser. There are 504 other projects in the npm registry using mailparser.

Gute NachtOP•13mo ago

if i do that the server no longer receives it

Chaika•13mo ago

hmm that's all Express stuff handling it. You could try setting it to application/octet-stream then, and if you're using the body-parser middleware it looks like you may need to change it to accept that On the worker side of things you could log the response it's getting as well if it helps, namely the status and any response. But yea, you're into Express land now. I'm no expert in that, Workers has no issue fetching/posting it though

Gute NachtOP•13mo ago

hey @Chaika message.raw is undefined

Gute NachtOP•13mo ago

it works sorta now tho

Gute NachtOP•13mo ago

other than that part

Chaika•13mo ago

What's your full worker code? (removing anything sensitive) If you're testing in the simulator it won't work I believe

Gute NachtOP•13mo ago

export default {
  async email(message, env, ctx) {
    const apiKey = "c708b14fc6752ab...";

    const response = await fetch("https://emails...../add/", {
      body: message.raw,
      method: "POST",
      headers: {
        'Accept': 'application/octet-stream',
        'Content-Type': 'application/octet-stream',
        "email-to": message.to,
        "email-from": message.from,
        "authorization": apiKey,
      }
    })

    console.log(response.status);

    console.log(message.raw);
  }
}

export default {
  async email(message, env, ctx) {
    const apiKey = "c708b14fc6752ab...";

    const response = await fetch("https://emails...../add/", {
      body: message.raw,
      method: "POST",
      headers: {
        'Accept': 'application/octet-stream',
        'Content-Type': 'application/octet-stream',
        "email-to": message.to,
        "email-from": message.from,
        "authorization": apiKey,
      }
    })

    console.log(response.status);

    console.log(message.raw);
  }
}

Chaika•13mo ago

If you tail your worker, does it also say undefined for the message?

Gute NachtOP•13mo ago

yes

Chaika•13mo ago

nah I mean the worker itself not on Express's end

Gute NachtOP•13mo ago

yes it does

Gute NachtOP•13mo ago

Chaika•13mo ago

the simulator won't work with it have to test on a live email and tail It's going to be an issue on Express's end recieving it, message.raw is almost certainly not undefined when using it in production, but can always double check

Chaika•13mo ago

ex: this simple code

export default {
  async email(message, env, ctx) {
    console.log(message.raw);

  }
}

export default {
  async email(message, env, ctx) {
    console.log(message.raw);

  }
}

works fine, it's not going to log the object but it does at least confirm it's not undefined

Chaika•13mo ago

(If it helps to clarify tailing, start the tail from your Worker in the dash under Logs -> Real-time logs -> Start log stream, wait a few secs and send the email, and then you should see the event after a bit from the email, and you can click to expand to see all logs and such)

Gute NachtOP•13mo ago

I see yea you're right that's weird well uh what might i do about that

Chaika•13mo ago

If it helps to have a direction to look into, when I was looking into body-parser earlier (keeping in mind I haven't used Express very much), I noticed there's an option to use it with raw bodies when the content-type matches (ex: can use it for octet-stream) https://www.npmjs.com/package/body-parser#bodyparserrawoptions I'm guessing right now if the middleware has no idea how to handle it, it just sends undefined

npm

body-parser

Node.js body parsing middleware. Latest version: 1.20.2, last published: 10 months ago. Start using body-parser in your project by running npm i body-parser. There are 23288 other projects in the npm registry using body-parser.

Gute NachtOP•13mo ago

im not even using body-parser anytmore

Chaika•13mo ago

where does that log come from?

Gute NachtOP•13mo ago

Chaika•13mo ago

looks like without body-parser, Express doesn't do anything with body because it has no idea how to use it (src: https://stackoverflow.com/a/66555975) could go back to body-parser or use something like this: https://stackoverflow.com/a/26736672

Gute NachtOP•13mo ago

ok it is now this

Chaika•13mo ago

nice, looks like the email

Chaika•13mo ago

not sure what your goal is, but you can use a library like https://www.npmjs.com/package/mailparser to parse it out further

npm

mailparser

Gute NachtOP•13mo ago

what's all this about

Chaika•13mo ago

hmm, how are you using the library? For me, this works fine

import express from 'express';
import bodyParser from 'body-parser';
import { simpleParser } from 'mailparser';

const app = express();

// Middleware to parse raw bodies
app.use(bodyParser.raw());

// Routes
app.post('/', async (req, res) => {
    try {
        // Parse the email body
        const parsedMail = await simpleParser(req.body);
        console.log(parsedMail)
        console.log(JSON.stringify(parsedMail, null, 2));
        // Grab the subject
        const subject = parsedMail.subject;
        console.log(subject);
        res.send(`Received the subject: ${subject}`);
    } catch (error) {
        console.error(error);
        res.status(500).send('An error occurred while parsing the email.');
    }
});

// Start the server
const port = 3000;
app.listen(port, () => {
    console.log(`Server is running on port ${port}`);
});

import express from 'express';
import bodyParser from 'body-parser';
import { simpleParser } from 'mailparser';

const app = express();

// Middleware to parse raw bodies
app.use(bodyParser.raw());

// Routes
app.post('/', async (req, res) => {
    try {
        // Parse the email body
        const parsedMail = await simpleParser(req.body);
        console.log(parsedMail)
        console.log(JSON.stringify(parsedMail, null, 2));
        // Grab the subject
        const subject = parsedMail.subject;
        console.log(subject);
        res.send(`Received the subject: ${subject}`);
    } catch (error) {
        console.error(error);
        res.status(500).send('An error occurred while parsing the email.');
    }
});

// Start the server
const port = 3000;
app.listen(port, () => {
    console.log(`Server is running on port ${port}`);
});

Gute NachtOP•13mo ago

Chaika•13mo ago

Interesting.. I tested the full thing end to end and it still seems to work fine to me. Can you share a more full code example? Is your handler async?

Gute NachtOP•13mo ago

Chaika•13mo ago

How are you importing that/which package are you using?

Gute NachtOP•13mo ago

same as you import { simpleParser } from "mailparser"; "mailparser": "^3.6.5",

Chaika•13mo ago

if it helps to explain, to me this output would indicate that whatever input you gave it was garbage/not a proper rfc822 email so would look up the stack and try to figure out why are you using bodyparser or the other hack to parse the body?

Gute NachtOP•13mo ago

yes i am app.use(bodyParser.raw()); im afool im srory its b ase64 of the thing because iw as testing and forgot to undo that 😅

Chaika•13mo ago

ahh ok, there's various email samples you can use online to test sending with, makes it easier to debug and rule out the worker part. I grabbed this one here https://github.com/FlexConfirmMail/Thunderbird/blob/master/sample.eml while I was testing, and then can just use a REST client like Insomnia to test Not sure if that's not already what you're doing, but just in case, it's a lot easier

GitHub

Thunderbird/sample.eml at master · FlexConfirmMail/Thunderbird

Thunderbird向けの誤送信防止アドオン. Contribute to FlexConfirmMail/Thunderbird development by creating an account on GitHub.

Gute NachtOP•13mo ago

now its smepty

Chaika•13mo ago

looks like that happens when you give it no input / empty file

Gute NachtOP•13mo ago

intersting

Gute NachtOP•13mo ago

Chaika•13mo ago

what happens when you try to convert an object into a string if you bypass the Worker side of things and send a request with a test email to your API, does it work?

Gute NachtOP•13mo ago

no still like this

Gute NachtOP•13mo ago

req.body is the thing below ☝️

Chaika•13mo ago

add logs for length and converting it to utf8, ex:

 console.log(req.body.length)
 console.log(req.body.toString());

 console.log(req.body.length)
 console.log(req.body.toString());

so you can see what you're getting in

Gute NachtOP•13mo ago

umm something interesting ahppened now i think it decided to work randomly

Chaika•13mo ago

Well that's something. No reason for it to not work in the first place anyway, if your code was almost the same as mine express/worker wise, mine was working without any issues

Gute NachtOP•13mo ago

well tysm chaika

Gaming

Programming

email worker webhook not working