[feat] Webhooks for incoming email: can be very big; plain text only?

[jamescridland]

Love this service. And particularly, I like the fact you've got logs! Excellent.

So, I've been using a webhook as a way of getting incoming mail (actually, to check that the recipients are subscribed to the newsletter that I make, and respond with a nice message asking them to if they don't). And I notice that the following simple line of PHP, which reads the message into an array...

$message = json_decode(file_get_contents('php://input'),true);

...is causing a good deal of memory exhaustion.

[13-Feb-2025 05:15:14 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 67108872 bytes) in /var/www/html/(snip).php on line 9
[13-Feb-2025 05:29:14 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388616 bytes) in /var/www/html/(snip).php on line 9
[13-Feb-2025 05:29:24 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388616 bytes) in /var/www/html/(snip).php on line 9
[13-Feb-2025 05:43:56 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 67108872 bytes) in /var/www/html/(snip).php on line 9
[13-Feb-2025 05:46:17 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 67108872 bytes) in /var/www/html/(snip).php on line 9

Of course, the issue is that people send me attachments. Those attachments are bloody massive, and this causes this PHP script to run out of memory quite fast.

Yes, I should handle this better. Somehow. Not quite sure how.

But - is there a feature request, perhaps, to be able to configure that attachments should NOT be sent in a webhook? That could be really helpful and useful if it were possible.

[titanism]

Could you use https://github.com/salsify/jsonstreamingparser?tab=readme-ov-file#usage or https://github.com/halaxa/json-machine to prevent these exceptions?

Another thing you could do is if these are saved to the file system, you could use jq to delete the key attachments and its value before you run it with your PHP script, e.g. https://unix.stackexchange.com/a/700681.

[jamescridland]

@titanism Thanks - interesting ideas. If I've understood both packages correctly, though, they're made for big database-like chunks of JSON row-by-row (it's more obvious with the json-machine package), not just one big object (which the email is).

Interesting idea to save it to the file system first and then do operations on it there. I'm literally only pulling out the "from" email address and the text message body; so that could be perfectly possible to treat the file first. I'll keep thinking.