pagerduty and sentry email integration

When adding a new generic email integration into PagerDuty, you are greeted with the following defaults:

Screenshot from 2015-12-15 13:54:18

If you are integrating with Sentry, then this default behavior will quickly annoy you because you will get a brand new PagerDuty incident each time someone makes a comment on the Sentry report.

Luckily, you can easily write rudimentary email filters in PagerDuty, but what to filter on?

It turns out that Sentry includes a plaintext section in the mails they send out, and there seems to be at least one regularly occurring pattern in new Sentry event mails, namely the string:

A new event has been recorded in Sentry:

Armed with this knowledge, we can write a simple PagerDuty filter so that new Sentry events create new PagerDuty incidents, but comments on existing Sentry events are ignored.

Screenshot from 2015-12-15 10:51:51

Things that make me sad:

  • no pre-canned Sentry integration in PagerDuty
  • PagerDuty email filters only work on subject line, body, and sender; why can’t I filter on the message headers?
  • Sentry includes an “ID” + a unique event identifier in their email, but only in the mail on the first event; on subsequent mails, the unique identifier is no longer present, which prevents me from writing a better filter in PagerDuty that could theoretically find the unique identifier and link followup emails together

So, I guess we are still stuck with “parsing email bodies as an API” 1990s Linux-landia. Won’t some nice VC throw a few hundred $million at this sure-to-be-a-unicorn market?

a love letter to instacart

I recently signed up for Instacart Express, which starts you off with a free two-week trial. After last night, I cancelled my subscription before they billed me.

Instacart sent me an automated email asking why, and this was my response:

I really wanted to love Instacart. I thought it was going to change my life.

But my experience during your two week trial has been a clown show.

The first order that was screwed up was XXXXXXX. We ordered shallots and the shopper gave us garlic instead. If you’ve ever cooked anything more complex than a Hot Pocket, you’d know that those items are not substitutes for each other.

On top of this, you charged an incredible $3.69 per shallot. I can only assume these shallots were eaten by civets, crapped out, and then harvested by the only migrant workers in America making a fair living wage.

To your credit, you did fix the issue by re-delivering these magical shallots for free the next morning, and in the end, it was a very minor incident after all.

But tonight was a very special experience indeed. And by “special” I mean, “not special”.

On order XXXXXXX, I placed it quite early in the day with a delivery time for 7 — 8pm.

At 8:30pm, after receiving no update from Instacart, I sent a ping to the shopper asking if there were any issues with the order, and received this text in response:


Pretty brilliant, I must say.

So, in summary, thank you for the free two week trial of your service. I learned that I would have been signing up for a year of disappointment for $99, and for that price, I could donate money to the Trump campaign and gotten far more entertainment value out of it.

Maybe you can make me feel better. But seriously. The Donald. Gotta be better than The Donald.

To Instacart’s credit, they responded quickly, offering me another free month of Instacart Express and a $25 coupon. So, I guess we’ll continue with the experiment, as I really want this service to work for us.

Thanks Michael P!