Hey there, in a previous life, I led the team that built the "new" webhook delivery system at Shopify (now 2+ years old).
A few notes / questions:
1. How do you deal with endpoints that are down or 500'ing? I saw "Smart Retry", but it doesn't say what kind of retry policy or backoff occurs. Related, how do you notify clients when their endpoints are having trouble? (I ask because that's PII that I then have to share with you).
2. Is there support for message signing (e.g., HMAC) to let clients verify that the webhook really came from us? How does it work?
3. Any kind of deduplication support? One use case we had was that certain webhooks only required the latest delivery. e.g., product inventory. If we previously failed to deliver a webhook, but have a newer version pending, our next attempt should try to send the newest version.
4. Is it possible to delay hydrating data in the payload? It seems the expected usage is that I send a blob of data and you take care of the last mile and send it. Is it possible to send you an id and then you call back to my service and fetch the latest version of that object just before delivery?
5. How are webhook subscriptions actually managed? e.g., my app lets users register webhook URLs? How do I get those URLs to your service?
6. The big question: What do I do if your service is down?
Congrats on the launch. 100% agree there are a lot of requirements in building a decent webhook delivery system. HootSuite covered a lot of them in their blog post from a few years back:
Hey Kenrose, At Hookdeck(https://hookdeck.io) We have built a system that deals with most of the questions you raised above. I am willing to chat more about Hookdeck & the problem webhook problem we are solving. At Hookdeck, we love Shopify & even have a platform guide which explores the "new" Shopify webhooks https://hookdeck.io/platforms/shopify-webhooks-features-and-...
Wish you'd made this two months ago - had to build a lot of this from scratch at my startup and would've totally used something like this! I tried looking on the internet, and found basically nothing.
[+] [-] kenrose|5 years ago|reply
A few notes / questions:
1. How do you deal with endpoints that are down or 500'ing? I saw "Smart Retry", but it doesn't say what kind of retry policy or backoff occurs. Related, how do you notify clients when their endpoints are having trouble? (I ask because that's PII that I then have to share with you).
2. Is there support for message signing (e.g., HMAC) to let clients verify that the webhook really came from us? How does it work?
3. Any kind of deduplication support? One use case we had was that certain webhooks only required the latest delivery. e.g., product inventory. If we previously failed to deliver a webhook, but have a newer version pending, our next attempt should try to send the newest version.
4. Is it possible to delay hydrating data in the payload? It seems the expected usage is that I send a blob of data and you take care of the last mile and send it. Is it possible to send you an id and then you call back to my service and fetch the latest version of that object just before delivery?
5. How are webhook subscriptions actually managed? e.g., my app lets users register webhook URLs? How do I get those URLs to your service?
6. The big question: What do I do if your service is down?
Congrats on the launch. 100% agree there are a lot of requirements in building a decent webhook delivery system. HootSuite covered a lot of them in their blog post from a few years back:
https://medium.com/hootsuite-engineering/a-scalable-reliable...
[+] [-] GhvstCode|5 years ago|reply
[+] [-] kenrose|5 years ago|reply
https://gowebhooks.github.io/learning-lab/tailwind-starter-k...
when I click on your logo.
[+] [-] davidchuchu|5 years ago|reply
[+] [-] mh44|5 years ago|reply
[+] [-] tom23444|5 years ago|reply