top | item 27598164

Show HN: DNS-powered website with no back end

83 points| elliottinvent | 4 years ago |companydirectory.uk | reply

89 comments

order
[+] elliottinvent|4 years ago|reply
All of the contact data on this website is fetched from DNS TXT records. It's an example of how NUM [0] data can be used.

A small team and I have created NUM because we believe that paid, restricted, rate-limited APIs provided by the giants of the web are holding back developers and we want to change that.

NUM provides: a simple way for businesses to provide machine-readable data direct to their users, bypassing data hoarding web giants; a forever free, unlimited and unrestricted source of data to developers.

The general-purpose semantic web has failed because the standards (microdata, JSON-LD, Open Graph) are too complicated and difficult for small businesses to adopt and often forgotten about by marketing teams in large businesses because data is too difficult to edit.

NUM is different because it can be adopted by any business using a simple online form. The protocol includes two DNS queries – the first to the authoritative DNS zone; the second failover query to our NUM Server, any business can claim their domain on NUM Server using a simple online process.

The Company Directory website linked to is just one example of how NUM data can be used. The website fetches all data from DNS using client-side Cloudflare DNS-over-HTTPS calls (see Network tab in Dev Tools). Barclays, or any other company listed can update the data on this site by adopting the NUM protocol [1] independently in their own DNS – for example, to replace the data for the Barclays listing, they can create a TXT record at 1._num.barclays.co.uk or claim the record shown by visiting numserver.com/claim/barclays.co.uk

DNS-based protocols like NUM (other examples are SPF and DMARC) typically suffer from the chicken-and-egg problem where no-one looks for this data in DNS because it's not there, and no one stores this data in DNS because no-one is looking for it.

To help overcome this, we've pre-populated the DNS with contact data for large UK companies and are in the process of automatically gathering contact data for all *.uk domain names (we expect to complete this in the next 10 days) before moving on to all other UK companies, then companies worldwide (the US, Canada, Australia next).

NUM can be adopted / edited with a simple online form. Anyone can create a record for their domain, either independently in their own DNS or using our service at https://app.numserver.com/tools/editor/add

To ensure that NUM is as efficient as possible, we store all data in DNS using MODL [2] – a compact, DNS-friendly data serialisation format; we make compact DNS objects developer friendly using Unpacker [3] – we have developed both in-house. I have written a very basic Notion document to explain how all of these technologies fit together [4].

To simplify all of this for front-end developers (who by and large don't care about DNS), we've packaged it up in a Typscript library called company-api which you can query for a domain and get a beautiful object back containing company contact data [5].

We are very keen for feedback, good or bad.

0. https://www.num.uk

1. https://www.numprotocol.com

2. https://www.modl.uk

3. https://www.unpacker.uk

4. https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d...

5. https://www.npmjs.com/package/company-api

Edit: to make it clearer what problem we're trying to solve.

[+] tptacek|4 years ago|reply
I guess I don't understand how DNS records work around the problem of metered APIs, since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records. Also: DNS is much, much slower than an HTTP API, which can fetch arbitrary, specific collections of records for a single query.

I'm not trying to shoot this down so much as figure out what I'm missing here.

[+] faeyanpiraat|4 years ago|reply
This won't work.

If companies cannot be bothered to make their websites user friendly in the first place, why would they do it in this new and obscure way?

It's like the episode on Dragon's Den where someone tried pitching a service that would call companies on your behalf, and would only call you to connect both ends when the music stops and a call center agent actually picks up the phone.

It's the job of the company to fix their long wait times, not of some 3rd party service, which may or may not work with a specific customer service number.

[+] bawolff|4 years ago|reply
But umm, why? Seems like you're in essence just using an https api to fetch dns data. So why not just use an https api and cut out the extra step?
[+] krakov|4 years ago|reply
Incidentally, it looks like Semantic Web is not dead but actually accelerating growth -thanks to current SEO practices abd Google showing semantic data within search results [1]

As for complicated , yeah, but business web platforms like Wix support those as part of SEO capabilities [2]. So you would you need the smb web dev platforms to support NUM to make it a reality?

1. http://webdatacommons.org/structureddata/#toc3

2. https://support.wix.com/en/article/adding-structured-data-to...

[+] chrisweekly|4 years ago|reply
Brilliant. Stoked to give this a try. Clever path to radically low latency for some use cases.
[+] high_byte|4 years ago|reply
> bypassing data hoarding web giants

but using cloudflare to resolve dns? hmm..

[+] o8r3oFTZPE|4 years ago|reply
"All of the contact data on this website is fetched from DNS TXT files."

I found contact data for a number of companies in the Javascript files.

    {"object_display_name":"Organisation","name":"Co-op Insurance","slogan":"Car, Home, Travel, Pet and Life Insurance from Co-op","contacts":[
    {"object_display_name":"Organisation","name":"Columbus Direct","slogan":"Award Winning Travel Insurance for 30 Years","contacts":[
    {"object_display_name":"Organisation","name":"First Direct","slogan":"Online and telephone banking 24 7 365","contacts":[
    {"object_display_name":"Organisation","name":"Churchill","slogan":null,"contacts":[
    {"object_display_name":"Organisation","name":"MORE THAN","slogan":"Car, home, pet, life, travel and landlord insurance quotes","contacts":[
    {"object_display_name":"Organisation","name":"Saga","slogan":"Over 50s Insurance, Holidays, Money and Magazine","contacts":[
    {"object_display_name":"Organisation","name":"LV=","slogan":"Liverpool Victoria","contacts":[
    {"object_display_name":"Organisation","name":"Sheila\'s Wheels","slogan":"Car & Home Insurance with Style","contacts":[
    {"object_display_name":"Organisation","name":"Swiftcover","slogan":"Super Fast Car and Home Insurance","contacts":[
    {"object_display_name":"Organisation","name":"Barclays Bank","slogan":"A big world needs a big bank","contacts":[
    {"object_display_name":"Organisation","name":"Hastings Direct","slogan":"Car, Van, Bike and Home Insurance","contacts":[
    {"object_display_name":"Organisation","name":"Co-operative Bank","slogan":"For People with Purpose","contacts":[
    {"object_display_name":"Organisation","name":"Halifax UK","slogan":"Halifax makes it happen","contacts":[
    {"object_display_name":"Organisation","name":"Lloyds Bank","slogan":"By Your Side","contacts":[
    {"object_display_name":"Organisation","name":"Royal Bank of Scotland","slogan":"Enjoy better banking with RBS where people matter","contacts":[
    {"object_display_name":"Organisation","name":"Admiral","slogan":"Car, MultiCar and MultiCover Insurance Quotes","contacts":[
    {"object_display_name":"Organisation","name":"Aviva","slogan":"Insurance, Savings and Investments","contacts":[
    {"object_display_name":"Organisation","name":"AXA","slogan":null,"contacts":[
[+] unknown_error|4 years ago|reply
I think you're leeching off someone else's infrastructure and using it to do things they never meant it to do. Sure, the technical capability is there, but your use case would drastically increase their costs. You are essentially cost-shifting your customers' costs onto theirs. Not cool.

It's like building a cloud storage solution off Gmail's free storage. It can be done, has been done, but that doesn't mean it's cool to do so.

Your system would increase costs for DNS providers all over the world, without their consent, just because you're using it as a loophole. It was a problem that wasn't there fixed in a way that leeches from rather than gives back to the community.

[+] elliottinvent|4 years ago|reply
Thanks for your point of view, I find it really interesting.

> I think you're leeching off someone else's infrastructure

Ok, who’s the victim here? CloudFlare? Since we use their DoH end point?

Google Cloud DNS? Since that’s where we’re storing the data in DNS?

All of this is just standard DNS - CloudFlare DoH and GCDNS can be switched out for any other because it’s just vanilla DNS.

Let’s say Barclays wanted to serve out data using NUM and stored data in their own DNS zone, would they be abusing their DNS provider’s infrastructure? I don’t think so.

If we’re successful with our plans for NUM, and it becomes mainstream then surely this presents a huge opportunity for DNS providers who will have increased query costs for clients.

DNS revolvers will make their own decisions about whether they cache NUM queries (or perhaps even answer them at all) but revolvers that answer them quickly will surely have an edge on those that don’t.

> and using it to do things they never meant it to do.

The DNS is a distributed database. It’s designed to convert human friendly data to machine friendly data and I think NUM fits this perfectly. I understand not everyone shares my point of view.

> Sure, the technical capability is there, but your use case would drastically increase their costs. You are essentially cost-shifting your customers' costs onto theirs. Not cool.

It increases the costs of CloudFlare / Google? Ok, if it’s significant, they have a commercial decision to make - support full DNS as per the protocol spec, or partial-DNS where they block certain use cases.

> It's like building a cloud storage solution off Gmail's free storage. It can be done, has been done, but that doesn't mean it's cool to do so.

No, it’s not. The DNS is owned by no one and everyone.

> Your system would increase costs for DNS providers all over the world, without their consent

Most will just pass this on to domain owners, DNS query costs are peanuts - 200 USD per billion at scale.

> It was a problem that wasn't there fixed in a way that leeches from rather than gives back to the community.

I respect your point or view but think the opposite is true. We’re freeing data, opening it up for developers so that they can build things far outside the jurisdiction of the giants of the web - I think this is a fantastic way to give back to the community.

[+] o8r3oFTZPE|4 years ago|reply
The idea I had for this over 10 years ago was to modify djb's dnstxt to output raw HTML with a MIME header. tinydns allows one to store arbitrary data so I could put anything in a 512-byte DNS packet, including "text/html" (newlines and carriage returns). I could request tiny web pages from tinydns. There have been other similar things by other folks like putting deCSS code in DNS, Wikipedia data, audio files, etc.

Now this might all sound silly, but since then 1. DNS packets are now massive and can carry much more data and 2. big companies are pushing a "next-gen" UDP-based protocol (reminds me of djb's CurveCP) for HTTP, serving HTML and other web junk. They also want to use this UDP-based protocol for other things, like DNS, eventually.

Whatever. HTML in DNS worked great. Tiny pages that fit in a CPU cache.

[+] o8r3oFTZPE|4 years ago|reply
Putting URLs in DNS. Could be useful.
[+] bawolff|4 years ago|reply
I don't get it.

You have to host the static page somewhere. Why not host the data in the same place (as a json blob or whatever)? It seems like the implication is all the data in dns is static, so what's the benefit of adding dns to the mix - you already have to use something else to host some of your static resources, why not just have a single static host instead of two separate static hosting systems?

(Yes in theory you could write a dns server to dynamically answer queries, but that doesn't seem like its what is being proposed here)

Dont get me wrong, its cool you can access dns from a browser, but i dont think this is a compelling usecase (something like web torrent in the browser with magnet links in dns for a decentralized web, seems like it would make a good demo for this technique)

[+] elliottinvent|4 years ago|reply
What we're trying to do here is standardise how machine-readable data is stored and retrieved, primarily for companies. Any of the companies on this page can adopt the standard and override the data on the website. So the point is, it's not static.

Sure, at the moment all this data is coming from one DNS zone (num.net) but that's only a failover query location, first we query e.g. barclays.co.uk for this data, so Barclays can override (or remove) this data if they want to.

[+] forgotmypw17|4 years ago|reply
Thank you for sharing your project, Elliott.

It is an intriguing technology, and a clever way of putting together existing pieces in a new way.

I think it is very much in the spirit of how things are done on the Internet, although I'm not entirely convinced that NUM wouldn't gum up the DNS system if widely adopted.

My question to you is, how would I take advantage of NUM as a small-time personal-website operator, if my website was largely about small text-file snippets?

[+] elliottinvent|4 years ago|reply
Thanks for your kind comment.

The standardised use cases of NUM (currently contact data, images [gravatar for domains] and some others) would allow you as a small website operator to publish any public data that you wanted to be machine-readable.

I think your question might be how to use NUM to serve out your small text-file snippets? If so, NUM could be used for that. NUM lookups are based on URIs, eg:

num://numexample.com:1

In this example module 1 is contact data and lookups for this uri involve a query to the dns for:

dig 1._num.numexample.com TXT

If instead of publishing contact data you wanted to publish general purpose data you can do this using “module zero”. For example data could be published form the NUM URI num://numexample.com:0/foo

The DNS location is:

foo.0._num.numexample.com

[+] ksec|4 years ago|reply
>CompanyDirectory.UK is a NUM Technology [1]

>What is NUM?

>NUM is a DNS-based alternative to the World Wide Web for storing and retrieving structured data. The web is amazing but websites are built for browsing and are an inefficient way to find precise pieces of data like telephone numbers, bank details and more.

[1] https://www.num.uk

[+] darkr|4 years ago|reply
[+] elliottinvent|4 years ago|reply
Thanks for your comment and for highlighting this.

As a summary for others: Telnic (and Telnames) allowed people to store contact data in the DNS of their .tel domain name.

> Sounds similar to Telnic

Telnic used NAPTR records and was only available on .tel domains, NUM uses TXT and can be used on any domain name, by publishing a "_num" zone.

Telnic was also "just" for contact data. What we're trying to do with NUM is make it possible to publish any kind of data. Contact data as just one "module".

[+] tedk-42|4 years ago|reply
Please no.

CI systems like GitHub actions and CircleCI use to have unlimited CI minutes until people started abusing them for things they weren't intended for (i.e. crypto mining).

I would hate if I had to pay for DNS services (or be forced into using an ISP one which does things like blacklist certain domains) just because people on the internet wanted to be 'cool' and show off how smort they are.

This is why we can't have nice things.

[+] zzyzxd|4 years ago|reply
I like the limit on GitHub actions. It won't be sustainable in long term otherwise. You just can't count on people's self discipline for this kind of stuff, and there will always be malicious usage as long as the system allows.

If putting large random TXT records can harm the global DNS infrastructure, DNS providers should put a stricter limit on it. And OP's project will help us get there.

[+] bawolff|4 years ago|reply
I don't see why this application would be abusive. Chrome does much worse things trying to detect if you are behind a captive portal.

Edit: oh i see, they arent using generic dns but cloudflare's specific DoH gateway. Guess that could maybe be problematic if everyone did that, although if any company would tolerate that it would be cloudflare.

[+] elliottinvent|4 years ago|reply
Thanks for your feedback. Obviously I think it’s cool but I doubt that’ll be a good enough reason for most people.

NUM is a protocol for storage and retrieval of data, that’s all. Hopefully the demonstration shows it’s pretty good at that. If people choose to store huge amounts of data using it, I think that would be cool too.

I don’t think there’s any risk of it being used for crypto mining. NUM doesn’t open up any new attacks vectors that aren’t already present in DNS and it’s an incredibly resilient and robust system as we know.

[+] adflux|4 years ago|reply
Your domain names are free?
[+] bfung|4 years ago|reply
In a cursory read through, I’m still not sure what problem NUM actually solves. It seems like it moves data storage into DNS TXT records and defines some parsers.

What benefits does NUM have over html over http? One can use some semantic html tags to organize the data for parsing, making it roughly equivalent to NUM modules. If a NUM client is running w/o a newly developed NUM module, how does it parse that data and how is it better than html without semantic meaning?

[+] elliottinvent|4 years ago|reply
> What benefits does NUM have over html over http?

Web standards (JSON-LD, microdata, RDFa) are inefficient, since you have to download the entire HTML to find a particular piece of data like a phone number.

Web standards are also not widely adopted enough for developers to build something on top of that data. For example, web standards can't be used to find a phone number for _any_ company with a phone number published to their website, since many small businesses don't mark up phone numbers with semantic web data.

We're pre-populating NUM with millions of pieces of data so if a phone number is on the public website, it'll be in the NUM record for that domain.

> One can use some semantic html tags to organize the data for parsing, making it roughly equivalent to NUM modules.

Of course, the problem is that not enough websites do this (especially those of small business).

> If a NUM client is running w/o a newly developed NUM module, how does it parse that data and how is it better than html without semantic meaning?

If NUM is used for general purpose data storage (instead of for a particular module), then parsing of that data would need to be handled in a bespoke client – not a general purpose NUM client. Any advantages of using NUM over HTTP in this example but would be related to scaling and cost savings only.

[+] yakshaving_jgt|4 years ago|reply
This is interesting. I run https://newbusinessmonitor.co.uk/ and I’m often asked if I can provide contact data for UK companies, but this is difficult because no contact data is published when a company registers. I wonder how you’ll be getting this data for smaller companies.
[+] elliottinvent|4 years ago|reply
The long game is for companies to store NUM data when they register a domain, potentially through their DNS provider or web designer. Maybe even storing machine readable data instead of a website, since many small business websites are cookie-cutter anyway – contact info, directions, menu etc.

Public data is available for most domain registries, e.g Nominet [0] and others [1] so we crawl new domains to discover contact data.

It could be good to chat (email elliott[dot][email protected]), we're in the process of mashing up Companies House data with our contact data in an effort to provide company numbers in NUM records too.

0. https://registrars.nominet.uk/uk-namespace/the-uk-zone-files... 1. https://czds.icann.org/

[+] rognjen|4 years ago|reply
I think it would be wise to slightly change the design so as not to look so similar to Companies House.
[+] chadxz|4 years ago|reply
If you are offering this to people for free, what is your company’s business model? Why centralize their info on a single domain instead of putting it on their own using a common format?
[+] elliottinvent|4 years ago|reply
The data is free to developers and users. We offer a freemium service to businesses that they can use to publish and manage NUM data.

We prepopulate data in our domain (NUM.net) so the protocol is useful from launch.

The protocol allows anyone to publish data to their own DNS, they can also delegate their “_num” zone to us so we can easily manage the data.

Our plan isn’t to centralise everyone’s data in one domain (num.net), but that’s a fallback. Realistically, many small businesses will use this hosted solution long term and our business model counts on that.

[+] nopcode|4 years ago|reply
Isn’t DNS de-facto centralised too?
[+] nine_k|4 years ago|reply
Yes in the sense that DNS is a tree with a single root.

No in the sense that you can put your data into leaves of your choice and under your control. You can easily run your own DNS server.

Halfway-yes in the sense that this thing uses HTTPS access to DNS which has few providers now, and not the classic DNS because UDP from a browser is hard, and even arbitrary TCP from a browser is hard.

I think that the use of DNS is a bit of a gimmick on this demo site; it could be more practically used in more interesting ways. A demo site should be and is very straightforward. But the idea of storing semantic-web-style information in the DNS looks interesting to me, and the suggested simplified format may be worth a look.

The service NUM is offering is also interesting, and one of the key parts of it is that it's not storing your data, so you are not beholden to it. It helps make them accessible, but you store the data yourself.

[+] bawolff|4 years ago|reply
I feel like saying something is "decentralized" is like saying something is "secure". Its a meaningless statement. Nothing is 100% secure and nothing is 100% decentralized. The relavent questions are from whom? Against what?
[+] elliottinvent|4 years ago|reply
I think you could argue DNS over HTTPS is centralised (for now) with Cloudflare, Quad9, Google making up the main players. But I think the DNS is the original decentralised system due to the amount of players in the domain registrar / DNS service provider / ISP [DNS resolver] space.