dominiek's comments

"Any application that could be done on a blockchain could be better done on a centralized database. Except crime."

What if the actual crime is committed by those that control our money...

Saifedean Ammous' book "The Bitcoin Standard" gives an excellent history of hard money and shows that any new form of hard money will inevitably replace the lesser form (e.g. from the silver standard to the gold standard).

Crypto is composed of interesting projects and scams. Gambling and speculations are symptoms of a wider problem of debased currency and a lack of hard money.

Bitcoin fixes this...

Talking about Bitcoin in the same vain as Crypto shows a profound lack of understanding of history both past and recent when looking at Bitcoin.

While people trade their little Pokemon NFTs Bitcoin will be doing what Bitcoin has been doing from the beginning: Staying exactly 21 million bitcoin - the scarcest and most accessible form of hard money the world has ever seen. It's a black hole sucking in monetary inflation.

1 Bitcoin = 1 Bitcoin.

Keep in mind that I have no control on the client. No manufacturer has implemented TLS let alone sFTP.

If I had any influence on the protocol it would be HTTPs. This is why I wasn't expecting to build an FTP server in 2020.

Exactly my reasoning. See my comment above for more info.

I'm making the maintenance of this less painful by doing a hacking/debugging session with manufacturers once a month where we hook up many devices and fix issues. After addressing most edge cases fewer are coming up now (despite a relentless stream of new cheaply manufactured devices)

I had considered this, but decided against this for a couple of reasons:

- Scaling requirements are relatively low. Even though we're dealing with 10 thousands of devices, the amount of firware updates at a given time to those devices is minimal. Our main scaling challenges are around OCPP over websockets. Story for another day.

- I have bad memories of ProFTPd etc buffer overflow exploits.

- I wanted something simple that could bridge between FTP and our cloud persistence (MongoDB and Cloud Storage).

- I found this Node.js library that I since then forked: https://github.com/autovance/ftp-srv - The great thing about this library is that it allows a quick implementation of a custom filesystem.

- For Kubernetes pods the file system should really be treated as a /tmp - which we are doing.

- When a charge station connects, the FTP username/password is a temporary generated set of tokens that is checked against our MongoDB.

Essentially, I'm using FTP as a throwaway here.

If you think through this you can imagine it would be quite a lift to accomplish this with an existing FTP server.

The main reason for using Node.js is because the rest of our stack is Node.js: https://bedrock.io

We use MongoDB as persistence and have existing wrappers for dealing with Google Cloud Storage.

Since it's an isolated service we could've used a different implementation language.

In our case Node.js in our existing Kubernetes environment was the least amount of friction

Yep VPN. The devices don't support TLS. (We have Cloudflare in front of other services)

Amazing.

I never thought I would say this, but I actually implemented an FTP server in 2020. This was needed to support firmware updates to specific hardware (Electric Vehicle charging stations). Apparently embedded software developers choose FTP whenever a spec doesn't specify how binary file transfers should work.

It was kind of amusing getting FTP to work in a modern cloud environment. I run a single Kubernetes pod with a Node.js based FTP server optimized for one thing: Transferring files between FTP and Google Cloud Storage. A series of ports are specified in the Docker file to enable passive FTP transfers.

Even more amusing was the number of varieties in which FTP was implemented by different hardware manufacturers. I regularly had to dive into the FTP libraries to add support for crazy edge cases (tcpflow in kubectl exec -it is your friend!). Example: one device added a newline in the middle of a command (USER\n myusername)..

The latest curve ball I received this week is that a certain firmware version of a Qualcomm modem chip cannot deal with the size of the IP packets coming from our FTP server... Fun stuff!

Unfortunately, the current state of web apps have many layers of frameworks, transpilers and tooling. So yes, this is relatively simple by today's standards.

Shoot. Looks like the .gitignore excluded the `public/js` dir which was used for the watchify output.

If you pull now it should work. Sorry about that.

I am a Dutch person and I’ve lived abroad for about half of my adult life in places like Japan and USA. I identify myself as both a Dutchman and a European. Just two weeks ago I shared a beer with an unknown Frenchman while watching the Eurocup in an Irish Pub while rooting for his country (Netherlands did not qualify). I am European and I love Europe. The things I miss most about home are my friends in Amsterdam (many of which have different nationalities) and I miss European food (the seasonal stuff you find in small villages in Southern Europe).

Even though I have these strong feelings towards Europe, I am against the EU as a super-state. I have been a “Euro Skeptic” ever since I started reading about liberty and became a libertarian. Unfortunately, being a Euro Skeptic and/or believing in smaller governments gets you cast into the “Right-wing Nutbag” camp in Europe (even by my own parents).

A lot of “Remainians” have strong personal feelings towards Europe just like me. They’ve benefited from some EU initiative at a university, or perhaps they married a fellow European person. Also, they enjoy the benefits of the Schengen treaty, being able to travel and settle freely - without spending years of legal fees and stress on immigration like me. All of these things can be accomplished by treaties and standards without a monolithic EU super nation. Trade and treaties (like Shengen) are great, and they are perfectly achievable without an EU super state).

There are countless of arguments around the democracy and economics of the EU, but at the end of the day the EU is an idealistic construct. It was created with noble intensions to create peace on a warn-torn continent - accelerated quickly after the fall of the Berlin Wall (to address a strong and unified Germany).

Yet once again, the idealism of Europe (and in my opinion the disregard for the fundamentals of liberty) is creating a new cycle of instability on the continent. Whether there’s going to be a Brexit or Bremain, a storm is coming.

Bottlenose (http://bottlenose.com) - LA or NYC preferred. Will require frequent trips to Amsterdam (the Netherlands). Remote might be possible.

We are looking for Big Data Engineers, Data Hackers and Machine Learning Pragmatists. Example job description here: http://bottlenose.com/jobs/senior-big-data-engineer. But if you are hungry for challenges in the intersection of Big Data / Machine Learning. Talk to us. Contact me at dominiek at bottlenose dot com (Recruiters NOT OK).

Bottlenose is headquartered in LA with a large part of the development team in Amsterdam. Technologies we use: Cassandra, Storm, Hadoop, MongoDB, Node.js.

Our Vision:

The Stream. It's what matters now. From email to tweets, from forum postings to customer complaints, from enterprise social to automated systems monitors, the drivers for business are now in vortex of voices reflecting markets, internal morale, innovation, brand equity, product acceptance, politics and threats. Rational intend has shifted to irrational actions. Understanding the past bares no guarantee for the future. Understanding what's actionable in streams after-the-fact is expensive. Even dangerous.

This new paradigm requires new tools and methods that can make sense out of the stream and understand human emotion. Bottlenose’s core technology - StreamSense - creates a virtual mind map of everything people pay attention to. It is a new kind of database to derive powerful new insights. StreamSense gives organizations the emotional intelligence that they didn’t have before. It allows them to sense what people are feeling and to anticipate the unknown unknowns - both the good and the bad.

Attempts at creating machine understanding of our world like the Semantic Web and the Knowledge Graph have failed due to two fundamental flaws: 1) A model of the world is not something that should be derived from documents written by experts; 2) In order to build a ‘global brain’ you need to understand human emotion and human intend.

Bottlenose’s ultimate mission is to create a global artificial brain. This global consciousness is build out of a continuous stream of human conversation and attention. By analyzing these vast amounts of data and by analyzing the emotions around it, we build up a mental model of the world. We think this will give us a shot at creating an incredibly powerful Artificial General Intelligence.

I'm not saying it's no longer relevant. I'm saying HTTP has become like other lower-level transport layers like TCP or Ethernet and that we should no longer build apps in an HTTP-centric way

I think that Berners-Lee and W3C get way too much credit for the web. And this is an incredibly arrogant statement by Mr. Berners-Lee.

Facebook has in-fact done some serious web innovation the last years and W3C has completely dropped the ball. Facebook and Twitter have been catalyzing internet adoption and the general spread of information.

Mr. Berners-Lee's obsession about content-silos shows that there is a serious disconnect between the current state of the web and W3C. The web was about content and documents fifteen years ago, now it's about the flow of data.

I know Berners-Lee is a big Linked Data advocate, but the approach that's being taken by the W3C is painfully slow and doesn't take into account the fluidity of information.

This is one of the reasons why developers (and even semantic web developers) have resorted to non-W3C technologies more and more: JSON, Javascript-wrappers, Webkit, client-side routing, non-REST HTTP requests, IOSockets/Coment, streaming apis, etc.

The web is emergent and out of control. Deal with it. Technologies and tools compete for attention and adoption. You snooze, you lose.

As for the 'content silos': Are you fucking kidding me? 'Content' being stuck in Facebook is not going to happen, in fact, the content is going to flow more and more. If you mark something as 'only my friends can see this', it will leak. Don't want to be tagged in a picture? Well, you have no choice. Face recognition will get you soon.

The internet, thanks to social web, is a giant copy machine. There's a huge shitstream of content and your attention and the activity around it is the thing that matters. Who cares about the damn content.

So maybe it's time for the 'Web Founder' and the W3 Web Museum to roll up their sleeves and do something, instead of bitch about the companies that actually advance the web.

So instead of bitching about the companies and people that actually advance the web and change the world, maybe it's time for the 'Web Founder' and the Web Museum to roll up their sleeves and do something...

If I was Twitter, I would keep deals like this on the down low. As soon as your company starts generating revenue, all investors and stakeholders will start looking at that first revenue stream.

This deal is just a little utility fee they're getting for providing priority access to their data pipe. At this point, Google and Microsoft will probably be the biggest 'data pipe' clients that Twitter can have. These deals have resulted in "several million dollars a month" which is absolutely nothing! These numbers, without factoring in anything else can never justify Twitter's current valuation.

That's why I think Twitter should make sure that people don't get distracted with this puny little revenue stream, but rather turn their heads to the real pot of gold: monetizing the activity ecosystem.

You bring up a very good point. Actually, I think this issue warrants an entire blog post in response. I think there are indeed huge dangers in informationization that could be catastrophic.

So I guess we need to determine whether informationization is a force that can be stopped, and if it's not, what can we do to minimize catastrophic failure. One thing could be to have certain physical/static baselines to fall back to (like shelters, food, minimum levels of non-liquid wealth). Or convert some of NNT's black-swan heuristics into well known practices (e.g. "Learn to be redundant, don’t be optimal.", "have small losses if you are wrong, and big profits if you are right", etc.)

I haven't put a lot of thought into the risks and downsides of informationization actually, and perhaps not enough people out there aren't either. At this year's Singularity Summit there was a real lack of skepticism and interest in debating some of these core downsides and dangers.

This is very interesting when working with B2B style consulting, but the real trick is to apply these MVP practices to the creation of entire new B2C markets.

Nice!

@vlad @run4yourlives @wheels Your comments make good sense. Reading up on the contest laws now.

Perhaps I should start a 'Help me out with legal issues' contest ;]

freshbooks.com