I never thought I would say this, but I actually implemented an FTP server in 2020. This was needed to support firmware updates to specific hardware (Electric Vehicle charging stations). Apparently embedded software developers choose FTP whenever a spec doesn't specify how binary file transfers should work.
It was kind of amusing getting FTP to work in a modern cloud environment. I run a single Kubernetes pod with a Node.js based FTP server optimized for one thing: Transferring files between FTP and Google Cloud Storage. A series of ports are specified in the Docker file to enable passive FTP transfers.
Even more amusing was the number of varieties in which FTP was implemented by different hardware manufacturers. I regularly had to dive into the FTP libraries to add support for crazy edge cases (tcpflow in kubectl exec -it is your friend!). Example: one device added a newline in the middle of a command (USER\n myusername)..
The latest curve ball I received this week is that a certain firmware version of a Qualcomm modem chip cannot deal with the size of the IP packets coming from our FTP server... Fun stuff!
Implementing an FTP server from scratch that had to be compatible with lots of clients in 2020 was an interesting choice. Just to have it in javascript? Perhaps security-motivated? There are probably battle tested implementations in e.g. Python, Java or other safe-ish languages to build on?
I learned this lesson in the mid 90s when fixing client compatibility bugs in an FTP server module we had built in an interpreted language, because, how hard could it be...
> The latest curve ball I received this week is that a certain firmware version of a Qualcomm modem chip cannot deal with the size of the IP packets coming from our FTP server... Fun stuff!
How much better have we gotten at specifying protocols? Have we learned how to make protocols less ambiguous and less susceptible to crazy edge cases which make it burdensome to implement support in practice once there are lots of sloppy implementations in the field?
Why not run a regular FTP server and have your application periodically look for new files to process? For horizontal scaling, you just take a distributed lock on the file name.
doesn't that open up like a ton of man-in-the-middle attacks? I realize it may just be one of those "no other option" things but dang gritted-teeth-emoji
From rfc801 (transition from NCP to TCP/IP) :
"FTP: This is specified in RFC 765. It is very similar to the FTP used with the NCP. The primary differences are that in addition to the changes for Telnet, that the data channel is limited to 8-bit bytes so FTP features to use other transmission byte sizes are eliminated."
So FTP is older than the standardisation of byte size :-)
This could just be auto tested as the code would rarely change. It's like Firefox devel has a dartboard of unique and useful features to be randomly removed.
Not that I think we should go back, but I do miss the wildwest days of web development when it was still acceptable to FTP untested code straight to production.
For my first dev job we would develop on production using an FTP client to push up changes on save. One day I was writing an SQL UPDATE statement and I forgot to include a WHERE clause. I basically nuked the entire product DB and it took days to recover because it was also not unheard of to not have regular db back ups. No one really questioned it too much though. Stuff like that just happened from time to time back then.
I rarely ever use FTP today. I wonder if students learning to code today even know what FTP is? It was one of the first things I learnt when learning to build websites as a teenager, but I don't that's the case anymore.
It's kind of interesting how processes in tech have evolved as much as the technology over the last few decades. It's hard to think of a good usecase for FTP anymore, but just a couple of decades ago it was used everywhere. Is anyone still using it for anything?
The backend team in one of my previous jobs (around 2010) had an interesting file locking mechanism.
All the developer were sat around one table but you couldn’t see each other behind the dual screens. So every hour or so, somebody stood up to inform the team that he is going to be editing file xyz, so please don’t touch.
It wasn’t production though, everyone was just working on a shared staging system.
Im a youngin’ and I regularly use it to transfer Roms to hacked portable systems (psp, vita, 3ds, wii). It’s convenient to transfer larger roms this way.
I still use it to sync folders to my iPad (from Documents Readdle).
Since walled gardens don't leave much options open, it's basically either that or a cloud provider -- and no free cloud plan has enough space to store these folders.
So FTP wins easily this one. It's free with unlimited storage, always has been, always will be.
I like the user-facing simplicity of FTP as a text-based interface for browsing and downloading/uploading files. No broken links, as the directory layout generally doesn’t change every other year, no eternal-beta web UI, diverse choice of powerful native clients. It’s unfortunate that the protocol itself didn’t held up so well.
FTP is still used by some photographers at major sporting events. The cameras they use actually have RJ jacks in the side, and they run Ethernet cables back to a router, or maybe a second operator with a computer. The cameras have built-in FTP clients, and can upload image files as they are taken. It's apparently important for stills images to appear in near-real-time on sports web sites.
Fun trivia: in the early days of the ARPAnet, mail was delivered (at least on PDP-10's) by FTP-appending a message in to the receipients mailbox file. (Which was protected as append-only to world (owner, group, world).)
Didn't last long, obviously, but that was back in the days when every site had a well-publicized guest login and you could telnet anywhere. (Well, OK, in 1972 there were only a few dozen nodes, but there were some really interesting ones to play with.)
We used to play a game with telnet from HARV-10 where we'd telnet-chain around the world until someone dropped the connection or it got too slow.
Back in the days of X.25 and the UK "coloured book" protocols, grey book email went over blue book "network independent FTP" (in batch). I don't know how NIFTP compared with ARPANET FTP as a protocol, though.
On reading the headline my first thought was "wow, FTP is way older than I expected" and when I realised it's not, my second thought was: "damn, I'm old..." :)
Tangentially related. About 10 years ago I considered an idea that would allow websites to accept large files. The web admin would either embed our company's page as an iframe or just link to our whitelabeled url, such as "mydomain.ftphub.com". The admin would then get an email with a download link. Or they could reverse the process and send a link to a customer the way yousendit does.
I still own the domains ftphub.com and ftphub.net, and I put the domains at auction since it looks like I'll never get around to it.
Should I work on this ? Or has this opportunity been commoditized to the point where I there is no way it could turn a profit ? Also has anyone under 30 even heard of "ftp" ? The abbreviation in our brand but not sure it's meaningful anymore. Thanks and sorry if this is too off topic.
under 30 and yes ive heard of FTP. I used them at school to submit assignments for some classes and I use FTPs (albeit sparingly) at my current place of work.
> In 1924, the CCITT introduced the International Telegraph Alphabet No. 2 (ITA2) code[14] as an international standard, which was based on the Western Union code with some minor changes.
> ITA2 is still used in telecommunications devices for the deaf (TDD), Telex, and some amateur radio applications, such as radioteletype ("RTTY").
It's interesting that the fundamentals of this were invented in 1870s. Who needs semiconductors when you have gears and levers?
Choosing a ftp server in 2021 is an decission between two options, bad and worse. Vsftpd is easier to setup than proftpd but it last release was 6 years ago...
And just the other month, we switched a transmission from SFTP to FTP because we couldn't be bothered to deal with the bottlenecks in the SFTP stack. HTTP? Maybe next year :-)
The worst part is not that its 50 years old. The worst part is that some places still INSIST on using it. For instance I have to deal with FOTA and it's a pain.
Older technologies are more reliable and last longer, because they have survived the changes of time. That also means if you learn older technology (my rule of thumb is 15+– think Elixir, Lisp, Haskell, Python, Java, C++), your knowledge is unlikely to expire.
I had a finance-industry client require we upload to their FTP server because they couldn't trust downloading the file over HTTPS from our server with a secure/unique URL.
It's still pretty huge in the enterprise behind-the-scenes world of EDI exchange. I work in the railroad industry and use it pretty much daily (in addition to sftp, ftps, and other protocols).
I'm 54, so news like this is a little bit of a scary reminder of my mortality. :-(
Still, a couple of years back now I had to setup an IP-limited SFTP server for our comms team. But I made them 'sign in blood' that it was a temporary solution until their supplier could implement a more secure alternative. Like I said, it was a few years ago...
FTP definitely still has its uses, it just needs careful thought and setup.
[+] [-] dominiek|5 years ago|reply
I never thought I would say this, but I actually implemented an FTP server in 2020. This was needed to support firmware updates to specific hardware (Electric Vehicle charging stations). Apparently embedded software developers choose FTP whenever a spec doesn't specify how binary file transfers should work.
It was kind of amusing getting FTP to work in a modern cloud environment. I run a single Kubernetes pod with a Node.js based FTP server optimized for one thing: Transferring files between FTP and Google Cloud Storage. A series of ports are specified in the Docker file to enable passive FTP transfers.
Even more amusing was the number of varieties in which FTP was implemented by different hardware manufacturers. I regularly had to dive into the FTP libraries to add support for crazy edge cases (tcpflow in kubectl exec -it is your friend!). Example: one device added a newline in the middle of a command (USER\n myusername)..
The latest curve ball I received this week is that a certain firmware version of a Qualcomm modem chip cannot deal with the size of the IP packets coming from our FTP server... Fun stuff!
[+] [-] tpmx|5 years ago|reply
I learned this lesson in the mid 90s when fixing client compatibility bugs in an FTP server module we had built in an interpreted language, because, how hard could it be...
> The latest curve ball I received this week is that a certain firmware version of a Qualcomm modem chip cannot deal with the size of the IP packets coming from our FTP server... Fun stuff!
Right.
[+] [-] rectang|5 years ago|reply
[+] [-] winrid|5 years ago|reply
Why not run a regular FTP server and have your application periodically look for new files to process? For horizontal scaling, you just take a distributed lock on the file name.
[+] [-] gogopuppygogo|5 years ago|reply
[+] [-] dheera|5 years ago|reply
Username: anonymous
Anonymous login accepted, enter e-mail address as password.
Password: [email protected]
I bet aoeu.com and asdf.com got a good amount of unwanted mail back then.
[+] [-] 1vuio0pswjnm7|5 years ago|reply
[+] [-] a-dub|5 years ago|reply
[+] [-] stjohnswarts|5 years ago|reply
[+] [-] marsven_422|5 years ago|reply
[deleted]
[+] [-] waynesonfire|5 years ago|reply
If you did this for work, junior engineer move in my opinion. This practice is called not invented here syndrome.
[+] [-] ajb|5 years ago|reply
So FTP is older than the standardisation of byte size :-)
[+] [-] mywittyname|5 years ago|reply
[+] [-] jandrese|5 years ago|reply
[+] [-] jbverschoor|5 years ago|reply
What’s a nibble of a byte is 8bits
[+] [-] molticrystal|5 years ago|reply
https://blog.mozilla.org/addons/2021/04/15/built-in-ftp-impl...
[+] [-] kwyjobojoe|5 years ago|reply
[+] [-] poyu|5 years ago|reply
[+] [-] kypro|5 years ago|reply
For my first dev job we would develop on production using an FTP client to push up changes on save. One day I was writing an SQL UPDATE statement and I forgot to include a WHERE clause. I basically nuked the entire product DB and it took days to recover because it was also not unheard of to not have regular db back ups. No one really questioned it too much though. Stuff like that just happened from time to time back then.
I rarely ever use FTP today. I wonder if students learning to code today even know what FTP is? It was one of the first things I learnt when learning to build websites as a teenager, but I don't that's the case anymore.
It's kind of interesting how processes in tech have evolved as much as the technology over the last few decades. It's hard to think of a good usecase for FTP anymore, but just a couple of decades ago it was used everywhere. Is anyone still using it for anything?
[+] [-] smoe|5 years ago|reply
All the developer were sat around one table but you couldn’t see each other behind the dual screens. So every hour or so, somebody stood up to inform the team that he is going to be editing file xyz, so please don’t touch.
It wasn’t production though, everyone was just working on a shared staging system.
[+] [-] formercoder|5 years ago|reply
[+] [-] aspaceman|5 years ago|reply
[+] [-] mtone|5 years ago|reply
So FTP wins easily this one. It's free with unlimited storage, always has been, always will be.
[+] [-] nly|5 years ago|reply
[+] [-] layer8|5 years ago|reply
[+] [-] SulphurCrested|5 years ago|reply
[+] [-] cpr|5 years ago|reply
Didn't last long, obviously, but that was back in the days when every site had a well-publicized guest login and you could telnet anywhere. (Well, OK, in 1972 there were only a few dozen nodes, but there were some really interesting ones to play with.)
We used to play a game with telnet from HARV-10 where we'd telnet-chain around the world until someone dropped the connection or it got too slow.
[+] [-] gnufx|5 years ago|reply
[+] [-] stephvd|5 years ago|reply
[+] [-] dcminter|5 years ago|reply
[+] [-] mrits|5 years ago|reply
[+] [-] throwawayftphub|5 years ago|reply
I still own the domains ftphub.com and ftphub.net, and I put the domains at auction since it looks like I'll never get around to it.
Should I work on this ? Or has this opportunity been commoditized to the point where I there is no way it could turn a profit ? Also has anyone under 30 even heard of "ftp" ? The abbreviation in our brand but not sure it's meaningful anymore. Thanks and sorry if this is too off topic.
[+] [-] cumwolf|5 years ago|reply
[+] [-] zabzonk|5 years ago|reply
Pedantry: It's "Jimi".
[+] [-] jedimastert|5 years ago|reply
[+] [-] SturgeonsLaw|5 years ago|reply
[+] [-] inglor_cz|5 years ago|reply
https://en.wikipedia.org/wiki/Abhay_Bhushan
[+] [-] goda90|5 years ago|reply
[+] [-] tpmx|5 years ago|reply
https://en.wikipedia.org/wiki/Baudot_code#ITA2
> In 1924, the CCITT introduced the International Telegraph Alphabet No. 2 (ITA2) code[14] as an international standard, which was based on the Western Union code with some minor changes.
> ITA2 is still used in telecommunications devices for the deaf (TDD), Telex, and some amateur radio applications, such as radioteletype ("RTTY").
It's interesting that the fundamentals of this were invented in 1870s. Who needs semiconductors when you have gears and levers?
https://en.wikipedia.org/wiki/Émile_Baudot
[+] [-] AdmiralAsshat|5 years ago|reply
[+] [-] jedimastert|5 years ago|reply
[+] [-] rfraile|5 years ago|reply
[+] [-] dekhn|5 years ago|reply
[+] [-] geenew|5 years ago|reply
It's one of those things that stands up to its name: it's a file transfer protocol, and is still, practically, the best.
[+] [-] lolc|5 years ago|reply
[+] [-] theflyinghorse|5 years ago|reply
[+] [-] crazypython|5 years ago|reply
[+] [-] spacemanmatt|5 years ago|reply
[+] [-] jvolkman|5 years ago|reply
[+] [-] cybert00th|5 years ago|reply
Still, a couple of years back now I had to setup an IP-limited SFTP server for our comms team. But I made them 'sign in blood' that it was a temporary solution until their supplier could implement a more secure alternative. Like I said, it was a few years ago...
FTP definitely still has its uses, it just needs careful thought and setup.