Oh this is cool! I recently wrapped libfuse in Nim and after porting the 'hello' filesystem example I made one which is more or less exactly this. However my version you pipe data and have to provide a mountpoint, then when it's done it writes the result over stdout. That means you can inline it in a pipe chain but also that you have to make sure to grab the output.
At the moment I'm exploring other stuff which could be made into file systems. I've got a statusbar thing for the Nimdow window manager which allows you to write contents to individual files and it creates a bar with blocks on them as the output. It makes it super easy to swap out what is on your bar which is pretty neat.
Another tool I've made is a music player. It uses libvlc and when given a folder it reads all the media with ID3 tags and sets up folders like 'by-artist', 'by-album', etc. Each file is named as '<track number> - <song title>' and contains the full path to the actual file. To play a song you cat one of these files into 'control/current' and write the word play to 'control/command'. There's a bit more to it like that like a playlist feature and some more commands, but that's the basic idea. The goal is to have a super-scriptable music player.
Here's an idea: recursively mount code files/projects. Use something like tree-sitter to extract class and function definitions and make each into a "file" within the directory representing the actual file. Need to get an idea for how a codebase is structured? Just `tree` it :)
Getting deeper into the rabbit hole, maybe imports could be resolved into symlinks and such. Plenty of interesting possibilities!
This makes me think, it would be nice if there was an easy built-in way to expose information about a process using the filesystem. Something like "cat /proc/$pid/fs/current_track" to get a name of a current song from a music player, or "ls /proc/$pid/fs/tabs" to list open tabs in my browser (and maybe use this to grab the html or embedded images).
I mean right now it's possible to do this using FUSE, but that's convoluted and nobody does it.
Useful enough that it should be an OS-level standard feature, imho.
Unix-like OSes allow mounting disk images to explore their contents. But there's many more file formats where exploring files-inside-files is useful. Compressed archives, for one. Some file managers support those, but (imho) application-level is not the optimal layer to put this functionality.
Could be implemented with a kind of driver-per-filetype.
Really what you'd like to see is a way to write the mount command for each file type (do one thing well) and another command to detect the file type and dispatch accordingly (probably similar to the `file` command), all in user space.
The only thing standing in the way of this today is that MacOS doesn't expose a user space file system API. You can do this on Linux, Windows, and BSDs today.
(No, file provider extensions don't cut it, Apple devs who read this, please give us a FUSE equivalent, we know it exists).
I thought archivemount already did that. Am I missing something?
Anyway, even if that's not what you are looking for, FUSE is a more general mechanism that will allow you to do what you want (well, it seems like, at least) and much more.
For zip archives, there are fuse-zip and mount-zip which are FUSE filesystem.
As an intermediate between OS level and application-level, there are desktop environment level: gvfs for GNOME and KIO for KDE, but they are compatible only in their own ecosystems.
This is really neat, but when I saw the headline I got excited that it was something I have been looking for / considering writing, and I figure the comments here would be a good place to ask if something like this exists:
Is there a FUSE filesystem that runs in-memory (like tmpfs) while mounted, and then when dismounted it serializes to a single file on disk? The closest I can find are FUSE drivers that mount archive files, but then you don't get things like symlinks.
> The libsqlfs library implements a POSIX style file system on top of an SQLite database. It allows applications to have access to a full read/write file system in a single file, complete with its own file hierarchy and name space. This is useful for applications which needs structured storage, such as embedding documents within documents, or management of configuration data or preferences. Libsqlfs can be used as an shared library, or it can be built as a FUSE (Linux File System in User Space) module to allow a libsqlfs database to be accessed via OS level file system interfaces by normal applications.
Not purely in-memory, but something like https://github.com/jrwwallis/qcow2fuse maybe? It's clunky compared to OSX's DMGs, but if you squint it achieves similar ends.
Otherwise you could achieve this with a tmpfs wrapped to serialize to a tarball (preserving symlinks) when unmounted.
I’m sure you’re already aware of this, but there are all kinds of very real scenarios that could lead to corrupted data if you’re only flushing the buffer upon unmounting.
Sounds like you’ve got an interesting problem you’re trying to solve though.
I can't think of anything _exactly_ like that, but I think you can get close by just copying some type of image file to /tmp and then moving it to disk when you're done after unmounting.
But wow what a clever idea. Not sure id ever need to reach for it personally as I do most data processing in a higher level language, but I can imagine people can find use cases.
It's an interesting idea but I think the usefulness would be greatly enhanced if it could handle json arrays; most needed json structures contain array elements in my experience
Neat. Now how about a filesystem that takes a directory of files and exposes it as a single json file? You could call it the Filesystem File, and mount it in the File Filesystem if you wanted...
this is cool but it's fuse.. which is not so cool.
these days i reach for jq.. I've recently became interested in duckdb too.
Using a tool that is specialized for the format is usually more ideal than a generic one treating everything as files.
There is a lot in JSON that can't be represented in a flexible way as files and directories.
For instance, what if a key has "/" in it..
What happens to lists and their order when you re-serialize
How are hashes represented..
how can you tell if a parent is a object or a list..
inserting a item into a list is a ton of error prone renames..
the list goes on
Daniel Stenberg, of cURL fame, co-write an Amiga-editor called FrexxEd where the open buffers were exposed as files in the filesystem.
Meaning you could write any shell script to manipulate an open buffer (not that important as it also exposed all editor functionality both via IPC via Arexx and via FPL - a C-like scripting language), and that you could e.g. compile without saving (that was very helpful on a system where a lot of people might only even have a single floppy drive, and where being able to have the compiler in the drive and compile straight from the in-memory version in RAM so you didn't have to keep swapping floppies was highly useful (just remember to save before actually trying to run the program - no memory protection...)
I guess support for XML would be tricky, because XML is just way more complex format than the ones already supported. It is still essentially a tree, but with additional structure.
Representing elements and their contents is easy enough. But attributes, comments, processing instructions, entities... And remember, an XML document can include a DTD (it does not have to be in a separate file).
To present it as a file system in a useful, non-convoluted way? I will be very, very interested if it's possible, but not holding my breath.
[+] [-] PMunch|1 year ago|reply
At the moment I'm exploring other stuff which could be made into file systems. I've got a statusbar thing for the Nimdow window manager which allows you to write contents to individual files and it creates a bar with blocks on them as the output. It makes it super easy to swap out what is on your bar which is pretty neat.
Another tool I've made is a music player. It uses libvlc and when given a folder it reads all the media with ID3 tags and sets up folders like 'by-artist', 'by-album', etc. Each file is named as '<track number> - <song title>' and contains the full path to the actual file. To play a song you cat one of these files into 'control/current' and write the word play to 'control/command'. There's a bit more to it like that like a playlist feature and some more commands, but that's the basic idea. The goal is to have a super-scriptable music player.
[+] [-] pedrovhb|1 year ago|reply
Getting deeper into the rabbit hole, maybe imports could be resolved into symlinks and such. Plenty of interesting possibilities!
[+] [-] fishyjoe|1 year ago|reply
No worries if not, I'm just curious!
[+] [-] lambdaxyzw|1 year ago|reply
I mean right now it's possible to do this using FUSE, but that's convoluted and nobody does it.
[+] [-] RetroTechie|1 year ago|reply
Unix-like OSes allow mounting disk images to explore their contents. But there's many more file formats where exploring files-inside-files is useful. Compressed archives, for one. Some file managers support those, but (imho) application-level is not the optimal layer to put this functionality.
Could be implemented with a kind of driver-per-filetype.
[+] [-] duped|1 year ago|reply
The only thing standing in the way of this today is that MacOS doesn't expose a user space file system API. You can do this on Linux, Windows, and BSDs today.
(No, file provider extensions don't cut it, Apple devs who read this, please give us a FUSE equivalent, we know it exists).
[+] [-] Sophira|1 year ago|reply
[0] https://avf.sourceforge.net/
[+] [-] lmm|1 year ago|reply
[+] [-] frizlab|1 year ago|reply
[+] [-] crabbone|1 year ago|reply
Anyway, even if that's not what you are looking for, FUSE is a more general mechanism that will allow you to do what you want (well, it seems like, at least) and much more.
[+] [-] jraph|1 year ago|reply
For zip archives, there are fuse-zip and mount-zip which are FUSE filesystem.
As an intermediate between OS level and application-level, there are desktop environment level: gvfs for GNOME and KIO for KDE, but they are compatible only in their own ecosystems.
[+] [-] xk3|1 year ago|reply
You can look inside of archives pretty easily with `lsar` (part of the unar package). It works with disk images like ISO 9660 files too
But yes, especially for nested archives, having deeper OS support would be nice.
[+] [-] kybernetikos|1 year ago|reply
[+] [-] jasonpeacock|1 year ago|reply
https://jvns.ca/blog/2023/12/04/mounting-git-commits-as-fold...
(previously: https://news.ycombinator.com/item?id=38527866 )
[+] [-] amiga386|1 year ago|reply
[+] [-] paulgb|1 year ago|reply
Is there a FUSE filesystem that runs in-memory (like tmpfs) while mounted, and then when dismounted it serializes to a single file on disk? The closest I can find are FUSE drivers that mount archive files, but then you don't get things like symlinks.
[+] [-] speps|1 year ago|reply
> The libsqlfs library implements a POSIX style file system on top of an SQLite database. It allows applications to have access to a full read/write file system in a single file, complete with its own file hierarchy and name space. This is useful for applications which needs structured storage, such as embedding documents within documents, or management of configuration data or preferences. Libsqlfs can be used as an shared library, or it can be built as a FUSE (Linux File System in User Space) module to allow a libsqlfs database to be accessed via OS level file system interfaces by normal applications.
[+] [-] Scaevolus|1 year ago|reply
Otherwise you could achieve this with a tmpfs wrapped to serialize to a tarball (preserving symlinks) when unmounted.
[+] [-] hnlmorg|1 year ago|reply
I’m sure you’re already aware of this, but there are all kinds of very real scenarios that could lead to corrupted data if you’re only flushing the buffer upon unmounting.
Sounds like you’ve got an interesting problem you’re trying to solve though.
[+] [-] khc|1 year ago|reply
[+] [-] ranger_danger|1 year ago|reply
[+] [-] compressedgas|1 year ago|reply
[+] [-] alephaleph|1 year ago|reply
[+] [-] sambeau|1 year ago|reply
I used it as a per-user database for a web-templating language for a giant web-site building tool.
[+] [-] purple-leafy|1 year ago|reply
But wow what a clever idea. Not sure id ever need to reach for it personally as I do most data processing in a higher level language, but I can imagine people can find use cases.
Nice out of the box thinking
[+] [-] freeney|1 year ago|reply
[+] [-] Edmond|1 year ago|reply
It uses file system mechanics to model objects, meaning you can design object based solutions that support file system style navigation.
Demo: https://youtu.be/XgTgubZQPHw
[+] [-] qazxcvbnm|1 year ago|reply
[+] [-] timrobinson333|1 year ago|reply
[+] [-] MadcapJake|1 year ago|reply
setfattr -n user.type -v list # use xattr on macOS
[1]: https://github.com/mgree/ffs/issues/66
[+] [-] planede|1 year ago|reply
[+] [-] chuckadams|1 year ago|reply
[+] [-] sigmonsays|1 year ago|reply
these days i reach for jq.. I've recently became interested in duckdb too.
Using a tool that is specialized for the format is usually more ideal than a generic one treating everything as files.
There is a lot in JSON that can't be represented in a flexible way as files and directories. For instance, what if a key has "/" in it.. What happens to lists and their order when you re-serialize How are hashes represented.. how can you tell if a parent is a object or a list.. inserting a item into a list is a ton of error prone renames.. the list goes on
(edited for formatting)
[+] [-] willlma|1 year ago|reply
[+] [-] isoprophlex|1 year ago|reply
(This is a joke. I love the idea and execution.)
[+] [-] vidarh|1 year ago|reply
Meaning you could write any shell script to manipulate an open buffer (not that important as it also exposed all editor functionality both via IPC via Arexx and via FPL - a C-like scripting language), and that you could e.g. compile without saving (that was very helpful on a system where a lot of people might only even have a single floppy drive, and where being able to have the compiler in the drive and compile straight from the in-memory version in RAM so you didn't have to keep swapping floppies was highly useful (just remember to save before actually trying to run the program - no memory protection...)
[+] [-] mypalmike|1 year ago|reply
[+] [-] secwang|1 year ago|reply
[+] [-] agumonkey|1 year ago|reply
[+] [-] ThinkBeat|1 year ago|reply
[+] [-] IlliOnato|1 year ago|reply
Representing elements and their contents is easy enough. But attributes, comments, processing instructions, entities... And remember, an XML document can include a DTD (it does not have to be in a separate file).
To present it as a file system in a useful, non-convoluted way? I will be very, very interested if it's possible, but not holding my breath.