There's many network filesystems that can be useful in many situations: NFS, SMB, DLNA and even SSH (perhaps using sshfs! However they can be a bit cumbersome to setup or use (some DLNA clients have issues refreshing their catalogue and they require so many dependencies such as ffmpeg, SMB and NFS are hard to configure and notoriously dangerous if misconfigured or a new CVE shows up, SSH is a weird choice ...) making me wish for a better alternative. While I was browsing github I came across an old repo that allowed HTTP mounts: httpfs. I used this for a while but it was a bit slow and cumbersome to use too. In the end I decided to fix it.
The principle is rather simple: HTTP allows for relatively simple data transfer (data blocks can be read using partial fetches, which the HTTP protocol allows) so it should be trivial to build a client using libcurl. The fuse client library makes things also very simple: only a few funcions must be implemented (ie. read, open, ...) and FUSE takes care of the rest for us.
Due to the nature of the HTTP protocol (mostly used to serve inmutable data) we simply dismiss all the write functionality by returning permission erros. The only challenging part is to implement the readdir() functionality, since there's no standard way to list files within a directory. For this I resorted to the listing functionality that many web servers provide (autoindex in nginx). The first implementation was parsing the HTTP response, since it contains a full list of files and directories and it is possible to tell files from directories (using the final slash character). This had some issues so in the end I resorted to a pretty cool nginx trick: autoindex_format json. This enables a JSON listing that contains information such as file size, timestamp, type, etc. It makes the implementation very fast due to the metadata being present in the listing (otherwise it is necesary to use HEAD requests to figure out file sizes).
{
{ "name":"Movies", "type":"directory", "mtime":"Mon, 05 Sep 2022 09:07:49 GMT" },
{ "name":"Music", "type":"directory", "mtime":"Mon, 28 Feb 2022 08:56:08 GMT" },
{ "name":"Software", "type":"directory", "mtime":"Mon, 28 Feb 2022 11:37:39 GMT" },
{ "name":"better.call.saul.s06e05.1080p.web.h264.mkv", "type":"file", "mtime":"Tue, 10 May 2022 20:21:42 GMT", "size":4123844878 }
}
The implementation turned out quite simple: an HTTP client (using curl), a FUSE interface and an LRU metadata cache to make the implementation even faster. I've been using this in a few low-powered devices for a while with great results: it's fast and reliable! Here goes the github repo: httpfs+