There is nothing exciting about filesystems
August 14th, 2009 by tglek
When I originally started at Mozilla, I only knew the people who interviewed me. But I quickly discovered beltzner when he uttered a sacrilegious statement that went something like: “….. nothing could be as boring as filesystems….”. Mike Beltzner is one of my favourite characters at Mozilla for his ability to speak his mind, but this quote has troubled me greatly. How can one not care about filesystems? Linux’s ability to do file stuff efficiently makes it magnitudes faster than other operating systems. Plan 9′s file-system-centric layout proved that OSes don’t have to consist of a series of poorly named and categorized system calls. In fact, a clean file layout allows many awesome optimizations. ZFS is one of the few things keeping Solaris relevant. HFS+ is one of the things keeping OSX from being fast.
Being a Linux user, I was disappointed by the pointlessness of optimizing application IO. Sure we inefficiently open tons of files on startup, sure we hit the filesystem 10-100x more than we could, why would one optimize when there when there is no more than a few percent of startup being take up by terrible io patterns?
Excitingly Crappy Filesystems
Luckily Firefox runs on OSX and we are making it run on WinCE. I was delighted to discover that on wince* we paid 1-5ms per file existence check, modification date, size, etc. I was shocked to see that the throughput while reading certain files could be expressed in bytes per second (most crappy flash media seems to be able to pull in >1mb/s). This brought upon switching our jar io to mmap, amalgamating jar files, moving more files into jars, etc. I’ll blog about the details later. My basic idea is that we can utilize jar files as “controlled filesystem environments” to deal with having to run on crappy OSes with exceptionally bad filesystems. OSes such as OSX where file IO is barely faster than that of a WinCE phone.
Beltzner, wouldn’t it be exciting if OSes like Mac OSX had file systems worth being excited about?
* MS likes to use puns for their product names
5 Responses to “There is nothing exciting about filesystems”
I take some of it back; filesystems can be exciting, especially when trying to optimize around them. I’m incredibly thankful that we have super-smart people like yourself who can figure out how to sidestep the performance problems of a platform in order to deliver better experiences for our users.
Way to go, Taras!
(ps: I actually did know a lot about filesystems at some point, and was astonished to discover how complex inodes and the like were … then I promptly forgot everything)
Nothing made me more sad than when any mention of ZFS started disappearing from the Snow Leopard (10.6) developer builds. HFS+ is such a bad file system, and they were in the process of at least giving us the option to have ZFS if we knew what we were doing — now it’s nowhere to be found.
Filesystems are awesome, I need to have a serious talk with Beltzner once I get back from vacation.
As a old Solaris hand, I too was greatly excited when the first hints of ZFS in 10.5 (and later 10.6) came out, and subsequently greatly disappointed when it seemed that it had been dropped.
> My basic idea is that we can utilize jar files as
> “controlled filesystem environments” to deal with
> having to run on crappy OSes with exceptionally bad
> filesystems
When I started understanding you were doing that though your newsgroup posts, I was a bit shocked. It really sounded like inappropriate optimization “hey, why is this guy trying to reimplement the file system !”
I could just not imagine the wince fs was that bad.
But is “jar” a good filesystem ? Maybe for read only files, but if you really swallow the pill, you’ll need writing in your filesystem. What about just including btrfs fully ? ;->
I sure hope sam sidler reads this post, then he will know how awesome linux is.:)