GopherCon is here, and it is time to reveal what Bazil is all about.
Bazil, also known as bazil.org/bazil
, is a file system that lets
your data reside where it is most convenient for it to reside.
Bazil is still under heavy development, but welcomes developers and curious power users. Here's a little teaser of what's coming.
Imagine you have
On the desktop, you naturally want to be able to use the whole 3TB disk. And you're not always using the desktop, even when you're home – the sofa is just so comfortable. You'd like to work with your files even when you're on the laptop.
So you install the currently fashionable large-corporate-backed cloud-sync solution.
A file sync based solution will try to copy all of your files from the desktop to the laptop – yet the laptop's smaller SSD just can't hold that much! You're forced to play games with picking-and-choosing what folders get synchronized, and just don't have the convenience of grabbing that 8-year-old wedding photo on a whim.
To modernize an aphorism, you can't put ten terabytes of files on a 500 GB SSD. Syncing between very disproportionate systems is fundamentally a problematic design, and is best for a small hand-picked set of files, not as an actual storage solution.
Don't take this the wrong way; you really should have some sort of remote backups for important data, in case the building burns down. S3 RRS/Glacier, Google Cloud Storage DRA seem very promising for backup cold storage; we'll come back to that later.
Rocking it old school? We're down with that.
A network file system like CIFS or NFS, or something like sshfs
,
would let you use the files from the desktop on the laptop – but your
wifi will never be as fast as the laptop's local SSD, in either
bandwidth or latency, so now all your file accesses are crawling, and
you end up hunting for an ethernet cable whenever you need to transfer
something bigger.
To speed things up, you end up copying often used files to the SSD. Now you have several copies of the same files, and no idea what was modified when, or whether you're looking at the last copy, or whether it's safe to delete to free up space on the cramped laptop.
A network file system will also require for you to stay within wifi range. For travel, you're once again reduced to up manually copying files around, and once again lose track of where's the latest copy of what file.
However, this leaves you installing & configuring cloud backup software in addition to your network file system woes; not the simplest ordeal, and don't expect any kind of file history browsing / recovery integration for you network file system clients.
or
You can choose to back up to your own disks – with the same caveats as aboveor
All of the bad parts of the computer in the closet, with the extra of needing to fiddle with the disks and remember things.Bazil separates knowledge of a file from the contents of the file, letting the laptop know about all of the files, without having to store the contents of the file.
With Bazil, the laptop SSD contents act as
And because Bazil keeps track of the changes, it can also keep track of changes and synchronize them between the different peers; no more confusion about what copy is the latest.
You try to read a file where the contents are not locally stored, the data will be fetched from desktop or cloud/closet server, whichever happens to be the fastest way. All the data is accessible even if it won't fit on the SSD.
You can pin files for travel, so you're no longer tied to your home network, or even any network connectivity.
Bazil is the archival solution, with the snapshot feature. Every Bazil peer can browse the earlier snapshots, making restoring files easy no matter what computer you're on. You don't have to manage both a network file system and a backup solution.
Bazil is the redundancy solution, with copies of file contents
stored on multiple computers. The CAS stores
immutable, write-once objects, so you can even mitigate software bugs
by taking an extra copy of the history with just rsync
, file system
snapshots, or any other file copy tool. A snapshot is just an object,
and refers to other objects; the objects contain everything needed to
regain access to your files.
All Bazil file storage can be encrypted to guarantee your privacy, whether in the cloud, on your own computers, or on external hard drives. Encryption is on by default.
Bazil is still under heavy development, and a lot of the functionality hinted at above is still not quite there. We welcome developers and curious power users.
See the documentation for more, and feel free to ask questions on the mailing list or Twitter @BazilFS.
The original gopher image was made by Renee French.
posted on 2014-04-24, tagged as bazil project announce