小文件存储系统seaweedfs

Posted on Edited on

SeaweedFS 是开源的，简单的，高伸缩性的分布式文件系统。SeaweedFS 作为支持全 POSIX 文件系统语义替代，Seaweed-FS 选择仅实现 key-file 的映射，类似 “NoSQL”，也可以说是 “NoFS”。

SeaweedFS 仅花费 40 字节的硬盘来存储每个文件的元数据。

与GlusterFS, Ceph相比较

System File Meta File Content Read POSIX REST API Optimized for small files
SeaweedFS lookup volume id, cacheable O(1) disk seek Yes Yes
SeaweedFS Filer Linearly Scalable, Customizable O(1) disk seek FUSE Yes Yes
GlusterFS hashing FUSE, NFS
Ceph hashing + rules FUSE Yes
MooseFS in memory FUSE No

体验seaweedfs

参考==>https://hub.docker.com/r/chrislusf/seaweedfs

  1. 拉取 docker-compose 文件，wget https://raw.githubusercontent.com/chrislusf/seaweedfs/master/docker/seaweedfs-compose.yml
  2. 启动，docker-compose -f seaweedfs-compose.yml -p seaweedfs up

参考 => https://github.com/chrislusf/seaweedfs/wiki/Volume-Server-API

1
2
3
4
5
6
➜  seaweedfs: curl http://localhost:9333/dir/assign
{"fid":"7,01248b7b86","url":"172.18.0.3:8080","publicUrl":"172.18.0.3:8080","count":1}%                                                                         
➜  seaweedfs: curl -F file=@/home/will/documents/zaq12wsxcde3--de4064dc15870163a9aab589a1ccf7900dd68ef4 http://127.0.0.1:8080/7,01248b7b86
{"name":"zaq12wsxcde3--de4064dc15870163a9aab589a1ccf7900dd68ef4","size":489,"eTag":"468de108"}%                                                                 
➜  seaweedfs: curl http://localhost:9333/dir/lookup\?volumeId\=7
{"volumeId":"7","locations":[{"url":"172.18.0.3:8080","publicUrl":"172.18.0.3:8080"}]}%
  1. curl http://localhost:9333/dir/assign 获取fid
  2. curl -F file=@/home/will/documents/zaq12wsxcde3--de4064dc15870163a9aab589a1ccf7900dd68ef4 http://127.0.0.1:8080/7,01248b7b86 上传文件
  3. curl http://localhost:9333/dir/lookup\?volumeId\=7 查询
  4. 在浏览器中打开 http://172.18.0.3:8080/7,01248b7b86，查看文档
  5. master 的url：http://127.0.0.1:9333/
  6. volume的url：http://172.18.0.3:8080/ui/index.html