1. Integration with HDFS
Using NFS and WebHDFS
Christophe Marchal | Software Architect

3. Access your files in HDFS

4. Network File System
WebHDFS

5. NFS support Motivations
1. File browsing and downloading
2. File uploading
3. Data streaming

6. Network File System Stack

7. NFS Gateway

8. NFS Gateway

9. Hands on
mount -t nfs -o vers=3,proto=tcp,nolock HOSTIP:/ /PATH/TO/MOUNTPOINT

10. WebHDFS

11. API
HTTP
http://<HOST>:<HTTP_PORT>/webhdfs/v1/<PATH>?op=
WebHDFS
webhdfs://<HOST>:<HTTP_PORT>/<PATH>
hdfs://<HOST>:<RPC_PORT>/<PATH>

12. API
HTTP GET
HTTP PUT
OPEN
GETFILESTATUS
LISTSTATUS
GETCONTENTSUMMARY
GETFILECHECKSUM
GETHOMEDIRECTORY
GETDELEGATIONTOKEN
CREATE
MKDIRS
RENAME
SETREPLICATION
SETOWNER
SETPERMISSION
SETTIMES
RENEWDELEGATIONTOKEN
CANCELDELEGATIONTOKEN
HTTP POST
HTTP DELETE
APPEND
DELETE

13. File status

14. Read File

15. Append
Append to a File

16. Append
Example of use

17. Thanks
Christophe Marchal | Software Architect
@toff63