File Writer - HDFS File Access
- For SFTP files, if you attempt to write a file into a directory where you do not have a write access right, the write operation will fail with "access denied" error. When you get an SFTP account credential, it is also important to know where your home directory is, for example, sftp://ftp.snaplogic.com/home/mrtest for username "mrtest"
- HDFS Example : For HDFS file access, please use a SnapLogic on-premises Groundplex and make sure that its instance is within the Hadoop cluster and SSH authentication has already been established. You can access HDFS files in the same way as other file protocols in File Reader and File Writer Snaps. There is no need to use any account in the Snap.
Warning:
HDFS 2.4.0 is supported for the hdfs protocol.
An example for HDFS is:
hdfs://<hostname>:<port number>/<path
to folder>/<filename> If Cloudera Hadoop Name node is installed in AWS EC2 and its hostname is "ec2-54-198-212-134.compute-1.amazonaws.com" and its port number is 8020, then you would enter:
hdfs://ec2-54-198-212-134.compute-1.amazonaws.com:8020/user/john/input/sample.csv