Starting HDFS

During the first time logon to the configured HDFS file system, open name node (HDFS server) and execute the following command to format it.

$ hadoop namenode -format 

After formatting, need to start the distributed file system. The command to start the name node and the data nodes is -

$ start-dfs.sh 

Listing Files in HDFS

To list of files in a directory, status of a file using ‘ls’. Below syntax of ls to list all files from a directory or a filename as an argument.

$HADOOP_HOME/bin/hadoop fs -ls <args>

hadoop fs -mkdir:

It will take path uri’s as argument and creates directory or directories.

$HADOOP_HOME/bin/hadoop fs -mkdir <paths>

Space utilization in a HDFS dir.

Displays sizes of files and directories contained in the given directory or the size of a file if it’s just a file.

$HADOOP_HOME/bin/hadoop fs -du URI

Upload:

Copy single src file, or multiple src files from local file system to the Hadoop data file system.

$HADOOP_HOME/bin/hadoop fs -put <localsrc>
 ... <HDFS_dest_Path>

• <localsrc> -- Local source path
• <HDFS_dest_Path> -- HDFS destination path

Download:

Copies/Downloads files to the local file system.

$HADOOP_HOME/bin/hadoop fs -get <hdfs_src> <localdst> 

• <hdfs_src> -- HDFS source path
• <localdst> -- Local Destination to copy

Getting help:

Help command to get list of commands supported by Hadoop Data File System(HDFS).

$HADOOP_HOME/bin/hadoop fs -help

Inserting Data into HDFS

The file called filename.txt in the local system to be saved in the hdfs file system. Follow the steps given below to upload the required file in the Hadoop file system.

Create an input directory.

Transfer and store a data file from local systems to the Hadoop file system using the put command.

$HADOOP_HOME/bin/hadoop fs -put /home/filename.txt /usr/input 

Verify the file using ls command.

$HADOOP_HOME/bin/hadoop fs -ls /usr/input

Retrieving Data from HDFS

File in HDFS called outfile required to download from Hadoop file system. Given below are the steps to do it.

Initially, view the data from HDFS using cat command.

$HADOOP_HOME/bin/hadoop fs -cat /usr/output/outfile 

Get the file from HDFS to the local file system using get command.

$HADOOP_HOME/bin/hadoop fs -get /usr/output/ /home/download/ 

Shutting Down the HDFS

Shut down the HDFS by using the following command.

$ stop-dfs.sh