org.apache.hadoop.dfs
Class NameNode

java.lang.Object
  org.apache.hadoop.dfs.NameNode

All Implemented Interfaces:

FSConstants

public class NameNode
extends Object
implements FSConstants

NameNode serves as both directory namespace manager and "inode table" for the Hadoop DFS. There is a single NameNode running in any DFS deployment. (Well, except when there is a second backup/failover NameNode.) The NameNode controls two critical tables: 1) filename->blocksequence (namespace) 2) block->machinelist ("inodes") The first table is stored on disk and is very precious. The second table is rebuilt every time the NameNode comes up. 'NameNode' refers to both this class as well as the 'NameNode server'. The 'FSNamesystem' class actually performs most of the filesystem management. The majority of the 'NameNode' class itself is concerned with exposing the IPC interface to the outside world, plus some configuration management. NameNode implements the ClientProtocol interface, which allows clients to ask for DFS services. ClientProtocol is not designed for direct use by authors of DFS client code. End-users should instead use the org.apache.nutch.hadoop.fs.FileSystem class. NameNode also implements the DatanodeProtocol interface, used by DataNode programs that actually store DFS data blocks. These methods are invoked repeatedly and automatically by all the DataNodes in a DFS deployment. NameNode also implements the NamenodeProtocol interface, used by secondary namenodes or rebalancing processes to get partial namenode's state, for example partial blocksMap etc.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.dfs.FSConstants

FSConstants.CheckpointStates, FSConstants.DatanodeReportType, FSConstants.NodeType, FSConstants.SafeModeAction, FSConstants.StartupOption, FSConstants.UpgradeAction

Field Summary

static int	DEFAULT_PORT
static int	DISK_ERROR
static int	DNA_BLOCKREPORT
static int	DNA_FINALIZE
static int	DNA_INVALIDATE
static int	DNA_RECOVERBLOCK
static int	DNA_REGISTER
static int	DNA_SHUTDOWN
static int	DNA_TRANSFER
static int	DNA_UNKNOWN Determines actions that data node should perform when receiving a datanode command.
static int	INVALID_BLOCK
static org.apache.commons.logging.Log	LOG
static int	NOTIFY
static org.apache.commons.logging.Log	stateChangeLog
static long	versionID Compared to the previous version the following changes have been introduced: (Only the latest change is reflected.
static long	versionID 16: Block parameter added to nextGenerationStamp().
static long	versionID 1: changed the serialization in DatanodeInfo

Fields inherited from interface org.apache.hadoop.dfs.FSConstants

BLOCK_INVALIDATE_CHUNK, BLOCKREPORT_INITIAL_DELAY, BLOCKREPORT_INTERVAL, BUFFER_SIZE, COMPLETE_SUCCESS, DATA_TRANSFER_VERSION, DEFAULT_BLOCK_SIZE, DEFAULT_DATA_SOCKET_SIZE, HEARTBEAT_INTERVAL, LAYOUT_VERSION, LEASE_HARDLIMIT_PERIOD, LEASE_SOFTLIMIT_PERIOD, MAX_PATH_DEPTH, MAX_PATH_LENGTH, MIN_BLOCKS_FOR_WRITE, OP_ACK, OP_BLOCKRECEIVED, OP_BLOCKREPORT, OP_CLIENT_ABANDONBLOCK, OP_CLIENT_ABANDONBLOCK_ACK, OP_CLIENT_ADDBLOCK, OP_CLIENT_ADDBLOCK_ACK, OP_CLIENT_COMPLETEFILE, OP_CLIENT_COMPLETEFILE_ACK, OP_CLIENT_DATANODE_HINTS, OP_CLIENT_DATANODE_HINTS_ACK, OP_CLIENT_DATANODEREPORT, OP_CLIENT_DATANODEREPORT_ACK, OP_CLIENT_DELETE, OP_CLIENT_DELETE_ACK, OP_CLIENT_EXISTS, OP_CLIENT_EXISTS_ACK, OP_CLIENT_ISDIR, OP_CLIENT_ISDIR_ACK, OP_CLIENT_LISTING, OP_CLIENT_LISTING_ACK, OP_CLIENT_MKDIRS, OP_CLIENT_MKDIRS_ACK, OP_CLIENT_OBTAINLOCK, OP_CLIENT_OBTAINLOCK_ACK, OP_CLIENT_OPEN, OP_CLIENT_OPEN_ACK, OP_CLIENT_RAWSTATS, OP_CLIENT_RAWSTATS_ACK, OP_CLIENT_RELEASELOCK, OP_CLIENT_RELEASELOCK_ACK, OP_CLIENT_RENAMETO, OP_CLIENT_RENAMETO_ACK, OP_CLIENT_RENEW_LEASE, OP_CLIENT_RENEW_LEASE_ACK, OP_CLIENT_STARTFILE, OP_CLIENT_STARTFILE_ACK, OP_CLIENT_TRYAGAIN, OP_COPY_BLOCK, OP_ERROR, OP_FAILURE, OP_HEARTBEAT, OP_INVALIDATE_BLOCKS, OP_READ_BLOCK, OP_READ_METADATA, OP_REPLACE_BLOCK, OP_STATUS_CHECKSUM_OK, OP_STATUS_ERROR, OP_STATUS_ERROR_CHECKSUM, OP_STATUS_ERROR_EXISTS, OP_STATUS_ERROR_INVALID, OP_STATUS_SUCCESS, OP_TRANSFERBLOCKS, OP_TRANSFERDATA, OP_WRITE_BLOCK, OPERATION_FAILED, READ_TIMEOUT, SIZE_OF_INTEGER, SMALL_BUFFER_SIZE, STILL_WAITING, WRITE_TIMEOUT, WRITE_TIMEOUT_EXTENSION

Constructor Summary

NameNode(Configuration conf)
Start NameNode.

NameNode(String bindAddress, Configuration conf)
Create a NameNode at the specified location and start it.

Method Summary

void	abandonBlock(org.apache.hadoop.dfs.Block b, String src, String holder) The client needs to give up on the block.
org.apache.hadoop.dfs.LocatedBlock	addBlock(String src, String clientName) A client that wants to write an additional block to the indicated filename (which must currently be open for writing) should call addBlock().
void	blockReceived(org.apache.hadoop.dfs.DatanodeRegistration nodeReg, org.apache.hadoop.dfs.Block[] blocks, String[] delHints) blockReceived() allows the DataNode to tell the NameNode about recently-received block data, with a hint for pereferred replica to be deleted when there is any excessive blocks.
org.apache.hadoop.dfs.DatanodeCommand	blockReport(org.apache.hadoop.dfs.DatanodeRegistration nodeReg, long[] blocks) blockReport() tells the NameNode about all the locally-stored blocks.
void	clearQuota(String path) Remove the quota for a directory
void	commitBlockSynchronization(org.apache.hadoop.dfs.Block block, long newgenerationstamp, long newlength, boolean closeFile, boolean deleteblock, DatanodeID[] newtargets) Commit block synchronization in lease recovery
boolean	complete(String src, String clientName) The client is done writing data to the given filename, and would like to complete it.
void	create(String src, FsPermission masked, String clientName, boolean overwrite, short replication, long blockSize) Create a new file entry in the namespace.
boolean	delete(String src) Deprecated.
boolean	delete(String src, boolean recursive) Delete the given file or directory from the file system.
UpgradeStatusReport	distributedUpgradeProgress(FSConstants.UpgradeAction action) Report distributed upgrade progress or force current upgrade to proceed.
void	errorReport(org.apache.hadoop.dfs.DatanodeRegistration nodeReg, int errorCode, String msg) errorReport() tells the NameNode about something that has gone awry.
void	finalizeUpgrade() Finalize previous upgrade.
static void	format(Configuration conf) Format a new filesystem.
void	fsync(String src, String clientName) Write all metadata for this file into persistent storage.
LocatedBlocks	getBlockLocations(String src, long offset, long length) Get locations of the blocks of the specified file within the specified range.
org.apache.hadoop.dfs.BlocksWithLocations	getBlocks(DatanodeInfo datanode, long size) return a list of blocks & their locations on datanode whose total size is size
ContentSummary	getContentSummary(String path) Get ContentSummary rooted at the specified directory.
DatanodeInfo[]	getDatanodeReport(FSConstants.DatanodeReportType type) Get a report on the system's current datanodes.
long	getEditLogSize() Returns the size of the current edit log.
org.apache.hadoop.dfs.DFSFileInfo	getFileInfo(String src) Get the file info for a specific file.
File	getFsImageName() Returns the name of the fsImage file
File[]	getFsImageNameCheckpoint() Returns the name of the fsImage file uploaded by periodic checkpointing
org.apache.hadoop.dfs.DFSFileInfo[]	getListing(String src) Get a listing of the indicated directory
InetSocketAddress	getNameNodeAddress() Returns the address on which the NameNodes is listening to.
static NameNodeMetrics	getNameNodeMetrics()
long	getPreferredBlockSize(String filename) Get the block size for the given file.
long	getProtocolVersion(String protocol, long clientVersion) Return protocol version corresponding to protocol interface.
long[]	getStats() Get a set of statistics about the filesystem.
boolean	isInSafeMode() Is the cluster currently in safe mode?
void	join() Wait for service to finish.
static void	main(String[] argv)
void	metaSave(String filename) Dumps namenode state into specified file
boolean	mkdirs(String src, FsPermission masked) Create a directory (or hierarchy of directories) with the given name and permission.
long	nextGenerationStamp(org.apache.hadoop.dfs.Block block)
org.apache.hadoop.dfs.UpgradeCommand	processUpgradeCommand(org.apache.hadoop.dfs.UpgradeCommand comm) This is a very general way to send a command to the name-node during distributed upgrade process.
void	refreshNodes() Tells the namenode to reread the hosts and exclude files.
org.apache.hadoop.dfs.DatanodeRegistration	register(org.apache.hadoop.dfs.DatanodeRegistration nodeReg) Register Datanode.
boolean	rename(String src, String dst) Rename an item in the file system namespace.
void	renewLease(String clientName) Client programs can cause stateful changes in the NameNode that affect other clients.
void	reportBadBlocks(org.apache.hadoop.dfs.LocatedBlock[] blocks) The client has detected an error on the specified located blocks and is reporting them to the server.
org.apache.hadoop.dfs.CheckpointSignature	rollEditLog() Roll the edit log.
void	rollFsImage() Roll the image
org.apache.hadoop.dfs.DatanodeCommand	sendHeartbeat(org.apache.hadoop.dfs.DatanodeRegistration nodeReg, long capacity, long dfsUsed, long remaining, int xmitsInProgress, int xceiverCount) Data node notify the name node that it is alive Return a block-oriented command for the datanode to execute.
void	setOwner(String src, String username, String groupname) Set owner of a path (i.e.
void	setPermission(String src, FsPermission permissions) Set permissions for an existing file/directory.
void	setQuota(String path, long quota) Set the quota for a directory.
boolean	setReplication(String src, short replication) Set replication for an existing file.
boolean	setSafeMode(FSConstants.SafeModeAction action) Enter, leave or get safe mode.
void	stop() Stop all NameNode threads and wait for all to finish.
void	verifyRequest(org.apache.hadoop.dfs.DatanodeRegistration nodeReg) Verify request.
void	verifyVersion(int version) Verify version.
org.apache.hadoop.dfs.NamespaceInfo	versionRequest()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_PORT

public static final int DEFAULT_PORT

See Also:

Constant Field Values

LOG

public static final org.apache.commons.logging.Log LOG

stateChangeLog

public static final org.apache.commons.logging.Log stateChangeLog

versionID

public static final long versionID

Compared to the previous version the following changes have been introduced: (Only the latest change is reflected. The log of historical changes can be retrieved from the svn). 35 : Quota-related RPCs are introduced: getQuota, clearQuota; Besides, getContentSummary also returns the quota of the directory.

See Also:

Constant Field Values

versionID

public static final long versionID

16: Block parameter added to nextGenerationStamp().

See Also:

Constant Field Values

NOTIFY

public static final int NOTIFY

See Also:

Constant Field Values

DISK_ERROR

public static final int DISK_ERROR

See Also:

Constant Field Values

INVALID_BLOCK

public static final int INVALID_BLOCK

See Also:

Constant Field Values

DNA_UNKNOWN

public static final int DNA_UNKNOWN

Determines actions that data node should perform when receiving a datanode command.

See Also:

Constant Field Values

DNA_TRANSFER

public static final int DNA_TRANSFER

See Also:

Constant Field Values

DNA_INVALIDATE

public static final int DNA_INVALIDATE

See Also:

Constant Field Values

DNA_SHUTDOWN

public static final int DNA_SHUTDOWN

See Also:

Constant Field Values

DNA_REGISTER

public static final int DNA_REGISTER

See Also:

Constant Field Values

DNA_FINALIZE

public static final int DNA_FINALIZE

See Also:

Constant Field Values

DNA_BLOCKREPORT

public static final int DNA_BLOCKREPORT

See Also:

Constant Field Values

DNA_RECOVERBLOCK

public static final int DNA_RECOVERBLOCK

See Also:

Constant Field Values

versionID

public static final long versionID

1: changed the serialization in DatanodeInfo

See Also:

Constant Field Values

Constructor Detail

NameNode

public NameNode(Configuration conf)
         throws IOException

Start NameNode.

The name-node can be started with one of the following startup options:

• REGULAR - normal startup
• FORMAT - format name node
• UPGRADE - start the cluster upgrade and create a snapshot of the current file system state
• ROLLBACK - roll the cluster back to the previous state

The option is passed via configuration field: dfs.namenode.startup The conf will be modified to reflect the actual ports on which the NameNode is up and running if the user passes the port as zero in the conf.

Parameters:

conf - confirguration

Throws:

IOException

NameNode

public NameNode(String bindAddress,
                Configuration conf)
         throws IOException

Create a NameNode at the specified location and start it. The conf will be modified to reflect the actual ports on which the NameNode is up and running if the user passes the port as zero.

Throws:

IOException

Method Detail

getProtocolVersion

public long getProtocolVersion(String protocol,
                               long clientVersion)
                        throws IOException

Description copied from interface: VersionedProtocol

Return protocol version corresponding to protocol interface.

Parameters:

protocol - The classname of the protocol interface

clientVersion - The version of the protocol that the client speaks

Returns:

the version that the server will speak

Throws:

IOException

format

public static void format(Configuration conf)
                   throws IOException

Format a new filesystem. Destroys any filesystem that may already exist at this location.

Throws:

IOException

getNameNodeMetrics

public static NameNodeMetrics getNameNodeMetrics()

join

public void join()

Wait for service to finish. (Normally, it runs forever.)

stop

public void stop()

Stop all NameNode threads and wait for all to finish.

getBlocks

public org.apache.hadoop.dfs.BlocksWithLocations getBlocks(DatanodeInfo datanode,
                                                           long size)
                                                    throws IOException

return a list of blocks & their locations on datanode whose total size is size

Parameters:

datanode - on which blocks are located

size - total size of blocks

Returns:

a list of blocks & their locations

Throws:

IOException

getBlockLocations

public LocatedBlocks getBlockLocations(String src,
                                       long offset,
                                       long length)
                                throws IOException

Get locations of the blocks of the specified file within the specified range. DataNode locations for each block are sorted by the proximity to the client.

Return LocatedBlocks which contains file length, blocks and their locations. DataNode locations for each block are sorted by the distance to the client's address.

The client will then have to contact one of the indicated DataNodes to obtain the actual data.

Parameters:

src - file name

offset - range start offset

length - range length

Returns:

file length and array of blocks with their locations

Throws:

IOException

create

public void create(String src,
                   FsPermission masked,
                   String clientName,
                   boolean overwrite,
                   short replication,
                   long blockSize)
            throws IOException

Create a new file entry in the namespace.

This will create an empty file specified by the source path. The path should reflect a full path originated at the root. The name-node does not have a notion of "current" directory for a client.

Once created, the file is visible and available for read to other clients. Although, other clients cannot delete(String), re-create or rename(String, String) it until the file is completed or explicitly as a result of lease expiration.

Blocks have a maximum size. Clients that intend to create multi-block files must also use addBlock(String, String).

Parameters:

src - path of the file being created.

masked - masked permission.

clientName - name of the current client.

overwrite - indicates whether the file should be overwritten if it already exists.

replication - block replication factor.

blockSize - maximum block size.

Throws:

AccessControlException - if permission to create file is denied by the system. As usually on the client side the exception will be wrapped into RemoteException.

QuotaExceededException - if the file creation violates any quota restriction

IOException - if other errors occur.

setReplication

public boolean setReplication(String src,
                              short replication)
                       throws IOException

Set replication for an existing file.

The NameNode sets replication to the new value and returns. The actual block replication is not expected to be performed during this method call. The blocks will be populated or removed in the background as the result of the routine block maintenance procedures.

Parameters:

src - file name

replication - new replication

Returns:

true if successful; false if file does not exist or is a directory

Throws:

IOException

setPermission

public void setPermission(String src,
                          FsPermission permissions)
                   throws IOException

Set permissions for an existing file/directory.

Throws:

IOException

setOwner

public void setOwner(String src,
                     String username,
                     String groupname)
              throws IOException

Set owner of a path (i.e. a file or a directory). The parameters username and groupname cannot both be null.

username - If it is null, the original username remains unchanged.

groupname - If it is null, the original groupname remains unchanged.

Throws:

IOException

addBlock

public org.apache.hadoop.dfs.LocatedBlock addBlock(String src,
                                                   String clientName)
                                            throws IOException

A client that wants to write an additional block to the indicated filename (which must currently be open for writing) should call addBlock(). addBlock() allocates a new block and datanodes the block data should be replicated to.

Returns:

LocatedBlock allocated block information.

Throws:

IOException

abandonBlock

public void abandonBlock(org.apache.hadoop.dfs.Block b,
                         String src,
                         String holder)
                  throws IOException

The client needs to give up on the block.

Throws:

IOException

complete

public boolean complete(String src,
                        String clientName)
                 throws IOException

The client is done writing data to the given filename, and would like to complete it. The function returns whether the file has been closed successfully. If the function returns false, the caller should try again. A call to complete() will not return true until all the file's blocks have been replicated the minimum number of times. Thus, DataNode failures may cause a client to call complete() several times before succeeding.

Throws:

IOException

reportBadBlocks

public void reportBadBlocks(org.apache.hadoop.dfs.LocatedBlock[] blocks)
                     throws IOException

The client has detected an error on the specified located blocks and is reporting them to the server. For now, the namenode will mark the block as corrupt. In the future we might check the blocks are actually corrupt.

Parameters:

blocks - Array of located blocks to report

Throws:

IOException

nextGenerationStamp

public long nextGenerationStamp(org.apache.hadoop.dfs.Block block)
                         throws IOException

Returns:

the next GenerationStamp to be associated with the specified block.

Throws:

IOException

commitBlockSynchronization

public void commitBlockSynchronization(org.apache.hadoop.dfs.Block block,
                                       long newgenerationstamp,
                                       long newlength,
                                       boolean closeFile,
                                       boolean deleteblock,
                                       DatanodeID[] newtargets)
                                throws IOException

Commit block synchronization in lease recovery

Throws:

IOException

getPreferredBlockSize

public long getPreferredBlockSize(String filename)
                           throws IOException

Get the block size for the given file.

Parameters:

filename - The name of the file

Returns:

The number of bytes in each block

Throws:

IOException

rename

public boolean rename(String src,
                      String dst)
               throws IOException

Rename an item in the file system namespace.

Parameters:

src - existing file or directory name.

dst - new name.

Returns:

true if successful, or false if the old name does not exist or if the new name already belongs to the namespace.

Throws:

IOException - if the new name is invalid.

QuotaExceededException - if the rename would violate any quota restriction

delete

@Deprecated
public boolean delete(String src)
               throws IOException

Deprecated.

Delete the given file or directory from the file system.

Any blocks belonging to the deleted files will be garbage-collected.

Parameters:

src - existing name.

Returns:

true only if the existing file or directory was actually removed from the file system.

Throws:

IOException

delete

public boolean delete(String src,
                      boolean recursive)
               throws IOException

Delete the given file or directory from the file system.

same as delete but provides a way to avoid accidentally deleting non empty directories programmatically.

Parameters:

src - existing name

recursive - if true deletes a non empty directory recursively, else throws an exception.

Returns:

true only if the existing file or directory was actually removed from the file system.

Throws:

IOException

mkdirs

public boolean mkdirs(String src,
                      FsPermission masked)
               throws IOException

Create a directory (or hierarchy of directories) with the given name and permission.

Parameters:

src - The path of the directory being created

masked - The masked permission of the directory being created

Returns:

True if the operation success.

Throws:

QuotaExceededException - if the operation would violate any quota restriction.

IOException

renewLease

public void renewLease(String clientName)
                throws IOException

Client programs can cause stateful changes in the NameNode that affect other clients. A client may obtain a file and neither abandon nor complete it. A client might hold a series of locks that prevent other clients from proceeding. Clearly, it would be bad if a client held a bunch of locks that it never gave up. This can happen easily if the client dies unexpectedly.

So, the NameNode will revoke the locks and live file-creates for clients that it thinks have died. A client tells the NameNode that it is still alive by periodically calling renewLease(). If a certain amount of time passes since the last call to renewLease(), the NameNode assumes the client has died.

Throws:

IOException

getListing

public org.apache.hadoop.dfs.DFSFileInfo[] getListing(String src)
                                               throws IOException

Get a listing of the indicated directory

Throws:

IOException

getFileInfo

public org.apache.hadoop.dfs.DFSFileInfo getFileInfo(String src)
                                              throws IOException

Get the file info for a specific file.

Parameters:

src - The string representation of the path to the file

Returns:

object containing information regarding the file or null if file not found

Throws:

IOException - if permission to access file is denied by the system

getStats

public long[] getStats()
                throws IOException

Get a set of statistics about the filesystem. Right now, only three values are returned.

• [0] contains the total storage capacity of the system, in bytes.
• [1] contains the total used space of the system, in bytes.
• [2] contains the available storage of the system, in bytes.

Throws:

IOException

getDatanodeReport

public DatanodeInfo[] getDatanodeReport(FSConstants.DatanodeReportType type)
                                 throws IOException

Get a report on the system's current datanodes. One DatanodeInfo object is returned for each DataNode. Return live datanodes if type is LIVE; dead datanodes if type is DEAD; otherwise all datanodes if type is ALL.

Throws:

IOException

setSafeMode

public boolean setSafeMode(FSConstants.SafeModeAction action)
                    throws IOException

Enter, leave or get safe mode.

Safe mode is a name node state when it

1. does not accept changes to name space (read-only), and
2. does not replicate or delete blocks.

Safe mode is entered automatically at name node startup. Safe mode can also be entered manually using setSafeMode(SafeModeAction.SAFEMODE_GET).

At startup the name node accepts data node reports collecting information about block locations. In order to leave safe mode it needs to collect a configurable percentage called threshold of blocks, which satisfy the minimal replication condition. The minimal replication condition is that each block must have at least dfs.replication.min replicas. When the threshold is reached the name node extends safe mode for a configurable amount of time to let the remaining data nodes to check in before it will start replicating missing blocks. Then the name node leaves safe mode.

If safe mode is turned on manually using setSafeMode(SafeModeAction.SAFEMODE_ENTER) then the name node stays in safe mode until it is manually turned off using setSafeMode(SafeModeAction.SAFEMODE_LEAVE). Current state of the name node can be verified using setSafeMode(SafeModeAction.SAFEMODE_GET)

Configuration parameters:

dfs.safemode.threshold.pct is the threshold parameter.
dfs.safemode.extension is the safe mode extension parameter.
dfs.replication.min is the minimal replication parameter.

Special cases:

The name node does not enter safe mode at startup if the threshold is set to 0 or if the name space is empty.
If the threshold is set to 1 then all blocks need to have at least minimal replication.
If the threshold value is greater than 1 then the name node will not be able to turn off safe mode automatically.
Safe mode can always be turned off manually.

Parameters:

action -

• 0 leave safe mode;
• 1 enter safe mode;
• 2 get safe mode state.

Returns:

• 0 if the safe mode is OFF or
• 1 if the safe mode is ON.

Throws:

IOException

isInSafeMode

public boolean isInSafeMode()

Is the cluster currently in safe mode?

refreshNodes

public void refreshNodes()
                  throws IOException

Tells the namenode to reread the hosts and exclude files.

Throws:

IOException

getEditLogSize

public long getEditLogSize()
                    throws IOException

Returns the size of the current edit log.

Returns:

The number of bytes in the current edit log.

Throws:

IOException

rollEditLog

public org.apache.hadoop.dfs.CheckpointSignature rollEditLog()
                                                      throws IOException

Roll the edit log.

Returns:

a unique token to identify this transaction.

Throws:

IOException

rollFsImage

public void rollFsImage()
                 throws IOException

Roll the image

Throws:

IOException

finalizeUpgrade

public void finalizeUpgrade()
                     throws IOException

Finalize previous upgrade. Remove file system state saved during the upgrade. The upgrade will become irreversible.

Throws:

IOException

distributedUpgradeProgress

public UpgradeStatusReport distributedUpgradeProgress(FSConstants.UpgradeAction action)
                                               throws IOException

Report distributed upgrade progress or force current upgrade to proceed.

Parameters:

action - FSConstants.UpgradeAction to perform

Returns:

upgrade status information or null if no upgrades are in progress

Throws:

IOException

metaSave

public void metaSave(String filename)
              throws IOException

Dumps namenode state into specified file

Throws:

IOException

getContentSummary

public ContentSummary getContentSummary(String path)
                                 throws IOException

Get ContentSummary rooted at the specified directory.

Parameters:

path - The string representation of the path

Throws:

IOException

setQuota

public void setQuota(String path,
                     long quota)
              throws IOException

Set the quota for a directory.

Parameters:

path - The string representation of the path to the directory

quota - The limit of the number of names in the tree rooted at the directory

Throws:

FileNotFoundException - if the path is a file or does not exist

QuotaExceededException - if the directory size is greater than the given quota

IOException

clearQuota

public void clearQuota(String path)
                throws IOException

Remove the quota for a directory

Parameters:

path - The string representation of the path to the directory

Throws:

FileNotFoundException - if the path is not a directory

IOException

fsync

public void fsync(String src,
                  String clientName)
           throws IOException

Write all metadata for this file into persistent storage. The file must be currently open for writing.

Parameters:

src - The string representation of the path

clientName - The string representation of the client

Throws:

IOException

register

public org.apache.hadoop.dfs.DatanodeRegistration register(org.apache.hadoop.dfs.DatanodeRegistration nodeReg)
                                                    throws IOException

Register Datanode.

Returns:

updated DatanodeRegistration, which contains new storageID if the datanode did not have one and registration ID for further communication.

Throws:

IOException

See Also:

DataNode.register(), FSNamesystem.registerDatanode(DatanodeRegistration)

sendHeartbeat

public org.apache.hadoop.dfs.DatanodeCommand sendHeartbeat(org.apache.hadoop.dfs.DatanodeRegistration nodeReg,
                                                           long capacity,
                                                           long dfsUsed,
                                                           long remaining,
                                                           int xmitsInProgress,
                                                           int xceiverCount)
                                                    throws IOException

Data node notify the name node that it is alive Return a block-oriented command for the datanode to execute. This will be either a transfer or a delete operation.

Throws:

IOException

blockReport

public org.apache.hadoop.dfs.DatanodeCommand blockReport(org.apache.hadoop.dfs.DatanodeRegistration nodeReg,
                                                         long[] blocks)
                                                  throws IOException

blockReport() tells the NameNode about all the locally-stored blocks. The NameNode returns an array of Blocks that have become obsolete and should be deleted. This function is meant to upload *all* the locally-stored blocks. It's invoked upon startup and then infrequently afterwards.

blocks - - the block list as an array of longs. Each block is represented as 2 longs. This is done instead of Block[] to reduce memory used by block reports.

Returns:

- the next command for DN to process.

Throws:

IOException

blockReceived

public void blockReceived(org.apache.hadoop.dfs.DatanodeRegistration nodeReg,
                          org.apache.hadoop.dfs.Block[] blocks,
                          String[] delHints)
                   throws IOException

blockReceived() allows the DataNode to tell the NameNode about recently-received block data, with a hint for pereferred replica to be deleted when there is any excessive blocks. For example, whenever client code writes a new Block here, or another DataNode copies a Block to this DataNode, it will call blockReceived().

Throws:

IOException

errorReport

public void errorReport(org.apache.hadoop.dfs.DatanodeRegistration nodeReg,
                        int errorCode,
                        String msg)
                 throws IOException

errorReport() tells the NameNode about something that has gone awry. Useful for debugging.

Throws:

IOException

versionRequest

public org.apache.hadoop.dfs.NamespaceInfo versionRequest()
                                                   throws IOException

Throws:

IOException

processUpgradeCommand

public org.apache.hadoop.dfs.UpgradeCommand processUpgradeCommand(org.apache.hadoop.dfs.UpgradeCommand comm)
                                                           throws IOException

This is a very general way to send a command to the name-node during distributed upgrade process. The generosity is because the variety of upgrade commands is unpredictable. The reply from the name-node is also received in the form of an upgrade command.

Returns:

a reply in the form of an upgrade command

Throws:

IOException

verifyRequest

public void verifyRequest(org.apache.hadoop.dfs.DatanodeRegistration nodeReg)
                   throws IOException

Verify request. Verifies correctness of the datanode version, registration ID, and if the datanode does not need to be shutdown.

Parameters:

nodeReg - data node registration

Throws:

IOException

verifyVersion

public void verifyVersion(int version)
                   throws IOException

Verify version.

Parameters:

version -

Throws:

IOException

getFsImageName

public File getFsImageName()
                    throws IOException

Returns the name of the fsImage file

Throws:

IOException

getFsImageNameCheckpoint

public File[] getFsImageNameCheckpoint()
                                throws IOException

Returns the name of the fsImage file uploaded by periodic checkpointing

Throws:

IOException

getNameNodeAddress

public InetSocketAddress getNameNodeAddress()

Returns the address on which the NameNodes is listening to.

Returns:

the address on which the NameNodes is listening to.

main

public static void main(String[] argv)
                 throws Exception

Throws:

Exception