org.apache.hadoop.dfs
Interface FSDatasetInterface

All Superinterfaces:

FSDatasetMBean

public interface FSDatasetInterface
extends FSDatasetMBean

This is an interface for the underlying storage that stores blocks for a data node. Examples are the FSDataset (which stores blocks on dirs) and SimulatedFSDataset (which simulates data).

Nested Class Summary

static class	FSDatasetInterface.BlockWriteStreams This class contains the output streams for the data and checksum of a block
static class	FSDatasetInterface.MetaDataInputStream This class provides the input stream and length of the metadata of a block

Method Summary

void	checkDataDir() Check if all the data directories are healthy
void	finalizeBlock(org.apache.hadoop.dfs.Block b) Finalizes the block previously opened for writing using writeToBlock.
InputStream	getBlockInputStream(org.apache.hadoop.dfs.Block b) Returns an input stream to read the contents of the specified block
InputStream	getBlockInputStream(org.apache.hadoop.dfs.Block b, long seekOffset) Returns an input stream at specified offset of the specified block
org.apache.hadoop.dfs.Block[]	getBlockReport() Returns the block report - the full list of blocks stored
long	getChannelPosition(org.apache.hadoop.dfs.Block b, FSDatasetInterface.BlockWriteStreams stream) Returns the current offset in the data stream.
long	getLength(org.apache.hadoop.dfs.Block b) Returns the specified block's on-disk length (excluding metadata)
FSDatasetInterface.MetaDataInputStream	getMetaDataInputStream(org.apache.hadoop.dfs.Block b) Returns metaData of block b as an input stream (and its length)
long	getMetaDataLength(org.apache.hadoop.dfs.Block b) Returns the length of the metadata file of the specified block
org.apache.hadoop.dfs.Block	getStoredBlock(long blkid)
void	invalidate(org.apache.hadoop.dfs.Block[] invalidBlks) Invalidates the specified blocks
boolean	isValidBlock(org.apache.hadoop.dfs.Block b) Is the block valid?
boolean	metaFileExists(org.apache.hadoop.dfs.Block b) Does the meta file exist for this block?
void	setChannelPosition(org.apache.hadoop.dfs.Block b, FSDatasetInterface.BlockWriteStreams stream, long dataOffset, long ckOffset) Sets the file pointer of the data stream and checksum stream to the specified values.
void	shutdown() Shutdown the FSDataset
String	toString() Stringifies the name of the storage
void	unfinalizeBlock(org.apache.hadoop.dfs.Block b) Unfinalizes the block previously opened for writing using writeToBlock.
void	updateBlock(org.apache.hadoop.dfs.Block oldblock, org.apache.hadoop.dfs.Block newblock) Update the block to the new generation stamp and length.
FSDatasetInterface.BlockWriteStreams	writeToBlock(org.apache.hadoop.dfs.Block b, boolean isRecovery) Creates the block and returns output streams to write data and CRC

Methods inherited from interface org.apache.hadoop.dfs.datanode.metrics.FSDatasetMBean

getCapacity, getDfsUsed, getRemaining, getStorageInfo

Method Detail

getMetaDataLength

long getMetaDataLength(org.apache.hadoop.dfs.Block b)
                       throws IOException

Returns the length of the metadata file of the specified block

Parameters:

b - - the block for which the metadata length is desired

Returns:

the length of the metadata file for the specified block.

Throws:

IOException

getMetaDataInputStream

FSDatasetInterface.MetaDataInputStream getMetaDataInputStream(org.apache.hadoop.dfs.Block b)
                                                              throws IOException

Returns metaData of block b as an input stream (and its length)

Parameters:

b - - the block

Returns:

the metadata input stream;

Throws:

IOException

metaFileExists

boolean metaFileExists(org.apache.hadoop.dfs.Block b)
                       throws IOException

Does the meta file exist for this block?

Parameters:

b - - the block

Returns:

true of the metafile for specified block exits

Throws:

IOException

getLength

long getLength(org.apache.hadoop.dfs.Block b)
               throws IOException

Returns the specified block's on-disk length (excluding metadata)

Parameters:

b -

Returns:

the specified block's on-disk length (excluding metadta)

Throws:

IOException

getStoredBlock

org.apache.hadoop.dfs.Block getStoredBlock(long blkid)
                                           throws IOException

Returns:

the generation stamp stored with the block.

Throws:

IOException

getBlockInputStream

InputStream getBlockInputStream(org.apache.hadoop.dfs.Block b)
                                throws IOException

Returns an input stream to read the contents of the specified block

Parameters:

b -

Returns:

an input stream to read the contents of the specified block

Throws:

IOException

getBlockInputStream

InputStream getBlockInputStream(org.apache.hadoop.dfs.Block b,
                                long seekOffset)
                                throws IOException

Returns an input stream at specified offset of the specified block

Parameters:

b -

seekOffset -

Returns:

an input stream to read the contents of the specified block, starting at the offset

Throws:

IOException

writeToBlock

FSDatasetInterface.BlockWriteStreams writeToBlock(org.apache.hadoop.dfs.Block b,
                                                  boolean isRecovery)
                                                  throws IOException

Creates the block and returns output streams to write data and CRC

Parameters:

b -

isRecovery - True if this is part of erro recovery, otherwise false

Returns:

a BlockWriteStreams object to allow writing the block data and CRC

Throws:

IOException

updateBlock

void updateBlock(org.apache.hadoop.dfs.Block oldblock,
                 org.apache.hadoop.dfs.Block newblock)
                 throws IOException

Update the block to the new generation stamp and length.

Throws:

IOException

finalizeBlock

void finalizeBlock(org.apache.hadoop.dfs.Block b)
                   throws IOException

Finalizes the block previously opened for writing using writeToBlock. The block size is what is in the parameter b and it must match the amount of data written

Parameters:

b -

Throws:

IOException

unfinalizeBlock

void unfinalizeBlock(org.apache.hadoop.dfs.Block b)
                     throws IOException

Unfinalizes the block previously opened for writing using writeToBlock. The temporary file associated with this block is deleted.

Parameters:

b -

Throws:

IOException

getBlockReport

org.apache.hadoop.dfs.Block[] getBlockReport()

Returns the block report - the full list of blocks stored

Returns:

- the block report - the full list of blocks stored

isValidBlock

boolean isValidBlock(org.apache.hadoop.dfs.Block b)

Is the block valid?

Parameters:

b -

Returns:

- true if the specified block is valid

invalidate

void invalidate(org.apache.hadoop.dfs.Block[] invalidBlks)
                throws IOException

Invalidates the specified blocks

Parameters:

invalidBlks - - the blocks to be invalidated

Throws:

IOException

checkDataDir

void checkDataDir()
                  throws DiskChecker.DiskErrorException

Check if all the data directories are healthy

Throws:

DiskChecker.DiskErrorException

toString

String toString()

Stringifies the name of the storage

Overrides:

toString in class Object

shutdown

void shutdown()

Shutdown the FSDataset

getChannelPosition

long getChannelPosition(org.apache.hadoop.dfs.Block b,
                        FSDatasetInterface.BlockWriteStreams stream)
                        throws IOException

Returns the current offset in the data stream.

Parameters:

b -

stream - The stream to the data file and checksum file

Returns:

the position of the file pointer in the data stream

Throws:

IOException

setChannelPosition

void setChannelPosition(org.apache.hadoop.dfs.Block b,
                        FSDatasetInterface.BlockWriteStreams stream,
                        long dataOffset,
                        long ckOffset)
                        throws IOException

Sets the file pointer of the data stream and checksum stream to the specified values.

Parameters:

b -

stream - The stream for the data file and checksum file

dataOffset - The position to which the file pointre for the data stream should be set

ckOffset - The position to which the file pointre for the checksum stream should be set

Throws:

IOException