org.apache.hadoop.conf
Class Configuration

java.lang.Object
  org.apache.hadoop.conf.Configuration

All Implemented Interfaces:

Iterable<Map.Entry<String,String>>

Direct Known Subclasses:

JobConf

public class Configuration
extends Object
implements Iterable<Map.Entry<String,String>>

Provides access to configuration parameters.

Resources

Configurations are specified by resources. A resource contains a set of name/value pairs as XML data. Each resource is named by either a String or by a Path. If named by a String, then the classpath is examined for a file with that name. If named by a Path, then the local filesystem is examined directly, without referring to the classpath.

Hadoop by default specifies two resources, loaded in-order from the classpath:

1. hadoop-default.xml : Read-only defaults for hadoop.
2. hadoop-site.xml: Site-specific configuration for a given hadoop installation.

Applications may add additional resources, which are loaded subsequent to these resources in the order they are added.

Final Parameters

Configuration parameters may be declared final. Once a resource declares a value final, no subsequently-loaded resource can alter that value. For example, one might define a final parameter with:

  <property>
    <name>dfs.client.buffer.dir</name>
    <value>/tmp/hadoop/dfs/client</value>
    <final>true</final>
  </property>

Administrators typically define parameters as final in hadoop-site.xml for values that user applications may not alter.

Variable Expansion

Value strings are first processed for variable expansion. The available properties are:

1. Other properties defined in this Configuration; and, if a name is undefined here,
2. Properties in System.getProperties().

For example, if a configuration resource contains the following property definitions:

  <property>
    <name>basedir</name>
    <value>/user/${user.name}</value>
  </property>
  
  <property>
    <name>tempdir</name>
    <value>${basedir}/tmp</value>
  </property>

When conf.get("tempdir") is called, then ${basedir} will be resolved to another property in this Configuration, while ${user.name} would then ordinarily be resolved to the value of the System property with that name.

Nested Class Summary

static class

Configuration.IntegerRanges A class that represents a set of positive integer ranges.

Constructor Summary

Configuration()
A new configuration.

Configuration(Configuration other)
A new configuration with the same settings cloned from another.

Method Summary

void	addResource(Path file) Add a configuration resource.
void	addResource(String name) Add a configuration resource.
void	addResource(URL url) Add a configuration resource.
String	get(String name) Get the value of the name property, null if no such property exists.
String	get(String name, String defaultValue) Get the value of the name property.
boolean	getBoolean(String name, boolean defaultValue) Get the value of the name property as a boolean.
Class<?>	getClass(String name, Class<?> defaultValue) Get the value of the name property as a Class.
<U> Class<? extends U>	getClass(String name, Class<? extends U> defaultValue, Class<U> xface) Get the value of the name property as a Class implementing the interface specified by xface.
Class<?>	getClassByName(String name) Load a class by name.
ClassLoader	getClassLoader() Get the ClassLoader for this job.
InputStream	getConfResourceAsInputStream(String name) Get an input stream attached to the configuration resource with the given name.
Reader	getConfResourceAsReader(String name) Get a Reader attached to the configuration resource with the given name.
File	getFile(String dirsProp, String path) Get a local file name under a directory named in dirsProp with the given path.
float	getFloat(String name, float defaultValue) Get the value of the name property as a float.
int	getInt(String name, int defaultValue) Get the value of the name property as an int.
Path	getLocalPath(String dirsProp, String path) Get a local file under a directory named by dirsProp with the given path.
long	getLong(String name, long defaultValue) Get the value of the name property as a long.
Configuration.IntegerRanges	getRange(String name, String defaultValue) Parse the given attribute as a set of integer ranges
String	getRaw(String name) Get the value of the name property, without doing variable expansion.
URL	getResource(String name) Get the URL for the named resource.
Collection<String>	getStringCollection(String name) Get the comma delimited values of the name property as a collection of Strings.
String[]	getStrings(String name) Get the comma delimited values of the name property as an array of Strings.
String[]	getStrings(String name, String... defaultValue) Get the comma delimited values of the name property as an array of Strings.
Iterator<Map.Entry<String,String>>	iterator() Get an Iterator to go through the list of String key-value pairs in the configuration.
static void	main(String[] args) For debugging.
void	set(String name, String value) Set the value of the name property.
void	setBoolean(String name, boolean value) Set the value of the name property to a boolean.
void	setClass(String name, Class<?> theClass, Class<?> xface) Set the value of the name property to the name of a theClass implementing the given interface xface.
void	setClassLoader(ClassLoader classLoader) Set the class loader that will be used to load the various objects.
void	setInt(String name, int value) Set the value of the name property to an int.
void	setLong(String name, long value) Set the value of the name property to a long.
void	setQuietMode(boolean quietmode) Set the quiteness-mode.
void	setStrings(String name, String... values) Set the array of string values for the name property as as comma delimited values.
String	toString()
void	write(OutputStream out) Write out the non-default properties in this configuration to the give OutputStream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Configuration

public Configuration()

A new configuration.

Configuration

public Configuration(Configuration other)

A new configuration with the same settings cloned from another.

Parameters:

other - the configuration from which to clone settings.

Method Detail

addResource

public void addResource(String name)

Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

Parameters:

name - resource to be added, the classpath is examined for a file with that name.

addResource

public void addResource(URL url)

Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

Parameters:

url - url of the resource to be added, the local filesystem is examined directly to find the resource, without referring to the classpath.

addResource

public void addResource(Path file)

Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

Parameters:

file - file-path of resource to be added, the local filesystem is examined directly to find the resource, without referring to the classpath.

get

public String get(String name)

Get the value of the name property, null if no such property exists. Values are processed for variable expansion before being returned.

Parameters:

name - the property name.

Returns:

the value of the name property, or null if no such property exists.

getRaw

public String getRaw(String name)

Get the value of the name property, without doing variable expansion.

Parameters:

name - the property name.

Returns:

the value of the name property, or null if no such property exists.

set

public void set(String name,
                String value)

Set the value of the name property.

Parameters:

name - property name.

value - property value.

get

public String get(String name,
                  String defaultValue)

Get the value of the name property. If no such property exists, then defaultValue is returned.

Parameters:

name - property name.

defaultValue - default value.

Returns:

property value, or defaultValue if the property doesn't exist.

getInt

public int getInt(String name,
                  int defaultValue)

Get the value of the name property as an int. If no such property exists, or if the specified value is not a valid int, then defaultValue is returned.

Parameters:

name - property name.

defaultValue - default value.

Returns:

property value as an int, or defaultValue.

setInt

public void setInt(String name,
                   int value)

Set the value of the name property to an int.

Parameters:

name - property name.

value - int value of the property.

getLong

public long getLong(String name,
                    long defaultValue)

Get the value of the name property as a long. If no such property is specified, or if the specified value is not a valid long, then defaultValue is returned.

Parameters:

name - property name.

defaultValue - default value.

Returns:

property value as a long, or defaultValue.

setLong

public void setLong(String name,
                    long value)

Set the value of the name property to a long.

Parameters:

name - property name.

value - long value of the property.

getFloat

public float getFloat(String name,
                      float defaultValue)

Get the value of the name property as a float. If no such property is specified, or if the specified value is not a valid float, then defaultValue is returned.

Parameters:

name - property name.

defaultValue - default value.

Returns:

property value as a float, or defaultValue.

getBoolean

public boolean getBoolean(String name,
                          boolean defaultValue)

Get the value of the name property as a boolean. If no such property is specified, or if the specified value is not a valid boolean, then defaultValue is returned.

Parameters:

name - property name.

defaultValue - default value.

Returns:

property value as a boolean, or defaultValue.

setBoolean

public void setBoolean(String name,
                       boolean value)

Set the value of the name property to a boolean.

Parameters:

name - property name.

value - boolean value of the property.

getRange

public Configuration.IntegerRanges getRange(String name,
                                            String defaultValue)

Parse the given attribute as a set of integer ranges

Parameters:

name - the attribute name

defaultValue - the default value if it is not set

Returns:

a new set of ranges from the configured value

getStringCollection

public Collection<String> getStringCollection(String name)

Get the comma delimited values of the name property as a collection of Strings. If no such property is specified then empty collection is returned.

This is an optimized version of getStrings(String)

Parameters:

name - property name.

Returns:

property value as a collection of Strings.

getStrings

public String[] getStrings(String name)

Get the comma delimited values of the name property as an array of Strings. If no such property is specified then null is returned.

Parameters:

name - property name.

Returns:

property value as an array of Strings, or null.

getStrings

public String[] getStrings(String name,
                           String... defaultValue)

Get the comma delimited values of the name property as an array of Strings. If no such property is specified then default value is returned.

Parameters:

name - property name.

defaultValue - The default value

Returns:

property value as an array of Strings, or default value.

setStrings

public void setStrings(String name,
                       String... values)

Set the array of string values for the name property as as comma delimited values.

Parameters:

name - property name.

values - The values

getClassByName

public Class<?> getClassByName(String name)
                        throws ClassNotFoundException

Load a class by name.

Parameters:

name - the class name.

Returns:

the class object.

Throws:

ClassNotFoundException - if the class is not found.

getClass

public Class<?> getClass(String name,
                         Class<?> defaultValue)

Get the value of the name property as a Class. If no such property is specified, then defaultValue is returned.

Parameters:

name - the class name.

defaultValue - default value.

Returns:

property value as a Class, or defaultValue.

getClass

public <U> Class<? extends U> getClass(String name,
                                       Class<? extends U> defaultValue,
                                       Class<U> xface)

Get the value of the name property as a Class implementing the interface specified by xface. If no such property is specified, then defaultValue is returned. An exception is thrown if the returned class does not implement the named interface.

Parameters:

name - the class name.

defaultValue - default value.

xface - the interface implemented by the named class.

Returns:

property value as a Class, or defaultValue.

setClass

public void setClass(String name,
                     Class<?> theClass,
                     Class<?> xface)

Set the value of the name property to the name of a theClass implementing the given interface xface. An exception is thrown if theClass does not implement the interface xface.

Parameters:

name - property name.

theClass - property value.

xface - the interface implemented by the named class.

getLocalPath

public Path getLocalPath(String dirsProp,
                         String path)
                  throws IOException

Get a local file under a directory named by dirsProp with the given path. If dirsProp contains multiple directories, then one is chosen based on path's hash code. If the selected directory does not exist, an attempt is made to create it.

Parameters:

dirsProp - directory in which to locate the file.

path - file-path.

Returns:

local file under the directory with the given path.

Throws:

IOException

getFile

public File getFile(String dirsProp,
                    String path)
             throws IOException

Get a local file name under a directory named in dirsProp with the given path. If dirsProp contains multiple directories, then one is chosen based on path's hash code. If the selected directory does not exist, an attempt is made to create it.

Parameters:

dirsProp - directory in which to locate the file.

path - file-path.

Returns:

local file under the directory with the given path.

Throws:

IOException

getResource

public URL getResource(String name)

Get the URL for the named resource.

Parameters:

name - resource name.

Returns:

the url for the named resource.

getConfResourceAsInputStream

public InputStream getConfResourceAsInputStream(String name)

Get an input stream attached to the configuration resource with the given name.

Parameters:

name - configuration resource name.

Returns:

an input stream attached to the resource.

getConfResourceAsReader

public Reader getConfResourceAsReader(String name)

Get a Reader attached to the configuration resource with the given name.

Parameters:

name - configuration resource name.

Returns:

a reader attached to the resource.

iterator

public Iterator<Map.Entry<String,String>> iterator()

Get an Iterator to go through the list of String key-value pairs in the configuration.

Specified by:

iterator in interface Iterable<Map.Entry<String,String>>

Returns:

an iterator over the entries.

write

public void write(OutputStream out)
           throws IOException

Write out the non-default properties in this configuration to the give OutputStream.

Parameters:

out - the output stream to write to.

Throws:

IOException

getClassLoader

public ClassLoader getClassLoader()

Get the ClassLoader for this job.

Returns:

the correct class loader.

setClassLoader

public void setClassLoader(ClassLoader classLoader)

Set the class loader that will be used to load the various objects.

Parameters:

classLoader - the new class loader.

toString

public String toString()

Overrides:

toString in class Object

setQuietMode

public void setQuietMode(boolean quietmode)

Set the quiteness-mode. In the the quite-mode error and informational messages might not be logged.

Parameters:

quietmode - true to set quiet-mode on, false to turn it off.

main

public static void main(String[] args)
                 throws Exception

For debugging. List non-default properties to the terminal and exit.

Throws:

Exception