org.apache.hadoop.net

Class NetworkTopologyWithNodeGroup

java.lang.Object

org.apache.hadoop.net.NetworkTopology

org.apache.hadoop.net.NetworkTopologyWithNodeGroup

@InterfaceAudience.LimitedPrivate(value={"HDFS","MapReduce"})
@InterfaceStability.Unstable
public class NetworkTopologyWithNodeGroup
extends NetworkTopology

The class extends NetworkTopology to represents a cluster of computer with a 4-layers hierarchical network topology. In this network topology, leaves represent data nodes (computers) and inner nodes represent switches/routers that manage traffic in/out of data centers, racks or physical host (with virtual switch).

See Also:

NetworkTopology

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.net.NetworkTopology

NetworkTopology.InvalidTopologyException

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_NODEGROUP

Fields inherited from class org.apache.hadoop.net.NetworkTopology

DEFAULT_HOST_LEVEL, DEFAULT_RACK, LOG, netlock, numOfRacks, r

Constructor Summary

Constructors

Constructor and Description
NetworkTopologyWithNodeGroup()

Method Summary

Methods

Modifier and Type	Method and Description
void	add(Node node) Add a leaf node Update node counter & rack counter if necessary
protected Node	getNodeForNetworkLocation(Node node) Return a reference to the node given its string representation.
String	getNodeGroup(String loc) Given a string representation of a node group for a specific network location
String	getRack(String loc) Given a string representation of a rack for a specific network location To be overridden in subclasses for specific NetworkTopology implementations, as alternative to overriding the full NetworkTopology.getRack(String) method.
boolean	isNodeGroupAware() Check if network topology is aware of NodeGroup
boolean	isOnSameNodeGroup(Node node1, Node node2) Check if two nodes are on the same node group (hypervisor) The assumption here is: each nodes are leaf nodes.
boolean	isOnSameRack(Node node1, Node node2) Check if two nodes are on the same rack
void	pseudoSortByDistance(Node reader, Node[] nodes) Sort nodes array by their distances to reader It linearly scans the array, if a local node is found, swap it with the first element of the array.
void	remove(Node node) Remove a node Update node counter and rack counter if necessary

Methods inherited from class org.apache.hadoop.net.NetworkTopology

chooseRandom, contains, countNumOfAvailableNodes, getDatanodesInRack, getDistance, getFirstHalf, getInstance, getLastHalf, getLeaves, getNode, getNumOfLeaves, getNumOfRacks, isSameParents, swap, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_NODEGROUP

public static final String DEFAULT_NODEGROUP

See Also:

Constant Field Values

Constructor Detail

NetworkTopologyWithNodeGroup

public NetworkTopologyWithNodeGroup()

Method Detail

getNodeForNetworkLocation

protected Node getNodeForNetworkLocation(Node node)

Description copied from class: NetworkTopology

Return a reference to the node given its string representation. Default implementation delegates to NetworkTopology.getNode(String).

To be overridden in subclasses for specific NetworkTopology implementations, as alternative to overriding the full NetworkTopology.add(Node) method.

Overrides:

getNodeForNetworkLocation in class NetworkTopology

Parameters:

node - The string representation of this node's network location is used to retrieve a Node object.

Returns:

a reference to the node; null if the node is not in the tree

See Also:

NetworkTopology.add(Node), NetworkTopology.getNode(String)

getRack

public String getRack(String loc)

Description copied from class: NetworkTopology

Given a string representation of a rack for a specific network location To be overridden in subclasses for specific NetworkTopology implementations, as alternative to overriding the full NetworkTopology.getRack(String) method.

Overrides:

getRack in class NetworkTopology

Parameters:

loc - a path-like string representation of a network location

Returns:

a rack string

getNodeGroup

public String getNodeGroup(String loc)

Given a string representation of a node group for a specific network location

Parameters:

loc - a path-like string representation of a network location

Returns:

a node group string

isOnSameRack

public boolean isOnSameRack(Node node1,
                   Node node2)

Description copied from class: NetworkTopology

Check if two nodes are on the same rack

Overrides:

isOnSameRack in class NetworkTopology

Parameters:

node1 - one node (can be null)

node2 - another node (can be null)

Returns:

true if node1 and node2 are on the same rack; false otherwise

isOnSameNodeGroup

public boolean isOnSameNodeGroup(Node node1,
                        Node node2)

Check if two nodes are on the same node group (hypervisor) The assumption here is: each nodes are leaf nodes.

Overrides:

isOnSameNodeGroup in class NetworkTopology

Parameters:

node1 - one node (can be null)

node2 - another node (can be null)

Returns:

true if node1 and node2 are on the same node group; false otherwise

Throws:

IllegalArgumentException - when either node1 or node2 is null, or node1 or node2 do not belong to the cluster

isNodeGroupAware

public boolean isNodeGroupAware()

Check if network topology is aware of NodeGroup

Overrides:

isNodeGroupAware in class NetworkTopology

add

public void add(Node node)

Add a leaf node Update node counter & rack counter if necessary

Overrides:

add in class NetworkTopology

Parameters:

node - node to be added; can be null

Throws:

IllegalArgumentException - if add a node to a leave or node to be added is not a leaf

remove

public void remove(Node node)

Remove a node Update node counter and rack counter if necessary

Overrides:

remove in class NetworkTopology

Parameters:

node - node to be removed; can be null

pseudoSortByDistance

public void pseudoSortByDistance(Node reader,
                        Node[] nodes)

Sort nodes array by their distances to reader It linearly scans the array, if a local node is found, swap it with the first element of the array. If a local node group node is found, swap it with the first element following the local node. If a local rack node is found, swap it with the first element following the local node group node. If neither local node, node group node or local rack node is found, put a random replica location at position 0. It leaves the rest nodes untouched.

Overrides:

pseudoSortByDistance in class NetworkTopology

Parameters:

reader - the node that wishes to read a block from one of the nodes

nodes - the list of nodes containing data for the reader