DistributedFileSystem (Apache Hadoop Main 2.2.0 API)

java.lang.Object
- org.apache.hadoop.conf.Configured
- - org.apache.hadoop.fs.FileSystem
  - - org.apache.hadoop.hdfs.DistributedFileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, Configurable
```
@InterfaceAudience.LimitedPrivate(value={"MapReduce","HBase"})
@InterfaceStability.Unstable
public class DistributedFileSystem
extends FileSystem
```
Implementation of the abstract FileSystem for the DFS system. This object is the way end-user code interacts with a Hadoop DistributedFileSystem.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class DistributedFileSystem.DiskStatus
Deprecated.
Use FsStatus instead
- Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
  FileSystem.Statistics

Nested Classes
Modifier and Type	Class and Description
`static class`	`DistributedFileSystem.DiskStatus` Deprecated. Use `FsStatus` instead

Field Summary
- Fields inherited from class org.apache.hadoop.fs.FileSystem
  DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics

Constructor Summary

Constructors
Constructor and Description

DistributedFileSystem()

DistributedFileSystem(InetSocketAddress namenode, Configuration conf)
Deprecated.

Constructors
Constructor and Description
`DistributedFileSystem()`
`DistributedFileSystem(InetSocketAddress namenode, Configuration conf)` Deprecated.

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`allowSnapshot(Path path)`
`FSDataOutputStream`	`append(Path f, int bufferSize, Progressable progress)` Append to an existing file (optional operation).
`void`	`cancelDelegationToken(Token<DelegationTokenIdentifier> token)` Deprecated. Use Token.cancel instead.
`protected URI`	`canonicalizeUri(URI uri)` Canonicalize the given URI.
`void`	`close()` No more filesystem operations are needed.
`void`	`concat(Path trg, Path[] psrcs)` Move blocks from srcs to trg and delete srcs afterwards.
`FSDataOutputStream`	`create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress)` Create an FSDataOutputStream at the indicated Path with write-progress reporting.
`HdfsDataOutputStream`	`create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress, InetSocketAddress[] favoredNodes)` Same as `create(Path, FsPermission, boolean, int, short, long, Progressable)` with the addition of favoredNodes that is a hint to where the namenode should place the file blocks.
`FSDataOutputStream`	`create(Path f, FsPermission permission, EnumSet<CreateFlag> cflags, int bufferSize, short replication, long blockSize, Progressable progress, Options.ChecksumOpt checksumOpt)` Create an FSDataOutputStream at the indicated Path with a custom checksum option
`FSDataOutputStream`	`createNonRecursive(Path f, FsPermission permission, EnumSet<CreateFlag> flag, int bufferSize, short replication, long blockSize, Progressable progress)` Same as create(), except fails if parent directory doesn't already exist.
`Path`	`createSnapshot(Path path, String snapshotName)` Create a snapshot
`void`	`createSymlink(Path target, Path link, boolean createParent)` See `FileContext.createSymlink(Path, Path, boolean)`
`boolean`	`delete(Path f, boolean recursive)` Delete a file.
`void`	`deleteSnapshot(Path snapshotDir, String snapshotName)` Delete a snapshot of a directory
`void`	`disallowSnapshot(Path path)`
`void`	`finalizeUpgrade()` Finalize previously upgraded files system state.
`String`	`getCanonicalServiceName()` Get a canonical service name for this file system.
`DFSClient`	`getClient()`
`ContentSummary`	`getContentSummary(Path f)` Return the `ContentSummary` of a given `Path`.
`long`	`getCorruptBlocksCount()` Returns count of blocks with at least one replica marked corrupt.
`DatanodeInfo[]`	`getDataNodeStats()`
`DatanodeInfo[]`	`getDataNodeStats(HdfsConstants.DatanodeReportType type)`
`long`	`getDefaultBlockSize()` Return the number of bytes that large input files should be optimally be split into to minimize i/o time.
`protected int`	`getDefaultPort()` Get the default port for this file system.
`short`	`getDefaultReplication()` Get the default replication.
`Token<DelegationTokenIdentifier>`	`getDelegationToken(String renewer)` Get a new delegation token for this file system.
`Token<DelegationTokenIdentifier>`	`getDelegationToken(Text renewer)` Deprecated. use `getDelegationToken(String)`
`DistributedFileSystem.DiskStatus`	`getDiskStatus()` Deprecated. Use `FileSystem.getStatus()` instead
`BlockLocation[]`	`getFileBlockLocations(FileStatus file, long start, long len)` Return an array containing hostnames, offset and size of portions of the given file.
`BlockLocation[]`	`getFileBlockLocations(Path p, long start, long len)` Return an array containing hostnames, offset and size of portions of the given file.
`BlockStorageLocation[]`	`getFileBlockStorageLocations(List<BlockLocation> blocks)` Used to query storage location information for a list of blocks.
`FileChecksum`	`getFileChecksum(Path f)` Get the checksum of a file.
`FileStatus`	`getFileLinkStatus(Path f)` See `FileContext.getFileLinkStatus(Path)`
`FileStatus`	`getFileStatus(Path f)` Returns the stat information about the file.
`Path`	`getHomeDirectory()` Return the current user's home directory in this filesystem.
`Path`	`getLinkTarget(Path f)` See `FileContext.getLinkTarget(Path)`
`long`	`getMissingBlocksCount()` Returns count of blocks with no good replicas left.
`long`	`getRawCapacity()` Deprecated. Use `FileSystem.getStatus()` instead
`long`	`getRawUsed()` Deprecated. Use `FileSystem.getStatus()` instead
`String`	`getScheme()` Return the protocol scheme for the FileSystem.
`FsServerDefaults`	`getServerDefaults()` Return a set of server default configuration values
`SnapshotDiffReport`	`getSnapshotDiffReport(Path snapshotDir, String fromSnapshot, String toSnapshot)` Get the difference between two snapshots, or between a snapshot and the current tree of a directory.
`SnapshottableDirectoryStatus[]`	`getSnapshottableDirListing()`
`FsStatus`	`getStatus(Path p)` Returns a status object describing the use and capacity of the file system.
`long`	`getUnderReplicatedBlocksCount()` Returns count of blocks with one of more replica missing.
`URI`	`getUri()` Returns a URI whose scheme and authority identify this FileSystem.
`Path`	`getWorkingDirectory()` Get the current working directory for the given file system
`void`	`initialize(URI uri, Configuration conf)` Called after a new FileSystem instance is constructed.
`boolean`	`isFileClosed(Path src)` Get the close status of a file
`boolean`	`isInSafeMode()` Utility function that returns if the NameNode is in safemode or not.
`RemoteIterator<Path>`	`listCorruptFileBlocks(Path path)`
`protected RemoteIterator<LocatedFileStatus>`	`listLocatedStatus(Path p, PathFilter filter)` Listing a directory The returned results include its block location if it is a file The results are filtered by the given path filter
`FileStatus[]`	`listStatus(Path p)` List all the entries of a directory Note that this operation is not atomic for a large directory.
`void`	`metaSave(String pathname)`
`boolean`	`mkdir(Path f, FsPermission permission)` Create a directory, only when the parent directories exist.
`boolean`	`mkdirs(Path f, FsPermission permission)` Create a directory and its parent directories.
`FSDataInputStream`	`open(Path f, int bufferSize)` Opens an FSDataInputStream at the indicated Path.
`protected HdfsDataOutputStream`	`primitiveCreate(Path f, FsPermission absolutePermission, EnumSet<CreateFlag> flag, int bufferSize, short replication, long blockSize, Progressable progress, Options.ChecksumOpt checksumOpt)`
`protected boolean`	`primitiveMkdir(Path f, FsPermission absolutePermission)` This version of the mkdirs method assumes that the permission is absolute.
`boolean`	`recoverLease(Path f)` Start the lease recovery of a file
`void`	`refreshNodes()` Refreshes the list of hosts and excluded hosts from the configured files.
`boolean`	`rename(Path src, Path dst)` Renames Path src to Path dst.
`void`	`rename(Path src, Path dst, Options.Rename... options)` This rename operation is guaranteed to be atomic.
`void`	`renameSnapshot(Path path, String snapshotOldName, String snapshotNewName)` Rename a snapshot
`long`	`renewDelegationToken(Token<DelegationTokenIdentifier> token)` Deprecated. Use Token.renew instead.
`boolean`	`reportChecksumFailure(Path f, FSDataInputStream in, long inPos, FSDataInputStream sums, long sumsPos)` Deprecated.
`protected Path`	`resolveLink(Path f)` See `AbstractFileSystem.getLinkTarget(Path)`
`boolean`	`restoreFailedStorage(String arg)` enable/disable/check restoreFaileStorage
`long`	`rollEdits()` Rolls the edit log on the active NameNode.
`void`	`saveNamespace()` Save namespace image.
`void`	`setBalancerBandwidth(long bandwidth)` Requests the namenode to tell all datanodes to use a new, non-persistent bandwidth value for dfs.balance.bandwidthPerSec.
`void`	`setOwner(Path p, String username, String groupname)` Set owner of a path (i.e.
`void`	`setPermission(Path p, FsPermission permission)` Set permission of a path.
`void`	`setQuota(Path src, long namespaceQuota, long diskspaceQuota)` Set a directory's quotas
`boolean`	`setReplication(Path src, short replication)` Set replication for an existing file.
`boolean`	`setSafeMode(HdfsConstants.SafeModeAction action)` Enter, leave or get safe mode.
`boolean`	`setSafeMode(HdfsConstants.SafeModeAction action, boolean isChecked)` Enter, leave or get safe mode.
`void`	`setTimes(Path p, long mtime, long atime)` Set access time of a file
`void`	`setVerifyChecksum(boolean verifyChecksum)` Set the verify checksum flag.
`void`	`setWorkingDirectory(Path dir)` Set the current working directory for the given file system.
`boolean`	`supportsSymlinks()` See `AbstractFileSystem.supportsSymlinks()`
`String`	`toString()`

Methods inherited from class org.apache.hadoop.fs.FileSystem
addDelegationTokens, append, append, cancelDeleteOnExit, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, createSnapshot, delete, deleteOnExit, enableSymlinks, exists, fixRelativePart, get, get, get, getAllStatistics, getBlockSize, getCanonicalUri, getChildFileSystems, getDefaultBlockSize, getDefaultReplication, getDefaultUri, getFileSystemClass, getFSofPath, getInitialWorkingDirectory, getLength, getLocal, getName, getNamed, getReplication, getServerDefaults, getStatistics, getStatistics, getStatus, getUsed, globStatus, globStatus, isDirectory, isFile, isSymlinksEnabled, listFiles, listLocatedStatus, listStatus, listStatus, listStatus, makeQualified, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveMkdir, printStatistics, processDeleteOnExit, resolvePath, setDefaultUri, setDefaultUri, setWriteChecksum, startLocalOutput

Methods inherited from class org.apache.hadoop.conf.Configured
getConf, setConf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - DistributedFileSystem
```
public DistributedFileSystem()
```
  - DistributedFileSystem
```
@Deprecated
public DistributedFileSystem(InetSocketAddress namenode,
                                Configuration conf)
                      throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
- Method Detail
  - getScheme
```
public String getScheme()
```
    Return the protocol scheme for the FileSystem.
    
    Overrides:
    
    getScheme in class FileSystem
    
    Returns:
    hdfs
  - getUri
```
public URI getUri()
```
    Description copied from class: FileSystem
    
    Returns a URI whose scheme and authority identify this FileSystem.
    
    Specified by:
    
    getUri in class FileSystem
  - initialize
```
public void initialize(URI uri,
              Configuration conf)
                throws IOException
```
    Description copied from class: FileSystem
    
    Called after a new FileSystem instance is constructed.
    
    Overrides:
    
    initialize in class FileSystem
    
    Parameters:
    uri - a uri whose authority section names the host, port, etc. for this FileSystem
    conf - the configuration
    
    Throws:
    
    IOException
  - getWorkingDirectory
```
public Path getWorkingDirectory()
```
    Description copied from class: FileSystem
    
    Get the current working directory for the given file system
    
    Specified by:
    
    getWorkingDirectory in class FileSystem
    
    Returns:
    the directory pathname
  - getDefaultBlockSize
```
public long getDefaultBlockSize()
```
    Description copied from class: FileSystem
    
    Return the number of bytes that large input files should be optimally be split into to minimize i/o time.
    
    Overrides:
    
    getDefaultBlockSize in class FileSystem
  - getDefaultReplication
```
public short getDefaultReplication()
```
    Description copied from class: FileSystem
    
    Get the default replication.
    
    Overrides:
    
    getDefaultReplication in class FileSystem
  - setWorkingDirectory
```
public void setWorkingDirectory(Path dir)
```
    Description copied from class: FileSystem
    
    Set the current working directory for the given file system. All relative paths will be resolved relative to it.
    
    Specified by:
    
    setWorkingDirectory in class FileSystem
  - getHomeDirectory
```
public Path getHomeDirectory()
```
    Description copied from class: FileSystem
    
    Return the current user's home directory in this filesystem. The default implementation returns "/user/$USER/".
    
    Overrides:
    
    getHomeDirectory in class FileSystem
  - getFileBlockLocations
```
public BlockLocation[] getFileBlockLocations(FileStatus file,
                                    long start,
                                    long len)
                                      throws IOException
```
    Description copied from class: FileSystem
    
    Return an array containing hostnames, offset and size of portions of the given file. For a nonexistent file or regions, null will be returned. This call is most helpful with DFS, where it returns hostnames of machines that contain the given file. The FileSystem will simply return an elt containing 'localhost'.
    
    Overrides:
    
    getFileBlockLocations in class FileSystem
    
    Parameters:
    file - FilesStatus to get data from
    start - offset into the given file
    len - length for which to get locations for
    
    Throws:
    
    IOException
  - getFileBlockLocations
```
public BlockLocation[] getFileBlockLocations(Path p,
                                    long start,
                                    long len)
                                      throws IOException
```
    Description copied from class: FileSystem
    
    Return an array containing hostnames, offset and size of portions of the given file. For a nonexistent file or regions, null will be returned. This call is most helpful with DFS, where it returns hostnames of machines that contain the given file. The FileSystem will simply return an elt containing 'localhost'.
    
    Overrides:
    
    getFileBlockLocations in class FileSystem
    
    Parameters:
    p - path is used to identify an FS since an FS could have another FS that it could be delegating the call to
    start - offset into the given file
    len - length for which to get locations for
    
    Throws:
    
    IOException
  - getFileBlockStorageLocations
```
@InterfaceStability.Unstable
public BlockStorageLocation[] getFileBlockStorageLocations(List<BlockLocation> blocks)
                                                    throws IOException,
                                                           UnsupportedOperationException,
                                                           InvalidBlockTokenException
```
    Used to query storage location information for a list of blocks. This list of blocks is normally constructed via a series of calls to getFileBlockLocations(Path, long, long) to get the blocks for ranges of a file. The returned array of BlockStorageLocation augments BlockLocation with a VolumeId per block replica. The VolumeId specifies the volume on the datanode on which the replica resides. The VolumeId has to be checked via VolumeId.isValid() before being used because volume information can be unavailable if the corresponding datanode is down or if the requested block is not found. This API is unstable, and datanode-side support is disabled by default. It can be enabled by setting "dfs.datanode.hdfs-blocks-metadata.enabled" to true.
    
    Parameters:
    blocks - List of target BlockLocations to query volume location information
    
    Returns:
    volumeBlockLocations Augmented array of BlockStorageLocations containing additional volume location information for each replica of each block.
    
    Throws:
    
    IOException
    
    UnsupportedOperationException
    
    InvalidBlockTokenException
  - setVerifyChecksum
```
public void setVerifyChecksum(boolean verifyChecksum)
```
    Description copied from class: FileSystem
    
    Set the verify checksum flag. This is only applicable if the corresponding FileSystem supports checksum. By default doesn't do anything.
    
    Overrides:
    
    setVerifyChecksum in class FileSystem
  - recoverLease
```
public boolean recoverLease(Path f)
                     throws IOException
```
    Start the lease recovery of a file
    
    Parameters:
    f - a file
    
    Returns:
    true if the file is already closed
    
    Throws:
    
    IOException - if an error occurs
  - open
```
public FSDataInputStream open(Path f,
                     int bufferSize)
                       throws IOException
```
    Description copied from class: FileSystem
    
    Opens an FSDataInputStream at the indicated Path.
    
    Specified by:
    
    open in class FileSystem
    
    Parameters:
    f - the file name to open
    bufferSize - the size of the buffer to be used.
    
    Throws:
    
    IOException
  - append
```
public FSDataOutputStream append(Path f,
                        int bufferSize,
                        Progressable progress)
                          throws IOException
```
    Description copied from class: FileSystem
    
    Append to an existing file (optional operation).
    
    Specified by:
    
    append in class FileSystem
    
    Parameters:
    f - the existing file to be appended.
    bufferSize - the size of the buffer to be used.
    progress - for reporting progress if it is not null.
    
    Throws:
    
    IOException
  - create
```
public FSDataOutputStream create(Path f,
                        FsPermission permission,
                        boolean overwrite,
                        int bufferSize,
                        short replication,
                        long blockSize,
                        Progressable progress)
                          throws IOException
```
    Description copied from class: FileSystem
    
    Create an FSDataOutputStream at the indicated Path with write-progress reporting.
    
    Specified by:
    
    create in class FileSystem
    
    Parameters:
    f - the file name to open
    overwrite - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.
    bufferSize - the size of the buffer to be used.
    replication - required block replication for the file.
    
    Throws:
    
    IOException
    See Also:
    FileSystem.setPermission(Path, FsPermission)
  - create
```
public HdfsDataOutputStream create(Path f,
                          FsPermission permission,
                          boolean overwrite,
                          int bufferSize,
                          short replication,
                          long blockSize,
                          Progressable progress,
                          InetSocketAddress[] favoredNodes)
                            throws IOException
```
    Same as create(Path, FsPermission, boolean, int, short, long, Progressable) with the addition of favoredNodes that is a hint to where the namenode should place the file blocks. The favored nodes hint is not persisted in HDFS. Hence it may be honored at the creation time only. HDFS could move the blocks during balancing or replication, to move the blocks from favored nodes. A value of null means no favored nodes for this create
    
    Throws:
    
    IOException
  - create
```
public FSDataOutputStream create(Path f,
                        FsPermission permission,
                        EnumSet<CreateFlag> cflags,
                        int bufferSize,
                        short replication,
                        long blockSize,
                        Progressable progress,
                        Options.ChecksumOpt checksumOpt)
                          throws IOException
```
    Description copied from class: FileSystem
    
    Create an FSDataOutputStream at the indicated Path with a custom checksum option
    
    Overrides:
    
    create in class FileSystem
    
    Parameters:
    f - the file name to open
    cflags - CreateFlags to use for this stream.
    bufferSize - the size of the buffer to be used.
    replication - required block replication for the file.
    checksumOpt - checksum parameter. If null, the values found in conf will be used.
    
    Throws:
    
    IOException
    See Also:
    FileSystem.setPermission(Path, FsPermission)
  - primitiveCreate
```
protected HdfsDataOutputStream primitiveCreate(Path f,
                                   FsPermission absolutePermission,
                                   EnumSet<CreateFlag> flag,
                                   int bufferSize,
                                   short replication,
                                   long blockSize,
                                   Progressable progress,
                                   Options.ChecksumOpt checksumOpt)
                                        throws IOException
```
    Overrides:
    
    primitiveCreate in class FileSystem
    
    Throws:
    
    IOException
  - createNonRecursive
```
public FSDataOutputStream createNonRecursive(Path f,
                                    FsPermission permission,
                                    EnumSet<CreateFlag> flag,
                                    int bufferSize,
                                    short replication,
                                    long blockSize,
                                    Progressable progress)
                                      throws IOException
```
    Same as create(), except fails if parent directory doesn't already exist.
    
    Overrides:
    
    createNonRecursive in class FileSystem
    
    Parameters:
    f - the file name to open
    flag - CreateFlags to use for this stream.
    bufferSize - the size of the buffer to be used.
    replication - required block replication for the file.
    
    Throws:
    
    IOException
    See Also:
    FileSystem.setPermission(Path, FsPermission)
  - setReplication
```
public boolean setReplication(Path src,
                     short replication)
                       throws IOException
```
    Description copied from class: FileSystem
    
    Set replication for an existing file.
    
    Overrides:
    
    setReplication in class FileSystem
    
    Parameters:
    src - file name
    replication - new replication
    
    Returns:
    true if successful; false if file does not exist or is a directory
    
    Throws:
    
    IOException
  - concat
```
public void concat(Path trg,
          Path[] psrcs)
            throws IOException
```
    Move blocks from srcs to trg and delete srcs afterwards. The file block sizes must be the same.
    
    Overrides:
    
    concat in class FileSystem
    
    Parameters:
    trg - existing file to append to
    psrcs - list of files (same block size, same replication)
    
    Throws:
    
    IOException
  - rename
```
public boolean rename(Path src,
             Path dst)
               throws IOException
```
    Description copied from class: FileSystem
    
    Renames Path src to Path dst. Can take place on local fs or remote DFS.
    
    Specified by:
    
    rename in class FileSystem
    
    Parameters:
    src - path to be renamed
    dst - new path after rename
    
    Returns:
    true if rename is successful
    
    Throws:
    
    IOException - on failure
  - rename
```
public void rename(Path src,
          Path dst,
          Options.Rename... options)
            throws IOException
```
    This rename operation is guaranteed to be atomic.
    
    Overrides:
    
    rename in class FileSystem
    
    Parameters:
    src - path to be renamed
    dst - new path after rename
    
    Throws:
    
    IOException - on failure
  - delete
```
public boolean delete(Path f,
             boolean recursive)
               throws IOException
```
    Description copied from class: FileSystem
    
    Delete a file.
    
    Specified by:
    
    delete in class FileSystem
    
    Parameters:
    f - the path to delete.
    recursive - if path is a directory and set to true, the directory is deleted else throws an exception. In case of a file the recursive can be set to either true or false.
    
    Returns:
    true if delete is successful else false.
    
    Throws:
    
    IOException
  - getContentSummary
```
public ContentSummary getContentSummary(Path f)
                                 throws IOException
```
    Description copied from class: FileSystem
    
    Return the ContentSummary of a given Path.
    
    Overrides:
    
    getContentSummary in class FileSystem
    
    Parameters:
    f - path to use
    
    Throws:
    
    IOException
  - setQuota
```
public void setQuota(Path src,
            long namespaceQuota,
            long diskspaceQuota)
              throws IOException
```
    Set a directory's quotas
    
    Throws:
    
    IOException
    See Also:
    ClientProtocol.setQuota(String, long, long)
  - listStatus
```
public FileStatus[] listStatus(Path p)
                        throws IOException
```
    List all the entries of a directory Note that this operation is not atomic for a large directory. The entries of a directory may be fetched from NameNode multiple times. It only guarantees that each name occurs once if a directory undergoes changes between the calls.
    
    Specified by:
    
    listStatus in class FileSystem
    
    Parameters:
    p - given path
    
    Returns:
    the statuses of the files/directories in the given patch
    
    Throws:
    
    FileNotFoundException - when the path does not exist; IOException see specific implementation
    
    IOException
  - listLocatedStatus
```
protected RemoteIterator<LocatedFileStatus> listLocatedStatus(Path p,
                                                  PathFilter filter)
                                                       throws IOException
```
    Description copied from class: FileSystem
    
    Listing a directory The returned results include its block location if it is a file The results are filtered by the given path filter
    
    Overrides:
    
    listLocatedStatus in class FileSystem
    
    Parameters:
    p - a path
    filter - a path filter
    
    Returns:
    an iterator that traverses statuses of the files/directories in the given path
    
    Throws:
    
    FileNotFoundException - if f does not exist
    
    IOException - if any I/O error occurred
  - mkdir
```
public boolean mkdir(Path f,
            FsPermission permission)
              throws IOException
```
    Create a directory, only when the parent directories exist. See FsPermission.applyUMask(FsPermission) for details of how the permission is applied.
    
    Parameters:
    f - The path to create
    permission - The permission. See FsPermission#applyUMask for details about how this is used to calculate the effective permission.
    
    Throws:
    
    IOException
  - mkdirs
```
public boolean mkdirs(Path f,
             FsPermission permission)
               throws IOException
```
    Create a directory and its parent directories. See FsPermission.applyUMask(FsPermission) for details of how the permission is applied.
    
    Specified by:
    
    mkdirs in class FileSystem
    
    Parameters:
    f - The path to create
    permission - The permission. See FsPermission#applyUMask for details about how this is used to calculate the effective permission.
    
    Throws:
    
    IOException
  - primitiveMkdir
```
protected boolean primitiveMkdir(Path f,
                     FsPermission absolutePermission)
                          throws IOException
```
    Description copied from class: FileSystem
    
    This version of the mkdirs method assumes that the permission is absolute. It has been added to support the FileContext that processes the permission with umask before calling this method. This a temporary method added to support the transition from FileSystem to FileContext for user applications.
    
    Overrides:
    
    primitiveMkdir in class FileSystem
    
    Throws:
    
    IOException
  - close
```
public void close()
           throws IOException
```
    Description copied from class: FileSystem
    
    No more filesystem operations are needed. Will release any held locks.
    
    Specified by:
    
    close in interface Closeable
    
    Specified by:
    
    close in interface AutoCloseable
    
    Overrides:
    
    close in class FileSystem
    
    Throws:
    
    IOException
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - getClient
```
@InterfaceAudience.Private
public DFSClient getClient()
```
  - getStatus
```
public FsStatus getStatus(Path p)
                   throws IOException
```
    Description copied from class: FileSystem
    
    Returns a status object describing the use and capacity of the file system. If the file system has multiple partitions, the use and capacity of the partition pointed to by the specified path is reflected.
    
    Overrides:
    
    getStatus in class FileSystem
    
    Parameters:
    p - Path for which status should be obtained. null means the default partition.
    
    Returns:
    a FsStatus object
    
    Throws:
    
    IOException - see specific implementation
  - getDiskStatus
```
@Deprecated
public DistributedFileSystem.DiskStatus getDiskStatus()
                                               throws IOException
```
    Deprecated. Use FileSystem.getStatus() instead
    
    Return the disk usage of the filesystem, including total capacity, used space, and remaining space
    
    Throws:
    
    IOException
  - getRawCapacity
```
@Deprecated
public long getRawCapacity()
                    throws IOException
```
    Deprecated. Use FileSystem.getStatus() instead
    
    Return the total raw capacity of the filesystem, disregarding replication.
    
    Throws:
    
    IOException
  - getRawUsed
```
@Deprecated
public long getRawUsed()
                throws IOException
```
    Deprecated. Use FileSystem.getStatus() instead
    
    Return the total raw used space in the filesystem, disregarding replication.
    
    Throws:
    
    IOException
  - getMissingBlocksCount
```
public long getMissingBlocksCount()
                           throws IOException
```
    Returns count of blocks with no good replicas left. Normally should be zero.
    
    Throws:
    
    IOException
  - getUnderReplicatedBlocksCount
```
public long getUnderReplicatedBlocksCount()
                                   throws IOException
```
    Returns count of blocks with one of more replica missing.
    
    Throws:
    
    IOException
  - getCorruptBlocksCount
```
public long getCorruptBlocksCount()
                           throws IOException
```
    Returns count of blocks with at least one replica marked corrupt.
    
    Throws:
    
    IOException
  - listCorruptFileBlocks
```
public RemoteIterator<Path> listCorruptFileBlocks(Path path)
                                           throws IOException
```
    Overrides:
    
    listCorruptFileBlocks in class FileSystem
    
    Returns:
    an iterator over the corrupt files under the given path (may contain duplicates if a file has more than one corrupt block)
    
    Throws:
    
    IOException
  - getDataNodeStats
```
public DatanodeInfo[] getDataNodeStats()
                                throws IOException
```
    Returns:
    datanode statistics.
    
    Throws:
    
    IOException
  - getDataNodeStats
```
public DatanodeInfo[] getDataNodeStats(HdfsConstants.DatanodeReportType type)
                                throws IOException
```
    Returns:
    datanode statistics for the given type.
    
    Throws:
    
    IOException
  - setSafeMode
```
public boolean setSafeMode(HdfsConstants.SafeModeAction action)
                    throws IOException
```
    Enter, leave or get safe mode.
    
    Throws:
    
    IOException
    See Also:
    ClientProtocol.setSafeMode( HdfsConstants.SafeModeAction,boolean)
  - setSafeMode
```
public boolean setSafeMode(HdfsConstants.SafeModeAction action,
                  boolean isChecked)
                    throws IOException
```
    Enter, leave or get safe mode.
    
    Parameters:
    action - One of SafeModeAction.ENTER, SafeModeAction.LEAVE and SafeModeAction.GET
    isChecked - If true check only for Active NNs status, else check first NN's status
    
    Throws:
    
    IOException
    See Also:
    org.apache.hadoop.hdfs.protocol.ClientProtocol#setSafeMode(SafeModeAction, boolean)
  - saveNamespace
```
public void saveNamespace()
                   throws AccessControlException,
                          IOException
```
    Save namespace image.
    
    Throws:
    
    AccessControlException
    
    IOException
    See Also:
    ClientProtocol.saveNamespace()
  - rollEdits
```
public long rollEdits()
               throws AccessControlException,
                      IOException
```
    Rolls the edit log on the active NameNode. Requires super-user privileges.
    
    Returns:
    the transaction ID of the newly created segment
    
    Throws:
    
    AccessControlException
    
    IOException
    See Also:
    ClientProtocol.rollEdits()
  - restoreFailedStorage
```
public boolean restoreFailedStorage(String arg)
                             throws AccessControlException,
                                    IOException
```
    enable/disable/check restoreFaileStorage
    
    Throws:
    
    AccessControlException
    
    IOException
    See Also:
    ClientProtocol.restoreFailedStorage(String arg)
  - refreshNodes
```
public void refreshNodes()
                  throws IOException
```
    Refreshes the list of hosts and excluded hosts from the configured files.
    
    Throws:
    
    IOException
  - finalizeUpgrade
```
public void finalizeUpgrade()
                     throws IOException
```
    Finalize previously upgraded files system state.
    
    Throws:
    
    IOException
  - metaSave
```
public void metaSave(String pathname)
              throws IOException
```
    Throws:
    
    IOException
  - getServerDefaults
```
public FsServerDefaults getServerDefaults()
                                   throws IOException
```
    Description copied from class: FileSystem
    
    Return a set of server default configuration values
    
    Overrides:
    
    getServerDefaults in class FileSystem
    
    Returns:
    server default configuration values
    
    Throws:
    
    IOException
  - reportChecksumFailure
```
@Deprecated
public boolean reportChecksumFailure(Path f,
                                       FSDataInputStream in,
                                       long inPos,
                                       FSDataInputStream sums,
                                       long sumsPos)
```
    Deprecated.
    
    We need to find the blocks that didn't match. Likely only one is corrupt but we will report both to the namenode. In the future, we can consider figuring out exactly which block is corrupt.
  - getFileStatus
```
public FileStatus getFileStatus(Path f)
                         throws IOException
```
    Returns the stat information about the file.
    
    Specified by:
    
    getFileStatus in class FileSystem
    
    Parameters:
    f - The path we want information from
    
    Returns:
    a FileStatus object
    
    Throws:
    
    FileNotFoundException - if the file does not exist.
    
    IOException
  - createSymlink
```
public void createSymlink(Path target,
                 Path link,
                 boolean createParent)
                   throws AccessControlException,
                          FileAlreadyExistsException,
                          FileNotFoundException,
                          ParentNotDirectoryException,
                          UnsupportedFileSystemException,
                          IOException
```
    Description copied from class: FileSystem
    
    See FileContext.createSymlink(Path, Path, boolean)
    
    Overrides:
    
    createSymlink in class FileSystem
    
    Throws:
    
    AccessControlException
    
    FileAlreadyExistsException
    
    FileNotFoundException
    
    ParentNotDirectoryException
    
    UnsupportedFileSystemException
    
    IOException
  - supportsSymlinks
```
public boolean supportsSymlinks()
```
    Description copied from class: FileSystem
    
    See AbstractFileSystem.supportsSymlinks()
    
    Overrides:
    
    supportsSymlinks in class FileSystem
  - getFileLinkStatus
```
public FileStatus getFileLinkStatus(Path f)
                             throws AccessControlException,
                                    FileNotFoundException,
                                    UnsupportedFileSystemException,
                                    IOException
```
    Description copied from class: FileSystem
    
    See FileContext.getFileLinkStatus(Path)
    
    Overrides:
    
    getFileLinkStatus in class FileSystem
    
    Throws:
    
    AccessControlException
    
    FileNotFoundException
    
    UnsupportedFileSystemException
    
    IOException
  - getLinkTarget
```
public Path getLinkTarget(Path f)
                   throws AccessControlException,
                          FileNotFoundException,
                          UnsupportedFileSystemException,
                          IOException
```
    Description copied from class: FileSystem
    
    See FileContext.getLinkTarget(Path)
    
    Overrides:
    
    getLinkTarget in class FileSystem
    
    Throws:
    
    AccessControlException
    
    FileNotFoundException
    
    UnsupportedFileSystemException
    
    IOException
  - resolveLink
```
protected Path resolveLink(Path f)
                    throws IOException
```
    Description copied from class: FileSystem
    
    See AbstractFileSystem.getLinkTarget(Path)
    
    Overrides:
    
    resolveLink in class FileSystem
    
    Throws:
    
    IOException
  - getFileChecksum
```
public FileChecksum getFileChecksum(Path f)
                             throws IOException
```
    Description copied from class: FileSystem
    
    Get the checksum of a file.
    
    Overrides:
    
    getFileChecksum in class FileSystem
    
    Parameters:
    f - The file path
    
    Returns:
    The file checksum. The default return value is null, which indicates that no checksum algorithm is implemented in the corresponding FileSystem.
    
    Throws:
    
    IOException
  - setPermission
```
public void setPermission(Path p,
                 FsPermission permission)
                   throws IOException
```
    Description copied from class: FileSystem
    
    Set permission of a path.
    
    Overrides:
    
    setPermission in class FileSystem
    
    Throws:
    
    IOException
  - setOwner
```
public void setOwner(Path p,
            String username,
            String groupname)
              throws IOException
```
    Description copied from class: FileSystem
    
    Set owner of a path (i.e. a file or a directory). The parameters username and groupname cannot both be null.
    
    Overrides:
    
    setOwner in class FileSystem
    
    Parameters:
    p - The path
    username - If it is null, the original username remains unchanged.
    groupname - If it is null, the original groupname remains unchanged.
    
    Throws:
    
    IOException
  - setTimes
```
public void setTimes(Path p,
            long mtime,
            long atime)
              throws IOException
```
    Description copied from class: FileSystem
    
    Set access time of a file
    
    Overrides:
    
    setTimes in class FileSystem
    
    Parameters:
    p - The path
    mtime - Set the modification time of this file. The number of milliseconds since Jan 1, 1970. A value of -1 means that this call should not set modification time.
    atime - Set the access time of this file. The number of milliseconds since Jan 1, 1970. A value of -1 means that this call should not set access time.
    
    Throws:
    
    IOException
  - getDefaultPort
```
protected int getDefaultPort()
```
    Description copied from class: FileSystem
    
    Get the default port for this file system.
    
    Overrides:
    
    getDefaultPort in class FileSystem
    
    Returns:
    the default port or 0 if there isn't one
  - getDelegationToken
```
public Token<DelegationTokenIdentifier> getDelegationToken(String renewer)
                                                    throws IOException
```
    Description copied from class: FileSystem
    
    Get a new delegation token for this file system. This is an internal method that should have been declared protected but wasn't historically. Callers should use FileSystem.addDelegationTokens(String, Credentials)
    
    Overrides:
    
    getDelegationToken in class FileSystem
    
    Parameters:
    renewer - the account name that is allowed to renew the token.
    
    Returns:
    a new delegation token
    
    Throws:
    
    IOException
  - getDelegationToken
```
@Deprecated
public Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
                                                    throws IOException
```
    Deprecated. use getDelegationToken(String)
    
    Get a valid Delegation Token.
    
    Parameters:
    renewer - Name of the designated renewer for the token
    
    Returns:
    Token
    
    Throws:
    
    IOException
  - renewDelegationToken
```
public long renewDelegationToken(Token<DelegationTokenIdentifier> token)
                          throws SecretManager.InvalidToken,
                                 IOException
```
    Deprecated. Use Token.renew instead.
    
    Renew an existing delegation token.
    
    Parameters:
    token - delegation token obtained earlier
    
    Returns:
    the new expiration time
    
    Throws:
    
    IOException
    
    SecretManager.InvalidToken
  - cancelDelegationToken
```
public void cancelDelegationToken(Token<DelegationTokenIdentifier> token)
                           throws IOException
```
    Deprecated. Use Token.cancel instead.
    
    Cancel an existing delegation token.
    
    Parameters:
    token - delegation token
    
    Throws:
    
    IOException
  - setBalancerBandwidth
```
public void setBalancerBandwidth(long bandwidth)
                          throws IOException
```
    Requests the namenode to tell all datanodes to use a new, non-persistent bandwidth value for dfs.balance.bandwidthPerSec. The bandwidth parameter is the max number of bytes per second of network bandwidth to be used by a datanode during balancing.
    
    Parameters:
    bandwidth - Balancer bandwidth in bytes per second for all datanodes.
    
    Throws:
    
    IOException
  - getCanonicalServiceName
```
public String getCanonicalServiceName()
```
    Get a canonical service name for this file system. If the URI is logical, the hostname part of the URI will be returned.
    
    Overrides:
    
    getCanonicalServiceName in class FileSystem
    
    Returns:
    a service string that uniquely identifies this file system.
    See Also:
    SecurityUtil.buildDTServiceName(URI, int)
  - canonicalizeUri
```
protected URI canonicalizeUri(URI uri)
```
    Description copied from class: FileSystem
    
    Canonicalize the given URI. This is filesystem-dependent, but may for example consist of canonicalizing the hostname using DNS and adding the default port if not specified. The default implementation simply fills in the default port if not specified and if the filesystem has a default port.
    
    Overrides:
    
    canonicalizeUri in class FileSystem
    
    Returns:
    URI
    See Also:
    NetUtils.getCanonicalUri(URI, int)
  - isInSafeMode
```
public boolean isInSafeMode()
                     throws IOException
```
    Utility function that returns if the NameNode is in safemode or not. In HA mode, this API will return only ActiveNN's safemode status.
    
    Returns:
    true if NameNode is in safemode, false otherwise.
    
    Throws:
    
    IOException - when there is an issue communicating with the NameNode
  - allowSnapshot
```
public void allowSnapshot(Path path)
                   throws IOException
```
    Throws:
    
    IOException
    See Also:
    HdfsAdmin.allowSnapshot(Path)
  - disallowSnapshot
```
public void disallowSnapshot(Path path)
                      throws IOException
```
    Throws:
    
    IOException
    See Also:
    HdfsAdmin.disallowSnapshot(Path)
  - createSnapshot
```
public Path createSnapshot(Path path,
                  String snapshotName)
                    throws IOException
```
    Description copied from class: FileSystem
    
    Create a snapshot
    
    Overrides:
    
    createSnapshot in class FileSystem
    
    Parameters:
    path - The directory where snapshots will be taken.
    snapshotName - The name of the snapshot
    
    Returns:
    the snapshot path.
    
    Throws:
    
    IOException
  - renameSnapshot
```
public void renameSnapshot(Path path,
                  String snapshotOldName,
                  String snapshotNewName)
                    throws IOException
```
    Description copied from class: FileSystem
    
    Rename a snapshot
    
    Overrides:
    
    renameSnapshot in class FileSystem
    
    Parameters:
    path - The directory path where the snapshot was taken
    snapshotOldName - Old name of the snapshot
    snapshotNewName - New name of the snapshot
    
    Throws:
    
    IOException
  - getSnapshottableDirListing
```
public SnapshottableDirectoryStatus[] getSnapshottableDirListing()
                                                          throws IOException
```
    Returns:
    All the snapshottable directories
    
    Throws:
    
    IOException
  - deleteSnapshot
```
public void deleteSnapshot(Path snapshotDir,
                  String snapshotName)
                    throws IOException
```
    Description copied from class: FileSystem
    
    Delete a snapshot of a directory
    
    Overrides:
    
    deleteSnapshot in class FileSystem
    
    Parameters:
    snapshotDir - The directory that the to-be-deleted snapshot belongs to
    snapshotName - The name of the snapshot
    
    Throws:
    
    IOException
  - getSnapshotDiffReport
```
public SnapshotDiffReport getSnapshotDiffReport(Path snapshotDir,
                                       String fromSnapshot,
                                       String toSnapshot)
                                         throws IOException
```
    Get the difference between two snapshots, or between a snapshot and the current tree of a directory.
    
    Throws:
    
    IOException
    See Also:
    DFSClient#getSnapshotDiffReport(Path, String, String)
  - isFileClosed
```
public boolean isFileClosed(Path src)
                     throws IOException
```
    Get the close status of a file
    
    Parameters:
    src - The path to the file
    
    Returns:
    return true if file is closed
    
    Throws:
    
    FileNotFoundException - if the file does not exist.
    
    IOException - If an I/O error occurred

Class DistributedFileSystem

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

Field Summary

Fields inherited from class org.apache.hadoop.fs.FileSystem

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.fs.FileSystem

Methods inherited from class org.apache.hadoop.conf.Configured

Methods inherited from class java.lang.Object

Constructor Detail

DistributedFileSystem

DistributedFileSystem

Method Detail

getScheme

getUri

initialize

getWorkingDirectory

getDefaultBlockSize

getDefaultReplication

setWorkingDirectory

getHomeDirectory

getFileBlockLocations

getFileBlockLocations

getFileBlockStorageLocations

setVerifyChecksum

recoverLease

open

append

create

create

create

primitiveCreate

createNonRecursive

setReplication

concat

rename

rename

delete

getContentSummary

setQuota

listStatus

listLocatedStatus

mkdir

mkdirs

primitiveMkdir

close

toString

getClient

getStatus

getDiskStatus

getRawCapacity

getRawUsed

getMissingBlocksCount

getUnderReplicatedBlocksCount

getCorruptBlocksCount

listCorruptFileBlocks

getDataNodeStats

getDataNodeStats

setSafeMode

setSafeMode

saveNamespace

rollEdits

restoreFailedStorage

refreshNodes

finalizeUpgrade

metaSave

getServerDefaults

reportChecksumFailure

getFileStatus

createSymlink

supportsSymlinks

getFileLinkStatus

getLinkTarget

resolveLink

getFileChecksum

setPermission

setOwner

setTimes

getDefaultPort