public class WebHdfsFileSystem extends FileSystem implements DelegationTokenRenewer.Renewable
| Modifier and Type | Class and Description |
|---|---|
static class |
WebHdfsFileSystem.DtRenewer
Delegation token renewer.
|
FileSystem.Statistics| Modifier and Type | Field and Description |
|---|---|
static org.apache.hadoop.hdfs.web.WebHdfsFileSystem.WebHdfsDelegationTokenSelector |
DT_SELECTOR
Token selector
|
static org.apache.commons.logging.Log |
LOG |
static String |
PATH_PREFIX
Http URI: http://namenode:port/{PATH_PREFIX}/path/to/file
|
static String |
SCHEME
File System URI: {SCHEME}://namenode:port/path/to/file
|
static Text |
TOKEN_KIND
Delegation token kind
|
static int |
VERSION
WebHdfs version.
|
DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics| Constructor and Description |
|---|
WebHdfsFileSystem() |
| Modifier and Type | Method and Description |
|---|---|
protected void |
addRenewAction(WebHdfsFileSystem webhdfs) |
FSDataOutputStream |
append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
protected URI |
canonicalizeUri(URI uri)
Canonicalize the given URI.
|
void |
close()
No more filesystem operations are needed.
|
void |
concat(Path trg,
Path[] srcs)
Concat existing files together.
|
FSDataOutputStream |
create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
void |
createSymlink(Path destination,
Path f,
boolean createParent)
Create a symlink pointing to the destination path.
|
boolean |
delete(Path f)
Delete a file
|
boolean |
delete(Path f,
boolean recursive)
Delete a file.
|
ContentSummary |
getContentSummary(Path p)
Return the
ContentSummary of a given Path. |
long |
getDefaultBlockSize()
Return the number of bytes that large input files should be optimally
be split into to minimize i/o time.
|
protected int |
getDefaultPort()
Get the default port for this file system.
|
short |
getDefaultReplication()
Get the default replication.
|
protected Token<?> |
getDelegationToken() |
Token<DelegationTokenIdentifier> |
getDelegationToken(String renewer)
Get a new delegation token for this file system.
|
BlockLocation[] |
getFileBlockLocations(FileStatus status,
long offset,
long length)
Return an array containing hostnames, offset and size of
portions of the given file.
|
BlockLocation[] |
getFileBlockLocations(Path p,
long offset,
long length)
Return an array containing hostnames, offset and size of
portions of the given file.
|
MD5MD5CRC32FileChecksum |
getFileChecksum(Path p)
Get the checksum of a file.
|
FileStatus |
getFileStatus(Path f)
Return a file status object that represents the path.
|
Path |
getHomeDirectory()
Return the current user's home directory in this filesystem.
|
static String |
getHomeDirectoryString(UserGroupInformation ugi) |
Token<?> |
getRenewToken() |
String |
getScheme()
Return the protocol scheme for the FileSystem.
|
URI |
getUri()
Returns a URI whose scheme and authority identify this FileSystem.
|
Path |
getWorkingDirectory()
Get the current working directory for the given file system
|
protected void |
initDelegationToken() |
void |
initialize(URI uri,
Configuration conf)
Called after a new FileSystem instance is constructed.
|
static boolean |
isEnabled(Configuration conf,
org.apache.commons.logging.Log log)
Is WebHDFS enabled in conf?
|
FileStatus[] |
listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
boolean |
mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into
directories.
|
FSDataInputStream |
open(Path f,
int buffersize)
Opens an FSDataInputStream at the indicated Path.
|
boolean |
rename(Path src,
Path dst)
Renames Path src to Path dst.
|
void |
rename(Path src,
Path dst,
Options.Rename... options)
Renames Path src to Path dst
|
protected Token<DelegationTokenIdentifier> |
selectDelegationToken(UserGroupInformation ugi) |
<T extends TokenIdentifier> |
setDelegationToken(Token<T> token)
Set delegation token.
|
void |
setOwner(Path p,
String owner,
String group)
Set owner of a path (i.e.
|
void |
setPermission(Path p,
FsPermission permission)
Set permission of a path.
|
boolean |
setReplication(Path p,
short replication)
Set replication for an existing file.
|
void |
setTimes(Path p,
long mtime,
long atime)
Set access time of a file
|
void |
setWorkingDirectory(Path dir)
Set the current working directory for the given file system.
|
addDelegationTokens, append, append, cancelDeleteOnExit, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, createNonRecursive, createSnapshot, createSnapshot, deleteOnExit, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAllStatistics, getBlockSize, getCanonicalServiceName, getCanonicalUri, getChildFileSystems, getDefaultBlockSize, getDefaultReplication, getDefaultUri, getFileLinkStatus, getFileSystemClass, getFSofPath, getInitialWorkingDirectory, getLength, getLinkTarget, getLocal, getName, getNamed, getReplication, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getUsed, globStatus, globStatus, isDirectory, isFile, isSymlinksEnabled, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, makeQualified, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, renameSnapshot, resolveLink, resolvePath, setDefaultUri, setDefaultUri, setVerifyChecksum, setWriteChecksum, startLocalOutput, supportsSymlinksgetConf, setConfpublic static final org.apache.commons.logging.Log LOG
public static final String SCHEME
public static final int VERSION
public static final String PATH_PREFIX
public static final Text TOKEN_KIND
public static final org.apache.hadoop.hdfs.web.WebHdfsFileSystem.WebHdfsDelegationTokenSelector DT_SELECTOR
protected void addRenewAction(WebHdfsFileSystem webhdfs)
public static boolean isEnabled(Configuration conf, org.apache.commons.logging.Log log)
public String getScheme()
getScheme in class FileSystemwebhdfspublic void initialize(URI uri, Configuration conf) throws IOException
FileSysteminitialize in class FileSystemuri - a uri whose authority section names the host, port, etc.
for this FileSystemconf - the configurationIOExceptionprotected void initDelegationToken()
throws IOException
IOExceptionprotected Token<?> getDelegationToken() throws IOException
IOExceptionprotected Token<DelegationTokenIdentifier> selectDelegationToken(UserGroupInformation ugi)
protected int getDefaultPort()
FileSystemgetDefaultPort in class FileSystempublic URI getUri()
FileSystemgetUri in class FileSystemprotected URI canonicalizeUri(URI uri)
FileSystemcanonicalizeUri in class FileSystemNetUtils.getCanonicalUri(URI, int)public static String getHomeDirectoryString(UserGroupInformation ugi)
public Path getHomeDirectory()
FileSystemgetHomeDirectory in class FileSystempublic Path getWorkingDirectory()
FileSystemgetWorkingDirectory in class FileSystempublic void setWorkingDirectory(Path dir)
FileSystemsetWorkingDirectory in class FileSystempublic FileStatus getFileStatus(Path f) throws IOException
FileSystemgetFileStatus in class FileSystemf - The path we want information fromFileNotFoundException - when the path does not exist;
IOException see specific implementationIOExceptionpublic boolean mkdirs(Path f, FsPermission permission) throws IOException
FileSystemmkdirs in class FileSystemf - path to createpermission - to apply to fIOExceptionpublic void createSymlink(Path destination, Path f, boolean createParent) throws IOException
createSymlink in class FileSystemIOExceptionHdfs.createSymlink(Path, Path, boolean)public boolean rename(Path src, Path dst) throws IOException
FileSystemrename in class FileSystemsrc - path to be renameddst - new path after renameIOException - on failurepublic void rename(Path src, Path dst, Options.Rename... options) throws IOException
FileSystemIf OVERWRITE option is not passed as an argument, rename fails if the dst already exists.
If OVERWRITE option is passed as an argument, rename overwrites the dst if it is a file or an empty directory. Rename fails if dst is a non-empty directory.
Note that atomicity of rename is dependent on the file system implementation. Please refer to the file system documentation for details. This default implementation is non atomic.
This method is deprecated since it is a temporary method added to support the transition from FileSystem to FileContext for user applications.
rename in class FileSystemsrc - path to be renameddst - new path after renameIOException - on failurepublic void setOwner(Path p, String owner, String group) throws IOException
FileSystemsetOwner in class FileSystemp - The pathowner - If it is null, the original username remains unchanged.group - If it is null, the original groupname remains unchanged.IOExceptionpublic void setPermission(Path p, FsPermission permission) throws IOException
FileSystemsetPermission in class FileSystemIOExceptionpublic boolean setReplication(Path p, short replication) throws IOException
FileSystemsetReplication in class FileSystemp - file namereplication - new replicationIOExceptionpublic void setTimes(Path p, long mtime, long atime) throws IOException
FileSystemsetTimes in class FileSystemp - The pathmtime - Set the modification time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set modification time.atime - Set the access time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set access time.IOExceptionpublic long getDefaultBlockSize()
FileSystemgetDefaultBlockSize in class FileSystempublic short getDefaultReplication()
FileSystemgetDefaultReplication in class FileSystempublic void concat(Path trg, Path[] srcs) throws IOException
FileSystemconcat in class FileSystemtrg - the path to the target destination.srcs - the paths to the sources to use for the concatenation.IOExceptionpublic FSDataOutputStream create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress) throws IOException
FileSystemcreate in class FileSystemf - the file name to openoverwrite - if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize - the size of the buffer to be used.replication - required block replication for the file.IOExceptionFileSystem.setPermission(Path, FsPermission)public FSDataOutputStream append(Path f, int bufferSize, Progressable progress) throws IOException
FileSystemappend in class FileSystemf - the existing file to be appended.bufferSize - the size of the buffer to be used.progress - for reporting progress if it is not null.IOExceptionpublic boolean delete(Path f) throws IOException
FileSystemdelete in class FileSystemIOExceptionpublic boolean delete(Path f, boolean recursive) throws IOException
FileSystemdelete in class FileSystemf - the path to delete.recursive - if path is a directory and set to
true, the directory is deleted else throws an exception. In
case of a file the recursive can be set to either true or false.IOExceptionpublic FSDataInputStream open(Path f, int buffersize) throws IOException
FileSystemopen in class FileSystemf - the file name to openbuffersize - the size of the buffer to be used.IOExceptionpublic void close()
throws IOException
FileSystemclose in interface Closeableclose in interface AutoCloseableclose in class FileSystemIOExceptionpublic FileStatus[] listStatus(Path f) throws IOException
FileSystemlistStatus in class FileSystemf - given pathFileNotFoundException - when the path does not exist;
IOException see specific implementationIOExceptionpublic Token<DelegationTokenIdentifier> getDelegationToken(String renewer) throws IOException
FileSystemFileSystem.addDelegationTokens(String, Credentials)getDelegationToken in class FileSystemrenewer - the account name that is allowed to renew the token.IOExceptionpublic Token<?> getRenewToken()
getRenewToken in interface DelegationTokenRenewer.Renewablepublic <T extends TokenIdentifier> void setDelegationToken(Token<T> token)
DelegationTokenRenewer.RenewablesetDelegationToken in interface DelegationTokenRenewer.Renewablepublic BlockLocation[] getFileBlockLocations(FileStatus status, long offset, long length) throws IOException
FileSystemgetFileBlockLocations in class FileSystemstatus - FilesStatus to get data fromoffset - offset into the given filelength - length for which to get locations forIOExceptionpublic BlockLocation[] getFileBlockLocations(Path p, long offset, long length) throws IOException
FileSystemgetFileBlockLocations in class FileSystemp - path is used to identify an FS since an FS could have
another FS that it could be delegating the call tooffset - offset into the given filelength - length for which to get locations forIOExceptionpublic ContentSummary getContentSummary(Path p) throws IOException
FileSystemContentSummary of a given Path.getContentSummary in class FileSystemp - path to useIOExceptionpublic MD5MD5CRC32FileChecksum getFileChecksum(Path p) throws IOException
FileSystemgetFileChecksum in class FileSystemp - The file pathIOExceptionCopyright © 2013 Apache Software Foundation. All rights reserved.