org.apache.hive.hcatalog.templeton.tool

Interface TempletonStorage

All Known Implementing Classes:

HDFSStorage, ZooKeeperStorage

public interface TempletonStorage

An interface to handle different Templeton storage methods, including ZooKeeper and HDFS. Any storage scheme must be able to handle being run in an HDFS environment, where specific file systems and virtual machines may not be available. Storage is done individually in a hierarchy: type (the data type, as listed below), then the id (a given jobid, jobtrackingid, etc.), then the key/value pairs. So an entry might look like: JOB jobid00035 user -> rachel datecreated -> 2/5/12 etc. Each field must be available to be fetched/changed individually.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	TempletonStorage.Type

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	STORAGE_CLASS
static java.lang.String	STORAGE_ROOT

Method Summary

Modifier and Type	Method and Description
void	closeStorage() For storage methods that require a connection, this is a hint that it's time to close the connection.
boolean	delete(TempletonStorage.Type type, java.lang.String id) Delete a data grouping (all data for a jobid, all tracking data for a job, etc.).
java.util.List<java.lang.String>	getAll() Get the id of each data grouping in the storage system.
java.util.List<java.lang.String>	getAllForKey(java.lang.String key, java.lang.String value) Get the id of each data grouping that has the specific key/value pair.
java.util.List<java.lang.String>	getAllForType(TempletonStorage.Type type) Get the id of each data grouping of a given type in the storage system.
java.util.List<java.lang.String>	getAllForTypeAndKey(TempletonStorage.Type type, java.lang.String key, java.lang.String value) Get the id of each data grouping of a given type that has the specific key/value pair.
java.lang.String	getField(TempletonStorage.Type type, java.lang.String id, java.lang.String key) Get the value of one field for a given data type.
java.util.Map<java.lang.String,java.lang.String>	getFields(TempletonStorage.Type type, java.lang.String id) Get all the name/value pairs stored for this id.
void	openStorage(Configuration config) For storage methods that require a connection, this is a hint that it's time to open a connection.
void	saveField(TempletonStorage.Type type, java.lang.String id, java.lang.String key, java.lang.String val) Save a single key/value pair for a specific job id.
void	startCleanup(Configuration config) Start the cleanup process for this storage type.

Field Detail

STORAGE_CLASS

static final java.lang.String STORAGE_CLASS

See Also:

Constant Field Values

STORAGE_ROOT

static final java.lang.String STORAGE_ROOT

See Also:

Constant Field Values

Method Detail

startCleanup

void startCleanup(Configuration config)

Start the cleanup process for this storage type.

Parameters:

config -

saveField

void saveField(TempletonStorage.Type type,
               java.lang.String id,
               java.lang.String key,
               java.lang.String val)
        throws NotFoundException

Save a single key/value pair for a specific job id.

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

key - The name of the field to save

val - The value of the field to save

Throws:

NotFoundException

getField

java.lang.String getField(TempletonStorage.Type type,
                          java.lang.String id,
                          java.lang.String key)

Get the value of one field for a given data type. If the type is UNKNOWN, search for the id in all types.

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

key - The name of the field to retrieve

Returns:

The value of the field requested, or null if not found.

getFields

java.util.Map<java.lang.String,java.lang.String> getFields(TempletonStorage.Type type,
                                                           java.lang.String id)

Get all the name/value pairs stored for this id. Be careful using getFields() -- optimistic locking will mean that your odds of a conflict are decreased if you read/write one field at a time. getFields() is intended for read-only usage. If the type is UNKNOWN, search for the id in all types.

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

Returns:

A Map of key/value pairs found for this type/id.

delete

boolean delete(TempletonStorage.Type type,
               java.lang.String id)
        throws NotFoundException

Delete a data grouping (all data for a jobid, all tracking data for a job, etc.). If the type is UNKNOWN, search for the id in all types.

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

Returns:

True if successful, false if not, throws NotFoundException if the id wasn't found.

Throws:

NotFoundException

getAll

java.util.List<java.lang.String> getAll()

Get the id of each data grouping in the storage system.

Returns:

An ArrayList of ids.

getAllForType

java.util.List<java.lang.String> getAllForType(TempletonStorage.Type type)

Get the id of each data grouping of a given type in the storage system.

Parameters:

type - The data type (as listed above)

Returns:

An ArrayList of ids.

getAllForKey

java.util.List<java.lang.String> getAllForKey(java.lang.String key,
                                              java.lang.String value)

Get the id of each data grouping that has the specific key/value pair.

Parameters:

key - The name of the field to search for

value - The value of the field to search for

Returns:

An ArrayList of ids.

getAllForTypeAndKey

java.util.List<java.lang.String> getAllForTypeAndKey(TempletonStorage.Type type,
                                                     java.lang.String key,
                                                     java.lang.String value)

Get the id of each data grouping of a given type that has the specific key/value pair.

Parameters:

type - The data type (as listed above)

key - The name of the field to search for

value - The value of the field to search for

Returns:

An ArrayList of ids.

openStorage

void openStorage(Configuration config)
          throws java.io.IOException

For storage methods that require a connection, this is a hint that it's time to open a connection.

Throws:

java.io.IOException

closeStorage

void closeStorage()
           throws java.io.IOException

For storage methods that require a connection, this is a hint that it's time to close the connection.

Throws:

java.io.IOException