Package org.apache.storm.hdfs.blobstore
Class HdfsClientBlobStore
- java.lang.Object
-
- org.apache.storm.blobstore.ClientBlobStore
-
- org.apache.storm.hdfs.blobstore.HdfsClientBlobStore
-
- All Implemented Interfaces:
AutoCloseable,Shutdownable
public class HdfsClientBlobStore extends ClientBlobStore
Client to access the HDFS blobStore. At this point, this is meant to only be used by the supervisor. Don't trust who the client says they are so pass null for all Subjects.The HdfsBlobStore implementation takes care of the null Subjects. It assigns Subjects based on what hadoop says who the users are. These users must be configured accordingly in the SUPERVISOR_ADMINS for ACL validation and for the supervisors to download the blobs. This API is only used by the supervisor in order to talk directly to HDFS.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.storm.blobstore.ClientBlobStore
ClientBlobStore.WithBlobstore
-
-
Constructor Summary
Constructors Constructor Description HdfsClientBlobStore()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidclose()AtomicOutputStreamcreateBlobToExtend(String key, SettableBlobMeta meta)Client facing API to create a blob.voidcreateStateInZookeeper(String key)Creates state inside a zookeeper.voiddeleteBlob(String key)Client facing API to delete a blob.InputStreamWithMetagetBlob(String key)Client facing API to read a blob.ReadableBlobMetagetBlobMeta(String key)Client facing API to read the metadata information.intgetBlobReplication(String key)Client facing API to read the replication of a blob.longgetRemoteBlobstoreUpdateTime()Client facing API to get the last update time of existing blobs in a blobstore.booleanisRemoteBlobExists(String blobKey)Decide if the blob is deleted from cluster.Iterator<String>listKeys()List keys.voidprepare(Map<String,Object> conf)Sets up the client API by parsing the configs.voidsetBlobMetaToExtend(String key, SettableBlobMeta meta)Client facing API to set the metadata for a blob.booleansetClient(Map<String,Object> conf, NimbusClient client)Client facing API to set a nimbus client.voidshutdown()AtomicOutputStreamupdateBlob(String key)Client facing API to update a blob.intupdateBlobReplication(String key, int replication)Client facing API to update the replication of a blob.-
Methods inherited from class org.apache.storm.blobstore.ClientBlobStore
createBlob, setBlobMeta, withConfiguredClient
-
-
-
-
Method Detail
-
prepare
public void prepare(Map<String,Object> conf)
Description copied from class:ClientBlobStoreSets up the client API by parsing the configs.- Specified by:
preparein classClientBlobStore- Parameters:
conf- The storm conf containing the config details
-
createBlobToExtend
public AtomicOutputStream createBlobToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException
Description copied from class:ClientBlobStoreClient facing API to create a blob.- Specified by:
createBlobToExtendin classClientBlobStore- Parameters:
key- blob key namemeta- contains ACL information- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyAlreadyExistsException
-
updateBlob
public AtomicOutputStream updateBlob(String key) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to update a blob.- Specified by:
updateBlobin classClientBlobStore- Parameters:
key- blob key name- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyNotFoundException
-
getBlobMeta
public ReadableBlobMeta getBlobMeta(String key) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to read the metadata information.- Specified by:
getBlobMetain classClientBlobStore- Parameters:
key- blob key name- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyNotFoundException
-
isRemoteBlobExists
public boolean isRemoteBlobExists(String blobKey) throws AuthorizationException
Description copied from class:ClientBlobStoreDecide if the blob is deleted from cluster.- Specified by:
isRemoteBlobExistsin classClientBlobStore- Parameters:
blobKey- blob key- Throws:
AuthorizationException
-
setBlobMetaToExtend
public void setBlobMetaToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to set the metadata for a blob.- Specified by:
setBlobMetaToExtendin classClientBlobStore- Parameters:
key- blob key namemeta- contains ACL information- Throws:
AuthorizationExceptionKeyNotFoundException
-
deleteBlob
public void deleteBlob(String key) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to delete a blob.- Specified by:
deleteBlobin classClientBlobStore- Parameters:
key- blob key name- Throws:
AuthorizationExceptionKeyNotFoundException
-
getBlob
public InputStreamWithMeta getBlob(String key) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to read a blob.- Specified by:
getBlobin classClientBlobStore- Parameters:
key- blob key name- Returns:
- an InputStream to read the metadata for a blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
listKeys
public Iterator<String> listKeys()
Description copied from class:ClientBlobStoreList keys.- Specified by:
listKeysin classClientBlobStore- Returns:
- Iterator for a list of keys currently present in the blob store.
-
getBlobReplication
public int getBlobReplication(String key) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to read the replication of a blob.- Specified by:
getBlobReplicationin classClientBlobStore- Parameters:
key- blob key name- Returns:
- int indicates the replication factor of a blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
updateBlobReplication
public int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException
Description copied from class:ClientBlobStoreClient facing API to update the replication of a blob.- Specified by:
updateBlobReplicationin classClientBlobStore- Parameters:
key- blob key namereplication- int indicates the replication factor a blob has to be set- Returns:
- int indicates the replication factor of a blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
setClient
public boolean setClient(Map<String,Object> conf, NimbusClient client)
Description copied from class:ClientBlobStoreClient facing API to set a nimbus client.- Specified by:
setClientin classClientBlobStore- Parameters:
conf- storm confclient- NimbusClient- Returns:
- indicates where the client connection has been setup.
-
createStateInZookeeper
public void createStateInZookeeper(String key)
Description copied from class:ClientBlobStoreCreates state inside a zookeeper. Required for blobstore to write to zookeeper when Nimbus HA is turned on in order to maintain state consistency.- Specified by:
createStateInZookeeperin classClientBlobStore
-
shutdown
public void shutdown()
-
close
public void close()
- Specified by:
closein interfaceAutoCloseable- Specified by:
closein classClientBlobStore
-
getRemoteBlobstoreUpdateTime
public long getRemoteBlobstoreUpdateTime() throws IOExceptionDescription copied from class:ClientBlobStoreClient facing API to get the last update time of existing blobs in a blobstore. This is only required for use on supervisors.- Specified by:
getRemoteBlobstoreUpdateTimein classClientBlobStore- Returns:
- the timestamp of when the blobstore was last updated. -1L if the blobstore does not support this.
- Throws:
IOException
-
-