Package org.apache.storm.blobstore
Class ClientBlobStore
- java.lang.Object
-
- org.apache.storm.blobstore.ClientBlobStore
-
- All Implemented Interfaces:
AutoCloseable,Shutdownable
- Direct Known Subclasses:
HdfsClientBlobStore,LocalModeClientBlobStore,NimbusBlobStore
public abstract class ClientBlobStore extends Object implements Shutdownable, AutoCloseable
The ClientBlobStore has two concrete implementations 1. NimbusBlobStore 2. HdfsClientBlobStore.Create, update, read and delete are some of the basic operations defined by this interface. Each operation is validated for permissions against an user. We currently have NIMBUS_ADMINS and SUPERVISOR_ADMINS configuration. NIMBUS_ADMINS are given READ, WRITE and ADMIN access whereas the SUPERVISOR_ADMINS are given READ access in order to read and download the blobs form the nimbus.
The ACLs for the blob store are validated against whether the subject is a NIMBUS_ADMIN, SUPERVISOR_ADMIN or USER who has read, write or admin privileges in order to perform respective operations on the blob.
For more detailed implementation
- See Also:
NimbusBlobStore
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interfaceClientBlobStore.WithBlobstore
-
Constructor Summary
Constructors Constructor Description ClientBlobStore()
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description abstract voidclose()AtomicOutputStreamcreateBlob(String key, SettableBlobMeta meta)Client facing API to create a blob.protected abstract AtomicOutputStreamcreateBlobToExtend(String key, SettableBlobMeta meta)Client facing API to create a blob.abstract voidcreateStateInZookeeper(String key)Creates state inside a zookeeper.abstract voiddeleteBlob(String key)Client facing API to delete a blob.abstract InputStreamWithMetagetBlob(String key)Client facing API to read a blob.abstract ReadableBlobMetagetBlobMeta(String key)Client facing API to read the metadata information.abstract intgetBlobReplication(String key)Client facing API to read the replication of a blob.abstract longgetRemoteBlobstoreUpdateTime()Client facing API to get the last update time of existing blobs in a blobstore.abstract booleanisRemoteBlobExists(String blobKey)Decide if the blob is deleted from cluster.abstract Iterator<String>listKeys()List keys.abstract voidprepare(Map<String,Object> conf)Sets up the client API by parsing the configs.voidsetBlobMeta(String key, SettableBlobMeta meta)Client facing API to set the metadata for a blob.protected abstract voidsetBlobMetaToExtend(String key, SettableBlobMeta meta)Client facing API to set the metadata for a blob.abstract booleansetClient(Map<String,Object> conf, NimbusClient client)Client facing API to set a nimbus client.abstract AtomicOutputStreamupdateBlob(String key)Client facing API to update a blob.abstract intupdateBlobReplication(String key, int replication)Client facing API to update the replication of a blob.static voidwithConfiguredClient(ClientBlobStore.WithBlobstore withBlobstore)-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.storm.daemon.Shutdownable
shutdown
-
-
-
-
Method Detail
-
withConfiguredClient
public static void withConfiguredClient(ClientBlobStore.WithBlobstore withBlobstore) throws Exception
- Throws:
Exception
-
prepare
public abstract void prepare(Map<String,Object> conf)
Sets up the client API by parsing the configs.- Parameters:
conf- The storm conf containing the config details
-
createBlobToExtend
protected abstract AtomicOutputStream createBlobToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException
Client facing API to create a blob.- Parameters:
key- blob key namemeta- contains ACL information- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyAlreadyExistsException
-
updateBlob
public abstract AtomicOutputStream updateBlob(String key) throws AuthorizationException, KeyNotFoundException
Client facing API to update a blob.- Parameters:
key- blob key name- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyNotFoundException
-
getBlobMeta
public abstract ReadableBlobMeta getBlobMeta(String key) throws AuthorizationException, KeyNotFoundException
Client facing API to read the metadata information.- Parameters:
key- blob key name- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyNotFoundException
-
isRemoteBlobExists
public abstract boolean isRemoteBlobExists(String blobKey) throws AuthorizationException
Decide if the blob is deleted from cluster.- Parameters:
blobKey- blob key- Throws:
AuthorizationException
-
setBlobMetaToExtend
protected abstract void setBlobMetaToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException
Client facing API to set the metadata for a blob.- Parameters:
key- blob key namemeta- contains ACL information- Throws:
AuthorizationExceptionKeyNotFoundException
-
deleteBlob
public abstract void deleteBlob(String key) throws AuthorizationException, KeyNotFoundException
Client facing API to delete a blob.- Parameters:
key- blob key name- Throws:
AuthorizationExceptionKeyNotFoundException
-
getBlob
public abstract InputStreamWithMeta getBlob(String key) throws AuthorizationException, KeyNotFoundException
Client facing API to read a blob.- Parameters:
key- blob key name- Returns:
- an InputStream to read the metadata for a blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
listKeys
public abstract Iterator<String> listKeys()
List keys.- Returns:
- Iterator for a list of keys currently present in the blob store.
-
getBlobReplication
public abstract int getBlobReplication(String key) throws AuthorizationException, KeyNotFoundException
Client facing API to read the replication of a blob.- Parameters:
key- blob key name- Returns:
- int indicates the replication factor of a blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
updateBlobReplication
public abstract int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException
Client facing API to update the replication of a blob.- Parameters:
key- blob key namereplication- int indicates the replication factor a blob has to be set- Returns:
- int indicates the replication factor of a blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
setClient
public abstract boolean setClient(Map<String,Object> conf, NimbusClient client)
Client facing API to set a nimbus client.- Parameters:
conf- storm confclient- NimbusClient- Returns:
- indicates where the client connection has been setup.
-
createStateInZookeeper
public abstract void createStateInZookeeper(String key)
Creates state inside a zookeeper. Required for blobstore to write to zookeeper when Nimbus HA is turned on in order to maintain state consistency.
-
close
public abstract void close()
- Specified by:
closein interfaceAutoCloseable
-
createBlob
public final AtomicOutputStream createBlob(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException
Client facing API to create a blob.- Parameters:
key- blob key namemeta- contains ACL information- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationExceptionKeyAlreadyExistsException
-
setBlobMeta
public final void setBlobMeta(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException
Client facing API to set the metadata for a blob.- Parameters:
key- blob key namemeta- contains ACL information- Throws:
AuthorizationExceptionKeyNotFoundException
-
getRemoteBlobstoreUpdateTime
public abstract long getRemoteBlobstoreUpdateTime() throws IOExceptionClient facing API to get the last update time of existing blobs in a blobstore. This is only required for use on supervisors.- Returns:
- the timestamp of when the blobstore was last updated. -1L if the blobstore does not support this.
- Throws:
IOException
-
-