Package org.apache.storm.daemon.nimbus
Class Nimbus
- java.lang.Object
-
- org.apache.storm.daemon.nimbus.Nimbus
-
- All Implemented Interfaces:
DaemonCommon,Shutdownable,Nimbus.Iface
public class Nimbus extends Object implements Nimbus.Iface, Shutdownable, DaemonCommon
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classNimbus.StandaloneINimbus
-
Field Summary
Fields Modifier and Type Field Description static SimpleVersionMIN_VERSION_SUPPORT_RPC_HEARTBEATstatic SubjectNIMBUS_SUBJECTstatic List<org.apache.storm.shade.org.apache.zookeeper.data.ACL>ZK_ACLS
-
Constructor Summary
Constructors Constructor Description Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry)Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry)Nimbus(Map<String,Object> conf, INimbus inimbus, StormMetricsRegistry metricsRegistry)
-
Method Summary
-
-
-
Field Detail
-
ZK_ACLS
public static final List<org.apache.storm.shade.org.apache.zookeeper.data.ACL> ZK_ACLS
-
MIN_VERSION_SUPPORT_RPC_HEARTBEAT
public static final SimpleVersion MIN_VERSION_SUPPORT_RPC_HEARTBEAT
-
NIMBUS_SUBJECT
public static final Subject NIMBUS_SUBJECT
-
-
Constructor Detail
-
Nimbus
public Nimbus(Map<String,Object> conf, INimbus inimbus, StormMetricsRegistry metricsRegistry) throws Exception
- Throws:
Exception
-
Nimbus
public Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception
- Throws:
Exception
-
Nimbus
public Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception
- Throws:
Exception
-
-
Method Detail
-
getNimbusAcls
public static List<org.apache.storm.shade.org.apache.zookeeper.data.ACL> getNimbusAcls(Map<String,Object> conf)
-
getVersionForKey
public static int getVersionForKey(String key, NimbusInfo nimbusInfo, org.apache.storm.shade.org.apache.curator.framework.CuratorFramework zkClient) throws KeyNotFoundException
- Throws:
KeyNotFoundException
-
topoIdsToClean
public static Set<String> topoIdsToClean(IStormClusterState state, BlobStore store, Map<String,Object> conf)
-
cleanInbox
public static void cleanInbox(String dirLoc, int seconds)
Deletes jar files in dirLoc older than seconds.- Parameters:
dirLoc- the location to look in for fileseconds- how old is too old and should be deleted
-
topologiesOnSupervisor
public static List<String> topologiesOnSupervisor(Map<String,Assignment> assignments, String supervisorId)
-
setAuthorizationHandler
public void setAuthorizationHandler(IAuthorizer authorizationHandler)
-
getHeartbeatsCache
public HeartbeatCache getHeartbeatsCache()
-
addSupervisor
public void addSupervisor(Supervisor supervisor)
Used for local cluster.- Parameters:
supervisor-Supervisor
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String,Object> topoConf, String operation) throws AuthorizationException
- Throws:
AuthorizationException
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String,Object> topoConf, String operation, ReqContext context) throws AuthorizationException
- Throws:
AuthorizationException
-
filterAuthorized
public Set<String> filterAuthorized(String operation, Collection<String> topoIds) throws NotAliveException, AuthorizationException, IOException
-
rmDependencyJarsInTopology
public void rmDependencyJarsInTopology(String topoId)
-
rmTopologyKeys
public void rmTopologyKeys(String topoId)
-
forceDeleteTopoDistDir
public void forceDeleteTopoDistDir(String topoId) throws IOException
- Throws:
IOException
-
doCleanup
public void doCleanup()
Cleanup topologies and Jars.
-
awaitLeadership
public boolean awaitLeadership(long timeout, TimeUnit timeUnit) throws InterruptedException- Throws:
InterruptedException
-
submitTopology
public void submitTopology(String name, String uploadedJarLocation, String jsonConf, StormTopology topology) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
submitTopologyin interfaceNimbus.Iface- Throws:
AlreadyAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
submitTopologyWithOpts
public void submitTopologyWithOpts(String topoName, String uploadedJarLocation, String jsonConf, StormTopology topology, SubmitOptions options) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
submitTopologyWithOptsin interfaceNimbus.Iface- Throws:
AlreadyAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
setUpAckerExecutorConfigs
public static void setUpAckerExecutorConfigs(String topoName, Map<String,Object> totalConfToSave, Map<String,Object> totalConf, int estimatedNumWorker)
-
killTopology
public void killTopology(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
killTopologyin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
killTopologyWithOpts
public void killTopologyWithOpts(String topoName, KillOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
killTopologyWithOptsin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
activate
public void activate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
activatein interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
deactivate
public void deactivate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
deactivatein interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
rebalance
public void rebalance(String topoName, RebalanceOptions options) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
rebalancein interfaceNimbus.Iface- Throws:
NotAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
setLogConfig
public void setLogConfig(String topoId, LogConfig config) throws org.apache.storm.thrift.TException
- Specified by:
setLogConfigin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getLogConfig
public LogConfig getLogConfig(String topoId) throws org.apache.storm.thrift.TException
- Specified by:
getLogConfigin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
debug
public void debug(String topoName, String componentId, boolean enable, double samplingPercentage) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceEnable/disable logging the tuples generated in topology via an internal EventLogger bolt. The component name is optional and if null or empty, the debug flag will apply to the entire topology. The 'samplingPercentage' will limit loggging to a percentage of generated tuples.- Specified by:
debugin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
setWorkerProfiler
public void setWorkerProfiler(String topoId, ProfileRequest profileRequest) throws org.apache.storm.thrift.TException
- Specified by:
setWorkerProfilerin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getComponentPendingProfileActions
public List<ProfileRequest> getComponentPendingProfileActions(String id, String componentId, ProfileAction action) throws org.apache.storm.thrift.TException
- Specified by:
getComponentPendingProfileActionsin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
uploadNewCredentials
public void uploadNewCredentials(String topoName, Credentials credentials) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
uploadNewCredentialsin interfaceNimbus.Iface- Throws:
NotAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
beginCreateBlob
public String beginCreateBlob(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException, org.apache.storm.thrift.TException
- Specified by:
beginCreateBlobin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyAlreadyExistsExceptionorg.apache.storm.thrift.TException
-
beginUpdateBlob
public String beginUpdateBlob(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
beginUpdateBlobin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
uploadBlobChunk
public void uploadBlobChunk(String session, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
uploadBlobChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
finishBlobUpload
public void finishBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
finishBlobUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
cancelBlobUpload
public void cancelBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
cancelBlobUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getBlobMeta
public ReadableBlobMeta getBlobMeta(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
getBlobMetain interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
setBlobMeta
public void setBlobMeta(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
setBlobMetain interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
beginBlobDownload
public BeginDownloadResult beginBlobDownload(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
beginBlobDownloadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
downloadBlobChunk
public ByteBuffer downloadBlobChunk(String session) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
downloadBlobChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
deleteBlob
public void deleteBlob(String key) throws AuthorizationException, KeyNotFoundException, IllegalStateException, org.apache.storm.thrift.TException
- Specified by:
deleteBlobin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionIllegalStateExceptionorg.apache.storm.thrift.TException
-
listBlobs
public ListBlobsResult listBlobs(String session) throws org.apache.storm.thrift.TException
- Specified by:
listBlobsin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getBlobReplication
public int getBlobReplication(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
getBlobReplicationin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
updateBlobReplication
public int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
updateBlobReplicationin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
createStateInZookeeper
public void createStateInZookeeper(String key) throws org.apache.storm.thrift.TException
- Specified by:
createStateInZookeeperin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
beginFileUpload
public String beginFileUpload() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
beginFileUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
uploadChunk
public void uploadChunk(String location, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
uploadChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
finishFileUpload
public void finishFileUpload(String location) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
finishFileUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
downloadChunk
public ByteBuffer downloadChunk(String id) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
downloadChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getNimbusConf
public String getNimbusConf() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getNimbusConfin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyInfo
public TopologyInfo getTopologyInfo(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyInfoByName
public TopologyInfo getTopologyInfoByName(String name) throws org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoByNamein interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getTopologyInfoByNameWithOpts
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoByNameWithOptsin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyInfoWithOpts
public TopologyInfo getTopologyInfoWithOpts(String topoId, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoWithOptsin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyPageInfo
public TopologyPageInfo getTopologyPageInfo(String topoId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyPageInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getSupervisorPageInfo
public SupervisorPageInfo getSupervisorPageInfo(String superId, String host, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getSupervisorPageInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getComponentPageInfo
public ComponentPageInfo getComponentPageInfo(String topoId, String componentId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getComponentPageInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyConf
public String getTopologyConf(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyConfin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopology
public StormTopology getTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceReturns the compiled topology that contains ackers and metrics consumsers. CompareNimbus.Iface.getUserTopology(String id).- Specified by:
getTopologyin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getUserTopology
public StormTopology getUserTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceReturns the user specified topology as submitted originally. CompareNimbus.Iface.getTopology(String id).- Specified by:
getUserTopologyin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyHistory
public TopologyHistoryInfo getTopologyHistory(String user) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyHistoryin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getClusterInfo
public ClusterSummary getClusterInfo() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getClusterInfoin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologySummaries
public List<TopologySummary> getTopologySummaries() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologySummariesin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologySummaryByName
public TopologySummary getTopologySummaryByName(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologySummaryByNamein interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologySummary
public TopologySummary getTopologySummary(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologySummaryin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getLeader
public NimbusSummary getLeader() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getLeaderin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
isTopologyNameAllowed
public boolean isTopologyNameAllowed(String name) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
isTopologyNameAllowedin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getOwnerResourceSummaries
public List<OwnerResourceSummary> getOwnerResourceSummaries(String owner) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getOwnerResourceSummariesin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getSupervisorAssignments
public SupervisorAssignments getSupervisorAssignments(String nodeId) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceGet assigned assignments for a specific supervisor- Specified by:
getSupervisorAssignmentsin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeats
public void sendSupervisorWorkerHeartbeats(SupervisorWorkerHeartbeats heartbeats) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceSend supervisor worker heartbeats for a specific supervisor- Specified by:
sendSupervisorWorkerHeartbeatsin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeat
public void sendSupervisorWorkerHeartbeat(SupervisorWorkerHeartbeat hb) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceSend supervisor local worker heartbeat when a supervisor is unreachable- Specified by:
sendSupervisorWorkerHeartbeatin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
shutdown
public void shutdown()
- Specified by:
shutdownin interfaceShutdownable
-
isWaiting
public boolean isWaiting()
- Specified by:
isWaitingin interfaceDaemonCommon
-
processWorkerMetrics
public void processWorkerMetrics(WorkerMetrics metrics) throws org.apache.storm.thrift.TException
- Specified by:
processWorkerMetricsin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
isRemoteBlobExists
public boolean isRemoteBlobExists(String blobKey) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.IfaceDecide if the blob is removed from cluster.- Specified by:
isRemoteBlobExistsin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
-