Class BasicContainer
- java.lang.Object
-
- org.apache.storm.daemon.supervisor.Container
-
- org.apache.storm.daemon.supervisor.BasicContainer
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.storm.daemon.supervisor.Container
Container.ContainerType
-
-
Field Summary
Fields Modifier and Type Field Description protected booleanexitedEarlyprotected doublehardMemoryLimitMultiplierprotected longhardMemoryLimitOverprotected LocalStatelocalStateprotected longlowMemoryThresholdMbprotected longmediumMemoryGracePeriodMsprotected longmediumMemoryThresholdMbprotected longmemoryLimitExceededStartprotected longmemoryLimitMbprotected StringprofileCmdprotected StringstormHome-
Fields inherited from class org.apache.storm.daemon.supervisor.Container
assignment, conf, containerMemoryTracker, ops, port, resourceIsolationManager, runAsUser, supervisorId, supervisorPort, symlinksDisabled, topoConf, topologyId, type, workerId
-
-
Constructor Summary
Constructors Constructor Description BasicContainer(Container.ContainerType type, Map<String,Object> conf, String supervisorId, int supervisorPort, int port, LocalAssignment assignment, ResourceIsolationInterface resourceIsolationManager, LocalState localState, String workerId, StormMetricsRegistry metricsRegistry, ContainerMemoryTracker containerMemoryTracker)Create a new BasicContainer.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidcleanUpForRestart()Clean up the container partly preparing for restart.protected voidcreateNewWorkerId()Create a new worker ID for this process and store in in this object and in the local state.booleandidMainProcessExit()Return true if the main process exited, else false.protected List<String>frameworkClasspath(SimpleVersion topoVersion)static List<String>getDependencyLocationsFor(Map<String,Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot)longgetMemoryReservationMb()Get the current memory reservation of this container.longgetMemoryUsageMb()Get the current memory usage of this container.static StringgetStormVersionFor(Map<String,Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot)protected StringgetWildcardDir(File dir)Returns a path with a wildcard as the final element, so that the JVM will expand that to all JARs in the directory.protected StringgetWorkerClassPath(String stormJar, List<String> dependencyLocations, SimpleVersion topoVersion)Compute the classpath for the worker process.protected StringgetWorkerLogWriter(SimpleVersion topoVersion)protected StringgetWorkerMain(SimpleVersion topoVersion)booleanisMemoryLimitViolated(LocalAssignment withUpdatedLimits)Check if the container is over its memory limit AND needs to be killed.protected StringjavaCmd(String cmd)protected StringjavaLibraryPath(String stormRoot, Map<String,Object> conf)Compute the java.library.path that should be used for the worker.voidlaunch()Launch the process for the first time.voidrelaunch()Restart the processes in this container.booleanrunProfiling(ProfileRequest request, boolean stop)Run a profiling request.protected List<String>substituteChildopts(Object value)protected List<String>substituteChildopts(Object value, int memOnheap, int memOffHeap)-
Methods inherited from class org.apache.storm.daemon.supervisor.Container
areAllProcessesDead, cleanUp, createArtifactsLink, createBlobstoreLinks, deleteSavedWorkerUser, forceKill, getTotalTopologyMemoryReserved, getTotalTopologyMemoryUsed, getTotalWorkersForThisTopology, getWorkerId, getWorkerUser, kill, readHeartbeat, readTopoConf, saveWorkerUser, setup, toString, updateMemoryAccounting, writeLogMetadata
-
-
-
-
Field Detail
-
localState
protected final LocalState localState
-
profileCmd
protected final String profileCmd
-
stormHome
protected final String stormHome
-
hardMemoryLimitMultiplier
protected final double hardMemoryLimitMultiplier
-
hardMemoryLimitOver
protected final long hardMemoryLimitOver
-
lowMemoryThresholdMb
protected final long lowMemoryThresholdMb
-
mediumMemoryThresholdMb
protected final long mediumMemoryThresholdMb
-
mediumMemoryGracePeriodMs
protected final long mediumMemoryGracePeriodMs
-
exitedEarly
protected volatile boolean exitedEarly
-
memoryLimitMb
protected volatile long memoryLimitMb
-
memoryLimitExceededStart
protected volatile long memoryLimitExceededStart
-
-
Constructor Detail
-
BasicContainer
public BasicContainer(Container.ContainerType type, Map<String,Object> conf, String supervisorId, int supervisorPort, int port, LocalAssignment assignment, ResourceIsolationInterface resourceIsolationManager, LocalState localState, String workerId, StormMetricsRegistry metricsRegistry, ContainerMemoryTracker containerMemoryTracker) throws IOException
Create a new BasicContainer.- Parameters:
type- the type of container being made.conf- the supervisor configsupervisorId- the ID of the supervisor this is a part of.supervisorPort- the thrift server port of the supervisor this is a part of.port- the port the container is on. Should be <= 0 if only a partial recoveryassignment- the assignment for this container. Should be null if only a partial recovery.resourceIsolationManager- used to isolate resources for a container can be null if no isolation is used.localState- the local state of the supervisor. May be null if partial recoveryworkerId- the id of the worker to use. Must not be null if doing a partial recovery.metricsRegistry- The metrics registry.containerMemoryTracker- The shared memory tracker for the supervisor's containers- Throws:
IOException
-
-
Method Detail
-
getDependencyLocationsFor
public static List<String> getDependencyLocationsFor(Map<String,Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot) throws IOException
- Throws:
IOException
-
getStormVersionFor
public static String getStormVersionFor(Map<String,Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot) throws IOException
- Throws:
IOException
-
createNewWorkerId
protected void createNewWorkerId()
Create a new worker ID for this process and store in in this object and in the local state. Never call this if a worker is currently up and running. We will lose track of the process.
-
cleanUpForRestart
public void cleanUpForRestart() throws IOExceptionDescription copied from class:ContainerClean up the container partly preparing for restart. By default delete all of the temp directories we are going to get a new worker_id anyways. POST CONDITION: the workerId will be set to null- Overrides:
cleanUpForRestartin classContainer- Throws:
IOException- on any error
-
relaunch
public void relaunch() throws IOExceptionDescription copied from class:ContainerRestart the processes in this container. PREREQUISITE: cleanUpForRestart has run and passed- Specified by:
relaunchin classContainer- Throws:
IOException- on any error
-
didMainProcessExit
public boolean didMainProcessExit()
Description copied from class:ContainerReturn true if the main process exited, else false. This is just best effort return false if unknown.- Specified by:
didMainProcessExitin classContainer
-
runProfiling
public boolean runProfiling(ProfileRequest request, boolean stop) throws IOException, InterruptedException
Description copied from class:ContainerRun a profiling request.- Specified by:
runProfilingin classContainer- Parameters:
request- the request to runstop- is this a stop request?- Returns:
- true if it succeeded, else false
- Throws:
IOException- on any errorInterruptedException- if running the command is interrupted.
-
javaLibraryPath
protected String javaLibraryPath(String stormRoot, Map<String,Object> conf)
Compute the java.library.path that should be used for the worker. This helps it to load JNI libraries that are packaged in the uber jar.- Parameters:
stormRoot- the root directory of the worker processconf- the config for the supervisor.- Returns:
- the java.library.path/LD_LIBRARY_PATH to use so native libraries load correctly.
-
getWildcardDir
protected String getWildcardDir(File dir)
Returns a path with a wildcard as the final element, so that the JVM will expand that to all JARs in the directory.- Parameters:
dir- the directory to which a wildcard will be appended- Returns:
- the path with wildcard ("*") suffix
-
frameworkClasspath
protected List<String> frameworkClasspath(SimpleVersion topoVersion)
-
getWorkerMain
protected String getWorkerMain(SimpleVersion topoVersion)
-
getWorkerLogWriter
protected String getWorkerLogWriter(SimpleVersion topoVersion)
-
getWorkerClassPath
protected String getWorkerClassPath(String stormJar, List<String> dependencyLocations, SimpleVersion topoVersion)
Compute the classpath for the worker process.- Parameters:
stormJar- the topology jardependencyLocations- any dependencies from the topologytopoVersion- the version of the storm framework to use- Returns:
- the full classpath
-
substituteChildopts
protected List<String> substituteChildopts(Object value, int memOnheap, int memOffHeap)
-
isMemoryLimitViolated
public boolean isMemoryLimitViolated(LocalAssignment withUpdatedLimits) throws IOException
Description copied from class:ContainerCheck if the container is over its memory limit AND needs to be killed. This does not necessarily mean that it just went over the limit.- Overrides:
isMemoryLimitViolatedin classContainer- Throws:
IOException- on any error
-
getMemoryUsageMb
public long getMemoryUsageMb()
Description copied from class:ContainerGet the current memory usage of this container.- Overrides:
getMemoryUsageMbin classContainer
-
getMemoryReservationMb
public long getMemoryReservationMb()
Description copied from class:ContainerGet the current memory reservation of this container.- Overrides:
getMemoryReservationMbin classContainer
-
launch
public void launch() throws IOExceptionDescription copied from class:ContainerLaunch the process for the first time. PREREQUISITE: setup has run and passed- Specified by:
launchin classContainer- Throws:
IOException- on any error
-
-