protected static class ThrottledFetcher.ThrottledConnection extends Object implements IThrottledConnection
| Modifier and Type | Field and Description |
|---|---|
protected AbortChecker |
abortCheck
Abort checker
|
protected PageCredentials |
authentication
Authentication
|
protected int |
connectionTimeoutMilliseconds
Connection timeout milliseconds
|
protected org.apache.http.conn.HttpClientConnectionManager |
connManager
The http connection manager.
|
protected long |
expireTime
This is when the connection will expire.
|
protected long |
fetchCounter
The current bytes in the current fetch
|
protected org.apache.http.client.methods.HttpRequestBase |
fetchMethod
The method object
|
protected IFetchThrottler |
fetchThrottler
Fetch throttler
|
protected String |
fetchType
The kind of fetch we are doing
|
protected org.apache.http.client.HttpClient |
httpClient
The http client object.
|
protected SSLSocketFactory |
httpsSocketFactory
Https protocol
|
protected LoginCookies |
lastFetchCookies
The cookies from the last fetch
|
protected ThrottledFetcher.ExecuteMethodThread |
methodThread
The thread that is actually doing the work
|
protected ThrottledFetcher.ConnectionPool |
myPool
Connection pool
|
protected String |
myUrl
The current URL being fetched
|
protected int |
port
Port
|
protected String |
protocol
Protocol
|
protected String |
proxyAuthDomain
Proxy auth domain
|
protected String |
proxyAuthPassword
Proxy auth password
|
protected String |
proxyAuthUsername
Proxy auth user name
|
protected String |
proxyHost
Proxy host
|
protected int |
proxyPort
Proxy port
|
protected String |
server
Server
|
protected int |
socketTimeoutMilliseconds
Socket timeout milliseconds
|
protected long |
startFetchTime
The start of the current fetch
|
protected int |
statusCode
The status code fetched, if any
|
protected boolean |
threadStarted
Set if thread has been started
|
protected Throwable |
throwable
The error trace, if any
|
_rcsid, FETCH_BAD_URI, FETCH_CIRCULAR_REDIRECT, FETCH_INTERRUPTED, FETCH_IO_ERROR, FETCH_NOT_TRIED, FETCH_SEQUENCE_ERROR, FETCH_UNKNOWN_ERROR| Constructor and Description |
|---|
ThrottledFetcher.ThrottledConnection(ThrottledFetcher.ConnectionPool myPool,
IFetchThrottler fetchThrottler,
String protocol,
String server,
int port,
PageCredentials authentication,
SSLSocketFactory httpsSocketFactory,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
int socketTimeoutMilliseconds,
int connectionTimeoutMilliseconds)
Constructor.
|
| Modifier and Type | Method and Description |
|---|---|
void |
beginFetch(String fetchType)
Begin the fetch process.
|
void |
close()
Close the connection.
|
void |
destroy()
Destroy the connection forever
|
void |
doneFetch(IVersionActivity activities)
Done with the fetch.
|
void |
executeFetch(String urlPath,
String userAgent,
String from,
boolean redirectOK,
String host,
FormData formData,
LoginCookies loginCookies)
Execute the fetch and get the return code.
|
LoginCookies |
getLastFetchCookies()
Get the last fetch cookies.
|
String |
getLimitedResponseBody(int maxSize,
String encoding)
Get limited response as a string.
|
InputStream |
getResponseBodyStream()
Get the response input stream.
|
int |
getResponseCode()
Get the http response code.
|
String |
getResponseHeader(String headerName)
Get a specified response header, if it exists.
|
Map<String,List<String>> |
getResponseHeaders()
Get response headers
|
protected void |
handleHTTPException(org.apache.http.HttpException e,
String activity) |
protected void |
handleIOException(IOException e,
String activity) |
boolean |
hasExpired(long currentTime)
Check whether the connection has expired.
|
void |
logFetchCount(int count)
Log the fetch of a number of bytes, from within a stream.
|
void |
noteInterrupted(Throwable e)
Note that the connection fetch was interrupted by something.
|
void |
setAbortChecker(AbortChecker abortCheck)
Set the abort checker.
|
protected final ThrottledFetcher.ConnectionPool myPool
protected final IFetchThrottler fetchThrottler
protected final String protocol
protected final String server
protected final int port
protected final PageCredentials authentication
protected long expireTime
protected org.apache.http.conn.HttpClientConnectionManager connManager
protected org.apache.http.client.HttpClient httpClient
protected org.apache.http.client.methods.HttpRequestBase fetchMethod
protected Throwable throwable
protected String myUrl
protected int statusCode
protected String fetchType
protected long fetchCounter
protected long startFetchTime
protected LoginCookies lastFetchCookies
protected final String proxyHost
protected final int proxyPort
protected final String proxyAuthDomain
protected final String proxyAuthUsername
protected final String proxyAuthPassword
protected final SSLSocketFactory httpsSocketFactory
protected final int socketTimeoutMilliseconds
protected final int connectionTimeoutMilliseconds
protected ThrottledFetcher.ExecuteMethodThread methodThread
protected boolean threadStarted
protected AbortChecker abortCheck
public ThrottledFetcher.ThrottledConnection(ThrottledFetcher.ConnectionPool myPool, IFetchThrottler fetchThrottler, String protocol, String server, int port, PageCredentials authentication, SSLSocketFactory httpsSocketFactory, String proxyHost, int proxyPort, String proxyAuthDomain, String proxyAuthUsername, String proxyAuthPassword, int socketTimeoutMilliseconds, int connectionTimeoutMilliseconds)
public void setAbortChecker(AbortChecker abortCheck)
setAbortChecker in interface IThrottledConnectionpublic boolean hasExpired(long currentTime)
hasExpired in interface IThrottledConnectioncurrentTime - is the current time to use to judge if a connection has expired.public void logFetchCount(int count)
public void destroy()
destroy in interface IThrottledConnectionpublic void beginFetch(String fetchType) throws ManifoldCFException, ServiceInterruption
beginFetch in interface IThrottledConnectionfetchType - is a short descriptive string describing the kind of fetch being requested. This
is used solely for logging purposes.ManifoldCFExceptionServiceInterruptionpublic void executeFetch(String urlPath, String userAgent, String from, boolean redirectOK, String host, FormData formData, LoginCookies loginCookies) throws ManifoldCFException, ServiceInterruption
executeFetch in interface IThrottledConnectionurlPath - is the path part of the url, e.g. "/robots.txt"userAgent - is the value of the userAgent header to use.from - is the value of the from header to use.redirectOK - should be set to true if you want redirects to be automatically followed.host - is the value to use as the "Host" header, or null to use the default.formData - describes additional form arguments and how to fetch the page.loginCookies - describes the cookies that should be in effect for this page fetch.ManifoldCFExceptionServiceInterruptionpublic int getResponseCode()
throws ManifoldCFException,
ServiceInterruption
getResponseCode in interface IThrottledConnectionManifoldCFExceptionServiceInterruptionpublic LoginCookies getLastFetchCookies() throws ManifoldCFException, ServiceInterruption
getLastFetchCookies in interface IThrottledConnectionManifoldCFExceptionServiceInterruptionpublic Map<String,List<String>> getResponseHeaders() throws ManifoldCFException, ServiceInterruption
getResponseHeaders in interface IThrottledConnectionManifoldCFExceptionServiceInterruptionpublic String getResponseHeader(String headerName) throws ManifoldCFException, ServiceInterruption
getResponseHeader in interface IThrottledConnectionheaderName - is the name of the header.ManifoldCFExceptionServiceInterruptionpublic InputStream getResponseBodyStream() throws ManifoldCFException, ServiceInterruption
getResponseBodyStream in interface IThrottledConnectionManifoldCFExceptionServiceInterruptionpublic String getLimitedResponseBody(int maxSize, String encoding) throws ManifoldCFException, ServiceInterruption
getLimitedResponseBody in interface IThrottledConnectionManifoldCFExceptionServiceInterruptionpublic void noteInterrupted(Throwable e)
noteInterrupted in interface IThrottledConnectionpublic void doneFetch(IVersionActivity activities) throws ManifoldCFException
doneFetch in interface IThrottledConnectionManifoldCFExceptionpublic void close()
close in interface IThrottledConnectionprotected void handleHTTPException(org.apache.http.HttpException e,
String activity)
throws ServiceInterruption,
ManifoldCFException
protected void handleIOException(IOException e, String activity) throws ServiceInterruption, ManifoldCFException