Package org.jsoup.helper
Class HttpConnection
- java.lang.Object
-
- org.jsoup.helper.HttpConnection
-
- All Implemented Interfaces:
Connection
public class HttpConnection extends java.lang.Object implements Connection
Implementation ofConnection
.- See Also:
Jsoup.connect(String)
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
HttpConnection.Base<T extends Connection.Base>
static class
HttpConnection.KeyVal
static class
HttpConnection.Request
static class
HttpConnection.Response
-
Nested classes/interfaces inherited from interface org.jsoup.Connection
Connection.Method
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
CONTENT_ENCODING
static java.lang.String
CONTENT_TYPE
static java.lang.String
DEFAULT_UA
Many users would get caught by not setting a user-agent and therefore getting different responses on their desktop vs in jsoup, which would otherwise default toJava
.private static java.lang.String
DefaultUploadType
static java.lang.String
FORM_URL_ENCODED
private static int
HTTP_TEMP_REDIR
static java.lang.String
MULTIPART_FORM_DATA
private Connection.Request
req
private Connection.Response
res
private static java.lang.String
USER_AGENT
-
Constructor Summary
Constructors Constructor Description HttpConnection()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static Connection
connect(java.lang.String url)
static Connection
connect(java.net.URL url)
Connection
cookie(java.lang.String name, java.lang.String value)
Set a cookie to be sent in the request.Connection
cookies(java.util.Map<java.lang.String,java.lang.String> cookies)
Adds each of the supplied cookies to the request.Connection.KeyVal
data(java.lang.String key)
Get the data KeyVal for this key, if anyConnection
data(java.lang.String... keyvals)
Add a number of request data parameters.Connection
data(java.lang.String key, java.lang.String value)
Add a request data parameter.Connection
data(java.lang.String key, java.lang.String filename, java.io.InputStream inputStream)
Add an input stream as a request data parameter.Connection
data(java.lang.String key, java.lang.String filename, java.io.InputStream inputStream, java.lang.String contentType)
Add an input stream as a request data parameter.Connection
data(java.util.Collection<Connection.KeyVal> data)
Adds all of the supplied data to the request data parametersConnection
data(java.util.Map<java.lang.String,java.lang.String> data)
Adds all of the supplied data to the request data parametersprivate static java.lang.String
encodeMimeName(java.lang.String val)
private static java.lang.String
encodeUrl(java.lang.String url)
Encodes the input URL into a safe ASCII URL string(package private) static java.net.URL
encodeUrl(java.net.URL u)
Connection.Response
execute()
Execute the request.Connection
followRedirects(boolean followRedirects)
Configures the connection to (not) follow server redirects.Document
get()
Execute the request as a GET, and parse the result.Connection
header(java.lang.String name, java.lang.String value)
Set a request header.Connection
headers(java.util.Map<java.lang.String,java.lang.String> headers)
Adds each of the supplied headers to the request.Connection
ignoreContentType(boolean ignoreContentType)
Ignore the document's Content-Type when parsing the response.Connection
ignoreHttpErrors(boolean ignoreHttpErrors)
Configures the connection to not throw exceptions when a HTTP error occurs.Connection
maxBodySize(int bytes)
Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated.Connection
method(Connection.Method method)
Set the request method to use, GET or POST.private static boolean
needsMultipart(Connection.Request req)
Connection
parser(Parser parser)
Provide an alternate parser to use when parsing the response to a Document.Document
post()
Execute the request as a POST, and parse the result.Connection
postDataCharset(java.lang.String charset)
Sets the default post data character set for x-www-form-urlencoded post dataConnection
proxy(java.lang.String host, int port)
Set the HTTP proxy to use for this request.Connection
proxy(java.net.Proxy proxy)
Set the proxy to use for this request.Connection
referrer(java.lang.String referrer)
Set the request referrer (aka "referer") header.Connection.Request
request()
Get the request object associated with this connectionConnection
request(Connection.Request request)
Set the connection's requestConnection
requestBody(java.lang.String body)
Set a POST (or PUT) request body.Connection.Response
response()
Get the response, once the request has been executedConnection
response(Connection.Response response)
Set the connection's responseConnection
sslSocketFactory(javax.net.ssl.SSLSocketFactory sslSocketFactory)
Set custom SSL socket factoryConnection
timeout(int millis)
Set the total request timeout duration.Connection
url(java.lang.String url)
Set the request URL to fetch.Connection
url(java.net.URL url)
Set the request URL to fetch.Connection
userAgent(java.lang.String userAgent)
Set the request user-agent header.
-
-
-
Field Detail
-
CONTENT_ENCODING
public static final java.lang.String CONTENT_ENCODING
- See Also:
- Constant Field Values
-
DEFAULT_UA
public static final java.lang.String DEFAULT_UA
Many users would get caught by not setting a user-agent and therefore getting different responses on their desktop vs in jsoup, which would otherwise default toJava
. So by default, use a desktop UA.- See Also:
- Constant Field Values
-
USER_AGENT
private static final java.lang.String USER_AGENT
- See Also:
- Constant Field Values
-
CONTENT_TYPE
public static final java.lang.String CONTENT_TYPE
- See Also:
- Constant Field Values
-
MULTIPART_FORM_DATA
public static final java.lang.String MULTIPART_FORM_DATA
- See Also:
- Constant Field Values
-
FORM_URL_ENCODED
public static final java.lang.String FORM_URL_ENCODED
- See Also:
- Constant Field Values
-
HTTP_TEMP_REDIR
private static final int HTTP_TEMP_REDIR
- See Also:
- Constant Field Values
-
DefaultUploadType
private static final java.lang.String DefaultUploadType
- See Also:
- Constant Field Values
-
req
private Connection.Request req
-
res
private Connection.Response res
-
-
Method Detail
-
connect
public static Connection connect(java.lang.String url)
-
connect
public static Connection connect(java.net.URL url)
-
encodeUrl
private static java.lang.String encodeUrl(java.lang.String url)
Encodes the input URL into a safe ASCII URL string- Parameters:
url
- unescaped URL- Returns:
- escaped URL
-
encodeUrl
static java.net.URL encodeUrl(java.net.URL u)
-
encodeMimeName
private static java.lang.String encodeMimeName(java.lang.String val)
-
url
public Connection url(java.net.URL url)
Description copied from interface:Connection
Set the request URL to fetch. The protocol must be HTTP or HTTPS.- Specified by:
url
in interfaceConnection
- Parameters:
url
- URL to connect to- Returns:
- this Connection, for chaining
-
url
public Connection url(java.lang.String url)
Description copied from interface:Connection
Set the request URL to fetch. The protocol must be HTTP or HTTPS.- Specified by:
url
in interfaceConnection
- Parameters:
url
- URL to connect to- Returns:
- this Connection, for chaining
-
proxy
public Connection proxy(java.net.Proxy proxy)
Description copied from interface:Connection
Set the proxy to use for this request. Set tonull
to disable.- Specified by:
proxy
in interfaceConnection
- Parameters:
proxy
- proxy to use- Returns:
- this Connection, for chaining
-
proxy
public Connection proxy(java.lang.String host, int port)
Description copied from interface:Connection
Set the HTTP proxy to use for this request.- Specified by:
proxy
in interfaceConnection
- Parameters:
host
- the proxy hostnameport
- the proxy port- Returns:
- this Connection, for chaining
-
userAgent
public Connection userAgent(java.lang.String userAgent)
Description copied from interface:Connection
Set the request user-agent header.- Specified by:
userAgent
in interfaceConnection
- Parameters:
userAgent
- user-agent to use- Returns:
- this Connection, for chaining
- See Also:
DEFAULT_UA
-
timeout
public Connection timeout(int millis)
Description copied from interface:Connection
Set the total request timeout duration. If a timeout occurs, anSocketTimeoutException
will be thrown.The default timeout is 30 seconds (30,000 millis). A timeout of zero is treated as an infinite timeout.
Note that this timeout specifies the combined maximum duration of the connection time and the time to read the full response.
- Specified by:
timeout
in interfaceConnection
- Parameters:
millis
- number of milliseconds (thousandths of a second) before timing out connects or reads.- Returns:
- this Connection, for chaining
- See Also:
Connection.maxBodySize(int)
-
maxBodySize
public Connection maxBodySize(int bytes)
Description copied from interface:Connection
Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated. The default maximum is 1MB. A max size of zero is treated as an infinite amount (bounded only by your patience and the memory available on your machine).- Specified by:
maxBodySize
in interfaceConnection
- Parameters:
bytes
- number of bytes to read from the input before truncating- Returns:
- this Connection, for chaining
-
followRedirects
public Connection followRedirects(boolean followRedirects)
Description copied from interface:Connection
Configures the connection to (not) follow server redirects. By default this is true.- Specified by:
followRedirects
in interfaceConnection
- Parameters:
followRedirects
- true if server redirects should be followed.- Returns:
- this Connection, for chaining
-
referrer
public Connection referrer(java.lang.String referrer)
Description copied from interface:Connection
Set the request referrer (aka "referer") header.- Specified by:
referrer
in interfaceConnection
- Parameters:
referrer
- referrer to use- Returns:
- this Connection, for chaining
-
method
public Connection method(Connection.Method method)
Description copied from interface:Connection
Set the request method to use, GET or POST. Default is GET.- Specified by:
method
in interfaceConnection
- Parameters:
method
- HTTP request method- Returns:
- this Connection, for chaining
-
ignoreHttpErrors
public Connection ignoreHttpErrors(boolean ignoreHttpErrors)
Description copied from interface:Connection
Configures the connection to not throw exceptions when a HTTP error occurs. (4xx - 5xx, e.g. 404 or 500). By default this is false; an IOException is thrown if an error is encountered. If set to true, the response is populated with the error body, and the status message will reflect the error.- Specified by:
ignoreHttpErrors
in interfaceConnection
- Parameters:
ignoreHttpErrors
- - false (default) if HTTP errors should be ignored.- Returns:
- this Connection, for chaining
-
ignoreContentType
public Connection ignoreContentType(boolean ignoreContentType)
Description copied from interface:Connection
Ignore the document's Content-Type when parsing the response. By default this is false, an unrecognised content-type will cause an IOException to be thrown. (This is to prevent producing garbage by attempting to parse a JPEG binary image, for example.) Set to true to force a parse attempt regardless of content type.- Specified by:
ignoreContentType
in interfaceConnection
- Parameters:
ignoreContentType
- set to true if you would like the content type ignored on parsing the response into a Document.- Returns:
- this Connection, for chaining
-
data
public Connection data(java.lang.String key, java.lang.String value)
Description copied from interface:Connection
Add a request data parameter. Request parameters are sent in the request query string for GETs, and in the request body for POSTs. A request may have multiple values of the same name.- Specified by:
data
in interfaceConnection
- Parameters:
key
- data keyvalue
- data value- Returns:
- this Connection, for chaining
-
sslSocketFactory
public Connection sslSocketFactory(javax.net.ssl.SSLSocketFactory sslSocketFactory)
Description copied from interface:Connection
Set custom SSL socket factory- Specified by:
sslSocketFactory
in interfaceConnection
- Parameters:
sslSocketFactory
- custom SSL socket factory- Returns:
- this Connection, for chaining
-
data
public Connection data(java.lang.String key, java.lang.String filename, java.io.InputStream inputStream)
Description copied from interface:Connection
Add an input stream as a request data parameter. For GETs, has no effect, but for POSTS this will upload the input stream.- Specified by:
data
in interfaceConnection
- Parameters:
key
- data key (form item name)filename
- the name of the file to present to the remove server. Typically just the name, not path, component.inputStream
- the input stream to upload, that you probably obtained from aFileInputStream
. You must close the InputStream in afinally
block.- Returns:
- this Connections, for chaining
- See Also:
if you want to set the uploaded file's mimetype.
-
data
public Connection data(java.lang.String key, java.lang.String filename, java.io.InputStream inputStream, java.lang.String contentType)
Description copied from interface:Connection
Add an input stream as a request data parameter. For GETs, has no effect, but for POSTS this will upload the input stream.- Specified by:
data
in interfaceConnection
- Parameters:
key
- data key (form item name)filename
- the name of the file to present to the remove server. Typically just the name, not path, component.inputStream
- the input stream to upload, that you probably obtained from aFileInputStream
.contentType
- the Content Type (aka mimetype) to specify for this file. You must close the InputStream in afinally
block.- Returns:
- this Connections, for chaining
-
data
public Connection data(java.util.Map<java.lang.String,java.lang.String> data)
Description copied from interface:Connection
Adds all of the supplied data to the request data parameters- Specified by:
data
in interfaceConnection
- Parameters:
data
- map of data parameters- Returns:
- this Connection, for chaining
-
data
public Connection data(java.lang.String... keyvals)
Description copied from interface:Connection
Add a number of request data parameters. Multiple parameters may be set at once, e.g.:.data("name", "jsoup", "language", "Java", "language", "English");
creates a query string like:?name=jsoup&language=Java&language=English
- Specified by:
data
in interfaceConnection
- Parameters:
keyvals
- a set of key value pairs.- Returns:
- this Connection, for chaining
-
data
public Connection data(java.util.Collection<Connection.KeyVal> data)
Description copied from interface:Connection
Adds all of the supplied data to the request data parameters- Specified by:
data
in interfaceConnection
- Parameters:
data
- collection of data parameters- Returns:
- this Connection, for chaining
-
data
public Connection.KeyVal data(java.lang.String key)
Description copied from interface:Connection
Get the data KeyVal for this key, if any- Specified by:
data
in interfaceConnection
- Parameters:
key
- the data key- Returns:
- null if not set
-
requestBody
public Connection requestBody(java.lang.String body)
Description copied from interface:Connection
Set a POST (or PUT) request body. Useful when a server expects a plain request body, not a set for URL encoded form key/value pairs. E.g.:
If any data key/vals are supplied, they will be sent as URL query params.Jsoup.connect(url) .requestBody(json) .header("Content-Type", "application/json") .post();
- Specified by:
requestBody
in interfaceConnection
- Returns:
- this Request, for chaining
-
header
public Connection header(java.lang.String name, java.lang.String value)
Description copied from interface:Connection
Set a request header.- Specified by:
header
in interfaceConnection
- Parameters:
name
- header namevalue
- header value- Returns:
- this Connection, for chaining
- See Also:
Connection.Base.headers()
-
headers
public Connection headers(java.util.Map<java.lang.String,java.lang.String> headers)
Description copied from interface:Connection
Adds each of the supplied headers to the request.- Specified by:
headers
in interfaceConnection
- Parameters:
headers
- map of headers name -> value pairs- Returns:
- this Connection, for chaining
- See Also:
Connection.Base.headers()
-
cookie
public Connection cookie(java.lang.String name, java.lang.String value)
Description copied from interface:Connection
Set a cookie to be sent in the request.- Specified by:
cookie
in interfaceConnection
- Parameters:
name
- name of cookievalue
- value of cookie- Returns:
- this Connection, for chaining
-
cookies
public Connection cookies(java.util.Map<java.lang.String,java.lang.String> cookies)
Description copied from interface:Connection
Adds each of the supplied cookies to the request.- Specified by:
cookies
in interfaceConnection
- Parameters:
cookies
- map of cookie name -> value pairs- Returns:
- this Connection, for chaining
-
parser
public Connection parser(Parser parser)
Description copied from interface:Connection
Provide an alternate parser to use when parsing the response to a Document. If not set, defaults to the HTML parser, unless the response content-type is XML, in which case the XML parser is used.- Specified by:
parser
in interfaceConnection
- Parameters:
parser
- alternate parser- Returns:
- this Connection, for chaining
-
get
public Document get() throws java.io.IOException
Description copied from interface:Connection
Execute the request as a GET, and parse the result.- Specified by:
get
in interfaceConnection
- Returns:
- parsed Document
- Throws:
java.net.MalformedURLException
- if the request URL is not a HTTP or HTTPS URL, or is otherwise malformedHttpStatusException
- if the response is not OK and HTTP response errors are not ignoredUnsupportedMimeTypeException
- if the response mime type is not supported and those errors are not ignoredjava.net.SocketTimeoutException
- if the connection times outjava.io.IOException
- on error
-
post
public Document post() throws java.io.IOException
Description copied from interface:Connection
Execute the request as a POST, and parse the result.- Specified by:
post
in interfaceConnection
- Returns:
- parsed Document
- Throws:
java.net.MalformedURLException
- if the request URL is not a HTTP or HTTPS URL, or is otherwise malformedHttpStatusException
- if the response is not OK and HTTP response errors are not ignoredUnsupportedMimeTypeException
- if the response mime type is not supported and those errors are not ignoredjava.net.SocketTimeoutException
- if the connection times outjava.io.IOException
- on error
-
execute
public Connection.Response execute() throws java.io.IOException
Description copied from interface:Connection
Execute the request.- Specified by:
execute
in interfaceConnection
- Returns:
- a response object
- Throws:
java.net.MalformedURLException
- if the request URL is not a HTTP or HTTPS URL, or is otherwise malformedHttpStatusException
- if the response is not OK and HTTP response errors are not ignoredUnsupportedMimeTypeException
- if the response mime type is not supported and those errors are not ignoredjava.net.SocketTimeoutException
- if the connection times outjava.io.IOException
- on error
-
request
public Connection.Request request()
Description copied from interface:Connection
Get the request object associated with this connection- Specified by:
request
in interfaceConnection
- Returns:
- request
-
request
public Connection request(Connection.Request request)
Description copied from interface:Connection
Set the connection's request- Specified by:
request
in interfaceConnection
- Parameters:
request
- new request object- Returns:
- this Connection, for chaining
-
response
public Connection.Response response()
Description copied from interface:Connection
Get the response, once the request has been executed- Specified by:
response
in interfaceConnection
- Returns:
- response
-
response
public Connection response(Connection.Response response)
Description copied from interface:Connection
Set the connection's response- Specified by:
response
in interfaceConnection
- Parameters:
response
- new response- Returns:
- this Connection, for chaining
-
postDataCharset
public Connection postDataCharset(java.lang.String charset)
Description copied from interface:Connection
Sets the default post data character set for x-www-form-urlencoded post data- Specified by:
postDataCharset
in interfaceConnection
- Parameters:
charset
- character set to encode post data- Returns:
- this Connection, for chaining
-
needsMultipart
private static boolean needsMultipart(Connection.Request req)
-
-