Package it.unimi.di.law.warc.records

Class AbstractWarcRecord

java.lang.Object

org.apache.http.message.AbstractHttpMessage

it.unimi.di.law.warc.records.AbstractWarcRecord

All Implemented Interfaces:

WarcRecord, org.apache.http.HttpMessage

Direct Known Subclasses:

HttpRequestWarcRecord, HttpResponseWarcRecord, InfoWarcRecord

public abstract class AbstractWarcRecord
extends org.apache.http.message.AbstractHttpMessage
implements WarcRecord

An abstract implementation of a basic WarcRecord.

Nested Class Summary

Nested classes/interfaces inherited from interface it.unimi.di.law.warc.records.WarcRecord

WarcRecord.Type

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	USE_BURL_PROPERTY
protected org.apache.http.message.HeaderGroup	warcHeaders

Fields inherited from class org.apache.http.message.AbstractHttpMessage

headergroup, params

Fields inherited from interface it.unimi.di.law.warc.records.WarcRecord

PROTOCOL_VERSION

Constructor Summary

Constructors

Constructor	Description
AbstractWarcRecord(java.net.URI targetURI, org.apache.http.message.HeaderGroup warcHeaders)	BUilds a record, optionally given the target URI and the warcHeaders.
AbstractWarcRecord(org.apache.http.message.HeaderGroup warcHeaders)	BUilds a record, optionally given the warcHeaders.

Method Summary

Modifier and Type	Method	Description
static WarcRecord	fromPayload(org.apache.http.message.HeaderGroup warcHeaders, BoundSessionInputBuffer payloadBuffer)
org.apache.http.ProtocolVersion	getProtocolVersion()
long	getWarcContentLength()	Returns the WARC Content-Length header.
java.util.Date	getWarcDate()	Returns the WARC-Date header.
org.apache.http.Header	getWarcHeader(WarcHeader.Name header)	Returns the specified WARC header.
org.apache.http.message.HeaderGroup	getWarcHeaders()	Returns the WARC headers.
java.util.UUID	getWarcRecordId()	Returns the WARC-Record-ID header.
java.net.URI	getWarcTargetURI()	Returns the WARC-Target-URI header as a URI.
WarcRecord.Type	getWarcType()	Returns the WARC-Type header.
void	write(java.io.OutputStream output, ByteArraySessionOutputBuffer buffer)	Writes the WARC record.
protected static void	writeHeaders(org.apache.http.message.HeaderGroup headers, java.io.OutputStream output)
protected abstract java.io.InputStream	writePayload(ByteArraySessionOutputBuffer buffer)

Methods inherited from class org.apache.http.message.AbstractHttpMessage

addHeader, addHeader, containsHeader, getAllHeaders, getFirstHeader, getHeaders, getLastHeader, getParams, headerIterator, headerIterator, removeHeader, removeHeaders, setHeader, setHeader, setHeaders, setParams

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

USE_BURL_PROPERTY

public static final java.lang.String USE_BURL_PROPERTY

See Also:

Constant Field Values

warcHeaders

protected final org.apache.http.message.HeaderGroup warcHeaders

Constructor Detail

AbstractWarcRecord

public AbstractWarcRecord(org.apache.http.message.HeaderGroup warcHeaders)

BUilds a record, optionally given the warcHeaders.

Parameters:

warcHeaders - the WARC headers, may be null.

See Also:

AbstractWarcRecord(URI,HeaderGroup)

AbstractWarcRecord

public AbstractWarcRecord(java.net.URI targetURI,
                          org.apache.http.message.HeaderGroup warcHeaders)

BUilds a record, optionally given the target URI and the warcHeaders. If the headers are null or the WarcHeader.Name.WARC_RECORD_ID header is absent, it will be generated at random, similarly if the headers are null or the WarcHeader.Name.WARC_DATE header absent, it will be set to the current time. If the target URI is not null and the WarcHeader.Name.WARC_TARGET_URI header is not set, it will be set to the given vaule.

Parameters:

targetURI - the target URI, may be null.

warcHeaders - the WARC headers, may be null.

Method Detail

getProtocolVersion

public org.apache.http.ProtocolVersion getProtocolVersion()

Specified by:

getProtocolVersion in interface org.apache.http.HttpMessage

getWarcHeaders

public org.apache.http.message.HeaderGroup getWarcHeaders()

Description copied from interface: WarcRecord

Returns the WARC headers.

Specified by:

getWarcHeaders in interface WarcRecord

Returns:

the WARC headers.

getWarcHeader

public org.apache.http.Header getWarcHeader(WarcHeader.Name header)

Description copied from interface: WarcRecord

Returns the specified WARC header.

Specified by:

getWarcHeader in interface WarcRecord

Parameters:

header - the name of the header to return.

Returns:

the header, or null if the header is not present.

getWarcRecordId

public java.util.UUID getWarcRecordId()

Description copied from interface: WarcRecord

Returns the WARC-Record-ID header.

Specified by:

getWarcRecordId in interface WarcRecord

Returns:

the record UUID.

getWarcType

public WarcRecord.Type getWarcType()

Description copied from interface: WarcRecord

Returns the WARC-Type header.

Specified by:

getWarcType in interface WarcRecord

Returns:

the record type.

getWarcDate

public java.util.Date getWarcDate()

Description copied from interface: WarcRecord

Returns the WARC-Date header.

Specified by:

getWarcDate in interface WarcRecord

Returns:

the record creation date.

getWarcContentLength

public long getWarcContentLength()

Description copied from interface: WarcRecord

Returns the WARC Content-Length header.

Specified by:

getWarcContentLength in interface WarcRecord

Returns:

the record payload length.

getWarcTargetURI

public java.net.URI getWarcTargetURI()

Returns the WARC-Target-URI header as a URI.

Parsing is performed by URI.create(String), unless the system property "it.unimi.di.law.warc.records.useburl" has been set to true, in which case BURL.parse(String) will be used (if BURL.parse(String) returns null, we throw an IllegalArgumentException as from the specification in WarcRecord.getWarcTargetURI()).

Specified by:

getWarcTargetURI in interface WarcRecord

Returns:

the record target URI.

Throws:

java.lang.IllegalStateException - if the header is not present.

java.lang.IllegalArgumentException - if the header value cannot be parsed into a URI.

See Also:

getWarcTargetURI()

writePayload

protected abstract java.io.InputStream writePayload(ByteArraySessionOutputBuffer buffer)
                                             throws java.io.IOException

Throws:

java.io.IOException

writeHeaders

protected static void writeHeaders(org.apache.http.message.HeaderGroup headers,
                                   java.io.OutputStream output)
                            throws java.io.IOException

Throws:

java.io.IOException

write

public void write(java.io.OutputStream output,
                  ByteArraySessionOutputBuffer buffer)
           throws java.io.IOException

Description copied from interface: WarcRecord

Writes the WARC record.

Specified by:

write in interface WarcRecord

Parameters:

output - the stream where to write the record.

buffer - a buffer that will be optionally used by the writer.

Throws:

java.io.IOException

fromPayload

public static WarcRecord fromPayload(org.apache.http.message.HeaderGroup warcHeaders,
                                     BoundSessionInputBuffer payloadBuffer)
                              throws java.io.IOException,
                                     WarcFormatException

Throws:

java.io.IOException

WarcFormatException