Package it.unimi.di.law.bubing.parser

Interface Parser<T>

All Superinterfaces:

Filter<URIResponse>, FlyweightPrototype<Filter<URIResponse>>, com.google.common.base.Predicate<URIResponse>, java.util.function.Predicate<URIResponse>

All Known Implementing Classes:

BinaryParser, HTMLParser

public interface Parser<T>
extends Filter<URIResponse>

A generic parser for responses. Every parser provides the following functionalities:

• it acts as a Filter that is able to decide whether it can parse a certain URIResponse or not (e.g., based on the declared content-type header etc.);
• while parsing, it will send the links found in the document to the specified Parser.LinkReceiver, that will typically accumulate them or send them to the appropriate class for processing;
• the parsing method will return a digest computed on a (possibly) suitably modified version of the document (the way in which the document it is actually modified and the way in which the hash is computed is implementation-dependent and should be commented by the implementing classes);
• after parsing, a guess of the charset used for the document will be made available.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	Parser.LinkReceiver	A class that can receive URLs discovered during parsing.
static interface	Parser.TextProcessor<T>	A class that can receive piece of text discovered during parsing.

Field Summary

Fields

Modifier and Type	Field	Description
static Parser.LinkReceiver	NULL_LINK_RECEIVER	A no-op implementation of Parser.LinkReceiver.

Fields inherited from interface it.unimi.di.law.warc.filters.Filter

FILTER_PACKAGE_NAME

Method Summary

Modifier and Type	Method	Description
Parser<T>	copy()	This method strengthens the return type of the method inherited from Filter.
java.lang.String	guessedCharset()	Returns a guessed charset for the document, or null if the charset could not be guessed.
byte[]	parse(java.net.URI uri, org.apache.http.HttpResponse response, Parser.LinkReceiver linkReceiver)	Parses a response.
T	result()	Returns the result of the processing.

Methods inherited from interface com.google.common.base.Predicate

apply, equals, test

Methods inherited from interface java.util.function.Predicate

and, negate, or

Field Detail

NULL_LINK_RECEIVER

static final Parser.LinkReceiver NULL_LINK_RECEIVER

A no-op implementation of Parser.LinkReceiver.

Method Detail

parse

byte[] parse(java.net.URI uri,
             org.apache.http.HttpResponse response,
             Parser.LinkReceiver linkReceiver)
      throws java.io.IOException

Parses a response.

Parameters:

response - a response to parse.

linkReceiver - a link receiver.

Returns:

a digest of the page content, or null if no digest has been computed.

Throws:

java.io.IOException

guessedCharset

java.lang.String guessedCharset()

Returns a guessed charset for the document, or null if the charset could not be guessed.

Returns:

a charset or null.

result

T result()

Returns the result of the processing.

Note that this method must be idempotent.

Returns:

the result of the processing.

copy

Parser<T> copy()

This method strengthens the return type of the method inherited from Filter.

Specified by:

copy in interface FlyweightPrototype<T>

Returns:

a copy of this object, sharing state with this object as much as possible.