Package it.unimi.di.law.bubing.parser

Class BinaryParser

java.lang.Object

it.unimi.di.law.bubing.parser.BinaryParser

All Implemented Interfaces:

com.google.common.base.Predicate<URIResponse>, Parser<java.lang.Void>, Filter<URIResponse>, FlyweightPrototype<Filter<URIResponse>>, java.util.function.Predicate<URIResponse>

public class BinaryParser
extends java.lang.Object
implements Parser<java.lang.Void>

A universal binary parser that just computes digests.

Nested Class Summary

Nested classes/interfaces inherited from interface it.unimi.di.law.bubing.parser.Parser

Parser.LinkReceiver, Parser.TextProcessor<T>

Field Summary

Fields inherited from interface it.unimi.di.law.warc.filters.Filter

FILTER_PACKAGE_NAME

Fields inherited from interface it.unimi.di.law.bubing.parser.Parser

NULL_LINK_RECEIVER

Constructor Summary

Constructors

Constructor	Description
BinaryParser(com.google.common.hash.HashFunction hashFunction)	Builds a parser for digesting a page.
BinaryParser(com.google.common.hash.HashFunction hashFunction, boolean crossAuthorityDuplicates)
BinaryParser(java.lang.String messageDigestAlgorithm)	Builds a parser for digesting a page.

Method Summary

Modifier and Type	Method	Description
boolean	apply(URIResponse response)
java.lang.Object	clone()
BinaryParser	copy()	This method strengthens the return type of the method inherited from Filter.
static com.google.common.hash.HashFunction	forName(java.lang.String messageDigest)	Return the hash function corresponding to a given message-digest algorithm given by name.
java.lang.String	guessedCharset()	Returns a guessed charset for the document, or null if the charset could not be guessed.
com.google.common.hash.Hasher	init(java.net.URI url)
byte[]	parse(java.net.URI uri, org.apache.http.HttpResponse httpResponse, Parser.LinkReceiver linkReceiver)	Parses a response.
java.lang.Void	result()	Returns the result of the processing.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.google.common.base.Predicate

equals, test

Methods inherited from interface java.util.function.Predicate

and, negate, or

Constructor Detail

BinaryParser

public BinaryParser(com.google.common.hash.HashFunction hashFunction)

Builds a parser for digesting a page.

Parameters:

hashFunction - the hash function used to digest, null if no digesting will be performed.

BinaryParser

public BinaryParser(com.google.common.hash.HashFunction hashFunction,
                    boolean crossAuthorityDuplicates)

BinaryParser

public BinaryParser(java.lang.String messageDigestAlgorithm)
             throws java.security.NoSuchAlgorithmException

Builds a parser for digesting a page.

Parameters:

messageDigestAlgorithm - the digesting algorithm (as a string).

Throws:

java.security.NoSuchAlgorithmException

Method Detail

forName

public static final com.google.common.hash.HashFunction forName(java.lang.String messageDigest)
                                                         throws java.security.NoSuchAlgorithmException

Return the hash function corresponding to a given message-digest algorithm given by name.

Parameters:

messageDigest - a message-digest algorithm (e.g., MurmurHash3 or MD5); null if messageDigest is the empty string.

Throws:

java.security.NoSuchAlgorithmException

init

public com.google.common.hash.Hasher init(java.net.URI url)

parse

public byte[] parse(java.net.URI uri,
                    org.apache.http.HttpResponse httpResponse,
                    Parser.LinkReceiver linkReceiver)
             throws java.io.IOException

Description copied from interface: Parser

Parses a response.

Specified by:

parse in interface Parser<java.lang.Void>

httpResponse - a response to parse.

linkReceiver - a link receiver.

Returns:

a digest of the page content, or null if no digest has been computed.

Throws:

java.io.IOException

apply

public boolean apply(URIResponse response)

Specified by:

apply in interface com.google.common.base.Predicate<URIResponse>

clone

public java.lang.Object clone()

Overrides:

clone in class java.lang.Object

guessedCharset

public java.lang.String guessedCharset()

Description copied from interface: Parser

Returns a guessed charset for the document, or null if the charset could not be guessed.

Specified by:

guessedCharset in interface Parser<java.lang.Void>

Returns:

a charset or null.

copy

public BinaryParser copy()

Description copied from interface: Parser

This method strengthens the return type of the method inherited from Filter.

Specified by:

copy in interface FlyweightPrototype<Filter<URIResponse>>

Specified by:

copy in interface Parser<java.lang.Void>

Returns:

a copy of this object, sharing state with this object as much as possible.

result

public java.lang.Void result()

Description copied from interface: Parser

Returns the result of the processing.

Note that this method must be idempotent.

Specified by:

result in interface Parser<java.lang.Void>

Returns:

the result of the processing.