Package it.unimi.di.law.warc.filters

Class IsProbablyBinary

java.lang.Object

it.unimi.di.law.warc.filters.AbstractFilter<org.apache.http.HttpResponse>

it.unimi.di.law.warc.filters.IsProbablyBinary

All Implemented Interfaces:

com.google.common.base.Predicate<org.apache.http.HttpResponse>, Filter<org.apache.http.HttpResponse>, FlyweightPrototype<Filter<org.apache.http.HttpResponse>>, java.util.function.Predicate<org.apache.http.HttpResponse>

public class IsProbablyBinary
extends AbstractFilter<org.apache.http.HttpResponse>

A filter accepting only http responses whose content stream appears to be binary.

Field Summary

Fields

Modifier and Type	Field	Description
static int	BINARY_CHECK_SCAN_LENGTH
static IsProbablyBinary	INSTANCE
static int	THRESHOLD	The number of zeroes that must appear to cause the page to be considered probably binary.

Fields inherited from interface it.unimi.di.law.warc.filters.Filter

FILTER_PACKAGE_NAME

Method Summary

Modifier and Type	Method	Description
boolean	apply(org.apache.http.HttpResponse httpResponse)	This method implements a simple heuristic for guessing whether a page is binary.
Filter<org.apache.http.HttpResponse>	copy()
java.lang.String	toString()	A string representation of the state of this filter.
static IsProbablyBinary	valueOf()	Get a new IsProbablyBinary that will accept only http responses whose content stream appears to be binary.
static IsProbablyBinary	valueOf(java.lang.String emptySpec)	Deprecated. Please use valueOf() instead.

Methods inherited from class it.unimi.di.law.warc.filters.AbstractFilter

toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.google.common.base.Predicate

equals, test

Methods inherited from interface java.util.function.Predicate

and, negate, or

Field Detail

INSTANCE

public static final IsProbablyBinary INSTANCE

BINARY_CHECK_SCAN_LENGTH

public static final int BINARY_CHECK_SCAN_LENGTH

See Also:

Constant Field Values

THRESHOLD

public static final int THRESHOLD

The number of zeroes that must appear to cause the page to be considered probably binary. Some misconfigured servers emit one or two ASCII NULs at the start of their pages, so we use a relatively safe value.

See Also:

Constant Field Values

Method Detail

apply

public boolean apply(org.apache.http.HttpResponse httpResponse)

This method implements a simple heuristic for guessing whether a page is binary.

The first BINARY_CHECK_SCAN_LENGTH bytes are scanned: if we find more than THRESHOLD zeroes, we deduce that this page is binary. Note that this works also with UTF-8, as no UTF-8 legal character encoding contains these characters (unless you're encoding 0, but this is not our case).

Returns:

true if this page has most probably a binary content.

Throws:

java.lang.NullPointerException - if the page has no byte content.

valueOf

@Deprecated
public static IsProbablyBinary valueOf(java.lang.String emptySpec)

Deprecated.

Please use valueOf() instead.

Get a new IsProbablyBinary that will accept only http responses whose content stream appears to be binary.

Parameters:

emptySpec - an empty string.

Returns:

a new IsProbablyBinary that will accept only http responses whose content stream appears to be binary.

valueOf

public static IsProbablyBinary valueOf()

Get a new IsProbablyBinary that will accept only http responses whose content stream appears to be binary.

Returns:

a new IsProbablyBinary that will accept only http responses whose content stream appears to be binary.

toString

public java.lang.String toString()

A string representation of the state of this filter.

Overrides:

toString in class java.lang.Object

Returns:

the class name of this + "()"

copy

public Filter<org.apache.http.HttpResponse> copy()