Package it.unimi.di.law.bubing.frontier

Class StatsThread

java.lang.Object

it.unimi.di.law.bubing.frontier.StatsThread

All Implemented Interfaces:

java.lang.Runnable

public final class StatsThread
extends java.lang.Object
implements java.lang.Runnable

A class isolating a number of ProgressLogger instances keeping track of a number of quantities of interest related to the Distributor, e.g., requests, transferred byets, etc.

Field Summary

Fields

Modifier and Type	Field	Description
long	brokenPathQueryCount	The number of path+queries living in a broken visit state.
long	brokenVisitStatesOnWorkbench	The number of broken visit states on the workbench.
int[]	dist	A variable used for exponentially-binned distribution of visit state sizes.
SummaryStats	entrySummaryStats	A variable accumulating statistics about the size (in visit states) of workbench entries.
ProgressLogger	receivedURLsLogger	A global progress logger, counting the URLs received from other agents.
ProgressLogger	requestLogger	A global progress logger, measuring the number of completed requests.
long	resolvedVisitStates	The number of resolved visit states.
ProgressLogger	resourceLogger	A global progress logger, measuring the number of non-duplicate resources actually stored.
ProgressLogger	transferredBytesLogger	A global progress logger, measuring the number of transferred bytes.
long	unresolved	The number of path+queries living in an unresolved visit state.

Constructor Summary

Constructors

Constructor	Description
StatsThread(Frontier frontier, Distributor distributor)	Creates the thread.

Method Summary

Modifier and Type	Method	Description
void	done()	Terminates the statistics, closing all the progress loggers.
void	emit()	Emits the statistics.
int	getVisitStates()	Returns the overall number of visit states.
long	getVisitStatesOnDisk()	Returns the number of visit states on disk.
void	run()
void	start(long previousCrawlDuration)	Starst all progress loggers.
static java.lang.String	toString(int[] a)	Returns an integer array as a string, but does not print trailing zeroes.
static java.lang.String	toString(java.util.concurrent.atomic.AtomicLongArray a)	Returns an AtomicLongArray array as a string, but does not print trailing zeroes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

requestLogger

public final ProgressLogger requestLogger

A global progress logger, measuring the number of completed requests.

resourceLogger

public final ProgressLogger resourceLogger

A global progress logger, measuring the number of non-duplicate resources actually stored.

transferredBytesLogger

public final ProgressLogger transferredBytesLogger

A global progress logger, measuring the number of transferred bytes.

receivedURLsLogger

public final ProgressLogger receivedURLsLogger

A global progress logger, counting the URLs received from other agents.

dist

public volatile int[] dist

A variable used for exponentially-binned distribution of visit state sizes.

unresolved

public volatile long unresolved

The number of path+queries living in an unresolved visit state.

brokenPathQueryCount

public volatile long brokenPathQueryCount

The number of path+queries living in a broken visit state.

entrySummaryStats

public volatile SummaryStats entrySummaryStats

A variable accumulating statistics about the size (in visit states) of workbench entries.

resolvedVisitStates

public volatile long resolvedVisitStates

The number of resolved visit states.

brokenVisitStatesOnWorkbench

public volatile long brokenVisitStatesOnWorkbench

The number of broken visit states on the workbench.

Constructor Detail

StatsThread

public StatsThread(Frontier frontier,
                   Distributor distributor)

Creates the thread.

Parameters:

frontier - the frontier instantiating the thread.

distributor - the distributor used.

Method Detail

start

public void start(long previousCrawlDuration)

Starst all progress loggers.

Parameters:

previousCrawlDuration - the duration of the previous crawl, or zero for a new crawl.

toString

public static java.lang.String toString(int[] a)

Returns an integer array as a string, but does not print trailing zeroes.

Parameters:

a - an array.

Returns:

Object.toString() of a, but without trailing zeroes.

toString

public static java.lang.String toString(java.util.concurrent.atomic.AtomicLongArray a)

Returns an AtomicLongArray array as a string, but does not print trailing zeroes.

Parameters:

a - an atomic array.

Returns:

Arrays.toString(long[]) of a, but without trailing zeroes.

emit

public void emit()

Emits the statistics.

run

public void run()

Specified by:

run in interface java.lang.Runnable

getVisitStatesOnDisk

public long getVisitStatesOnDisk()

Returns the number of visit states on disk.

Returns:

the number of visit states on disk.

getVisitStates

public int getVisitStates()

Returns the overall number of visit states.

Returns:

the overall number of visit states.

done

public void done()

Terminates the statistics, closing all the progress loggers.