Package it.unimi.di.law.bubing.frontier

Class WorkbenchVirtualizer

java.lang.Object

it.unimi.di.law.bubing.frontier.WorkbenchVirtualizer

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public class WorkbenchVirtualizer
extends java.lang.Object
implements java.io.Closeable

A workbench virtualizer based on a Berkeley DB database.

An instance of this class acts as a thin layer between the workbench and a set of disk queues, possibly one for each visit state, stored in a Database. Each queue is associated with a scheme+authority (the key). Values are given by an increasing timestamp (written as a vByte-encoded integer) followed by a path+query.

Path+queries are enqueued using the enqueueURL(VisitState, ByteArrayList) method. They can be dequeued in batches (the method uses cursors). When a queue is no longer needed, it can be removed.

Author:

Sebastiano Vigna

Constructor Summary

Constructors

Constructor	Description
WorkbenchVirtualizer(Frontier frontier)	Creates the virtualizer.

Method Summary

Modifier and Type	Method	Description
void	close()
void	collectIf(double threshold, double targetRatio)	Performs a garbage collection if the space used is below a given threshold, reaching a given target ratio.
long	count(VisitState visitState)	Returns the number of path+queries associated with the given visit state.
int	dequeuePathQueries(VisitState visitState, int maxUrls)	Dequeues at most the given number of path+queries into the given visit state.
void	enqueueURL(VisitState visitState, ByteArrayList url)	Enqueues the given URL as a path+query associated to the scheme+authority of the given visit state.
int	onDisk()	Returns the number of visit states on disk.
void	readMetadata()
void	remove(VisitState visitState)	Removes all path+queries associated with the given visit state.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

WorkbenchVirtualizer

public WorkbenchVirtualizer(Frontier frontier)

Creates the virtualizer.

Parameters:

frontier - the frontier instantiating this virtualizer.

Method Detail

dequeuePathQueries

public int dequeuePathQueries(VisitState visitState,
                              int maxUrls)
                       throws java.io.IOException

Dequeues at most the given number of path+queries into the given visit state.

Note that the path+queries are directly enqueued into the visit state using VisitState.enqueuePathQuery(byte[]).

Parameters:

visitState - the visitState in which path+queries will be moved.

maxUrls - the maximum number of path+queries to move.

Returns:

the number of actually dequeued path+queries.

Throws:

java.io.IOException

count

public long count(VisitState visitState)

Returns the number of path+queries associated with the given visit state.

Parameters:

visitState - the visitState whose path+queries are to be counted.

Returns:

the number of path+queries associated with the given visit state.

onDisk

public int onDisk()

Returns the number of visit states on disk.

Returns:

the number of visit states on disk.

remove

public void remove(VisitState visitState)
            throws java.io.IOException

Removes all path+queries associated with the given visit state.

Parameters:

visitState - the visitState whose path+queries are to be removed.

Throws:

java.io.IOException

enqueueURL

public void enqueueURL(VisitState visitState,
                       ByteArrayList url)
                throws java.io.IOException

Enqueues the given URL as a path+query associated to the scheme+authority of the given visit state.

Parameters:

visitState - the visitState to which the URL must be added.

url - a BUbiNG URL.

Throws:

java.io.IOException

collectIf

public void collectIf(double threshold,
                      double targetRatio)
               throws java.io.IOException

Performs a garbage collection if the space used is below a given threshold, reaching a given target ratio.

Parameters:

threshold - if ByteArrayDiskQueues.ratio() is below this value, a garbage collection will be performed.

targetRatio - passed to ByteArrayDiskQueues.count(Object).

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

readMetadata

public void readMetadata()
                  throws java.io.IOException,
                         java.lang.ClassNotFoundException

Throws:

java.io.IOException

java.lang.ClassNotFoundException