Package it.unimi.di.law.bubing.tool

Class BuildRepetitionSet

java.lang.Object

it.unimi.di.law.bubing.tool.BuildRepetitionSet

public class BuildRepetitionSet
extends java.lang.Object

Builds and saves the repetition set of a crawl.

The input format for the tool are TAB-separated triples <store,position,URL>, which are assumed to be stably sorted by URL (the position is the ordinal position in the store, as generated by the chosen filters). The triples must contain all the URLs overall appearing in all involved stores. For each URL that appears more than once, the pairs <store,position> of the copies following the first appearance are saved in a LongOpenHashSet as

        store << 48 | position
 

Constructor Summary

Constructors

Constructor	Description
BuildRepetitionSet()

Method Summary

Modifier and Type	Method	Description
static void	main(java.lang.String[] arg)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BuildRepetitionSet

public BuildRepetitionSet()

Method Detail

main

public static void main(java.lang.String[] arg)
                 throws java.io.IOException

Throws:

java.io.IOException