Package it.unimi.di.law.bubing.tool

Class GenerateGraphMap

java.lang.Object

it.unimi.di.law.bubing.tool.GenerateGraphMap

public class GenerateGraphMap
extends java.lang.Object

Builds and saves the graph map, that is, a text file containing all URLs ever crawled, and a binary file containing the corresponding nodes (duplicates are mapped to their archetype position).

The input format for the tool are a number of TAB-separated files (one per store), each containing triples <URL,digest,position>, which are assumed to be stably reverse-sorted starting at the digest (the position is the local final position). The positions are set to -1 in correspondence to duplicate pages.

The output is given by a text URL list, a corresponding binary list of long values (the node assigned to each URL) and a list of nodes and corresponding archetype URL, TAB separated.

This tool will print on standard output the number of nodes of the resulting graph.

Constructor Summary

Constructors

Constructor	Description
GenerateGraphMap()

Method Summary

Modifier and Type	Method	Description
static void	main(java.lang.String[] arg)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GenerateGraphMap

public GenerateGraphMap()

Method Detail

main

public static void main(java.lang.String[] arg)
                 throws java.io.IOException,
                        JSAPException

Throws:

java.io.IOException

JSAPException