nutch install
on ubuntu
http://wiki.apache.org/nutch/NutchTutorial#A1_Setup_Nutch_from_binary_distribution
还是要看官方的文档，之前google了一个早期1.0版本的中文安装教程，直接误入歧途
Steps
1 Setup Nutch from binary distribution Unzip your binary Nutch package to $HOME/nutch-1.3 cd $HOME/nutch-1.3/runtime/local
From now on, we am going to use ${NUTCH_RUNTIME_HOME} to refer to the current directory.
2. Verify your Nutch installation run "bin/nutch" - You can confirm a correct installation if you seeing the following:
Usage: nutch [-core] COMMAND
Some troubleshooting tips:
Run the following command if you are seeing "Permission denied":
chmod +x bin/nutchSetup JAVA_HOME if you are seeing JAVA_HOME not set. On Mac, you can run the following command or add it to ~/.bashrc:
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
3. Crawl your first website Add your agent name in the value field of the http.agent.name property in conf/nutch-site.xml, for example:
<property><br /> <name>http.agent.name</name><br /> <value>My Nutch Spider</value><br /></property>mkdir -p urls create a text file nutch under /urls with the following content (1 url per line for each site you want Nutch to crawl).
http://nutch.apache.org/
* Edit the file conf/regex-urlfilter.txt and replace
# accept anything else<br />+.
with a regular expression matching the domain you wish to crawl. For example, if you wished to limit the crawl to the nutch.apache.org domain, the line should read:
+^http://([a-z0-9]*\.)*nutch.apache.org/
This will include any url in the domain nutch.apache.org.
3.1 Using the Crawl Command
Now we are ready to initiate a crawl, use the following parameters:
-dir dir names the directory to put the crawl in.
-threads threads determines the number of threads that will fetch in parallel.
-depth depth indicates the link depth from the root page that should be crawled.
-topN N determines the maximum number of pages that will be retrieved at each level up to the depth.
Run the following command:
bin/nutch crawl urls -dir crawl -depth 3 -topN 5Now you should be able to see the following directories created:
crawl/crawldb<br />Crawl/linkdb<br />crawl/segments