org.seasar.robot.helper
クラス RobotsTxtHelper

java.lang.Object
  org.seasar.robot.helper.RobotsTxtHelper

public class RobotsTxtHelper
extends Object

フィールドの概要

protected static Pattern	ALLOW
protected static Pattern	CRAWL_DELAY
protected static Pattern	DISALLOW
protected static Pattern	USER_AGENT

コンストラクタの概要

RobotsTxtHelper()

メソッドの概要

protected String	getValue(Pattern pattern, String line)
RobotsTxt	parse(InputStream stream)
RobotsTxt	parse(InputStream stream, String charsetName)
protected RobotsTxt	parse(List<String> lines)
RobotsTxt	parse(Reader reader)
RobotsTxt	parse(String text)
protected String	stripComment(String line)

クラス java.lang.Object から継承されたメソッド

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

フィールドの詳細

USER_AGENT

protected static final Pattern USER_AGENT

DISALLOW

protected static final Pattern DISALLOW

ALLOW

protected static final Pattern ALLOW

CRAWL_DELAY

protected static final Pattern CRAWL_DELAY

コンストラクタの詳細

RobotsTxtHelper

public RobotsTxtHelper()

メソッドの詳細

parse

public RobotsTxt parse(String text)

parse

public RobotsTxt parse(InputStream stream)

parse

public RobotsTxt parse(InputStream stream,
                       String charsetName)

parse

public RobotsTxt parse(Reader reader)

parse

protected RobotsTxt parse(List<String> lines)

getValue

protected String getValue(Pattern pattern,
                          String line)

stripComment

protected String stripComment(String line)