org.eclipse.smila.importing.crawler.web.filter

Class DefaultLinkFilter

java.lang.Object

org.eclipse.smila.importing.crawler.web.filter.DefaultLinkFilter

All Implemented Interfaces:

LinkFilter

public class DefaultLinkFilter
extends java.lang.Object
implements LinkFilter

Default LinkFilter implementation.

Constructor Summary

Constructors

Constructor and Description
DefaultLinkFilter()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	allowRedirectLink(java.lang.String url, java.lang.String originalUrl, WebCrawlingContext context) Check if it is allowed to follow a given redirect link.
java.util.Collection<Record>	filterExtractedLinks(java.util.Collection<Record> extractedLinks, java.lang.String sourceUrl, WebCrawlingContext context) filter links extracted from given source URL.
void	setJobRunDataProvider(JobRunDataProvider jobRunDataProvider) DS service reference injection method.
void	unsetJobRunDataProvider(JobRunDataProvider jobRunDataProvider) DS service reference removal method.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultLinkFilter

public DefaultLinkFilter()

Method Detail

filterExtractedLinks

public java.util.Collection<Record> filterExtractedLinks(java.util.Collection<Record> extractedLinks,
                                                java.lang.String sourceUrl,
                                                WebCrawlingContext context)
                                                  throws WebCrawlerException

Description copied from interface: LinkFilter

filter links extracted from given source URL.

Specified by:

filterExtractedLinks in interface LinkFilter

Parameters:

extractedLinks - result from LinkExtractor service.

sourceUrl - the source URL from which the links were extracted

context - the WebCrawlingContext.

Returns:

links to follow in follow-up tasks

Throws:

WebCrawlerException

allowRedirectLink

public boolean allowRedirectLink(java.lang.String url,
                        java.lang.String originalUrl,
                        WebCrawlingContext context)
                          throws WebCrawlerException

Description copied from interface: LinkFilter

Check if it is allowed to follow a given redirect link.

Specified by:

allowRedirectLink in interface LinkFilter

Parameters:

url - a String containing the link to be checked

originalUrl - the original URL that was redirected.

context - the WebCrawlingContext.

Returns:

true if the link is allowed to be followed, false otherwise

Throws:

WebCrawlerException

setJobRunDataProvider

public void setJobRunDataProvider(JobRunDataProvider jobRunDataProvider)

DS service reference injection method.

unsetJobRunDataProvider

public void unsetJobRunDataProvider(JobRunDataProvider jobRunDataProvider)

DS service reference removal method.