Functions

appendConfig()

appendConfig($options) 

Parameters

$options

echoConfig()

echoConfig() 

writeConfig()

writeConfig($options) 

Parameters

$options

Classes and interfaces

DatabaseInterface

« More »

DatabaseResultInterface

« More »

FilterInterface

Filters are scripts that need to run while the crawler is busy crawling pages.

« More »

TweakInterface

Tweaks are scripts that need to run while the crawler is busy crawling pages.

« More »

CSVExport

crawler command to export crawler table to a csv file

« More »

FilterUrls

« More »

FromSitemap

« More »

FromUrl

« More »

SitemapExport

« More »

Crawler

Crawls one or more websites for url

« More »

CrawlerSettings

holds the configuration data for the crawler

« More »

PHPCurlCrawler

Curl based class to crawl webpages

« More »

PageCrawler

Analyse a single page

« More »

CrawlerDB

Class to help with read/write crawler database

« More »

CrawlerResultsDB

Class to help with read/write crawler database

« More »

DBPDO

extends PDO with extra methods

« More »

DB_PdoStatement

extends PDOStatement

« More »

CSVExport

exports crawler table to a csv file takes a filename as imput an exports the table to "output/filename.csv"

« More »

ExportAbstract

abstract class for exporters

« More »

SitemapExport

Class to create a sitemap for your site $sitemap = new Sitemap($url); $sitemap->generateSitemap(); $sitemap->writeSitemap("sitemap.xml"); $submit = $sitemap->pingSearchEngines($url.'sitemap.xml'); foreach ($submit as $searchengine) { echo $searchengine[0], " -> ",$searchengine[1], "
\n"; }
option to send a ping to searchengines, telling them you updated the sitemap

« More »

FilterExternalUrls

filters urls based on a given set of strings

« More »

FilterRunner

factory for filters

« More »

ImportAbstract

abstract class for exporters

« More »

SitemapImport

abstract class for exporters

« More »

KeepOnlyPages

class to ping urls

« More »

TweakRunner

factory for filters

« More »

htmlEscape

escapes special characters in urls

« More »

removeDuplicates

removes duplicate array entries

« More »

removeParameters

remove parameters from a url

« More »

stripHash

« More »

stripIndex

class to strip the index (expl index.php,index.asp) from urls

« More »

toLowercase

tweak to alter urls to lowercase

« More »

SplClassLoader

SplClassLoader implementation that implements the technical interoperability standards for PHP 5.3 namespaces and class names.

« More »