Clusters divide your site into parts, and deliver crawl results for each part you defined. This allows for
Here's an example of what clusters may look like:
The system always creates one default cluster that contains all URLs.
Defining clusters is done through a set of rules that can be entered when creating a crawl. To create a cluster, first expand the advanced settings and scroll down to the Clusters section.
Click the Add Cluster button. This will add a new cluster, which appears as a line of input elements.
You can click on Add Cluster as many times as you want to create new clusters. However, the total number of clusters is limited to 25 at the moment.
By clicking the Remove button right next to a rule, you can delete this cluster.
A cluster definition consists of two parts. These parts are:
These are the drop down boxes you may have already noticed. A match takes a parameter, so it is accompanied by a text input field. We will cover this later
Additionally, a cluster can be given a name and a description. This is optional. These are the two text input fields forming the second line.
Note that neither name nor description may contain line breaks.
Scopes and matches work the same as for URL rewriting. They are covered in depth there.
There are two additional scopes, though, which are available for clustering only:
cache: no-cache- note there always is a whitespace between the double colon and the header value. Both name and value are lower case