Clusters

Aggregate metrics for any subset you like.

About Clusters

Clusters divide your site into parts, and deliver crawl results for each part you defined. This allows for

  • Localization of problems
  • Identify important and less important portions of your site
  • Better optimization
  • ...and much more.

Here's an example of what clusters may look like:

The cluster report main page

Default Clusters

The system always creates one default cluster that contains all URLs.

Creating Your Own Clusters

Defining clusters is done through a set of rules that can be entered when creating a crawl. To create a cluster, first expand the advanced settings and scroll down to the Clusters section.

The Add Cluster button

Click the Add Cluster button. This will add a new cluster, which appears as a line of input elements.

You can click on Add Cluster as many times as you want to create new clusters. However, the total number of clusters is limited to 25 at the moment.

A line of input forming a cluster

By clicking the Remove button right next to a rule, you can delete this cluster.

Configuring Clusters

A cluster definition consists of two parts. These parts are:

  • A scope
  • A match

These are the drop down boxes you may have already noticed. A match takes a parameter, so it is accompanied by a text input field. We will cover this later

Additionally, a cluster can be given a name and a description. This is optional. These are the two text input fields forming the second line.

Note that neither name nor description may contain line breaks.

Scopes and Matches

Scopes and matches work the same as for URL rewriting. They are covered in depth there.

There are two additional scopes, though, which are available for clustering only:

  • HTML: The full HTML of the current page
  • HTTP Header: Single element of HTTP header in the format "name: value" - like cache: no-cache - note there always is a whitespace between the double colon and the header value. Both name and value are lower case