Aggregate metrics for any subset you like.

About Clusters

Clusters divide your site into segments, and deliver crawl results for each part you defined. This allows for

  • Faster localization of problems by page type
  • Filter many analyses by segment, e.g. drill down of response times for category vs. product pages
  • Identify important and less important segments of your site, e.g. PageRank share of cluster
  • Better structural optimization
  • ...and much more.

Default Clusters

The system always creates three default clusters: all URLs, internal URLs and external URLs. Here's an example of what clusters may look like:

The cluster report main page

Creating Your Own Clusters

Defining clusters is done through a set of rules that can be entered when creating a crawl or editing the master settings of a project. To create a cluster, navigate to the Clusters tab in the settings.

Click the Add Cluster button. This will add a new cluster within the visual editor, which appears as a line of input elements.

A line of input forming a cluster

The first input is used to set the cluster's name, the second to enter an additional description (optional). Following is the condition editor, to enter the cluster's conditions.

Within a cluster you can create logical AND/OR/XOR patterns and also create a nested +Group if useful. The +Rule will add a new line to add more conditions. Important to note: all conditions are case sensitive.

You can click on Add Cluster as many times as you want to create new clusters. However, the total amount is by default limited to 70 clusters. Using the Remove (trashbin) button right next to a rule or cluster will delete it.

For faster error research and the best overview its advised to cluster most of your important page types and segments. A cluster overview - and further data drill down dashboards for each cluster - can be accessed via the sidebar navigation: Structure -> Clusters. You can click on each cluster to access the cluster specific report.

Tips and best-practise examples for working with clusters

Keep in mind that patterns you create could sometimes be match external URLs, therefore it is best-practise to always add a condition targeting the host to avoid false positives:

Condition targeting a host

If you want to match a folder or part of your URL structure, for the best calculating performance of your crawl, use the Path as a scope:

Condition targeting a path

You can not only create clusters to target specific page types by keywords, but of course also to create technical segments. If you want to target all URLs with a 3xx HTTP status code use:

Condition targeting a status code

The logic construction of groups follows boolean algebra rules. If you want to combine logical conditions, follow e.g. this example. Pay special attention to the mix of AND/OR: The first two conditions must match, but the second OR group matches if any condition is true.

Groups using multiple conditions

Fast creation of similar clusters with the text editor

If you need to create several similar clusters (e.g. according to your folder structure) you could create the first one using the visual editor. After all conditions are set, switch to the text editor - the button is located at the top/bottom of the view. Once within the text editor you can copy & paste the cluster and quickly adjust e.g. the folder/cluster name. This can be much faster than re-creating a cluster with several conditions many times. See note about REFID below!

Tip: You can use the text editor to create a quick backup of clusters you want to use for similar projects.

Note: If you do any copy & paste operations with clusters, make sure the REFID is empty. When you create a new cluster this is always the case, but once you save your settings the cluster will own a unique REFID. If you delete the REFID our software will automatically generate a new unique ID, therefore you can just delete the REFID from your copies.