The Audisto Industry Monitor benchmarks portals, shops and websites within an industry by current technological web-standards and best-practise with regards to security, performance, web-technology and content.

To compare different portals we crawl up to 10.000 URLs for each one of them with identical crawl settings and perform a number of checks.

Security

HTTPS

:::todo Extend Cluster to check all URLs, not just crawled URLs :::

Each URL must use the HTTPS protocol to pass this check. This check is performed agains all discovered URLs of the portal.

CHECK "SECURITY: HTTPS" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "HTTPS" IS:
      Scheme Equals "https"

Strict-Transport-Security

To pass this check the HTTP-Header needs to contain a single, valid Strict-Transport-Security header with a duration of at least one year. This check is performed against all crawled URLs of the portal.

CHECK "SECURITY: Strict Transport Security" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "Strict Transport Security" IS:
      (Hint Does Not Equal "130: Strict-Transport-Security HTTP header missing" AND Hint Does Not Equal "131: Strict-Transport-Security HTTP header has short duration" AND Hint Does Not Equal "133: Strict-Transport-Security HTTP header is invalid" AND Hint Does Not Equal "132: Strict-Transport-Security HTTP header sent more than once")

Secure Cookies

Cookies should only be send over secure connections. Cookies need to have a secure flag set to pass this check. No cookies should be send over HTTP connections. This check is performed agains all crawled URLs of the portal.

CHECK "SECURITY: Secure Cookies" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "Secure Cookies" IS:
      (Hint Does Not Equal "135: Cookies set without secure flag" AND Hint Does Not Equal "141: Cookies send over insecure connection")

No unsafe resources

:::todo Check should cover unsecure Ressources on any page :::

Secure pages should not contain unsecure resources. All resources need to be referenced using HTTPS to pass this check. This check is performed against all crawled HTML pages.

CHECK "SECURITY: No unsafe resources" IS:
  IF (MIME Type Equals "text/html") THEN
    REQUIREMENT "No unsafe resources" IS:
      Hint Does Not Equal "56: Safe HTTPS URL loads unsafe resource"

No unsafe forms

User input data should only be transferred using a secure connections. To pass this check the URL containing the form and the URL receiving the form data need to be secure. This check is performed against all crawled HTML-pages.

CHECK "SECURITY: No unsafe forms" IS:
  IF (MIME Type Equals "text/html") THEN
    REQUIREMENT "No unsafe forms" IS:
      (Hint Does Not Equal "65: <form> POST to HTTPS from HTTP" AND Hint Does Not Equal "92: <form> Unsafe GET to HTTP from HTTPS" AND Hint Does Not Equal "81: <form> Unsafe POST to HTTP from HTTPS")

Performance

Redirects

Redirects within the internal link graph cause additional requests and therefore additional latencies. To pass this check no internal redirects and meta refresh redirects are allowed. This check is performed against all crawled URLs.

CHECK "PERFORMANCE: Redirects" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "HTTP Redirects" IS:
      (HTTP Status Does Not Equal "301" AND HTTP Status Does Not Equal "302" AND HTTP Status Does Not Equal "303" AND HTTP Status Does Not Equal "307" AND HTTP Status Does Not Equal "308")
  
  
    REQUIREMENT "Meta Refresh" IS:
      Hint Does Not Equal "19: <meta refresh> found"

Responsetimes

For a good user experience all URLs should have a good responsetime. To pass this check all URLs need to have a responsetime below 200 ms which is Google's PageSpeed recommendation. This check is performed against all crawled URLs.

CHECK "PERFORMANCE: Responsetimes" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "Responsetimes" IS:
      Response Time Less Than "200"

Filesizes

:::todo Come up with better metrics for this check :::

For a good user experience all HTML documents should have a reasonable filesize. To pass this check all HTML pages need to have an uncompressed filesize below 150 KB and a compressed filesize below 50 KB. This check is performed against all crawled HTML pages.

CHECK "PERFORMANCE: Filesizes" IS:
  IF (HTTP Status Exists AND MIME Type Equals "text/html") THEN
    REQUIREMENT "Filesizes" IS:
      (Compressed Content Size Less Than Or Equals "50000" AND Uncompressed Content Size Less Than Or Equals "150000")

Compression

To improve page load speed all transferred data should be compressed. To pass this test all crawled URLs with a compressable mime type should return gzip/deflate compressed data. This check is performed against all crawled URLs with a mime-type that could benefit from data compression.

CHECK "PERFORMANCE: Compression" IS:
  IF (MIME Type Equals "text/css" OR MIME Type Equals "text/plain" OR MIME Type Equals "text/javascript" OR MIME Type Equals "application/javascript" OR MIME Type Equals "application/json" OR MIME Type Equals "application/x-javascript" OR MIME Type Equals "application/xml" OR MIME Type Equals "application/xml+rss" OR MIME Type Equals "application/xhtml+xml" OR MIME Type Equals "application/x-font-ttf" OR MIME Type Equals "application/x-font-opentype" OR MIME Type Equals "application/vnd.ms-fontobject" OR MIME Type Equals "image/svg+xml" OR MIME Type Equals "image/x-icon" OR MIME Type Equals "application/rss+xml" OR MIME Type Equals "application/atom_xml" OR MIME Type Equals "text/html") THEN
    REQUIREMENT "Compression" IS:
      Hint Does Not Equal "67: Compression not enabled"

Technology

HTTP Error Codes

For a good user experience no URL should result in a server error. To pass this test no URL should return a 4xx or 5xx HTTP status code. This check is performed against all crawled URLs.

CHECK "TECHNOLOGY: HTTP Error codes" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "HTTP Error codes" IS:
      (HTTP Status Does Not Start With "4" AND HTTP Status Does Not Start With "5")

Canonicals

:::todo Check should cover problems of canonical groups in addition to the current problems :::

When Canonicals are used, they must meet certain minimum requirements. This includes that they are specified in the <head> of the page or in the HTTP header, contain a valid URL and don't have conflicting target URLs when specified more than once. This check is performed against all crawled URLs with a 200 HTTP status code.

CHECK "TECHNOLOGY: Canonicals" IS:
  IF HTTP Status Equals "200" THEN
    REQUIREMENT "Canonicals" IS:
      (Hint Does Not Equal "49: <link rel=canonical> contains malformed or empty href" AND Hint Does Not Equal "62: <link rel=canonical> found outside <head>" AND Hint Does Not Equal "48: <link rel=canonical> found twice and differs")

Hreflang

:::todo Check should cover problems of hreflang groups. Page is not allowed to be in a broken group. :::

When Hreflang is used, a complex set of rules must be fullfilled. To pass this check all URLs with hreflang definitions must have a self link and the hreflang URLs are not allowed to be empty or malformed. This check is performed against all crawled URLs.

CHECK "TECHNOLOGY: Hreflang" IS:
  IF HTTP Status Exists THEN
    REQUIREMENT "Hreflang" IS:
      (Hint Does Not Equal "123: Hreflang: Self link missing" AND Hint Does Not Equal "126: Hreflang: URL empty or malformed")

Encoding

Every document should specify a charset and correctly encode all characters in that charset. To pass this test a charset needs to be specified and in case of multiple specifications the definitions should not conflict. In addition there should not be any non printable characters in the documents. This check is performed against all crawled HTML documents.

CHECK "TECHNOLOGY: Encoding" IS:
  IF (MIME Type Equals "text/html") THEN
    REQUIREMENT "Encoding" IS:
      (Hint Does Not Equal "104: <html> contains too many uncommon non-printable characters" AND Hint Does Not Equal "103: Charset: Charset set in HTTP Content-Type header and in document differ." AND Hint Does Not Equal "102: Charset: Not set" AND Hint Does Not Equal "101: Charset: Invalid charset in Content-Type HTTP header")

Links

All links within a HTML document should be well formed and links within the network should be follow. To pass this check all links need to point to a valid URI and should not start or end with whitespace characters. In addition all internal links within the domain need to be set to "follow". This check is performed against all crawled HTML documents.

CHECK "TECHNOLOGY: Links" IS:
  IF MIME Type Equals "text/html" THEN
    REQUIREMENT "Links" IS:
      (Hint Does Not Equal "31: <a> has malformed href" AND Hint Does Not Equal "98: <a> href attribute has leading or trailing whitespace characters" AND Hint Does Not Equal "38: <base> contains malformed or empty href" AND Hint Does Not Equal "83: <base> found more than once and differs" AND Hint Does Not Equal "142: Linking: nofollow link to network")

Content

Search Snippet

:::todo Check should ignore URLs with Canonical to other URLs, but not on mobile/amp pages :::

Proper titles and meta-descriptions are best practises for SEO. To pass this test titles and meta-descriptions need to be specified. Both should be placed within the <head> of the html, occur a single time and should not be too short or too long. This check is performed against all indexable HTML documents with a 200 HTTP status code.

CHECK "CONTENT: Search Snippet" IS:
  IF (MIME Type Equals "text/html" AND HTTP Status Equals "200" AND Meta Robots Equals "index") THEN
    REQUIREMENT "Title" IS:
      (Hint Does Not Equal "4: <title> missing or empty" AND Hint Does Not Equal "61: <title> found outside <head>" AND Hint Does Not Equal "6: <title> occurs more than once" AND Hint Does Not Equal "16: <title> short or single word" AND Hint Does Not Equal "5: <title> too long for Google snippet")
  
  
    REQUIREMENT "Meta-Description" IS:
      (Hint Does Not Equal "7: <meta description> missing or empty" AND Hint Does Not Equal "9: <meta description> occurs more than once")

Headlines

:::todo Check should ignore URLs with Canonical to other URLs, but not on mobile/amp pages OR be performed on all HTML pages :::

Headings add to structure in a document. Missing headings are indicators of poorly structured content and therefore indicate lower content quality. To pass this check at least one <h1> or <h2> headline needs to be specified. This check is performed against all indexable HTML documents with a 200 HTTP status code.

CHECK "CONTENT: Headlines" IS:
  IF (MIME Type Equals "text/html" AND HTTP Status Equals "200" AND Meta Robots Equals "index") THEN
    REQUIREMENT "Headlines" IS:
      (Hint Does Not Equal "21: <h1> and <h2> not found" AND Hint Does Not Equal "20: <h1> not found")

Alt attributes

:::todo Check should ignore URLs with Canonical to other URLs, but not on mobile/amp pages OR be performed on all HTML pages :::

Valid HTML requires an alt attribute for all images. In addition alt attributes are one of the factors used by search engines to determine the topic of the image. To pass this test all images need at least an empty alt attribute. This check is performed against all indexable HTML documents with a 200 HTTP status code.

CHECK "CONTENT: Alt attributes" IS:
  IF (MIME Type Equals "text/html" AND HTTP Status Equals "200" AND Meta Robots Equals "index") THEN
    REQUIREMENT "Alt attributes" IS:
      Hint Does Not Equal "26: <img> has no alt attribute"