Mobile crawling and new hints

Thank you for all your positive feedback regarding our relaunch. Keep the feedback coming. We just released some new features you requested.

Here is a short summary of the changes:

Mobile Crawling

With mobile becoming more and more important we focused on having a solution to crawl mobile websites as well. With today's release we introduce two new User-agents, one for phones and one for tablets.

You can target the new User-agents in your application and in your robots.txt. All the details about the new User-agents is available in our bot reference.

Mobile Crawling is available in the basic, premium and ultimate edition. If you are still using one of the old editions you might want to upgrade.

New Hints

The new release brings a number of new hints. From now on we could for example detect hidden characters. We also have new hints for problems regarding anchors, frames, forms and compression.

The new hints are available in all editions.

Best Regards
Gerd and Tobias

 

Author