add SemrushBot
authorRobert Kaiser <kairo@kairo.at>
Thu, 14 Dec 2017 13:50:03 +0000 (14:50 +0100)
committerRobert Kaiser <kairo@kairo.at>
Thu, 14 Dec 2017 13:50:03 +0000 (14:50 +0100)
tests/ua_list_raw.txt

index 9fa1dfe..7bbb8dc 100755 (executable)
@@ -469,6 +469,7 @@ Mozilla/5.0 (compatible;picmole/1.0 +http://www.picmole.com)
 Mozilla/5.0 (X11; compatible; crawler@newstin.com; HTTPClient 3.1)
 Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 FAKE (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
+Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)
 Nokia6681/1.0 (2.30.0) Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (compatible; baiduspider; +http://www.baidu.com/search/spider.html)
 Nokia3650/1.0 SymbianOS/6.1 Series60/1.2 Profile/MIDP-1.0 Configuration/CLDC-1.0/ (compatible; YodaoBot-Mobile/1.0; http://www.youdao.com/help/webmaster/spider/; )
 NokiaN70/. FASTMobileCrawl/6.6 Profile/MIDP-2.0 Configuration/CLDC-1.1