-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathindex.json
More file actions
325 lines (325 loc) · 25.4 KB
/
index.json
File metadata and controls
325 lines (325 loc) · 25.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
[
"'Mozilla/5.0 (compatible; DuckDuckBot-Https/1.1; https://duckduckgo.com/duckduckbot)'",
"@LinkArchiver twitter bot",
"2ip bot/1.1 (+http://2ip.io)",
"adbeat_bot",
"APIs-Google (+https://developers.google.com/webmasters/APIs-Google.html)",
"ArchiveTeam ArchiveBot/20170106.02 (wpull 2.0.2)",
"Audisto Crawler (desktop; +https://audisto.com/bot)",
"Audisto Crawler (desktop; essential; +https://audisto.com/bot)",
"Audisto Crawler (mobile; +https://audisto.com/bot)",
"Audisto Crawler (mobile; essential; +https://audisto.com/bot)",
"AwarioRssBot/1.0 (+https://awario.com/bots.html; [email protected])",
"AwarioSmartBot/1.0 (+https://awario.com/bots.html; [email protected])",
"B2B Bot",
"Baidu-YunGuanCe-Bot(ce.baidu.com)",
"Baidu-YunGuanCe-PerfBot(ce.baidu.com)",
"Baidu-YunGuanCe-ScanBot(ce.baidu.com)",
"Better Uptime Bot Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36",
"blogmuraBot (+http://www.blogmura.com)",
"bot-pge.chlooe.com/1.0.0 (+http://www.chlooe.com/)",
"BublupBot (+https://www.bublup.com/bublup-bot.html)",
"Clickagy Intelligence Bot v2",
"Cliqzbot/0.1 (+http://cliqz.com [email protected])",
"Cliqzbot/0.1 (+http://cliqz.com/company/cliqzbot)",
"coccoc",
"coccoc/1.0 ()",
"coccoc/1.0 (http://help.coccoc.com/)",
"coccoc/1.0 (http://help.coccoc.vn/)",
"ContextAd Bot 1.0",
"CriteoBot/0.1 (+https://www.criteo.com/criteo-crawler/)",
"CrunchBot/1.0 (+http://www.leadcrunch.com/crunchbot)",
"deepnoc - https://deepnoc.com/bot",
"Domain Re-Animator Bot (http://domainreanimator.com) - [email protected]",
"DomainStatsBot/1.0 (http://domainstats.io/our-bot)",
"DuckAssistBot/1.2; (+http://duckduckgo.com/duckassistbot.html)",
"facebookexternalhit/1.1",
"Facebot/1.0",
"Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=728742641706423)",
"FeedlyBot/1.0 (http://feedly.com)",
"Flamingo_SearchEngine (+http://www.flamingosearch.com/bot)",
"FreeWebMonitoring SiteChecker/0.2 (+https://www.freewebmonitoring.com/bot.html)",
"FreshpingBot/1.0 (+https://freshping.io/)",
"FreshRSS/1.11.2 (Linux; https://freshrss.org) like Googlebot",
"g2reader-bot/1.0 (+http://www.g2reader.com/)",
"Google-CloudVertexBot",
"GroupMeBot/1.0",
"Gwene/1.0 (The gwene.org rss-to-news gateway) Googlebot",
"http://seewithkids.com/bot",
"INETDEX-BOT/1.5 (Mozilla/5.0; https://inetdex.com/bot.html)",
"infoobot/0.1 (https://www.infoo.nl/bot.html)",
"jpg-newsbot/2.0; (+https://vipnytt.no/bots/)",
"Landau-Media-Spider/1.0(http://bots.landaumedia.de/bot.html)",
"Linguee Bot (http://www.linguee.com/bot; [email protected])",
"Linguee Bot (http://www.linguee.com/bot)",
"LivelapBot/0.2 (http://site.livelap.com/crawler)",
"MauiBot ([email protected])",
"Mediapartners-Google",
"MojeekBot/0.2 (archi; http://www.mojeek.com/bot.html)",
"MoodleBot/1.0",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://www.changedetection.com/bot.html )",
"Mozilla/5.0 (compatible; adbeat_bot; [email protected]; [email protected])",
"Mozilla/5.0 (compatible; AddSearchBot/0.9; +http://www.addsearch.com/bot; [email protected])",
"Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html)",
"Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)",
"Mozilla/5.0 (compatible; AhrefsBot/5.2; News; +http://ahrefs.com/robot/)",
"Mozilla/5.0 (compatible; AhrefsBot/6.1; +http://ahrefs.com/robot/)",
"Mozilla/5.0 (compatible; AhrefsBot/6.1; News; +http://ahrefs.com/robot/)",
"Mozilla/5.0 (compatible; AlphaBot/3.2; +http://alphaseobot.com/bot.html)",
"Mozilla/5.0 (compatible; AndersPinkBot/1.0; +http://anderspink.com/bot.html)",
"Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot)",
"Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)",
"Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; +http://archive.org/details/archive.org_bot)",
"Mozilla/5.0 (compatible; archive.org_bot/heritrix-1.15.4 +http://www.archive.org)",
"Mozilla/5.0 (compatible; AwarioBot/1.0; +https://awario.com/bots.html)",
"Mozilla/5.0 (compatible; BehloolBot/beta; +http://www.webeaver.com/bot)",
"Mozilla/5.0 (compatible; BlogTraffic/1.4 Feed-Fetcher; +http://www.blogtraffic.de/rss-bot.html)",
"Mozilla/5.0 (compatible; bnf.fr_bot; +http://bibnum.bnf.fr/robot/bnf.html)",
"Mozilla/5.0 (compatible; bnf.fr_bot; +http://www.bnf.fr/fr/outils/a.dl_web_capture_robot.html)",
"Mozilla/5.0 (compatible; BomboraBot/1.0; +http://www.bombora.com/bot)",
"Mozilla/5.0 (compatible; BoxcarBot/1.1; [email protected])",
"Mozilla/5.0 (compatible; Cliqzbot/0.1 +http://cliqz.com/company/cliqzbot)",
"Mozilla/5.0 (compatible; Cliqzbot/1.0 +http://cliqz.com/company/cliqzbot)",
"Mozilla/5.0 (compatible; Cliqzbot/2.0; +http://cliqz.com/company/cliqzbot)",
"Mozilla/5.0 (compatible; coccoc/1.0; +http://help.coccoc.com/)",
"Mozilla/5.0 (compatible; coccoc/1.0; +http://help.coccoc.com/searchengine)",
"Mozilla/5.0 (compatible; coccocbot-image/1.0; +http://help.coccoc.com/searchengine)",
"Mozilla/5.0 (compatible; coccocbot-web/1.0; +http://help.coccoc.com/searchengine)",
"Mozilla/5.0 (compatible; Cocolyzebot/1.0; https://cocolyze.com/bot)",
"Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot)",
"Mozilla/5.0 (compatible; Digincore bot; https://www.digincore.com/crawler.html for rules and instructions.)",
"Mozilla/5.0 (compatible; DnyzBot/1.0)",
"Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)",
"Mozilla/5.0 (compatible; EveryoneSocialBot/1.0; [email protected] http://everyonesocial.com/)",
"Mozilla/5.0 (compatible; ExtLinksBot/1.5 +https://extlinks.com/Bot.html)",
"Mozilla/5.0 (compatible; FacebookBot/1.0; +https://developers.facebook.com/docs/sharing/webmasters/facebookbot/)",
"Mozilla/5.0 (compatible; Feedspotbot/1.0; +http://www.feedspot.com/fs/bot)",
"Mozilla/5.0 (compatible; FemtosearchBot/1.0; http://femtosearch.com)",
"Mozilla/5.0 (compatible; GeedoBot; +http://www.geedo.com/bot.html)",
"Mozilla/5.0 (compatible; Google-Extended/1.0; +http://www.google.com/bot.html)",
"Mozilla/5.0 (compatible; Google-InspectionTool/1.0)",
"Mozilla/5.0 (compatible; Google-Structured-Data-Testing-Tool +http://developers.google.com/structured-data/testing-tool/)",
"Mozilla/5.0 (compatible; heritrix/3.1.1-SNAPSHOT-20120116.200628 +http://www.archive.org/details/archive.org_bot)",
"Mozilla/5.0 (compatible; heritrix/3.3.0-SNAPSHOT-20140702-2247 +http://archive.org/details/archive.org_bot)",
"Mozilla/5.0 (compatible; hypestat/1.0; +https://hypestat.com/bot)",
"Mozilla/5.0 (compatible; image.coccoc/1.0; +http://help.coccoc.com/)",
"Mozilla/5.0 (compatible; imagecoccoc/1.0; +http://help.coccoc.com/)",
"Mozilla/5.0 (compatible; imagecoccoc/1.0; +http://help.coccoc.com/searchengine)",
"Mozilla/5.0 (compatible; ImagesiftBot; +imagesift.com)",
"Mozilla/5.0 (compatible; IstellaBot/1.23.15 +http://www.tiscali.it/)",
"Mozilla/5.0 (compatible; Jooblebot/2.0; Windows NT 6.1; WOW64; +http://jooble.org/jooble-bot) AppleWebKit/537.36 (KHTML, like Gecko) Safari/537.36",
"Mozilla/5.0 (compatible; KeybaseBot; +https://keybase.io)",
"Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/about/bots/)",
"Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/)",
"Mozilla/5.0 (compatible; linkdexbot/2.1; +http://www.linkdex.com/about/bots/)",
"Mozilla/5.0 (compatible; linkdexbot/2.1; +http://www.linkdex.com/bots/)",
"Mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)",
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)",
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots/2.0; +http://go.mail.ru/help/robots)",
"Mozilla/5.0 (compatible; Mappy/1.0; +http://mappydata.net/bot/)",
"Mozilla/5.0 (compatible; memorybot/1.21.14 +http://mignify.com/bot.html)",
"Mozilla/5.0 (compatible; MojeekBot/0.2; http://www.mojeek.com/bot.html)",
"Mozilla/5.0 (compatible; MojeekBot/0.2; http://www.mojeek.com/bot.html#relaunch)",
"Mozilla/5.0 (compatible; MojeekBot/0.5; http://www.mojeek.com/bot.html)",
"Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html)",
"Mozilla/5.0 (compatible; MojeekBot/0.6; http://www.mojeek.com/bot.html)",
"Mozilla/5.0 (compatible; Monsidobot/2.2; +http://monsido.com/bot.html; [email protected])",
"Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0; WOW64; Trident/6.0; BingPreview/1.0b)",
"Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1) Streamline3Bot/1.0",
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; WOW64; Trident/5.0; BingPreview/1.0b)",
"Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa/4.0 (Following Mediapartners-Google)",
"Mozilla/5.0 (compatible; Neevabot/1.0; +https://neeva.com/neevabot)",
"Mozilla/5.0 (compatible; NerdByNature.Bot; http://www.nerdbynature.net/bot)",
"Mozilla/5.0 (compatible; Nimbostratus-Bot/v1.3.2; http://cloudsystemnetworks.com)",
"Mozilla/5.0 (compatible; NIXStatsbot/1.1; +http://www.nixstats.com/bot.html)",
"Mozilla/5.0 (compatible; OdklBot/1.0 like Linux; [email protected])",
"Mozilla/5.0 (compatible; online-webceo-bot/1.0; +http://online.webceo.com)",
"Mozilla/5.0 (compatible; OpenHoseBot/2.1; +http://www.openhose.org/bot.html)",
"Mozilla/5.0 (compatible; PrivacyAwareBot/1.1; +http://www.privacyaware.org)",
"Mozilla/5.0 (compatible; Qwantify/Bleriot/1.1; +https://help.qwant.com/bot)",
"Mozilla/5.0 (compatible; Qwantify/Bleriot/1.2.1; +https://help.qwant.com/bot)",
"Mozilla/5.0 (compatible; RankActiveLinkBot; +https://rankactive.com/resources/rankactive-linkbot)",
"Mozilla/5.0 (compatible; redditbot/1.0; +http://www.reddit.com/feedback)",
"Mozilla/5.0 (compatible; RegionStuttgartBot/1.0; +http://it.region-stuttgart.de/competenzatlas/unternehmen-suchen/)",
"Mozilla/5.0 (compatible; ReverseEngineeringBot/0.1; +https://torus.company/bot.html)",
"Mozilla/5.0 (compatible; RidderBot/1.0; [email protected])",
"Mozilla/5.0 (compatible; rogerBot/1.0; UrlCrawler; http://www.seomoz.org/dp/rogerbot)",
"Mozilla/5.0 (compatible; SBIntuitionsBot/0.1; +https://www.sbintuitions.co.jp/bot/)",
"Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/)",
"Mozilla/5.0 (compatible; Semanticbot/1.0; +http://sempi.tech/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot-BA; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot-SA/0.97; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot-SI/0.97; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot/0.98~bl; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot/3~bl; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)",
"Mozilla/5.0 (compatible; SERankingBacklinksBot/1.0; +https://seranking.com/backlinks-crawler)",
"Mozilla/5.0 (compatible; SeznamBot/3.2-test1-1; +http://napoveda.seznam.cz/en/seznambot-intro/)",
"Mozilla/5.0 (compatible; SeznamBot/3.2-test1; +http://napoveda.seznam.cz/en/seznambot-intro/)",
"Mozilla/5.0 (compatible; SeznamBot/3.2-test2; +http://napoveda.seznam.cz/en/seznambot-intro/)",
"Mozilla/5.0 (compatible; SeznamBot/3.2-test4; +http://napoveda.seznam.cz/en/seznambot-intro/)",
"Mozilla/5.0 (compatible; SeznamBot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)",
"Mozilla/5.0 (compatible; SeznamBot/4.0; +http://napoveda.seznam.cz/seznambot-intro/)",
"Mozilla/5.0 (compatible; spbot/1.0; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/1.1; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/1.2; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/2.0; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/2.0.1; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/2.0.2; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/2.0.3; +http://www.seoprofiler.com/bot/ )",
"Mozilla/5.0 (compatible; spbot/2.0.4; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/2.1; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/3.0; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/3.1; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.1; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.2; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.3; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.4; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.5; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.6; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.7; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.7; +https://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.8; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.0.9; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.0a; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.0b; +http://www.seoprofiler.com/bot )",
"Mozilla/5.0 (compatible; spbot/4.1.0; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.2.0; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.3.0; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.4.0; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.4.1; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/4.4.2; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/5.0; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/5.0.1; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/5.0.2; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; spbot/5.0.3; +http://OpenLinkProfiler.org/bot )",
"Mozilla/5.0 (compatible; special_archiver/3.1.1 +http://www.archive.org/details/archive.org_bot)",
"Mozilla/5.0 (compatible; startmebot/1.0; +https://start.me/bot)",
"Mozilla/5.0 (compatible; StorygizeBot; http://www.storygize.com)",
"Mozilla/5.0 (compatible; StractBot/0.1; open source search engine; +https://trystract.com/webmasters)",
"Mozilla/5.0 (compatible; t3versionsBot/1.0; +https://www.t3versions.com/bot)",
"Mozilla/5.0 (compatible; TinEye-bot/1.31; +http://www.tineye.com/crawler.html)",
"Mozilla/5.0 (compatible; trovitBot 1.0; +http://www.trovit.com/bot.html)",
"Mozilla/5.0 (compatible; URLAppendBot/1.0; +http://www.profound.net/urlappendbot.html)",
"Mozilla/5.0 (compatible; Vigil/1.0; +http://vigil-app.com/bot.html)",
"Mozilla/5.0 (compatible; VoluumDSP-content-bot/2.0; [email protected])",
"Mozilla/5.0 (compatible; vuhuvBot/1.0; +http://vuhuv.com/bot.html)",
"Mozilla/5.0 (compatible; WellKnownBot/0.1; +https://well-known.dev/about/#bot)",
"Mozilla/5.0 (compatible; XoviBot/2.0; +http://www.xovibot.net/)",
"Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)",
"Mozilla/5.0 (compatible; Yellowbrandprotectionbot/1.0; +https://www.yellowbp.com/bot.html)",
"Mozilla/5.0 (compatible; Yeti/1.1; +http://naver.me/bot)",
"Mozilla/5.0 (compatible; yoozBot-2.2; http://yooz.ir; [email protected])",
"Mozilla/5.0 (compatible; zenback bot; powered by logly +http://corp.logly.co.jp/)",
"Mozilla/5.0 (compatible; ZuperlistBot/1.0)",
"Mozilla/5.0 (compatible;acapbot/0.1;treat like Googlebot)",
"Mozilla/5.0 (compatible;acapbot/0.1.;treat like Googlebot)",
"Mozilla/5.0 (compatible;FindITAnswersbot/1.0;+http://search.it-influentials.com/bot.htm)",
"Mozilla/5.0 (compatible) AI2Bot (+https://www.allenai.org/crawler)",
"Mozilla/5.0 (compatible) Ai2Bot-Dolma (+https://www.allenai.org/crawler)",
"Mozilla/5.0 (compatible) SemanticScholarBot (+https://www.semanticscholar.org/crawler)",
"Mozilla/5.0 (Linux) (compatible; AlexandriaOrgBot/1.0; +https://www.alexandria.org/bot.html)",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36 (compatible; KosmioBot/1.0; +http://kosm.io/bot.html)",
"Mozilla/5.0 (TweetmemeBot/4.0; +http://datasift.com/bot.html) Gecko/20100101 Firefox/31.0",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20170101 Firefox/67.0",
"Mozilla/5.0 (Windows NT 6.1; rv:38.0) Gecko/20100101 Firefox/38.0 (IndeedBot 1.1)",
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; +http://url-classification.io/wiki/index.php?title=URL_server_crawler) KStandBot/1.0",
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; +http://www.komodia.com/newwiki/index.php/URL_server_crawler) KomodiaBot/1.0",
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; +https://www.ubtsupport.com/legal/Streamline3Bot.php) Streamline3Bot/1.0",
"Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0; BingPreview/1.0b) like Gecko",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 - James BOT - WebCrawler http://cognitiveseo.com/bot.html",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools)",
"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36 (compatible; Google-Read-Aloud; +https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers)",
"Mozilla/5.0 (X11; U; Linux Core i7-4980HQ; de; rv:32.0; compatible; JobboerseBot; http://www.jobboerse.com/bot.htm) Gecko/20100101 Firefox/38.0",
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0 / GnowitNewsbot / Contact information at http://www.gnowit.com",
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36",
"Mozilla/5.0+(compatible;+PiplBot;+http://www.pipl.com/bot/)",
"Neticle Crawler v1.0 ( https://neticle.com/bot/en/ )",
"NINJA bot",
"NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected])",
"OdklBot/1.0 ([email protected])",
"OutclicksBot/2 +https://www.outclicks.net/agent/gIYbZ38dfAuhZkrFVl7sJBFOUhOVct6J1SvxgmBZgCe",
"OutclicksBot/2 +https://www.outclicks.net/agent/p2i4sNUh7eylJF1S6SGgRs5mP40ExlYvsr9GBxVQG6h",
"OutclicksBot/2 +https://www.outclicks.net/agent/PryJzTl8POCRHfvEUlRN5FKtZoWDQOBEvFJ2wh6KH5J",
"OutclicksBot/2 +https://www.outclicks.net/agent/VjzDygCuk4ubNmg40ZMbFqT0sIh7UfOKk8s8ZMiupUR",
"Pinterest/0.2 (+http://www.pinterest.com/bot.html)",
"PiplBot (+http://www.pipl.com/bot/)",
"psbot-image (+http://www.picsearch.com/bot.html)",
"psbot-page (+http://www.picsearch.com/bot.html)",
"psbot/0.1 (+http://www.picsearch.com/bot.html)",
"Quantcastbot/1.0 (+https://www.quantcast.com/bot)",
"Quantcastbot/2.0 (+https://www.quantcast.com/bot)",
"RSSingBot (http://www.rssing.com)",
"RyteBot/1.0.0 (+https://bot.ryte.com/)",
"SBL-BOT (http://sbl.net)",
"Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.2",
"SemanticScholarBot/1.0 (+http://s2.allenai.org/bot.html)",
"SEMrushBot",
"SentiBot www.sentibot.eu (compatible with Googlebot)",
"SentryUptimeBot/1.0 (+http://docs.sentry.io/product/uptime-monitoring/)",
"SenutoBot/1.0 (compatible; SenutoBot/1.0; +https://www.senuto.com/)",
"SeobilityBot (SEO Tool; https://www.seobility.net/sites/bot.html)",
"SEOlizer/1.1 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 (+https://www.seolizer.de/bot.html)",
"SerendeputyBot/0.8.6 (http://serendeputy.com/about/serendeputy-bot)",
"serpstatbot/1.0 (advanced backlink tracking bot; curl/7.58.0; http://serpstatbot.com/; [email protected])",
"serpstatbot/1.0 (advanced backlink tracking bot; http://serpstatbot.com/; [email protected])",
"SlaccaleBot",
"SummalyBot/5.1.0",
"Superfeedr bot/2.0 http://superfeedr.com - Make your feeds realtime: get in touch - feed-id:1162088860",
"Synapse (bot; +https://github.com/matrix-org/synapse)",
"TangibleeBot/1.0.0.0 (http://tangiblee.com/bot)",
"TurnitinBot (https://turnitin.com/robot/crawlerinfo.html)",
"webzio (+https://webz.io/bot.html)",
"webzio-extended (+https://webz.io/bot.html)",
"WeSEE:Search/0.1 (Alpha, http://www.wesee.com/en/support/bot/)",
"Wotbox/2.01 (+http://www.wotbox.com/bot/)",
"yacybot (-global; amd64 FreeBSD 9.2-RELEASE-p10; java 1.7.0_65; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 2.6.32-042stab111.11; java 1.7.0_79; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 2.6.32-042stab116.1; java 1.7.0_79; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.10.0-229.4.2.el7.x86_64; java 1.7.0_79; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.10.0-229.4.2.el7.x86_64; java 1.8.0_45; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.13.0-61-generic; java 1.7.0_79; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.14.32-xxxx-grs-ipv6-64; java 1.8.0_111; Europe/de) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_75; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.19.0-15-generic; java 1.8.0_45-internal; Europe/de) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.2.0-4-amd64; java 1.7.0_65; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 3.2.0-4-amd64; java 1.7.0_67; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 4.4.0-57-generic; java 9-internal; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 5.2.11-Jinsol; java 12.0.2; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Linux 5.2.9-Jinsol; java 12.0.2; Europe/en) http://yacy.net/bot.html",
"yacybot (-global; amd64 Windows 8 6.2; java 1.7.0_55; Europe/de) http://yacy.net/bot.html",
"yacybot (-global; amd64 Windows 8.1 6.3; java 1.7.0_55; Europe/de) http://yacy.net/bot.html",
"yacybot (/global; amd64 FreeBSD 10.3-RELEASE-p7; java 1.7.0_95; GMT/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 FreeBSD 10.3-RELEASE; java 1.8.0_77; GMT/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 2.6.32-042stab093.4; java 1.7.0_65; Etc/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 2.6.32-042stab094.8; java 1.7.0_79; America/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 2.6.32-042stab108.8; java 1.7.0_91; America/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 2.6.32-573.3.1.el6.x86_64; java 1.7.0_85; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.10.0-229.7.2.el7.x86_64; java 1.8.0_45; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.10.0-327.22.2.el7.x86_64; java 1.7.0_101; Etc/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.11.10-21-desktop; java 1.7.0_51; America/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-042stab093.4; java 1.7.0_79; Europe/de) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-042stab093.4; java 1.7.0_79; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-45-generic; java 1.7.0_75; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-74-generic; java 1.7.0_91; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-83-generic; java 1.7.0_95; Europe/de) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-83-generic; java 1.7.0_95; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-85-generic; java 1.7.0_101; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-85-generic; java 1.7.0_95; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.13.0-88-generic; java 1.7.0_101; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.14-0.bpo.1-amd64; java 1.7.0_55; Europe/de) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16-0.bpo.2-amd64; java 1.7.0_65; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_75; America/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_75; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_79; Europe/de) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_79; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_91; Europe/de) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.7.0_95; Europe/en) http://yacy.net/bot.html",
"yacybot (/global; amd64 Linux 3.16.0-4-amd64; java 1.8.0_111; Europe/en) http://yacy.net/bot.html",
"Yanga WorldSearch Bot v1.1/beta (http://www.yanga.co.uk/)",
"ZoomBot (Linkbot 1.0 http://suite.seozoom.it/bot.html)",
"ZoominfoBot (zoominfobot at zoominfo dot com)"
]