Why are the main crawler worker threads called “ToeThreads”??
While the mascots of web crawlers have usually been spider-related, I’d rather think of Heritrix as a centipede or millipede: fast and many-segmented. Anything that “crawls” over many things at once would presumably have a lot of feet and toes. Heritrix will often use many hundreds of worker threads to “crawl”, but ‘WorkerThread’ or ‘CrawlThread’ seem mundane. So instead, we have ‘ToeThreads’.
