A distributed (and polyglot) web crawler admin platform ↦
There are other spider management systems out there, but most of them limit you to Python because Scrapyd. Crawlab has no such dependency, so you can use whichever language/framework you’d like.
(This looks like a pain to operate, but docker-compose helps quite a bit in that arena.)
Discussion
Sign in or Join to comment or subscribe