🧪 Skills
Crawler
Web crawler toolkit for site crawling, link extraction, content scraping, sitemap generation, rate limiting, and local data export.
v1.0.0
Description
name: crawler version: 1.0.0 author: BytesAgain license: MIT-0 tags: [crawler, tool, utility]
Crawler
Web crawler toolkit — site crawling, link extraction, content scraping, sitemap generation, rate limiting, and data export.
Commands
| Command | Description |
|---|---|
crawler run |
Execute main function |
crawler list |
List all items |
crawler add <item> |
Add new item |
crawler status |
Show current status |
crawler export <format> |
Export data |
crawler help |
Show help |
Usage
# Show help
crawler help
# Quick start
crawler run
Examples
# Run with defaults
crawler run
# Check status
crawler status
# Export results
crawler export json
How It Works
Processes input with built-in logic and outputs structured results. All data stays local.
Tips
- Run
crawler helpfor all commands - Data stored in
~/.local/share/crawler/ - No API keys required for basic features
- Works offline
Powered by BytesAgain | bytesagain.com
Reviews (0)
Sign in to write a review.
No reviews yet. Be the first to review!
Comments (0)
No comments yet. Be the first to share your thoughts!