# firecrawl-policy-guardrails

## Skill Scaffold

```
firecrawl-policy-guardrails/
  SKILL.md
```

## File Descriptions

### 1. SKILL.md

**Purpose:** Implement robots.txt compliance, rate limiting policies, content filtering, and ethical scraping guardrails.
**Workflow:** Compliance skill - ensures scraping operations are ethical, legal, and respect target site policies.
**Relates to:** Extends firecrawl-security-basics; supports firecrawl-data-handling for responsible data use

## Summary

This skill implements ethical and legal guardrails for web scraping. It covers robots.txt parsing and compliance, respecting crawl-delay directives, domain blocklists for protected content, rate limiting to avoid site disruption, noindex/nofollow respect, terms of service awareness, and content licensing considerations. These guardrails ensure scraping operations are responsible, legal, and maintain good standing with content providers.
