Skip to content

qinnovates/clawfish

BranchesTags

Repository files navigation

ClawFish

License Resources Quorum Evaluated Myelin8 Indexed

Tip

263 curated SEO resources across 24 categories — every one evaluated by Quorum multi-persona review

ClawFish is a retrieval-augmented, multi-perspective, integrity-verified SEO resource system. Every resource is evaluated by Quorum census populations — not one person's opinion. Powered by Myelin8 for tiered memory, hybrid search, and Merkle integrity.

Version-Controlled Editorial Consensus (i.e. Wikipedia and GitHub-Style Review): Resources go through delegated research, peer review, citation validation (3x minimum), formal voting, and council oversight. See quorum/research-protocol.md.

Contents

SEO Tools & Frameworks

Open Source SEO Suites

  • SEOnaut — Open-source SEO auditing tool built in Go with MySQL. Analyzes websites for issues impacting rankings, generates severity-organized reports.
  • OpenSEO — Self-hostable SEO tool. Pay-as-you-go alternative to Semrush/Ahrefs. Keyword research, competitor analysis, no subscriptions.
  • OpenSEO API — Comprehensive suite of SEO tools through accessible APIs. Mission: make advanced SEO insights available without high subscription costs.
  • SEO Audits Toolkit (OSAT) — Python-based SEO & security audit toolkit. Lighthouse crawler, sitemap/keyword/image extractor, summarizer.
  • SEOstats — PHP library with 50+ methods to fetch SEO metrics from Alexa, Google, Mozscape, and SEMRush.
  • ContentSwift — Free content research and optimization tool. Alternative to Surfer, NeuronWriter, Frase.
  • Curated SEO Tools — Best SEO tools stash, organized by category.
  • AI SEO Tools — Collection of open-source tools and prompts to optimize websites for AI crawlers and modern search engines.

Awesome Lists

Commercial Platforms (Free Tiers Available)

  • Ahrefs Webmaster Tools — Free site audit and backlink data for verified site owners.
  • Google Search Console — Free. Monitor site performance in Google Search, fix issues, view search analytics.
  • Bing Webmaster Tools — Free. SEO reports, site scan, backlinks, keyword research for Bing.
  • Moz Free Tools — Domain analysis, keyword explorer (limited), link explorer, MozBar.
  • Semrush — All-in-one marketing toolkit. Keyword research, site audit, competitor analysis, content marketing.
  • Ahrefs — Backlink analysis, keyword research, content explorer, rank tracking.
  • Screaming Frog SEO Spider — Desktop website crawler. Free up to 500 URLs. Industry standard for technical audits.
  • Ubersuggest — Keyword suggestions, content ideas, backlink data. Free tier available.
  • SE Ranking — All-in-one SEO platform with rank tracking, site audit, competitor analysis.
  • Mangools — SEO toolset: KWFinder, SERPChecker, SERPWatcher, LinkMiner, SiteProfiler.

Crawlers & Scrapers

SEO-Focused Crawlers

  • Screaming Frog SEO Spider — Industry-standard desktop crawler. Finds broken links, audits redirects, analyzes page titles/meta, generates XML sitemaps.
  • SiteOne Crawler — Cross-platform website crawler for SEO, security, accessibility, and performance. Windows, macOS, Linux.
  • LibreCrawl — Free desktop SEO crawler. Open-source alternative to Screaming Frog. Crawl websites, analyze links, extract SEO data.
  • Greenflare — Lightweight open-source Python-based SEO web crawler for Linux, Mac, and Windows.
  • SEOnaut — Go-based open-source SEO crawler and auditor.

General-Purpose Crawlers (SEO-Applicable)

  • Scrapy ⭐ 53k+ — Fast high-level web crawling and scraping framework for Python. The standard for custom SEO crawlers.
  • Crawl4AI ⭐ 40k+ — LLM-friendly web crawler and scraper. Agentic crawler for complex multi-step operations.
  • Crawlee (Python) — Web scraping and browser automation library. Works with BeautifulSoup, Playwright, and raw HTTP. Proxy rotation.
  • Crawlee (JavaScript) — Web scraping library for JavaScript. Handles blocking, proxies, browsers. Works with Puppeteer, Playwright, Cheerio.
  • Scrapling — Adaptive web scraping framework. Bypasses anti-bot systems like Cloudflare Turnstile. Spider framework with pause/resume.
  • Colly ⭐ 23k+ — Fast and elegant scraping framework for Go. Clean API, supports distributed crawling.
  • Katana ⭐ 13k+ — Next-generation crawling and spidering framework by ProjectDiscovery. Standard and headless mode.
  • Heritrix — Internet Archive's web crawler. Designed for archival-quality crawling at scale.
  • Apache Nutch — Highly extensible and scalable web crawler. Integrates with Apache Solr for search.
  • MechanicalSoup — Python library for automating interaction with websites. Built on Requests and BeautifulSoup.

Headless Browsers

  • Playwright ⭐ 70k+ — Browser automation library. Chromium, Firefox, WebKit. Essential for JS-rendered SEO auditing.
  • Puppeteer ⭐ 89k+ — Node.js library for controlling Chrome/Chromium. Headless browser automation.
  • Selenium ⭐ 31k+ — Browser automation framework. Multi-language. For testing JS-heavy sites.

Analytics & Monitoring

Rank Tracking

  • SerpBear ⭐ 3k+ — Open-source search engine position rank tracking app. Self-hosted, unlimited keywords, Google Search Console integration, built-in SERP API.
  • Google Rank Tracker — Python script + cron job to check Google ranks on a daily basis.
  • SE Ranking — Real-time rank tracking with API access. 100K free API credits trial.
  • AccuRanker — Enterprise rank tracker. On-demand refreshes, SERP feature tracking.
  • SERPWatcher (Mangools) — Visual rank tracking with performance index scoring.

Web Analytics

  • Plausible ⭐ 21k+ — Lightweight, privacy-friendly web analytics. Open source alternative to Google Analytics.
  • Umami ⭐ 24k+ — Simple, fast, privacy-focused alternative to Google Analytics. Self-hosted.
  • Matomo ⭐ 20k+ — Google Analytics alternative. Full data ownership. GDPR compliant.
  • PostHog ⭐ 23k+ — Product analytics, session replay, feature flags. Self-hosted or cloud.
  • Google Analytics 4 — Free web analytics from Google. Event-based tracking, cross-platform.
  • Google Search Console — Free. The single most important SEO monitoring tool. Search performance, index coverage, Core Web Vitals.

Log File Analysis

  • GoAccess ⭐ 19k+ — Real-time web log analyzer. Terminal and browser dashboard. Analyze how bots crawl your site.
  • Screaming Frog Log File Analyser — Analyze search engine bot behavior from server logs.

Keyword Research

Open Source Tools

Commercial Tools

  • Ahrefs Keywords Explorer — Keyword difficulty, search volume, click metrics, SERP analysis for 10+ search engines.
  • Semrush Keyword Magic Tool — 26B+ keywords database. Keyword clustering, intent analysis.
  • KWFinder (Mangools) — Long-tail keyword research with accurate difficulty scores.
  • Ubersuggest — Keyword suggestions, content ideas, and competitive analysis. Free tier.
  • AnswerThePublic — Visual keyword research tool. Questions, prepositions, comparisons people search.
  • AlsoAsked — Maps "People Also Ask" data in a tree structure. Great for content ideation.
  • Keyword Surfer — Free Chrome extension showing search volumes in Google SERP.
  • Google Keyword Planner — Free with Google Ads account. Search volume ranges and competition data.
  • Google Trends — Free. Search interest over time, regional interest, related queries.

Link Building & Analysis

Open Source

  • Moz Link Explorer (Free) — Domain authority, page authority, linking domains. Limited free queries.
  • Majestic — Trust Flow, Citation Flow, backlink history. Free tier for verified sites.

Commercial Platforms

  • Ahrefs Backlink Checker — Most comprehensive backlink index. Free limited version available.
  • Semrush Backlink Analytics — Backlink audit, toxic score, competitor gap analysis.
  • Moz Link Explorer — Domain/page authority metrics, spam score, linking domains.
  • Majestic SEO — Trust Flow and Citation Flow metrics. Historical backlink data.
  • BuzzSumo — Content discovery and link prospecting. Find most-shared content by topic.
  • HARO (Connectively) — Help A Reporter Out. Earn backlinks through journalist outreach.
  • Hunter.io — Find email addresses for outreach. Domain search, email verification.
  • Pitchbox — Outreach and link building platform for agencies.
  • Respona — AI-powered link building and digital PR outreach.

Performance & Core Web Vitals

Open Source Tools

  • Lighthouse ⭐ 28k+ — Automated auditing for performance, accessibility, SEO. Built into Chrome DevTools.
  • WebPageTest — Performance testing from multiple locations. Detailed waterfalls and filmstrips.
  • Sitespeed.io ⭐ 5k+ — Open-source tool for monitoring web performance. Coach, Browsertime, PageXray.
  • Unlighthouse ⭐ 4k+ — Scan entire site with Google Lighthouse. Per-route reports.
  • yellowlab.tools — Online test to help speed up heavy web pages.

Commercial & Free Services

  • PageSpeed Insights — Free. Google's Core Web Vitals test. Lab + field data.
  • GTmetrix — Performance testing with Lighthouse scores, waterfalls, video playback.
  • Cloudflare Web Analytics — Free, privacy-first analytics with Core Web Vitals.
  • CrUX Dashboard — Chrome User Experience Report. Real-world Core Web Vitals data.
  • web.dev — Google's web development best practices. Test any URL.

Content Optimization

Open Source

  • ContentSwift — Free content research and optimization tool. Alternative to Surfer and Frase.
  • textstat ⭐ 2k+ — Python library for text readability metrics. Flesch score, grade levels, etc.
  • Natural — General natural language processing for Node.js. Tokenization, stemming, classification.

Commercial AI-Powered

  • Surfer SEO — Real-time content scoring, NLP analysis, Google Docs integration. Gold standard for content optimization.
  • Clearscope — Content optimization with NLP-driven recommendations. Used by major content teams.
  • Frase — Agentic SEO and GEO platform. Content briefs, optimization, SERP analysis.
  • MarketMuse — AI content planning and optimization. Topic authority modeling.
  • Rankability — IBM Watson + Google NLP for keyword recommendations. Monthly SEO coaching calls.
  • GrowthBar — AI-powered keyword research, content outlines, competitive analysis.
  • NeuronWriter — Content optimization with NLP and SERP competition analysis.
  • Dashword — Content optimization tool. Analyzes top-ranking content for optimization signals.

Technical SEO

Validation & Testing

Open Source Tools

  • Reppy — Modern robots.txt parser for Python. Wildcard matching, RFC support.
  • html-meta-tags — Extract and validate HTML meta tags from URLs.
  • canonical-path — Validate canonical URL implementations.

Guides & References

Mobile SEO

International SEO

Local SEO

Platforms & Tools

  • Google Business Profile — Free. The single most important local SEO listing. Feeds Google Maps and local pack.
  • Whitespark — Local SEO tools, citation building, review management. Built by local SEO expert Darren Shaw.
  • BrightLocal — Citation tracking, review monitoring, local SEO audits, citation building services.
  • Localo — Local SEO optimization tools including citation management software.
  • Merchynt (Paige) — AI-powered Google Business Profile management and citation management.
  • Moz Local — Local listing management, distribution to directories, review monitoring.
  • Yext — Enterprise local SEO. Listing management across 200+ directories.

Citation Aggregators

  • Data Axle — Distributes business data to directories and platforms.
  • Neustar Localeze — Citation data distribution across multiple outlets simultaneously.

Local SEO Guides

E-commerce SEO

Schema & Structured Data

Tools & Generators

Validators

Sitemaps & Robots

Sitemap Generators

Robots.txt

AI-Powered SEO Tools

Open Source

  • AI SEO Tools — Open-source tools and prompts for optimizing websites for AI crawlers and modern search engines.
  • Crawl4AI — LLM-friendly crawler with agentic capabilities and automated schema generation.

Commercial Platforms

  • Surfer SEO — AI-powered content optimization with NLP analysis.
  • Frase — Agentic SEO platform. AI content briefs, optimization, SERP analysis.
  • Clearscope — NLP-driven content optimization recommendations.
  • Rankability — IBM Watson + Google NLP for keyword recommendations.
  • AthenaHQ — AEO and GEO platform for AI search optimization.
  • MarketMuse — AI content planning with topic authority modeling.
  • ChatGPT/Claude for SEO — Using LLMs for keyword ideation, content drafting, meta description generation.

GEO (Generative Engine Optimization)

  • AthenaHQ — Optimize for AI search engines (ChatGPT, Perplexity, Google AI Overviews).
  • Frase — "Rank on Google. Get cited by AI." GEO + traditional SEO.

Learning Resources

Official Documentation

Free Learning

Courses & Certifications

Free Certifications

Paid Courses

Blogs & Newsletters

SEO Blogs

SEO Newsletters

Podcasts

Books

Beginner

  • The Art of SEO — The 1,000+ page definitive guide. Beginner to advanced. By Eric Enge, Stephan Spencer, Jessie Stricchiola.
  • SEO for Dummies — Explains complex concepts in simple terms with actionable steps. By Peter Kent.
  • SEO Workbook — Step-by-step with companion videos and worksheets. By Jason McDonald.
  • The Illustrated SEO — Visual learning approach to SEO fundamentals. Clear diagrams and memorable concepts.
  • SEO 2025 — Latest strategies including Google's 2024 algorithm updates. By Adam Clarke.

Advanced

  • Entity SEO — Entity-based optimization and schema markup for advanced professionals. By Dixon Jones.
  • Product-Led SEO — User-centric SEO approach. Product before optimization. By Eli Schwartz.
  • SEO Writing — Keyword research integration, search intent, content optimization. By Victoria Kurichenko.

Link Building

Communities

APIs & Datasets

SEO APIs

  • Google Search Console API — Free. Programmatic access to search performance data.
  • Google PageSpeed Insights API — Free. Lighthouse scores and Core Web Vitals via API.
  • DataForSEO — Comprehensive SEO data APIs: SERP, keywords, backlinks, on-page, domain analytics.
  • SerpApi — Google, Bing, Yahoo, Baidu, Yandex SERP scraping API.
  • SE Ranking API — Rank tracking, site audit, backlink monitoring API.
  • Moz API — Domain authority, page authority, backlinks API.
  • Ahrefs API — Backlink data, organic keywords, referring domains API.
  • Majestic API — Trust Flow, Citation Flow, backlink data API.
  • CognitiveSEO API — Backlinks, rankings, social signals API.

Datasets

Browser Extensions

Free SEO Extensions

SEO Requirements Checklist

Technical SEO

  • Crawlability — Site is crawlable by search engine bots. No critical pages blocked by robots.txt
  • Indexability — Important pages are indexable (no accidental noindex tags)
  • HTTPS — Site served over HTTPS with valid SSL certificate
  • Site Speed — Core Web Vitals passing (LCP < 2.5s, INP < 200ms, CLS < 0.1)
  • Mobile-First — Site is responsive and passes mobile-friendly test
  • XML Sitemap — Valid XML sitemap submitted to Search Console and Bing Webmaster Tools
  • Robots.txt — Properly configured robots.txt allowing important pages, blocking irrelevant ones
  • Canonical Tags — Self-referencing canonicals on all important pages. No duplicate content issues
  • Hreflang Tags — Correct hreflang implementation for multi-language sites
  • Structured Data — Relevant Schema.org markup (Organization, BreadcrumbList, Article, Product, FAQ, etc.)
  • Internal Linking — Logical site structure with clear internal link paths
  • Redirect Chains — No redirect chains or loops. Clean 301 redirects
  • 404 Handling — Custom 404 page. No important pages returning 404
  • Server Response — Time to First Byte (TTFB) under 200ms
  • JavaScript Rendering — Critical content not dependent on JavaScript rendering
  • Log File Analysis — Regular analysis of how search bots crawl the site

On-Page SEO

  • Title Tags — Unique, descriptive, 50-60 characters, primary keyword included
  • Meta Descriptions — Unique, compelling, 150-160 characters, includes CTA
  • Heading Structure — Single H1, logical H2-H6 hierarchy
  • Content Quality — Original, comprehensive, satisfies search intent, E-E-A-T aligned
  • Image Optimization — Descriptive alt text, compressed file sizes, modern formats (WebP, AVIF)
  • URL Structure — Short, descriptive, hyphenated, lowercase
  • Internal Links — Relevant internal links with descriptive anchor text

Off-Page SEO

  • Backlink Profile — Growing, diverse, high-quality backlink profile
  • Brand Mentions — Active brand presence across relevant platforms
  • Digital PR — Ongoing outreach for earned media and editorial links
  • Social Signals — Active social media presence (indirect ranking factor)
  • Toxic Link Audit — Regular audit and disavow of toxic/spammy backlinks

Content SEO

  • Keyword Research — Data-driven keyword strategy targeting relevant terms
  • Content Strategy — Topic clusters with pillar and supporting content
  • Search Intent — Content matches informational, navigational, transactional, or commercial intent
  • Content Freshness — Regular updates to important content
  • E-E-A-T — Demonstrate Experience, Expertise, Authoritativeness, Trustworthiness

Local SEO (if applicable)

  • Google Business Profile — Claimed, verified, and optimized
  • NAP Consistency — Name, Address, Phone consistent across all citations
  • Local Citations — Listed in relevant local and industry directories
  • Reviews — Active review management on Google, Yelp, industry platforms
  • Local Schema — LocalBusiness structured data with opening hours, geo-coordinates

Analytics & Monitoring

  • Google Search Console — Set up, sitemap submitted, monitoring impressions/clicks/CTR
  • Google Analytics / Alternative — Tracking organic traffic, conversions, user behavior
  • Rank Tracking — Monitoring keyword positions over time
  • Core Web Vitals Monitoring — Regular checks via CrUX or PageSpeed Insights
  • Backlink Monitoring — Tracking new and lost backlinks

Quorum Evaluation

Every resource in ClawFish is evaluated using Quorum multi-persona review. See the quorum/ directory for:

  • personas.yaml — 10 SEO expert personas (Technical Architect, Content Strategist, Link Builder, Local Expert, E-commerce Specialist, Data Analyst, International Strategist, Tool Developer, Agency Director, Search Researcher)
  • census-populations.yaml — 6 evaluation panels for different resource types
  • research-protocol.md — Delegated discovery, editorial consensus, citation validation (3x minimum), anti-boxing, anti-echoing, anti-faking
  • judicial-system.md — Formal voting protocol, judicial scoring, improvement loops, sentiment slider

How It Works

  1. Research is delegated by MECE partition — no duplicate crawling
  2. Resources go through Wikipedia/GitHub-style editorial review with multiple rounds
  3. Citations are validated 3x minimum by different personas
  4. Uncertain resources go through formal 1-by-1 blind voting
  5. External councils (Governance, Neuromodest, Ethics) audit the process
  6. Default stance is intentionally skeptical — adjustable via sentiment slider

Star History

ClawFish Mascot

Star History Chart

Contributing

Contribute to ClawFish, get immortalized as a persona.

Every contributor who submits a meaningful contribution gets a permanent AI persona created in their name within the ClawFish persona library. Your persona becomes part of the Quorum evaluation system — your digital identity lives on, reviewing and debating resources alongside other contributor personas.

See CONTRIBUTING.md for submission guidelines and docs/PERSONAS.md for the persona specification.

License

MIT © 2026 Kevin Qi

Built with Quorum | Powered by Myelin8

About

The SEO Resource Library. 263+ curated resources. Quorum multi-persona evaluated. Myelin8-indexed.

Resources

Readme

License

MIT license

Contributing

Contributing
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors