Technical SEO Cheat Sheet

Crawl, index, canonical — in the order search engines apply them

Technical SEO is a pipeline: a page must be crawlable before it can be indexed, and indexable before ranking is even a question. Most problems are one stage failing silently.

Category: SEO Difficulty: Intermediate Version: 1.0 Updated: June 6, 2025 Author: Sabir

Directives

Directive	Says	Common mistake
`robots.txt Disallow`	Do not CRAWL this path	Expecting it to de-index — a blocked URL can still be listed
`<meta name="robots" content="noindex">`	Do not INDEX this page	Blocking the page in robots.txt so the tag is never read
`rel="canonical"`	This is the preferred version	Every page pointing at the homepage
`301`	Moved permanently	Chaining several hops instead of pointing at the final URL
`302`	Moved temporarily	Using it for a permanent move, so equity never transfers
`410`	Gone, deliberately	Serving 404 for content you intentionally removed
`hreflang`	This page in another language	Missing the required self-referencing tag

Audit order

Can it be crawled? — robots.txt, then server response, then internal links pointing at it
Can it be indexed? — No noindex, canonical points at itself, returns 200
Does it render? — If content only appears after JavaScript, confirm it survives rendering
Is it unique? — Near-duplicate pages compete with each other, not with rivals
Is it fast on a phone? — Core Web Vitals are measured on mobile first
Is the structured data truthful? — Marking a page as something it is not gets rich results suppressed site-wide

FAQs

Why is a page still in the index after I blocked it in robots.txt?

Because Disallow blocks CRAWLING, not indexing. The engine can still list a URL it has never fetched. To remove it, allow the crawl and serve a noindex tag.

Should every page have a canonical tag?

Yes, and it should usually point at itself. The common failure is pointing every page at the homepage, which asks search engines to drop the entire site but one page.