SEO · Practical learning
Crawling, Indexing and Ranking: Diagnose the Right SEO Problem
Understand three different search stages and build a troubleshooting sequence before rewriting a page or adding keywords.
“My page is not ranking” can describe several different problems. The URL may not be accessible, Google may not have indexed it, or it may be indexed but appear below stronger results. Those situations need different checks. This lesson separates discovery, crawling, indexing and serving results through a simple diagnostic exercise. You will learn what evidence to collect before choosing an action, rather than responding to every search problem by adding more words to the page.
Start with the URL a visitor can actually load
Open the exact page address, including the expected https version. Check whether it loads successfully, redirects somewhere else or returns an error. A page that looks fine after clicking through the homepage may behave differently when loaded directly. Record the final URL and status. If the site has both www and non-www versions, check the preferred destination. You are establishing a reliable starting point. No amount of keyword editing will fix a URL that consistently sends visitors to a missing page.
Check discovery and crawl access
Search engines need a way to discover the URL, such as a link from an accessible page or a sitemap. They also need permission and technical access to fetch it. Review internal links, robots.txt and hosting restrictions. Remember that blocking crawling and requesting no indexing are different controls. Avoid changing either casually on a live site. For a student exercise, work on a test page or document the intended change and ask the site owner to review it before implementation.
Look for indexing evidence
Search Console’s URL Inspection can provide information about Google’s view of a verified website. Check the reported indexing status and canonical information, then compare with the live page. A site search can be a useful quick observation, but it is not a complete diagnostic substitute. Indexing is not guaranteed simply because you submitted a sitemap. If Google selects another canonical URL, investigate duplication and signals instead of creating yet another page to compete with the one you already have.
Treat ranking as a separate question
Once the page is indexed, examine whether its purpose fits the searches you want to attract. Review the actual content, competing results, useful supporting evidence and how the page connects with the rest of the site. A technically accessible page can still be less useful than alternatives. Avoid promising a deadline for a position change. Search systems and competing websites change, and a small sample of impressions does not provide enough evidence to diagnose every content or authority problem.
Write a focused action log
For each issue, record an observation, likely explanation, proposed action and success check. “Improve SEO” is too vague. “Restore a broken internal link, then verify the target loads and is discoverable” is actionable. Limit changes when you are trying to understand cause and effect. If you edit the template, content, redirects and sitemap on the same day, interpreting the result becomes harder. Keep dates and before-and-after evidence so another person can understand what you actually changed.
Worked example
- URL returns 404
- Check the route, file and redirect plan first.
- URL loads but has noindex
- Confirm whether exclusion is intentional.
- URL is indexed but attracts irrelevant queries
- Review page purpose, content and linking.
- URL is indexed with low positions
- Assess relevance and competing evidence; do not assume a technical fault.
Try it yourself
Choose one page you control and create a diagnostic sheet. Record its final URL, response status, indexing directive, internal entry point, sitemap presence and Search Console status if available. Write one likely issue and one testable action. If there is no clear fault, say so. The point of the exercise is to distinguish evidence from guesswork, not manufacture a problem to justify a large checklist of changes.
Questions learners ask
Does submitting a sitemap guarantee indexing?
No. A sitemap helps discovery, but indexing and serving results remain search-engine decisions.
Should I keep requesting indexing every day?
Repeated requests are not a substitute for resolving accessibility, content or canonical problems.
Keep learning
Further reading: Google: how Search works.