Crawl budget refers to the limit on how many times a search engine crawler (such as Googlebot) will crawl a given site within a certain period. Google adjusts crawl frequency per site to avoid placing excessive load on servers.
Crawl budget is determined by two factors. The first is the crawl capacity limit, also known as hostload, which follows from how many connections Googlebot opens in parallel and how long it waits between fetches; it drops when responses slow down or when the server starts returning 5xx errors or 429 responses. The second is crawl demand, which follows from the number of URLs Google has discovered, the popularity of the site and its pages, and how long it has been since a page was last crawled. Older Google material called the first factor the crawl rate limit; the current guide calls it the crawl capacity limit.
Page count alone does not decide whether crawl budget deserves attention. Google's guide addresses sites with more than one million unique pages whose content changes about once a week, and sites with more than ten thousand unique pages whose content changes very frequently (daily), while stating that these figures are rough guidance rather than exact thresholds. The practical test is whether a newly published page is crawled the same day. If it is, keeping the sitemap current and checking indexing status from time to time is enough. When a large site leaves new pages waiting a long time to be crawled, crawl budget optimization does become an SEO concern.
The relationship between URL shortening services and crawl budget is indirect but important. Services that generate large volumes of shortened URLs may have each shortened URL's page (such as preview pages) become a crawl target. What matters here is that robots.txt and noindex do different jobs. Robots.txt stops crawling and does not guarantee that a URL stays out of the index. Noindex keeps a page out of the index and only takes effect if the crawler can fetch the page. Applying both to the same URL backfires: crawling stops, so the noindex is never read and the intended effect never happens. Use robots.txt for URLs whose crawling you want to reduce, use noindex for URLs you only want kept out of search results, and pick one or the other.
Key techniques for optimizing crawl budget include: keeping server response times fast (raising the crawl capacity limit), removing duplicate and low-quality pages (narrowing the crawl scope), using XML sitemaps to highlight important pages (guiding crawl priority), and blocking unnecessary paths with robots.txt (eliminating wasted crawls).