# crawls.in

> A visual domain search engine. It holds a screenshot of the homepage of every domain it has crawled, across every
> extension, and lets you search them by keyword or by Google-style operators — then shows the extension landscape
> of each name (which TLDs exist for the same base name, and which of them are developed sites).

Built by Harmony Infotech (https://dom.to). Free to use, no account.

## Searching

- [Search](https://crawls.in/?q=spices): `https://crawls.in/?q=<query>` — a plain word or phrase matches domain names and page titles.
- Operators can be mixed into the query:
  - `tld:in` — restrict to one extension (`tld:co.in`, `tld:com`, …)
  - `bucket:developed` — `developed`, `active`, `basic`, `minimal` or `parked`, by how much real content the screenshot shows
  - `entropy:>=5` — the same signal as a number (higher = more developed page)
  - `cms:wordpress`, `tech:shopify` — the detected platform / tech stack
  - `intitle:bakery` — the page title must contain the word
  - `len:<=6`, `ext:>=3`, `extcount:>=3` — base-name length, and how many extensions of the same base name exist
  - `sort:newest` — also `oldest`, `short`, `long`, `entropy`, `ext`
- [Changelog](https://crawls.in/changelog): what changed in the engine and when.

## For agents and scripts

- `GET https://crawls.in/api/visual-search?q=<query>&limit=<n>` returns JSON: `count`, `total_indexed`, `time_ms`, `mode`
  and `results[]` with `domain`, `base_name`, `best_tld`, `ext_list`, `ext_count`, `entropy`, `cms`, `page_title`,
  `thumb_url` (the screenshot) and `nsfw`. The same operators work in `q`. Please keep requests modest; it is the
  endpoint behind the page, not a bulk export.

## Related

- [dom.to](https://dom.to): the domain toolkit this engine is part of.
- About: https://crawls.in/about — what it is, who it is for, how to search, the API, fair use
- Privacy: https://crawls.in/privacy — what is recorded when you search, for how long, your rights
- Terms: https://crawls.in/terms — fair use of the search and the API, other people's websites, removal requests