The story
There are a lot of therapists in the world. That's the good news. The bad news: finding the right fit can be weirdly hard — and most directory UX turns into "open 47 tabs and forget what you read." So I did what any reasonable person would do: scraped a bunch of listings, pulled out what each provider *does* and *doesn't* do, then ran it through an LLM to score/rank matches against my criteria. Nothing flashy. Just a small tool that turns a painful search into a shortlist.
Approach
- Collect profile URLs from directory search results
- Scrape + normalize key fields (focus areas, modalities, logistics, availability)
- Extract 'does / doesn't' and a short summary per provider
- Rank based on criteria, then sanity-check the top candidates manually
Tech Stack
PythonWeb scrapingLLM parsing + rankingCSV/JSON
Notes
- This is not medical advice. It's a personal research helper.
- LLMs are great for summarizing and filtering, but you still verify by reading the source and talking to humans.
- Scraping is a 'be respectful' sport: rate limits, robots, and terms matter.
If you’ve got a messy problem…
I love these kinds of “small tools that save real time.” They’re not always a business — but they’re always useful.