Nidhi Dumian @DumianNidhi · Yesterday 37 Followers
really fun exercise to check if your website answers high-intent, bottom of the funnel questions people are asking on reddit (and if they are doing so, they are asking those exact or very similar questions on LLMs too).
I created an agent using @craftdocs (so easy), asked it to scrape the @getclarify website using @firecrawl api and create a google sheet with these columns-
url, title, meta description, h1, h2, h3 and faqs
step2- asked craft to use firecrawl api again to scrape specific reddit threads (saas, crm, small business, startups) for the last 90 days and create another tab 'reddit-crm-research' within the same sheet with these columns-
post title, subreddit, url, main question/pain point, category, similar questions count (frequency)
step 3- asked the agent to create a 'gap analysis' tab to do this-
1- for each reddit query, cross reference against the website map.
2- fill gap analysis tab with these columns:
reddit question, category, frequency, verdict, matching url, what's missing, suggested action and suggested headings to add for each reddit query.
and finally, asked it to follow this exact process:
first, find any pages in the website map that seem relevant to that reddit query.
second, for each relevant page, check if it clearly addresses ALL the specific pain points.
for example: if someone asks "Salesforce is killing my budget, team of 5 needs alternative"- check if the page talks about ALL of these: Salesforce as an alternative, pricing transparency, small team fit, and cost predictability. Not just one of them- all of them.
third, give one of these verdicts:
-fully answered if page clearly covers all pain points, or
- partially answered if page exists but missing some pain points. if so, list exactly what's missing and suggest a specific H2 or H3 heading title for each missing point.
- and if not answered, i.e.,no relevant page exists, then suggest a new page with a proposed url, h1, and 3-4 h2 sections to create.
big thing to keep in mind- credits go really really really fast if the agent is trying to map each reddit query with each webpage and its content.
