Chapter 8: Research and Competitive Intelligence#

What It Is#

Claude Code can function as a research analyst. It can search the web, fetch and read entire web pages, scrape multi-page websites, analyze PDF reports, pull data from external sources, interact with live web pages through a browser, and synthesize everything into structured deliverables. The combination of web access, file system access, and analytical capability makes it uniquely suited for research tasks that involve gathering, processing, and synthesizing information from multiple sources.

Why It Matters for Non-Technical Professionals#

Research is the foundation of strategy, but it is also one of the most time-consuming activities in any professional’s week. Competitive analysis, market sizing, brand positioning research, content strategy audits, investor due diligence — these tasks require gathering information from many sources, reading large volumes of material, identifying patterns, and synthesizing findings.

The traditional options are: do it yourself (slow), hire a research firm (expensive), or ask an analyst (creates dependency). Claude Code offers a fourth option: delegate to an agent that can access the same sources you would, read them faster, and synthesize them more systematically.

What You Need#

Plugins: - playwright — for browser-based research and scraping - context7 — for accessing up-to-date documentation and frameworks

MCPs: - Playwright — browser automation for interacting with live websites - Claude-in-Chrome — for researching within authenticated pages (e.g., logged-in LinkedIn, Google Reviews) - MoSPI Stats (optional) — for Indian government economic data

Permissions: - WebSearch — for searching the internet - WebFetch(domain:*) — approve specific domains as needed

The Workflows#

Workflow 1: Deep Brand Research#

As a Claude prompt:

I need comprehensive brand positioning research on [Brand Name],
a company entering the [category] space. Conduct deep research:

1. What is their current positioning? How do they describe themselves?
2. Who are their direct and indirect competitors?
3. What whitespace exists in the market that they could own?
4. Analyze their founder's public communications (interviews,
   podcast appearances, blog posts) for messaging patterns
5. What is the broader market narrative they're operating within?

Use web search to find and analyze multiple sources. Save each
section as a separate file in a new folder:
/Users/krishna/Desktop/Research/brand-research/

Create a final synthesis document that combines all findings.

What Claude does: 1. Searches the web for the brand, its founders, competitors, and market 2. Reads multiple web pages, articles, and interviews 3. Analyzes messaging patterns and positioning 4. Identifies competitive landscape and whitespace 5. Produces multiple research documents plus a synthesis

Real example: I conducted deep brand research for a company entering a new consumer category. Claude downloaded and transcribed a founder podcast, analyzed investor blog posts, scraped the company’s website for messaging patterns, researched competitors, and produced five separate documents covering brand positioning, competitive landscape, storytelling frameworks, and recommended messaging strategies. The research was thorough enough to be used directly in a strategy meeting.

Workflow 2: Web Scraping Entire Websites#

As a Claude prompt:

Scrape the complete content from this website:
https://example.com/growth-guide/

Include:
- All text content from every page
- Image URLs (don't download images, just capture the URLs)
- Any embedded YouTube video links
- Preserve the original page structure and headings

Save the scraped content as organized markdown files in:
/Users/krishna/Desktop/Research/scraped-content/

What Claude does: 1. Navigates to the site and identifies all sub-pages 2. Fetches each page’s content 3. Extracts text, image URLs, and video links 4. Saves everything as structured markdown files

Real example: I scraped an entire multi-page growth guide from a marketing education website, preserving all content including embedded YouTube links. Claude then converted the raw scraped content into a clean, designed single-page HTML file that I could host locally for reference.

Workflow 3: Competitive Intelligence and Content Strategy#

As a Claude prompt:

Research [Company/Brand]'s LinkedIn content strategy.

I need to understand:
1. What topics do they post about most frequently?
2. What formats do they use (text only, carousels, video, articles)?
3. What is their posting frequency?
4. Which posts get the most engagement?
5. How do they position their brand voice?
6. What content gaps exist that a competitor could exploit?

Search their LinkedIn page, their website blog, any press coverage,
and their team members' public profiles.

Compile a strategy brief I can use for a content planning meeting.
Save as a markdown file.

What Claude does: 1. Searches for the company’s public content across platforms 2. Analyzes posting patterns, topics, and engagement signals 3. Identifies content strategy patterns (what they cover, what they avoid) 4. Produces a strategic brief with observations and recommended actions

Real example: I researched a competing firm’s LinkedIn content strategy before a content planning session. Claude analyzed their posting history, identified their content pillars, assessed which types of posts drove the most engagement, and produced a competitive brief that informed our own strategy. The research included specific recommendations for content themes we could own that the competitor was not covering.

Workflow 4: Using the Claude-in-Chrome Extension for Live Research#

Some research requires interacting with live web pages — reading Google reviews, navigating authenticated platforms, or extracting data from dynamic web applications. The Claude-in-Chrome extension handles this. It is now generally available on every direct Anthropic plan, so if you skipped it earlier because it was preview-only, it works on Pro today — you still install the extension and grant per-site permissions before Claude can act on a page.

As a Claude prompt (after connecting Claude-in-Chrome):

I have a Google Maps page open with reviews for [Business Name]
in [City]. Read through the latest 50 reviews and give me:

1. Overall sentiment breakdown (positive/negative/neutral)
2. Top 3 recurring praise themes
3. Top 3 recurring complaint themes
4. Any notable outlier reviews (unusually detailed or specific)
5. An overall assessment of customer satisfaction trends

What Claude does: 1. Reads the currently open browser page through the Chrome extension 2. Scrolls through reviews and extracts content 3. Analyzes sentiment patterns 4. Produces a structured summary

Workflow 5: Government Data APIs (MoSPI Stats)#

If you work with Indian economic data, the MoSPI Stats MCP connects Claude directly to government statistical databases.

As a Claude prompt:

Pull the latest Consumer Price Index (CPI) data from MoSPI for
the "Food and Beverages" category for the last 12 months.

Show the month-over-month trend and calculate the year-over-year
inflation rate. Create a chart showing the trend.

What Claude does: 1. Queries the MoSPI API through the MCP 2. Retrieves the requested data series 3. Computes derived metrics (month-over-month, year-over-year) 4. Creates a visualization

This is a niche but powerful capability. For anyone working with Indian economic data — investors, policy researchers, journalists, business analysts — having direct API access to government statistics through a conversational interface is transformative.

Workflow 6: Academic Literature Research with Free Government APIs#

For researchers, analysts, and anyone who needs to review published literature, Claude Code can search academic databases that are completely free and require no API keys.

What you need: - No plugins, no MCPs, no API keys. Claude calls these free APIs directly. - PubMed (NCBI E-utilities) — the US National Library of Medicine’s database of biomedical literature - ClinicalTrials.gov — the NIH’s registry of clinical studies - Unpaywall — checks if free full-text versions of papers are available

As a Claude prompt:

I need a literature review on [topic]. Search PubMed for the
most relevant papers published in the last 3 years.

For each paper found:
1. Title, authors, journal, and publication date
2. Abstract summary in 2-3 sentences
3. Check if a free full-text version is available via Unpaywall

Then write a synthesis of the key themes, noting:
- Areas of consensus across papers
- Contradictory findings
- Gaps in the current research

Save the raw paper data as JSON and the synthesis as a markdown file.

What Claude does: 1. Queries PubMed’s free E-utilities API with your search terms 2. Retrieves paper metadata and abstracts 3. Checks Unpaywall for open-access full-text availability 4. Optionally searches ClinicalTrials.gov for relevant ongoing studies 5. Synthesizes findings into a structured literature review

This workflow is not limited to biomedical research — PubMed covers a wide range of scientific literature. And the pattern (free API → data collection → synthesis) transfers to any domain with public databases.

Workflow 7: Parallel Research with Sub-Agents#

For large research projects that need multiple angles investigated simultaneously, Claude Code can dispatch specialized research sub-agents that work in parallel.

As a Claude prompt:

I need comprehensive research on [company/topic] from three angles.
Dispatch these as parallel research tasks:

1. Market positioning — What is their market position? Who are the
   key competitors? What differentiates them?
2. Recent developments — What has happened in the last 6 months?
   Any funding, launches, partnerships, or controversies?
3. Content and public presence — What is their content strategy?
   How do they show up on LinkedIn, Twitter, and their blog?

Run all three in parallel. When complete, synthesize the findings
into a single research brief.

What Claude does: 1. Spawns three research sub-agents, each with its own context window 2. Each sub-agent conducts independent web searches, reads sources, and compiles findings 3. Sub-agents work simultaneously — three five-minute research tasks complete in five minutes, not fifteen 4. The parent agent synthesizes all three reports into a unified brief

The sub-agents can use cheaper models (Sonnet instead of Opus), making this cost-effective. More importantly, each sub-agent starts with fresh context — no biases from the parent agent’s earlier reasoning — which often produces more objective findings.

Workflow 8: The Pre-Meeting Intelligence Brief#

Before any high-stakes conversation, I run the same play: deep research on the other side. Before an interview at a company I was excited about, I asked Claude to research the company, its sector, its competitors, and its positioning. It dispatched multiple research agents in parallel — the same mechanic as Workflow 7 — and came back with a cited brief plus talking points.

This is the most universal research prompt there is: “I am meeting X on Thursday — brief me.” A job interview, a partnership call, a sales meeting, an investor conversation — the shape never changes, only the names.

Do not leave the ask open-ended. Structure it: company overview, the people you will meet, recent news, competitive position, and three questions to ask them. The three questions are the underrated part — a sharp question signals preparation better than any fact you can recite.

Say to Claude:

I am meeting [company/person] on [day] about [context]. Run deep
research: company overview, the people involved, recent news,
competitors and positioning. Give me a one-page brief with sources
and three sharp questions I should ask.

What Claude does: 1. Spawns parallel research agents covering the company, the people, and the competitive picture 2. Reads and cross-checks sources, keeping citations for every claim 3. Synthesizes everything into a one-page brief you can absorb in five minutes 4. Drafts questions grounded in what it actually found — not generic filler

Most people walk into important meetings having skimmed a homepage. A researched brief is a preparation gap you can open in fifteen minutes.

Pro Tips#

Tip 1: Create a dedicated research folder for each project. Before starting a research task, create a folder (e.g., /Desktop/Research/project-name/). Tell Claude to save all outputs there. This keeps research organized and makes it easy to reference later.

Tip 2: Ask Claude to cite its sources. “For each finding, note the source URL so I can verify.” This is especially important for competitive intelligence and market research where accuracy matters.

Tip 3: Use web search for breadth, web fetch for depth. WebSearch finds relevant pages. WebFetch reads them in detail. A good research workflow starts with search to identify sources, then fetch to read the most promising ones.

Tip 4: For authenticated content, use Claude-in-Chrome. The Playwright browser opens a fresh, logged-out browser. Claude-in-Chrome uses your actual Chrome with your logins. Use it when you need to research content behind a login wall.

Tip 5: Save research as structured files, not just summaries. Ask Claude to save raw data separately from analysis. This lets you re-analyze later without re-scraping.

Workflow 9: Content Strategy from Internal Transcripts#

A less obvious but highly valuable research workflow: feeding Claude internal meeting transcripts or brainstorming recordings and having it produce structured content strategy documents.

As a Claude prompt:

I am attaching the transcript of a strategy discussion about our
content plans for [sector/vertical]:
/Users/krishna/Desktop/Strategy/brainstorming-call-transcript.txt

From this transcript, extract:
1. All content ideas mentioned (with the person who suggested them)
2. The strategic priorities that emerged
3. Any competitive examples referenced
4. Action items and next steps discussed

Then research the competitive examples mentioned. For each
competitor referenced, find what they have actually done (events,
reports, podcasts, content series) in this space.

Finally, synthesize everything into a content strategy brief with:
- Recommended content pillars
- Suggested IP names for recurring content properties
- A phased rollout plan (what to launch first, what to build
  toward)
- Competitive gaps we can own

Save each section as a separate file, plus a master strategy
document.

What Claude does: 1. Reads the transcript and extracts all discussion points 2. Researches every competitor referenced using web search 3. Identifies content gaps between what competitors have done and what was discussed 4. Produces a structured strategy document with pillar recommendations, naming options, and a phased timeline

Real example: I uploaded two call transcripts from a content strategy planning process for a deep tech vertical. Claude extracted all the discussed ideas, then researched competitor VC firms’ events and reports in the same space — finding actual event pages, published reports, and podcast series. The output included a phased content roadmap, ten naming options for a new podcast IP (constrained to a specific thematic direction I specified), and a competitive landscape showing which content angles were uncontested. What would have been days of competitive research and strategy synthesis became a single session.

Why this works well with transcripts: Meeting transcripts are messy — ideas are interspersed with tangents, agreements are implicit, and action items are scattered. Claude excels at extracting structure from unstructured conversation. The combination of transcript analysis plus competitive web research produces strategy documents that are grounded in both internal thinking and external reality.