Project Structure

The Social Crawler Index is organized into the following directory structure:

src/
├── mastra/
│   ├── agents/
│   │   └── twitter-crawler.agent.ts    # Main agent implementation
│   ├── db/
│   │   ├── schema.ts                   # Database schema definitions
│   │   └── client.ts                   # Database client implementation
│   └── tools/
│       ├── twitter-scraper.tool.ts     # Twitter scraping implementation
│       ├── coin-detection.tool.ts      # Coin detection logic
│       └── sentiment-analysis.tool.ts   # Gemini sentiment analysis
├── scripts/
│   ├── add-account.ts                  # Account management utility
│   └── view-results.ts                 # Results viewing utility
└── index.ts                            # Application entry point

Key Components

• src/mastra/agents/twitter-crawler.agent.ts: This is the core of the application, where the main agent logic for crawling Twitter is implemented.

• src/mastra/db/: This directory contains the database schema and client.

  • schema.ts: Defines the structure of the database tables.
  • client.ts: Implements the client for interacting with the database.

• src/mastra/tools/: This directory contains the tools used by the agent.

  • twitter-scraper.tool.ts: Handles the scraping of tweets from Twitter.
  • coin-detection.tool.ts: Implements the logic for detecting cryptocurrency mentions in tweets.
  • sentiment-analysis.tool.ts: Uses Gemini for sentiment analysis of tweets.

• src/scripts/: This directory contains utility scripts for managing the application.

  • add-account.ts: A script for adding Twitter accounts to monitor.
  • view-results.ts: A script for viewing the analysis results.

• src/index.ts: The main entry point of the application.