DEX8 Mascot

Platform

DEX8 is a powerful web crawler and automation platform.
With its unlimited number of deployed robots, it can reach Googlebot or Bingbot strength . Deploy an infinite number of web crawlers/scrapers/data-miners and extract any data from anywhere.

Features:

  • Software installation is not required. Everything runs in the Cloud.
  • Extract data from Amazon, eBay, or similar websites.
  • Extract data from Forex - stock markets - in real-time.
  • Deploy infinite ∞ number of web robots in the Cloud.
  • Huge capacity - each robot is deployed on a different IP address.
  • Powerful extraction: the robots work simultaneously in asynchronous mode.
  • Each robot can run several scraping tasks. Alternatively, a single task can be carried out by multiple robots.
  • Integrate DEX8 platform with Web proxy and prevent crawler bouncing.
  • Robots are controlled via a Web Panel, which can start, stop and pause the robot(s).
  • Code your own scraping files & tasks in NodeJS by using NPM libraries.
  • Use premade robot tasks to extract data easily without coding/programming.
  • Real-time monitoring - use browser to track extracted data in real-time.
  • Scraped data can be filtered and modified before being transferred to a database.
  • Extracted data is stored in the NoSQL database.
  • HTTP client error reporting in real-time.
  • Errors are stored in databases for more effective debugging.
  • Schedule tasks - define the exact time when you want your robot to start/stop automatically.
  • Export extracted data to a CSV or XLS file.
  • Use extracted data via DEX8 API Connector.

Technology:

  • NodeJS (latest version)
  • HTTP clients: dex8-sdk httpClient, chrome headless with puppeteer
  • Socket.io for real-time messaging between robot and Web panel
  • Angular 8+ (frontend - Web Panel)
  • NoSQL Database - MongoDB
DEX8 Schema