DuckDuckGo Search Engine Scraper

searchEngineDuckDuckGo

Turnkey task searchEngineDuckDuckGo extracts search results from DuckDuckGo.com search engine.
It will extract URL, title, text and determine if the search result is paid or organic.
The extracted data is saved into the database with option to be classified under category and subcategory.

Features:

• lists DuckDuckGo search engine results by entering a specific search term
• search a specific website by defining site: search operator
• generic search results from All type (web results)
• scrolls down the web page and clicks the "More Results" button automatically
• classifies the extracted data under category and subcategory
• saves the extracted data in NoSQL database as JSON document
• track results in real-time

Usage examples:

• SERP - search engine result page
• SEO - search engine optimisation
• Aggregated Search Engine - presents search results from different information sources
• detect position of a web page and monitor its competition
• determine if search result is organic or paid result
• access DuckDuckGo search results via DEX8 API
• schedule this task and execute periodically

Input parameters:

Input File

The example of input file (input_default.js):

module.exports = {
  site: 'www.adsuu.com',
  search_term: 'laptop',
  mo_uri: 'mongodb://dex8_freeuser:xxx@5.189.161.70:27017/dex8-dev-pool-free01',
  mo_coll: 'adverts_duckduckgo',
  category: 'IT',
  subcategory: 'computers'
};

Output Result

A document stored in the MongoDB database:

{
  "_id": "5ea2ffaefceea8a3ae6912f3",
  "url": "https://www.adsuu.com/business-offer-2/",
  "updated_at": "2020-04-26T06:05:23.753Z",
  "search_term": "laptop",
  "site": "www.adsuu.com",
  "se": "duckduckgo",
  "subcategory": "computers",
  "category": "IT",
  "title": "Browse ads in Business Offer",
  "__v": 0,
  "created_at": "2020-04-24T15:03:15.526Z",
  "is_paid": false,
  "text": "HP LAPTOP REPAIR CENTRE Are you having issues while accessing your Hp laptop or computer ...:: Australia, Melbourne :: 10.09.2019. :: business - offer. 60. HOW TO DISABLE MCAFEE REAL-TIME SCANNING ON W There is quite a time when users want to disable McAfee real-time..."
}

PRICE: 30.00 EUR /month

To buy this product you need to sign up for a free account and login .

  Click here to Sign Up for free !