Mongo

Save, list, delete, update extracted data

Introduction

Mongo is a DEX8 helper to manage extracted data in MongoDB database.
It can connect to the mongo server, disconnect from it, save, list, update and delete documents in mongo database.

Mongoose library is used to create mongodb models and methods.
mongoose


The Generic.js mongoose schema is used as default model. It inserts user_id, robot_id and task_id automatically every time new document is inserted.

const collection = 'generic';

// options
const opts = {
  collection, // default collection
  _id: true, // disable _id
  id: false, // set virtual id property
  autoIndex: true, // auto-create indexes in mognodb collection on mongoose restart
  minimize: true, // remove empty objects
  // safe: true, // pass errors to callback
  strict: false, // values not defined in schema will not be saved in db
  validateBeforeSave: true, // validate doc before saving. prevent saving false docs
  timestamps: { // create timestamps for each doc 'created_at' & 'updated_at'
    createdAt: 'created_at',
    updatedAt: 'updated_at'
  },
  new: true,
  // skipVersioning: { myProp: true }, // prevent changing version __v when 'myProp' is updated
  // versionKey: '_myV', // replace __v with _myV (not needed in most cases)
};


// mongoose schema
const moSchema = new Schema({
  user_id: Schema.Types.ObjectId, // users._id
  robot_id: Schema.Types.ObjectId, // robots._id
  task_id: Schema.Types.ObjectId // tasks._id
}, opts);

Properties

Property Description Type Default
user_id Unique mongoDB ObjectId of web panel user (DEX8 customer). object
robot_id Unique mongoDB ObjectId of DEX8 robot. object
task_id Unique mongoDB ObjectId of DEX8 task. object
conn mongo connection object object
model mongoose model object
compiledModels An array of all compiled models array

Class "Mongo"

new Mongo(user_id, robot_id, task_id)

const { Mongo } = require('dex8-sdk');
const mongo = new Mongo('5b9645b3594b162dbf5ce1a4', '7b9645b3594b162dbf5ce1c5', '3e9645b3594b162dbf5ce1ff');
const results = mongo.list({}, limit, skip, sort, select);
console.log(results);

Mongo class is already instantiated by the DEX8 system so usage is even more simple than example above. See 080database example.
dbconnect.js
----------------------------
module.exports = async (x, lib) => {
  const mongo = lib.mongo;

  const mo_uri = 'mongodb://dex8_freeuser:pass@5.189.161.70:27017/dex8-dev-pool-free01';
  const collection1 = 'coll_1';
  const collection2 = 'coll_2';

  await mongo.connect(mo_uri); // connect to mongodb server

  mongo.compileModel(collection1); // compile model
  mongo.compileModel(collection2); // compile model

  return x;
};

Methods

async connect(mo_uri)

Connects to the mongoDB server via mongo URI. Used database is also defined.
mongo.connect('mongodb+srv://user:pass@cluster0-n4qix.mongodb.net/dex8-dev-pool-free01?retryWrites=true&w=majority')

disconnect()

Disconnect from already connected mongoDB server via mongo URI.
mongo.disconnect()

deleteDatabase()

Removes whole database.
mongo.deleteDatabase()

showCollections()

Lists all collections in the database.
mongo.showCollections()

deleteCollection(collectionName)

Removes collection from the database.
mongo.deleteCollection(collectionName)

async compileModel(moSchema)

Compiles mongoose model. Parameter must be valid mongoose schema like this one. The model object is pushed into the "compiledModels" array.
mongo.compileModel(moSchema)

useModel(collectionName)

Takes model on which operations will be performed. It defines on which collection documents will be added, listed, updated, removed ...
mongo.useModel(collectionName)

save(doc)

Adds document into collection. Mongoose save function is used.
mongo.save({x: 23, y: 'something'})

add(doc | docs)

Adds document into collection. Parameter can be a single object or array of objects. Mongoose create function is used.
mongo.add({x: 23, y: 'something'}) or
mongo.add([{x: 23, y: 'something'}, {x: 25, y: 'aaa'}]) - database called two times

insertMulti(docs, insOpts)

Bulk insertions. While add() will send multiple requests for every doc, this method will call database only once. Mongoose create function is used.
const insOpts = {
  ordered: false, // (true) if true, will fail fast on the first error encountered and don't execute the remaining writes
  rawResult: true, // (false) will return inserted docs
  lean: false, // (false) if true skip schema validation
  limit: null // (null) limits the number of documents being processed
};

mongo.insertMulti([{x: 23, y: 'something'}, {x: 25, y: 'aaa'}], insOpts) //- database called only one time

list(moQuery, limit, skip, sort, select)

List and count documents from collection by using mongo search query criteria. Mongoose find function is used.
mongo.list({x: {$gt3: 3}}, 5, 0, '-x', '-_id')

listFast(moQuery, limit, skip, sort, select)

List documents from collection by using mongo search query criteria. This is useful for big collections because list() method is using count and is much slower. Mongoose find function is used.
mongo.listFast({x: {$gt3: 3}}, 5, 0, '-x', '-_id')

getOne(moQuery, sort, select)

Fetch one document from collection. Mongoose findOne function is used.
mongo.getOne({x: 3})

deleteOne(moQuery)

Delete just one document. Mongoose findOneAndDelete function is used.
mongo.deleteOne({x: 3})

deleteMulti(moQuery)

Delete multiple documents. Mongoose deleteMany function is used.
mongo.deleteMulti({x: {$lt: 5}})

editOne(moQuery, docNew, updOpts)

Update one document. Mongoose findOneAndUpdate function is used.
const updOpts = {
  new: true, // return updated document as 'result'
  upsert: false, // whether to create the doc if it doesn't match (false)
  runValidators: false, // validators validate the update operation against the model's schema
  strict: false, // values not defined in schema will not be saved in db (default is defined in schema options, and can be overwritten here)
  // sort: {created_at: -1} // if multiple results are found, sets the sort order to choose which doc to update
}
mongo.editOne({x: 3}, {x: 5, y: 'some thing'}, updOpts)

countDocs(moQuery)

Count documents according to the given mongo query. Returned value is a number. Mongoose countDocuments function is used.
mongo.count({x: {$gt: 10}})