In the past few days the temperature in the Twin Cities of Minneapolis and St. Paul has been lower than average. The days have been cloudy and rainy. Today the sun might peak in the afternoon and the temperature will hopefully reach 55 F. On the positive side the forecast for Saturday is sunny and temperatures in the low 70s. Hopefully will be able to grill and spend a few hours outside walking.
In this post I will cover interacting with MongoDB using Node.js via the Mongoose module. I will not say much about Installing MongoDB on Windows or Using the MongoDB Shell due to the fact that I have covered such topics on previous posts. I am using the Windows 10 OS due to the fact that several tools I use to generate the post are installed in this Windows machine. You can follow just as well if you decide to use Linux. The installation instructions provided by MongoDB are well written and are easy to follow.
Please note that you first need to install MongoDB and make sure it is up and running before proceeding to experiment with the exercises in this post.
Let’s start a new Node.js project as illustrated by the following console output:
C:\Users\John>mkdir mongo-demo
C:\Users\John>cd mongo-demo
C:\Users\John\mongo-demo>code .
C:\Users\John\mongo-demo>npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.
See `npm help json` for definitive documentation on these fields
and exactly what they do.
Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.
Press ^C at any time to quit.
package name: (mongo-demo)
version: (1.0.0)
description:
entry point: (index.js)
test command:
git repository:
keywords:
author: John C Canessa
license: (ISC)
About to write to C:\Users\John\mongo-demo\package.json:
{
"name": "mongo-demo",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "John C Canessa",
"license": "ISC"
}
Is this OK? (yes) y
The steps here mentioned are typical when starting a new project.
Next we need to install Mongoose. Mongoose is an ORM (Object-Relational Mapping) for MongoDB written for Node.js. To read more about ORM you can do it here. The following console capture illustrates the step needed to install Mongoose:
C:\Users\John\mongo-demo>npm install mongoose <==== npm notice created a lockfile as package-lock.json. You should commit this file. npm WARN mongo-demo@1.0.0 No description npm WARN mongo-demo@1.0.0 No repository field. + mongoose@5.5.4 added 24 packages from 18 contributors and audited 39 packages in 9.89s found 0 vulnerabilities C:\Users\John\mongo-demo>type package.json
{
"name": "mongo-demo",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "John C Canessa",
"license": "ISC",
"dependencies": {
"mongoose": "^5.5.4" <====
}
}
After installation completes we verify that the mongoose package has been installed.
When I was reading and experimenting it was recommended to use mongoose 5.0.1. The reason for that is to make sure all packages match. Initially I followed the suggestion as shown below:
C:\Users\John\mongo-demo>npm install mongoose@5.0.1 <==== npm WARN mongo-demo@1.0.0 No description npm WARN mongo-demo@1.0.0 No repository field. + mongoose@5.0.1 added 2 packages from 4 contributors, removed 6 packages, updated 12 packages and audited 23 packages in 4.721s found 1 moderate severity vulnerability run `npm audit fix` to fix them, or `npm audit` for details C:\Users\John\mongo-demo>type package.json
{
"name": "mongo-demo",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "John C Canessa",
"license": "ISC",
"dependencies": {
"mongoose": "^5.0.1" <====
}
}
I then decided to go back to the latest mongoose version which at the time was 5.5.5. I edited the code to work with 5.5.5. The issues were related to deprecated functions and methods.
The first thing to do with any database is to verify that we can connect to it. The following JavaScript code in index.js illustrates how to connect to MongoDB using Mongoose:
// **** load mongoose ****
const mongoose = require("mongoose");
// **** connect to MongoDB (from configuration file) ****
console.log("connecting to MongoDB...");
mongoose
.connect("mongodb://localhost/images")
.then(() => {
console.log("connected to MongoDB!!!");
})
.catch(err => {
console.error("could NOT connect to MongoDB", err);
});
Note that the MongoDB database we will use is named images and we have not created such database so far. The following screen dump shows the program in action:
C:\Users\John\mongo-demo>nodemon index.js [nodemon] 1.18.11 [nodemon] to restart at any time, enter `rs` [nodemon] watching: *.* [nodemon] starting `node index.js` connecting to MongoDB... connected to MongoDB!!!
In relational databases (i.e.., MySQL, SQL Server) we need to create a database and a table before we can connect to a table. In MongoDB the database and the collection (name given to a document which is equivalent to a table) are created when the first time you write to it. So far we have not written data to the collection so if you use the mongo shell (which we will use shortly) or MongoDB Compass which is a user interface used to interact with MongoDB databases, collections and documents, the database and collections will not have been created.
Something that one expects from a relational but not from a No-SQL database is the use of a schema. The first time I used Mongoose I was surprised that it requires a schema. So far I have not found mentioning of a schema in the MongoDB documentation. A schema is an artifact created by Mongoose. In the Data Models page of MongoDB it states:
“Data in MongoDB has a flexible schema. Collections do not enforce document structure by default. This flexibility gives you data-modeling choices to match your application and its performance requirements.”
On more than one occasion I have added new documents to an existing collection which included new additional fields. Adding non-existing fields to a relational database on the fly is something one is not allowed. It is fine in a NoSQL database like MongoDB.
Mongoose supports the following field types when defining a schema:
| Array |
| Boolean |
| Buffer |
| Date |
| Number |
| ObjectID |
| String |
In the following code snippet we will define a schema, compile it to a model and then use it to insert our first document into the image collection.
// **** define a schema (completely against No-SQL and MongoDB) ****
const imageSchema = new mongoose.Schema({
name: String,
patient: String,
tags: [String],
date: { type: Date, default: Date.now },
reviewed: { type: Boolean, default: false }
});
// **** compile the schema to a model ****
const Image = mongoose.model("image", imageSchema);
// **** instantiate an image ****
const image = new Image({
name: "11111111112222222222333333333344",
patient: "John Doe",
tags: ["right leg", "lateral"]
});
We defined a schema for an image. This is done following the DICOM storage server theme in the RESTful service. We have a name for the image, a name for the patient, some tags for image metadata, date and time when the image was captured, and a flag to indicate if the image has been reviewed by a radiologist. Please note that this is an oversimplification of what would be needed in a production DICOM storage server. I just wished to associate the schema data types with field names that show references to medical images.
// **** define a schema (completely against No-SQL and MongoDB) ****
const imageSchema = new mongoose.Schema({
name: String,
patient: String,
tags: [String],
date: { type: Date, default: Date.now },
reviewed: { type: Boolean, default: false }
});
// **** compile the schema to a model ****
const Image = mongoose.model("image", imageSchema);
/*
* Should pass parameters.
* Insert a document into the database.
*/
async function createImage() {
// **** instantiate an image ****
const image = new Image({
name: "11111111111111111111111111111111",
patient: "John Doe",
tags: ["right leg", "lateral"]
});
// **** save the image to the database (async method - returns a promise) ****
const result = await image.save();
// **** display result ****
console.log("result: " + result);
}
// **** create an image *****
createImage();
In this last screen capture, we inserted our first image associate with patient “Jane Doe”. Because we are using nodemon we were able to modify the data in our program and when saved inserted a new record for patient “John Doe”. Healthcare facilities tend to use such names for patients that required imaging but their names at the time were not known. This typically happens when patients have been in an accident, are run to emergency and no documents are available yet for positive identification.
[nodemon] starting `node index.js`
connecting to MongoDB...
connected to MongoDB!!!
result: { tags: [ 'hip', 'frontal' ],
date: 2019-04-29T14:10:51.404Z,
reviewed: false,
_id: 5cc705ebe05a6911b0c11d8b,
name: '11111111112222222222333333333344',
patient: 'Jane Doe',
__v: 0 }
[nodemon] restarting due to changes...
[nodemon] restarting due to changes...
[nodemon] starting `node index.js`
connecting to MongoDB...
connected to MongoDB!!!
result: { tags: [ 'right leg', 'lateral' ],
date: 2019-04-29T14:13:23.730Z,
reviewed: false,
_id: 5cc70683e6626c32b8ebb0b6,
name: '11111111111111111111111111111111',
patient: 'John Doe',
__v: 0 }
We will now use the mongo shell to take a look at the data we wrote to MongoDB.
C:\Users\John>mongo
MongoDB shell version v3.6.4
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.4
Server has startup warnings:
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** Start the server with --bind_ip
<address> to specify which IP
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** WARNING: The file system cache of this machine is configured to be greater than 40% of the total memory. This can lead to increased memory pressure and poor performance.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] See http://dochub.mongodb.org/core/wt-windows-system-file-cache
2019-04-29T08:06:15.459-0500 I CONTROL [initandlisten]
> use images
switched to db images
> db.images.find().pretty()
{
"_id" : ObjectId("5cc705ebe05a6911b0c11d8b"),
"tags" : [
"hip",
"frontal"
],
"date" : ISODate("2019-04-29T14:10:51.404Z"),
"reviewed" : false,
"name" : "11111111112222222222333333333344",
"patient" : "Jane Doe",
"__v" : 0
}
{
"_id" : ObjectId("5cc70683e6626c32b8ebb0b6"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T14:13:23.730Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"__v" : 0
}
From a command prompt we start the mongo shell. We then switch to the images database. This was specified in the connect mongoose statement. We then issue a query to find and display all the data available in the images collection. Sorry that I used the same name for the database as the collection. One is free to use different names.
The two images that were inserted are in the database. All documents in a MongoDB collection must have a unique id. The same holds true in relational databases. In MongoDB we can specify a unique value for each document id or in these cases, allow MongoDB to assign unique values. If you are interested in the contents of the _id field you can learn more about it here. I find the contents of the _id field interesting because long time ago I developed a storage server and had to create unique values for each object stored.
The next step is to query the documents using JavaScript. The following code snippet allows us to query the first 10 documents:
/*
* Get all image documents from MongoDB.
*/
async function getImages() {
// ???? inform the user what is going on ????
console.log("finding images ...");
// **** find all images in the database ****
// const images = await Image.find();
// **** find image(s) for specified patient(s) ****
const images = await Image.find({ patient: /Doe$/, reviewed: false })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 0, patient: 1, name: 1 });
// ???? inform the user what is going on ????
console.log("images found:\n" + images);
}
// **** create an image *****
// createImage();
// **** get all images ****
getImages();
When we ran the code we are able to see the same records as illustrated here:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ name: '11111111112222222222333333333344',
patient: 'Jane Doe' },{ name: '11111111111111111111111111111111',
patient: 'John Doe' }
The query was more specific than it was needed. We stored two patients with the last name “Doe” so there was no need to specify that the name ended in “Doe”. The field reviewed was set to false, so there was no need to specify it in the query. We inserted two records so the limit was not required either.
Note that the query just returned the image name and the patient name. This was specified in the .select() clause. If we omit it, all fields will be included in the results.
We can use comparison operators in a query. MongoDB supports:
| Name | Description |
| $eq | Matches values that are equal to a specified value. |
| $gt | Matches values that are greater than a specified value. |
| $gte | Matches values that are greater than or equal to a specified value. |
| $in | Matches any of the values specified in an array. |
| $lt | Matches values that are less than a specified value. |
| $lte | Matches values that are less than or equal to a specified value. |
| $ne | Matches all values that are not equal to a specified value. |
| $nin | Matches none of the values specified in an array. |
You can find additional documentation here.
After populating additional data in the database, let’s issue a query using comparison operators as illustrated by the following code snippet:
/*
* Get all image documents from MongoDB.
*/
async function getImages() {
// ???? inform the user what is going on ????
console.log("finding images ...");
// **** find all images in the database ****
// const images = await Image.find();
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: /$/, reviewed: false })
.find({ width: { $gt: 128 } })
// .find({ width: { $gte: 128 } })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
// ???? inform the user what is going on ????
console.log("images found:\n" + images);
}
The results follow:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc7227237c5612ba896038b,
name: '11111111111111111111111111111111',
patient: 'James Bond',
width: 256,
height: 256 },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 }
It would be nice for me to make the data that I have in the images database available. I will document the process and put the file in my GitHub repository.
After some modifications:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: /$/, reviewed: false })
// .find({ width: { $gt: 128 } })
.find({ width: { $gte: 128 } }) <====
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
The results follow:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc7227237c5612ba896038b,
name: '11111111111111111111111111111111',
patient: 'James Bond',
width: 256,
height: 256 },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 },{ _id: 5cc72293d56b582c60f63742,
name: '11111111111111111111111111111199',
patient: 'Money Penney',
width: 128, <====
height: 256 }
One more example of comparison:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: /$/, reviewed: false })
// .find({ width: { $gt: 128 } })
// .find({ width: { $gte: 128 } })
.find({ width: { $gte: 128, $lte: 256 } }) <====
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
And the associated run:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc7227237c5612ba896038b,
name: '11111111111111111111111111111111',
patient: 'James Bond',
width: 256,
height: 256 },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 },{ _id: 5cc72293d56b582c60f63742,
name: '11111111111111111111111111111199',
patient: 'Money Penney',
width: 128,
height: 256 }
The next code snippet shows the use of the $in operator. If you are familiar with SQL you can draw the similarities:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: /$/, reviewed: false })
// .find({ width: { $gt: 128 } })
// .find({ width: { $gte: 128 } })
// .find({ width: { $gte: 128, $lte: 256 } })
.find({ width: { $in: [128, 256] } })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
The run for this code follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc7227237c5612ba896038b,
name: '11111111111111111111111111111111',
patient: 'James Bond',
width: 256,
height: 256 },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 },{ _id: 5cc72293d56b582c60f63742,
name: '11111111111111111111111111111199',
patient: 'Money Penney',
width: 128,
height: 256 }
MongoDB supports logical operator in queries. The following list illustrates the ones supported at this time:
| Name | Description |
| $and | Joins query clauses with a logical AND returns all documents that match the conditions of both clauses. |
| $not | Inverts the effect of a query expression and returns documents that do not match the query expression. |
| $nor | Joins query clauses with a logical NOR returns all documents that fail to match both clauses. |
| $or | Joins query clauses with a logical OR returns all documents that match the conditions of either clause. |
To read more about these operators you can do so here.
The following code snippet illustrates the use of some logical operators:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: /$/, reviewed: false })
// .find({ width: { $gt: 128 } })
// .find({ width: { $gte: 128 } })
// .find({ width: { $gte: 128, $lte: 256 } })
// .find({ width: { $in: [128, 256] } })
.find()
.or([{ patient: "John Doe" }, { patient: "James Bond" }])
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
The run for the last query is illustrated as follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc7227237c5612ba896038b,
name: '11111111111111111111111111111111',
patient: 'James Bond', <====
width: 256,
height: 256 },{ _id: 5cc70683e6626c32b8ebb0b6,
name: '11111111111111111111111111111111',
patient: 'John Doe' },{ _id: 5cc72254a72d243298a2a16a, <====
name: '11111111111111111111111111111111',
patient: 'John Doe', <====
width: 256,
height: 256 }
In a previous example we saw the use of regular expressions when we looked for images associated with a patients with last name “Doe”. In this example we look for patients whose first name is “James”:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: "James Bond" })
.find({ patient: /^James/ })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
The result of the last query follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc7227237c5612ba896038b,
name: '11111111111111111111111111111111',
patient: 'James Bond', <====
width: 256,
height: 256 },{ _id: 5cc72e2dcf67480dfc23aa1e,
name: '11111111112222222222333333333399',
patient: 'James Smith', <====
width: 512,
height: 512 }
Note that JavaScript uses the Perl-like regular expression support.
The following code snippet we look for patients with the last name “Doe”:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: "James Bond" })
.find({ patient: /Doe$/ })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
The results follow:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc705ebe05a6911b0c11d8b,
name: '11111111112222222222333333333344',
patient: 'Jane Doe' },{ _id: 5cc70683e6626c32b8ebb0b6,
name: '11111111111111111111111111111111',
patient: 'John Doe' },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 }
In practice, a DICOM storage server will use separate fields for first, middle and last names. In these examples we assume (and we know what “assume” stands for) that the first name may or may not be followed by middle and ends with the last name. That is a poor design assumption which is not acceptable in production.
One flag one can use in regular expressions is to specify that the match to be done disregarding case. This is illustrated in the following example:
// **** find image(s) for specified patient(s) ****
const images = await Image
// .find({ patient: "James Bond" })
.find({ patient: /doe$/i })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
The run of the last code snippet follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc705ebe05a6911b0c11d8b,
name: '11111111112222222222333333333344',
patient: 'Jane Doe' },{ _id: 5cc70683e6626c32b8ebb0b6,
name: '11111111111111111111111111111111',
patient: 'John Doe' },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 }
It is often required to get a count of records. The count may be used for statistical purposes or to paginate pages with results from a query. The following code illustrates how to get a count:
// **** find image(s) for specified patient(s) ****
const images = await Image.find({ patient: /.*Doe.*/ })
.limit(10)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
// .select({ _id: 1, patient: 1, name: 1, width: 1, height: 1 });
.count();
The run follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ _id: 5cc705ebe05a6911b0c11d8b,
name: '11111111112222222222333333333344',
patient: 'Jane Doe' },{ _id: 5cc70683e6626c32b8ebb0b6,
name: '11111111111111111111111111111111',
patient: 'John Doe' },{ _id: 5cc72254a72d243298a2a16a,
name: '11111111111111111111111111111111',
patient: 'John Doe',
width: 256,
height: 256 }
[nodemon] restarting due to changes...
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
3
Apparently our database has 3 images that match the query criteria which specify the last name as being “Doe”.
To use counts we mentioned the use of it in the previous example. In the following code we use a page number and a page size to query the database and have it return the appropriate records.
async function getImages() {
// ???? inform the user what is going on ????
console.log("finding images ...");
// **** for paging ****
// e.g., /api/images?pageNumber=2&pageSize=10
const pageNumber = 2; <====
const pageSize = 5; <====
// **** find all images in the database ****
// const images = await Image.find();
// **** find image(s) for specified patient(s) ****
const images = await Image.find({ patient: /$/ })
.skip((pageNumber - 1) * pageSize)
.limit(pageSize)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, tags: 1, width: 1, height: 1 });
// ???? inform the user what is going on ????
console.log("images found:\n" + images);
}
The results of the previous code snippet follow:
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ tags: [ 'left arm', 'side' ],
_id: 5cc7585f285dd410a4131495,
name: '12345111112222222222333333333399',
patient: 'Jane Smith',
width: 512,
height: 1024 },{ tags: [ 'left arm', 'side' ],
_id: 5cc7586b62b47f2bfcefe820,
name: '12345111112222222222333333333399',
patient: 'Jane Smith',
width: 512,
height: 1024 },{ tags: [ 'left arm', 'side' ],
_id: 5cc7589c66c6353d703ba985,
name: '12345111112222222222333333333399',
patient: 'Jane Smith',
width: 512,
height: 1024 },{ tags: [ 'hand', 'lateral' ],
_id: 5cc757d77525e33e7816b803,
name: '12311111112222222222333333333399',
patient: 'John Canessa',
width: 512,
height: 512 },{ tags: [ 'hand', 'lateral' ],
_id: 5cc75789c4523c19504f7acd,
name: '12111111112222222222333333333399',
patient: 'John Canessa Smith',
width: 512,
height: 512 }
There are two main approaches used to update records in a MongoDB collection. The first is called query first and the second is update first.
Let’s start with a view of the database at this point in time as illustrated by the following console capture:
C:\Users\John\mongo-demo>nodemon index.js <====
[nodemon] 1.18.11
[nodemon] to restart at any time, enter `rs`
[nodemon] watching: *.*
[nodemon] starting `node index.js`
connecting to MongoDB...
finding images ...
connected to MongoDB!!!
images found:
{ tags: [ 'left arm', 'side' ],
_id: 5cc7585f285dd410a4131495,
name: '12345111112222222222333333333399',
patient: 'Jane Smith',
width: 512,
height: 1024 },{ tags: [ 'left arm', 'side' ],
_id: 5cc7586b62b47f2bfcefe820,
name: '12345111112222222222333333333399',
patient: 'Jane Smith',
width: 512,
height: 1024 },{ tags: [ 'left arm', 'side' ],
_id: 5cc7589c66c6353d703ba985,
name: '12345111112222222222333333333399',
patient: 'Jane Smith',
width: 512,
height: 1024 },{ tags: [ 'hand', 'lateral' ],
_id: 5cc757d77525e33e7816b803, <====
name: '12311111112222222222333333333399',
patient: 'John Canessa',
width: 512,
height: 512 },{ tags: [ 'hand', 'lateral' ],
_id: 5cc75789c4523c19504f7acd,
name: '12111111112222222222333333333399',
patient: 'John Canessa Smith',
width: 512,
height: 512 }
We will now add support for mongoose as illustrated in the following code snippet:
// **** load mongoose ****
const mongoose = require("mongoose");
// **** connect to MongoDB (from configuration file) ****
console.log("connecting to MongoDB...");
mongoose
.connect("mongodb://localhost/images")
.then(() => {
console.log("connected to MongoDB!!!");
})
.catch(err => {
console.error("could NOT connect to MongoDB", err);
});
// **** define a schema (completely against No-SQL and MongoDB) ****
const imageSchema = new mongoose.Schema({
name: String,
patient: String,
tags: [String],
date: { type: Date, default: Date.now },
reviewed: { type: Boolean, default: false },
width: Number,
height: Number
});
// **** compile the schema to a model ****
const Image = mongoose.model("image", imageSchema);
/*
* Should pass parameters.
* Insert a document into the database.
*/
async function createImage() {
// **** instantiate an image ****
const image = new Image({
name: "12345111112222222222333333333399",
patient: "Jane Smith",
tags: ["left arm", "side"],
width: 512,
height: 1024
});
// **** save the image to the database (async method - returns a promise) ****
const result = await image.save();
// **** display result ****
console.log("result: " + result);
}
/*
* Get all image documents from MongoDB.
* Must decorate with async due to await.
*/
async function getImages() {
// ???? inform the user what is going on ????
console.log("finding images ...");
// **** for paging ****
// e.g., /api/images?pageNumber=2&pageSize=10
const pageNumber = 2;
const pageSize = 5;
// **** find all images in the database ****
// const images = await Image.find();
// **** find image(s) for specified patient(s) ****
const images = await Image.find({ patient: /$/ })
.skip((pageNumber - 1) * pageSize)
.limit(pageSize)
.sort({ patient: 1 }) // -1 == desc, 1 == asc
.select({ _id: 1, patient: 1, name: 1, tags: 1, width: 1, height: 1 });
// ???? inform the user what is going on ????
console.log("images found:\n" + images);
}
// **** create an image *****
// createImage();
// // **** get all images ****
// getImages();
/*
* Update the metadata for the specified image.
* id image id to be updated
*
* Query first approach:
* findByID()
* modify its properties
* save()
*
* Update first approach:
* update directly
* optionally: get the updated document
*/
async function updateImage(id) {
// ???? display image id ????
console.log("updateImage <<< id: " + id);
// **** query for the image ****
const image = await Image.findById(id);
// **** check if image not found ****
if (!image) {
console.log(`updateImage <<< id: ${id} not found`);
return;
}
// **** update the image (flag image has been reviewed) approach 1 ****
image.reviewed = true;
image.patient = "Mike Pence";
// **** update the image (flag image has been reviewed) approach 2 ****
// image.set({
// reviewed: true,
// patient: "Mike Pence"
// });
// **** save to db updated record ****
const result = await image.save();
// ???? display updated record ????
console.log("updateImage <<< result: " + result);
}
// **** call the update image function ****
updateImage("5cc75789c4523c19504f7acd");
The code illustrates how to connect to MongoDB, define the schema, instantiate a model and use it to populate a record and save it to MongoDB.
The output from the code follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
updateImage <<< id: 5cc75789c4523c19504f7acd
connected to MongoDB!!!
updateImage <<< result: { tags: [ 'hand', 'lateral' ],
date: 2019-04-29T19:59:05.801Z,
reviewed: true,
_id: 5cc75789c4523c19504f7acd,
name: '12111111112222222222333333333399',
patient: 'Mike Pence',
width: 512,
height: 512,
__v: 0 }
We can also take a look at the state of the images collection by using the mongo shell as illustrated by:
> db.images.find().pretty()
{
"_id" : ObjectId("5cc705ebe05a6911b0c11d8b"),
"tags" : [
"hip",
"frontal"
],
"date" : ISODate("2019-04-29T14:10:51.404Z"),
"reviewed" : false,
"name" : "11111111112222222222333333333344",
"patient" : "Jane Doe",
"__v" : 0
}
{
"_id" : ObjectId("5cc70683e6626c32b8ebb0b6"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T14:13:23.730Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"__v" : 0
}
{
"_id" : ObjectId("5cc72254a72d243298a2a16a"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T16:12:04.976Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"width" : 256,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc7227237c5612ba896038b"),
"tags" : [
"chest",
"lateral"
],
"date" : ISODate("2019-04-29T16:12:34.832Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "James Bond",
"width" : 256,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc72293d56b582c60f63742"),
"tags" : [
"chest",
"lateral"
],
"date" : ISODate("2019-04-29T16:13:07.326Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111199",
"patient" : "Money Penney",
"width" : 128,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc72e2dcf67480dfc23aa1e"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T17:02:37.311Z"),
"reviewed" : false,
"name" : "11111111112222222222333333333399",
"patient" : "James Smith",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc75789c4523c19504f7acd"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T19:59:05.801Z"),
"reviewed" : true,
"name" : "12111111112222222222333333333399",
"patient" : "Mike Pence",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc757d77525e33e7816b803"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T20:00:23.601Z"),
"reviewed" : false,
"name" : "12311111112222222222333333333399",
"patient" : "John Canessa",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc757f24b23593dc4f18107"),
"tags" : [
"hand",
"front"
],
"date" : ISODate("2019-04-29T20:00:50.272Z"),
"reviewed" : false,
"name" : "12311111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 256,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc75814f2e4d435f06cbee7"),
"tags" : [
"right leg",
"side"
],
"date" : ISODate("2019-04-29T20:01:24.501Z"),
"reviewed" : false,
"name" : "12341111112222222222333333333399",
"patient" : "John Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7583de26d8038f86588a6"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:05.573Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7585f285dd410a4131495"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:39.622Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7586b62b47f2bfcefe820"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:51.324Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7589c66c6353d703ba985"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:03:40.890Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
Note that the find() command has been enhanced by the use of the pretty() method. The pretty() method produces a nicer and easier to read output for the selected documents.
Now let’s take a look at the update first approach. We will make us of some update operators. A list of them follows:
| $currentDate | Sets the value of a field to current date, either as a Date or a Timestamp. |
| $inc | Increments the value of the field by the specified amount. |
| $min | Only updates the field if the specified value is less than the existing field value. |
| $max | Only updates the field if the specified value is greater than the existing field value. |
| $mul | Multiplies the value of the field by the specified amount. |
| $rename | Renames a field. |
| $set | Sets the value of a field in a document. |
| $setOnInsert | Sets the value of a field if an update results in an insert of a document. Has no effect on update operations that modify existing documents. |
| $unset | Removes the specified field from a document. |
To learn more about the update operators you can do so here.
In the following code snippet, we will use a different version of the updateImage() method used in the last run. The code follows:
/*
* Update the metadata for the specified image.
* id image id to be updated
*
* Update first approach:
* update directly
* optionally: get the updated document
*/
async function updateImage2(id) {
// ???? display the id ????
console.log(`updateImage2 <<< id: ${id}`);
// **** update the specified image (returns update info) ****
const result = await Image.updateOne(
{ _id: id },
{
$set: {
reviewed: false,
patient: "John Canessa Coca Cola"
}
}
);
// // **** update the specified image (returns updated image) ****
// // const image = await Image.findByIdAndUpdate(id, {
// const image = await Image.findOneAndUpdate(
// id,
// {
// $set: {
// patient: "John Coca Cola",
// reviewed: false
// }
// },
// { new: true }
// );
// ???? display the update results ????
console.log("updateImage2 <<< result: ", result);
// console.log("updateImage2 <<< image: ", image);
The run showing results and the modified / updated image follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
updateImage2 <<< id: 5cc705ebe05a6911b0c11d8b
connected to MongoDB!!!
updateImage2 <<< result: { n: 1, nModified: 0, ok: 1 }
:::: :::: ::::
[nodemon] starting `node index.js`
connecting to MongoDB...
updateImage2 <<< id: 5cc705ebe05a6911b0c11d8b
connected to MongoDB!!!
updateImage2 <<< image: { tags: [ 'hip', 'frontal' ],
reviewed: false,
_id: 5cc705ebe05a6911b0c11d8b,
date: 2019-04-29T14:10:51.404Z,
name: '11111111112222222222333333333344',
patient: 'John Coca Cola',
__v: 0 }
Finally we can remove / delete documents from the database. The following snippet illustrates how to delete an image specified by id:
/*
* Remove the specified image from the database.
*/
async function removeImage(id) {
// ???? display the id ????
console.log(`removeImage <<< id: ${id}`);
// **** delete the image from the database ****
const result = await Image.deleteOne({ _id: id });
// ???? ????
console.log("removeImage <<< result: ", result);
}
// **** remove the specified image ****
removeImage("5cc705ebe05a6911b0c11d8b");
The run for it follows:
[nodemon] starting `node index.js`
connecting to MongoDB...
removeImage <<< id: 5cc705ebe05a6911b0c11d8b
connected to MongoDB!!!
removeImage <<< result: { n: 1, ok: 1, deletedCount: 1 }
This completes the code examples. I mentioned earlier in this post, I am going to provide the document data so you can populate the images database with the image collection as I have it in my machine. We first need to export using mongoexport and then import using mongoimport data. The reason for this is the fact that you will encounter the need to export and import data as you develop and test code.
Both utilities must be executed from the command prompt, not the mongo shell.
Let’s first export the data in the images database found in the images collection. Once again, I apologize by using the same name for the database and the collection. I did not want to change it at this time because it would take time to verify that the screen captures match.
The step to export the images data follows:
C:\Users\John\mongo-demo>mongo
MongoDB shell version v3.6.4
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.4
Server has startup warnings:
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** Start the server with --bind_ip
<address> to specify which IP
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** WARNING: The file system cache of this machine is configured to be greater than 40% of the total memory. This can lead to increased memory pressure and poor performance.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] See http://dochub.mongodb.org/core/wt-windows-system-file-cache
2019-04-29T08:06:15.459-0500 I CONTROL [initandlisten]
> use images
switched to db images
> db.images.find().pretty()
{
"_id" : ObjectId("5cc70683e6626c32b8ebb0b6"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T14:13:23.730Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"__v" : 0
}
{
"_id" : ObjectId("5cc72254a72d243298a2a16a"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T16:12:04.976Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"width" : 256,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc7227237c5612ba896038b"),
"tags" : [
"chest",
"lateral"
],
"date" : ISODate("2019-04-29T16:12:34.832Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "James Bond",
"width" : 256,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc72293d56b582c60f63742"),
"tags" : [
"chest",
"lateral"
],
"date" : ISODate("2019-04-29T16:13:07.326Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111199",
"patient" : "Money Penney",
"width" : 128,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc72e2dcf67480dfc23aa1e"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T17:02:37.311Z"),
"reviewed" : false,
"name" : "11111111112222222222333333333399",
"patient" : "James Smith",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc75789c4523c19504f7acd"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T19:59:05.801Z"),
"reviewed" : false,
"name" : "12111111112222222222333333333399",
"patient" : "John Canessa Smith",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc757d77525e33e7816b803"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T20:00:23.601Z"),
"reviewed" : false,
"name" : "12311111112222222222333333333399",
"patient" : "John Canessa",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc757f24b23593dc4f18107"),
"tags" : [
"hand",
"front"
],
"date" : ISODate("2019-04-29T20:00:50.272Z"),
"reviewed" : false,
"name" : "12311111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 256,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc75814f2e4d435f06cbee7"),
"tags" : [
"right leg",
"side"
],
"date" : ISODate("2019-04-29T20:01:24.501Z"),
"reviewed" : false,
"name" : "12341111112222222222333333333399",
"patient" : "John Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7583de26d8038f86588a6"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:05.573Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7585f285dd410a4131495"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:39.622Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7586b62b47f2bfcefe820"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:51.324Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7589c66c6353d703ba985"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:03:40.890Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
> exit
bye
C:\Users\John\mongo-demo>mongoexport --help
Usage:
mongoexport<options>
Export data from MongoDB in CSV or JSON format.
See http://docs.mongodb.org/manual/reference/program/mongoexport/ for more information.
general options:
/help print usage
/version print the tool version and
exit
verbosity options:
/v, /verbose:<level> more detailed log output
(include multiple times for
more verbosity, e.g. -vvvvv,
or specify a numeric value,
e.g. --verbose=N)
/quiet hide all log output
connection options:
/h, /host:<hostname> mongodb host to connect to
(setname/host1,host2 for
replica sets)
/port:<port> server port (can also use
--host hostname:port)
ssl options:
/ssl connect to a mongod or mongos
that has ssl enabled
/sslCAFile:<filename> the .pem file containing the
root certificate chain from
the certificate authority
/sslPEMKeyFile:<filename> the .pem file containing the
certificate and key
/sslPEMKeyPassword:<password> the password to decrypt the
sslPEMKeyFile, if necessary
/sslCRLFile:<filename> the .pem file containing the
certificate revocation list
/sslAllowInvalidCertificates bypass the validation for
server certificates
/sslAllowInvalidHostnames bypass the validation for
server name
/sslFIPSMode use FIPS mode of the
installed openssl library
authentication options:
/u, /username:<username> username for authentication
/p, /password:<password> password for authentication
/authenticationDatabase:<database-name> database that holds the
user's credentials
/authenticationMechanism:<mechanism> authentication mechanism to
use
namespace options:
/d, /db:<database-name> database to use
/c, /collection:<collection-name> collection to use
uri options:
/uri:mongodb-uri mongodb uri connection string
output options:
/f, /fields:<field>[,<field>]* comma separated list of field
names (required for exporting
CSV) e.g. -f "name,age"
/fieldFile:<filename> file with field names - 1 per
line
/type:<type> the output format, either
json or csv (defaults to
'json') (default: json)
/o, /out:<filename> output file; if not
specified, stdout is used
/jsonArray output to a JSON array rather
than one object per line
/pretty output JSON formatted to be
human-readable
/noHeaderLine export CSV data without a
list of field names at the
first line
querying options:
/q, /query:<json> query filter, as a JSON
string, e.g., '{x:{$gt:1}}'
/queryFile:<filename> path to a file containing a
query filter (JSON)
/k, /slaveOk allow secondary reads if
available (default true)
(default: false)
/readPreference:<string>|<json> specify either a preference
name or a preference json
object
/forceTableScan force a table scan (do not
use $snapshot)
/skip:<count> number of documents to skip
/limit:<count> limit the number of documents
to export
/sort:<json> sort order, as a JSON string,
e.g. '{x:1}'
/assertExists if specified, export fails if
the collection does not exist
(default: false)
C:\Users\John\mongo-demo>mongoexport --db images --collection images --out images.json --jsonArray
2019-05-02T09:48:18.454-0500 connected to: localhost
2019-05-02T09:48:18.457-0500 exported 13 records
C:\Users\John\mongo-demo>type images.json
[{"_id":{"$oid":"5cc70683e6626c32b8ebb0b6"},"tags":["right leg","lateral"],"date":{"$date":"2019-04-29T14:13:23.730Z"},"reviewed":false,"name":"11111111111111111111111111111111","patient":"John Doe","__v":0},{"_id":{"$oid":"5cc72254a72d243298a2a16a"},"tags":["right leg","lateral"],"date":{"$date":"2
019-04-29T16:12:04.976Z"},"reviewed":false,"name":"11111111111111111111111111111111","patient":"John Doe","width":256,"height":256,"__v":0},{"_id":{"$oid":"5cc7227237c5612ba896038b"},"tags":["chest","lateral"],"date":{"$date":"2019-04-29T16:12:34.832Z"},"reviewed":false,"name":"111111111111111111111
11111111111","patient":"James Bond","width":256,"height":256,"__v":0},{"_id":{"$oid":"5cc72293d56b582c60f63742"},"tags":["chest","lateral"],"date":{"$date":"2019-04-29T16:13:07.326Z"},"reviewed":false,"name":"11111111111111111111111111111199","patient":"Money Penney","width":128,"height":256,"__v":0
},{"_id":{"$oid":"5cc72e2dcf67480dfc23aa1e"},"tags":["hand","lateral"],"date":{"$date":"2019-04-29T17:02:37.311Z"},"reviewed":false,"name":"11111111112222222222333333333399","patient":"James Smith","width":512,"height":512,"__v":0},{"_id":{"$oid":"5cc75789c4523c19504f7acd"},"tags":["hand","lateral"]
,"date":{"$date":"2019-04-29T19:59:05.801Z"},"reviewed":false,"name":"12111111112222222222333333333399","patient":"John Canessa Smith","width":512,"height":512,"__v":0},{"_id":{"$oid":"5cc757d77525e33e7816b803"},"tags":["hand","lateral"],"date":{"$date":"2019-04-29T20:00:23.601Z"},"reviewed":false,"
name":"12311111112222222222333333333399","patient":"John Canessa","width":512,"height":512,"__v":0},{"_id":{"$oid":"5cc757f24b23593dc4f18107"},"tags":["hand","front"],"date":{"$date":"2019-04-29T20:00:50.272Z"},"reviewed":false,"name":"12311111112222222222333333333399","patient":"Jane Smith","width"
:256,"height":512,"__v":0},{"_id":{"$oid":"5cc75814f2e4d435f06cbee7"},"tags":["right leg","side"],"date":{"$date":"2019-04-29T20:01:24.501Z"},"reviewed":false,"name":"12341111112222222222333333333399","patient":"John Smith","width":512,"height":1024,"__v":0},{"_id":{"$oid":"5cc7583de26d8038f86588a6"
},"tags":["left arm","side"],"date":{"$date":"2019-04-29T20:02:05.573Z"},"reviewed":false,"name":"12345111112222222222333333333399","patient":"Jane Smith","width":512,"height":1024,"__v":0},{"_id":{"$oid":"5cc7585f285dd410a4131495"},"tags":["left arm","side"],"date":{"$date":"2019-04-29T20:02:39.622
Z"},"reviewed":false,"name":"12345111112222222222333333333399","patient":"Jane Smith","width":512,"height":1024,"__v":0},{"_id":{"$oid":"5cc7586b62b47f2bfcefe820"},"tags":["left arm","side"],"date":{"$date":"2019-04-29T20:02:51.324Z"},"reviewed":false,"name":"12345111112222222222333333333399","patie
nt":"Jane Smith","width":512,"height":1024,"__v":0},{"_id":{"$oid":"5cc7589c66c6353d703ba985"},"tags":["left arm","side"],"date":{"$date":"2019-04-29T20:03:40.890Z"},"reviewed":false,"name":"12345111112222222222333333333399","patient":"Jane Smith","width":512,"height":1024,"__v":0}]
To get an idea of what we have we use the mongo shell and display all 13 records. We then exit the mongo shell. Back on the console we display the usage of the mongoexport command. We then export the data to the images.json file. The contents of the file are displayed.
We have a file with the data. You will now have to import it. In the following screen capture we can see the steps to import the data:
C:\Users\John\mongo-demo>mongoimport --help
Usage:
mongoimport<options> <file>
Import CSV, TSV or JSON data into MongoDB. If no file is provided, mongoimport reads from stdin.
See http://docs.mongodb.org/manual/reference/program/mongoimport/ for more information.
general options:
/help print usage
/version print the tool version and
exit
verbosity options:
/v, /verbose:<level> more detailed log output
(include multiple times for
more verbosity, e.g. -vvvvv,
or specify a numeric value,
e.g. --verbose=N)
/quiet hide all log output
connection options:
/h, /host:<hostname> mongodb host to connect to
(setname/host1,host2 for
replica sets)
/port:<port> server port (can also use
--host hostname:port)
ssl options:
/ssl connect to a mongod or mongos
that has ssl enabled
/sslCAFile:<filename> the .pem file containing the
root certificate chain from
the certificate authority
/sslPEMKeyFile:<filename> the .pem file containing the
certificate and key
/sslPEMKeyPassword:<password> the password to decrypt the
sslPEMKeyFile, if necessary
/sslCRLFile:<filename> the .pem file containing the
certificate revocation list
/sslAllowInvalidCertificates bypass the validation for
server certificates
/sslAllowInvalidHostnames bypass the validation for
server name
/sslFIPSMode use FIPS mode of the
installed openssl library
authentication options:
/u, /username:<username> username for authentication
/p, /password:<password> password for authentication
/authenticationDatabase:<database-name> database that holds the
user's credentials
/authenticationMechanism:<mechanism> authentication mechanism to
use
namespace options:
/d, /db:<database-name> database to use
/c, /collection:<collection-name> collection to use
uri options:
/uri:mongodb-uri mongodb uri connection string
input options:
/f, /fields:<field>[,<field>]* comma separated list of
fields, e.g. -f name,age
/fieldFile:<filename> file with field names - 1 per
line
/file:<filename> file to import from; if not
specified, stdin is used
/headerline use first line in input
source as the field list (CSV
and TSV only)
/jsonArray treat input source as a JSON
array
/parseGrace:<grace> controls behavior when type
coercion fails - one of:
autoCast, skipField, skipRow,
stop (defaults to 'stop')
(default: stop)
/type:<type> input format to import: json,
csv, or tsv (defaults to
'json') (default: json)
/columnsHaveTypes indicated that the field list
(from --fields, --fieldsFile,
or --headerline) specifies
types; They must be in the
form of
'<colName>.<type>(<arg>)'.
The type can be one of: auto,
binary, bool, date, date_go,
date_ms, date_oracle, double,
int32, int64, string. For
each of the date types, the
argument is a datetime layout
string. For the binary type,
the argument can be one of:
base32, base64, hex. All
other types take an empty
argument. Only valid for CSV
and TSV imports. e.g.
zipcode.string(),
thumbnail.binary(base64)
ingest options:
/drop drop collection before
inserting documents
/ignoreBlanks ignore fields with empty
values in CSV and TSV
/maintainInsertionOrder insert documents in the order
of their appearance in the
input source
/j, /numInsertionWorkers:<number> number of insert operations
to run concurrently (defaults
to 1) (default: 1)
/stopOnError stop importing at first
insert/upsert error
/mode:[insert|upsert|merge] insert: insert only. upsert:
insert or replace existing
documents. merge: insert or
modify existing documents.
defaults to insert
/upsertFields:<field>[,<field>]* comma-separated fields for
the query part when --mode is
set to upsert or merge
/writeConcern:<write-concern-specifier> write concern options e.g.
--writeConcern majority,
--writeConcern '{w: 3,
wtimeout: 500, fsync: true,
j: true}'
/bypassDocumentValidation bypass document validation
C:\Users\John\mongo-demo>mongoimport --db images --collection images --drop --file images.json --jsonArray
2019-05-02T09:52:33.241-0500 connected to: localhost
2019-05-02T09:52:33.243-0500 dropping: images.images
2019-05-02T09:52:34.607-0500 imported 13 documents
C:\Users\John\mongo-demo>mongo
MongoDB shell version v3.6.4
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.4
Server has startup warnings:
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.432-0500 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** Start the server with --bind_ip
<address> to specify which IP
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten]
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] ** WARNING: The file system cache of this machine is configured to be greater than 40% of the total memory. This can lead to increased memory pressure and poor performance.
2019-04-29T08:06:15.433-0500 I CONTROL [initandlisten] See http://dochub.mongodb.org/core/wt-windows-system-file-cache
2019-04-29T08:06:15.459-0500 I CONTROL [initandlisten]
> use images
switched to db images
> db.images.find().pretty()
{
"_id" : ObjectId("5cc72293d56b582c60f63742"),
"tags" : [
"chest",
"lateral"
],
"date" : ISODate("2019-04-29T16:13:07.326Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111199",
"patient" : "Money Penney",
"width" : 128,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc7227237c5612ba896038b"),
"tags" : [
"chest",
"lateral"
],
"date" : ISODate("2019-04-29T16:12:34.832Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "James Bond",
"width" : 256,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc757d77525e33e7816b803"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T20:00:23.601Z"),
"reviewed" : false,
"name" : "12311111112222222222333333333399",
"patient" : "John Canessa",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc72e2dcf67480dfc23aa1e"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T17:02:37.311Z"),
"reviewed" : false,
"name" : "11111111112222222222333333333399",
"patient" : "James Smith",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc75814f2e4d435f06cbee7"),
"tags" : [
"right leg",
"side"
],
"date" : ISODate("2019-04-29T20:01:24.501Z"),
"reviewed" : false,
"name" : "12341111112222222222333333333399",
"patient" : "John Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc72254a72d243298a2a16a"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T16:12:04.976Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"width" : 256,
"height" : 256,
"__v" : 0
}
{
"_id" : ObjectId("5cc757f24b23593dc4f18107"),
"tags" : [
"hand",
"front"
],
"date" : ISODate("2019-04-29T20:00:50.272Z"),
"reviewed" : false,
"name" : "12311111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 256,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc75789c4523c19504f7acd"),
"tags" : [
"hand",
"lateral"
],
"date" : ISODate("2019-04-29T19:59:05.801Z"),
"reviewed" : false,
"name" : "12111111112222222222333333333399",
"patient" : "John Canessa Smith",
"width" : 512,
"height" : 512,
"__v" : 0
}
{
"_id" : ObjectId("5cc70683e6626c32b8ebb0b6"),
"tags" : [
"right leg",
"lateral"
],
"date" : ISODate("2019-04-29T14:13:23.730Z"),
"reviewed" : false,
"name" : "11111111111111111111111111111111",
"patient" : "John Doe",
"__v" : 0
}
{
"_id" : ObjectId("5cc7586b62b47f2bfcefe820"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:51.324Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7589c66c6353d703ba985"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:03:40.890Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7583de26d8038f86588a6"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:05.573Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
{
"_id" : ObjectId("5cc7585f285dd410a4131495"),
"tags" : [
"left arm",
"side"
],
"date" : ISODate("2019-04-29T20:02:39.622Z"),
"reviewed" : false,
"name" : "12345111112222222222333333333399",
"patient" : "Jane Smith",
"width" : 512,
"height" : 1024,
"__v" : 0
}
> exit
bye
We first get information about the mongoimport command. We exported the data as a JSON array so we will have to import it as JSON array. We specify the database and collection to use, in this case “images”. We request that the collection be dropped before we insert the data. If we do not, we may end with additional data if the collection was not empty.
When that is done, we go back to the mongo shell and display all the data in the images collection. Seems it is the same we exported.
Hope you enjoyed reading and more important practicing as you went over this post.
If you are interested in getting to the code and data you can find it in my GitHub repository.
If you have comments or questions regarding this or any other post in this blog, or if you would like me to help with any phase in the SDLC (Software Development Life Cycle) of a product or service, please do not hesitate and leave me a note below. Requests for help will remain private.
Keep on reading and experimenting. It is the best way to learn!
John
Follow me on Twitter: @john_canessa