Import Metadata and indexes

Migration Module (2.3.3) - Migrate from MongoDB to Amazon DocumentDB using Database Migration Service (DMS)

Although AWS DMS handles the migration of data, it does not migrate indexes. In section ”2.2.1 Connect to MongoDB instance using AWS Cloud9“ you created indexes on your MongoDB instance. In this section you will use the Amazon DocumentDB Index Tool to create these index on your Amazon DocumentDB cluster.

(1) From your AWS Cloud9 environment, download Amazon DocumentDB tools from the awslabs amazon-documentdb-tools github with the following commands at the OS prompt.

git clone https://github.com/awslabs/amazon-documentdb-tools.git
ls -l

installation

(2) Next, create a dump directory to dump the MongoDB indexes into.

export PATH=$PATH:/usr/local/bin/
sudo /usr/local/bin/pip install bson
mkdir dump

(3) Use the Amazon DocumentDB Index Tool to export the indexes from the source mongoDB database to the dump directory using Amazon DocumentDB Migration Tool.

Note: the next command requires the OS variable mongoIp to have been set in 2.2.1 Connect to MongoDB instance using AWS Cloud9.

sudo python amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py --host $mongoIp --port 27017 --username dmsuser --password 202xDoc --auth-db admin --dir dump --dump-indexes

(4) Run the commands below to clean up configuration files that aren’t needed for the next steps, and to view the metadata files that the index tool created:

sudo rm -rf dump/config
ls -lrt dump/labdb/

dumpindexes

(5) If you want to see the indexes, they are located in the dump directory on the left side of your Cloud9 browser window. If you expand the directory, you will see the two metadata files and can double click them to view them. Double click the customers.metadata.json file to view the contents:

cloud9indexes

(6) Run the following command against the target Amazon DocumentDB cluster to show any compatibility issues there may be with indexes before importing the indexes into Amazon DocumentDB.

Note: the next command requires the OS variables: *docdbEndpoint *, username, password, all to have been set in 2.2.2 Connect to the DocumentDB cluster.

sudo python amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py --show-issues --dir dump --host $docdbEndpoint:27017  --tls --tls-ca-file rds-combined-ca-bundle.pem --username $username --password $password

indexresult

The Amazon DocumentDB Index Tool has identified that a “2dsphere” geospatial index is not yet supported by Amazon DocumentDB.

(7) “The Amazon DocumentDB Index Tool allows you to skip incompatible indexes via the ”—skip-incompatible“ argument. Run the following command to use the Index Tool metadata files created above to create the same indexes in your Amazon DocumentDB cluster destination, ignoring the indexes that are not yet compatible:

sudo rm -rf dump/config
sudo python amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py --restore-indexes --dir dump --host $docdbEndpoint:27017  --tls --tls-ca-file rds-combined-ca-bundle.pem --username $username --password $password --skip-incompatible

imported

You will next validate that the indexes are created in Amazon DocumentDB .

Log in your Amazon DocumentDB endpoint:

mongo --ssl --host $docdbEndpoint:27017 --sslCAFile rds-combined-ca-bundle.pem --username $username --password $password

and run the following queries:

use labdb
db.locations.getIndexes()
db.customers.getIndexes()

indexes

You have successfully created metadata and the supported indexes. Please proceed to the next section to create a DMS migration task and move the data.