ETL Framework for Document Databases & Relational Databases

The following two research papers are targeted to achieve ETL functionality between Document and RDBMS databases. For both researches, MongoDB and SQL Server were used as proof of concept.

the first research is about building a replication layer between the document and document databases. The basic architecture was displayed in the below figure.

During this research, CPU processes were compared as shown in the below figure.

You can see that when SQL Server processing % in the range of 70 - 100 while MongoDB is not even reached 1 %.

By using the findings in the above research, another research was done to define a ETL framework between DocumentDB and SQL Server. 

Microsoft Announces GA of Azure DocumentDB Managed NoSQL Document Database Service

Microsoft has announced that the Azure DocumentDB database service is now generally available offering a fully managed NoSQL document database service that allows applications to query and process JSON data at scale. Microsoft has also released a new open source data migration tool to facilitate the movement of existing data from other database systems to DocumentDB.

https://appdevelopermagazine.com/2650/2015/4/15/Microsoft-Announces-GA-of-Azure-DocumentDB-Managed-NoSQL-Document-Database-Service/

Search in MongoDB

I came across this incidents while working on some MongoDB. I am not sure it is the expected behavior, so thought of posting it.

I have a collection called zipcodes which has following attributes.

So, I need to get the count for two state , I executed following script.

it returned me  1596 which was incorrect. Then started to go more into details, I started with by interchanging the values and then I got a different results.

Then I remove first row and executed it again.

Oops, I got the same values as before. Then I realize that, whenever we pass values for same attribute only the last one will be taken in. So for the above case I should us $in.

This behavior is same for insert, update, find etc.

UI Features in RoboMongo

I posted about RoboMongo tool around one month back. Thought of writing about two UI features of the tool.

Intelli-sence

This tol provides the intellisense feature so that users have the option of selecting the command rather than typing it.

Query Formatting

When large queries are typed, it will format the queries with different colors as shown below.

RavenDB

Schema free

Forget about tables, rows, mappings or complicated data-layers. RavenDB is a document-oriented database you can just dump all your objects into. Queries are amazingly fast and flexible.

Scalable

Sharding, replication and multi-tenancy are supported out-of-the-box. Scaling out is as easy as it gets.

Transactional

ACID transactions are fully supported, even between different nodes. If you put data in, it is going to stay there. We care about your data and we keep it safe.

High performance

RavenDB is a very fast persistence layer for every type of data model. Skip creating complicated mapping or multi-layer DALs, just persist your entities. It Just Works, and it does the Right Thing.

APIs

RavenDB comes with a carefully designed client API for .NET, Silverlight, Javascript and REST over HTTP.

Feature rich and extensible

Built with extensibility in mind, RavenDB can be easily extended via bundles. Many integration points ensure you can always squeeze more out of RavenDB. You aren't shackled to a One Size Fits None solution.

Carefully designed

Every bit of code was carefully considered. RavenDB was designed with best-practices in mind, and it ensures that everything Just Works.

Get the details from http://ravendb.net/

DROP and REMOVE Mongo documents

In MongoDB you have two options to remove all the documents in a collection.

db.users.drop ()

db.users.remove()

thought the output of above two statements ( removing the all the documents in a collection) is same, there are couple of differences with two options.

remove will remove document one by one where as drop will remove all the documents at once. So in case you are dropping documents of a large collation, drop will be much faster than the remove.

However, drop will remove the meta data related to the collection as well. for example, it will remove the indexes associated with the collection.

So if you want to remove all the documents in a large collection, better option is use drop statement and recreate indexes again.

NoSQL Benchmarking

In this article it will reveal the characteristics of Cassandra, HBase and MongoDB identified through multiple workloads.

Webinars on MongoDB

Webinar: Best Practices for MongoDB on AWS

Thursday March 28th 11:00am PDT / 2:00pm EDT / 6:00pm UTC

In this session we will look at best practices for administering large MongoDB deployments in the cloud.

Click here for more information.

Webinar: How Telcos Use MongoDB

Tuesday April 2nd 7:00am PDT / 10:00am EDT / 2:00pm UTC

In this webinar, you will learn how operators are increasingly leveraging MongoDB to develop new applications quickly and secure new revenue streams.

Click here for more information.

Webinar: Deployment Best Practices

Tuesday April 9th 11:00am PDT / 2:00pm EDT / 6:00pm GMT

(We suggest registering at least 30 minutes before the start of the webinar.)

The last bugs are finished, testing is complete, and business is ready. What do you do next? In this talk we will cover the topics to ensure that you are prepared for a successful launch of your MongoDB based product, including:

Click here for more information.

Webinar: How Banks Use MongoDB as a Tick Database

Tuesday April 16th 7:00am PDT / 10:00am EDT / 2:00pm GMT

(We suggest registering at least 30 minutes before the start of the webinar.)

Click here for more information.

Webinar: MongoDB 2.4 Feature Demo and Q&A on Geo Capabilities

Thursday April 18th 11:00am PDT / 2:00pm EDT / 6:00pm UTC

In version 2.4, there have been significant enhancements to the geospatial indexing capabilities in MongoDB, such as polygon intersections, a more accurate spherical model, and better integration with MongoDB's aggregation framework.

Click here for more information.

Webinar: NoSQL as the New Normal

Tuesday April 23rd 11:00am PDT / 2:00pm EDT / 6:00pm GMT

What started as a way for web giants to solve problems of serious scale has become the default way all enterprises manage Big Data. Despite having a catchy, if inaccurate title, there really isn't a coherent "NoSQL" category, nor is there a simple future for the range of NoSQL databases. In this presentation, Matt Asay will outline the reasons for NoSQL's existence and persistence, how the different NoSQL technologies help enterprises get control of Big Data, and will identify the trends that point to a bright future for post-relational databases.

Click here for more information.

Webinar: MongoDB 2.4 Feature Demo and Q&A on Security

Thursday April 25th 11:00am PDT / 2:00pm EDT / 6:00pm UTC

In version 2.4, MongoDB Enterprise includes Kerberos support for integration into existing enterprise security systems, as well as role-based privileges to provide more granular security for your cluster.

Click here for more information.

Webinar: How Banks Manage Risk with MongoDB

Tuesday April 30th 8:00am PDT / 11:00am EDT / 3:00pm GMT

(We suggest registering at least 30 minutes before the start of the webinar.)

Click here for more information.

Webinar: MongoDB 2.4 Feature Demo and Q&A on Hash-based Sharding

Thursday May 2nd 11:00am PDT / 2:00pm EDT / 6:00pm UTC

In version 2.4, MongoDB introduces hash-based sharding, allowing the user to shard based on a randomized shard key to spread documents