Role Playing Teaching Method

 

I have been teaching at various places, and I have been using a few different techniques. I had never used Role Playing teaching technique until yesterday. It was totally sudden and was never in the plan. 

I was teaching Data warehousing and wanted to explain the difficulty in requirement elicitation for data warehousing. The whole idea was to tell students that due to the ad-hoc data analysis in data warehousing it is difficult to collect requirements. 

Roles

This is a postgraduate course where participants have some level of industry experience. In the class, there was a person who was a Human Resources Manager from a software development organization and there were two Business Analysts. There were a few developers as well. I asked Human Resource Manager to come up with a scenario, and instantly, he decided on employee attrition which seems to be a headache for the software industry these days. 

So two BA were asked to find requirements for attrition analysis. Both parties started the discussion and no one knew where to start and what is the direction. During the 5 - 10 minutes discussion, it was very clear that the discussion was not focused. After completing the requirement elicitation, I asked BA to explain the requirements to the developer. Even though it was done, I am pretty much sure that the developer will not be able to develop an analytical solution for attrition.

Then the second role-playing started. I asked the same BA team to capture requirements to implement an exit interview procedure. Then the BA team said that they can be a passive participant at the exit interview and they can collect exsitng document with related to the exit interviews. Then it was very clear that requirement capturing for that task is much simpler than the before task. 

Experience Sharing

After the role playing was completed, I asked from each party, and the viewers' experience in the role playing. All agreed that requirement elicitation in analytical systems is difficult and needs more experience. 

For me this was a new experience and it was sudden. Next time I need to plan for such events more methodically. 

I am not sure whether they enjoyed, but I really did enjoy.

Theory Behind Role Playing Teaching Method

Source: https://roleplayasaninstructionalmethodforadultlearners.weebly.com/index.html

How to Confuse Machine Learning

This is a post that was shared in LinkedIn and thought of exploring this with my pet tool Orange Data Mining tool.

First, separate these images into single images randomly. Since there is a small number of images, thought of employing clustering over classification. 

Following is the very simple Orange flow and it can be found at GitHub as well. 

As you could see, it is a very simple Orange workflow. Inception v3 was used as the embedded and Cosine distance was used as the distance calculation. Then Hierarchical Clustering was used. it is as very clear clusters which can be seen in the following figure.

Then the cluster distribution as below. 

So, you can't confuse Machine Learning as you think.

StatModels in SQL Server Python

Python in SQL Server is an important feature that is provided especially for data scientists. However, there are additional libraries that you need to install to achieve your goals. statsmodels is a Python module that provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests and statistical data exploration.

In statmodels, there are multiple versions and you need to pick the right version for you. For example, STL is available only after 0.12.0 statmodels. By default SQL Server 2019 will get you the statmodels 0.9.0 which may not be enough so you have to upgrade which can be done from the following python pip.

python -m pip install statsmodels

However, this will give you an error saying that you need to upgrade the python to the higher version of 3.8 as SQL Server as the python 3.7.1 version. Now you need to upgrade python for SQL Server. 

But the better thing to do is to upgrading relevant statsmodels version which is 0.12.0. Therefore you need to run the command python -m pip install statsmodels==0.12.0 at the folder.

C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES

Article: Testing Type 2 Slowly Changing Dimensions in a Data Warehouse

Testing is a challenging task in Data Warehousing. This article provides methods and scripts for performing testing on the Slowly Changing Dimensions (SCD). 

Read the article at Testing Type 2 Slowly Changing Dimensions in a Data Warehouse (sqlshack.com)

With this, the following is the list of articles that are related to Data Warehouse.

Infrastructure Planning for a SQL Server Data Warehouse (mssqltips.com)

Why Surrogate Keys are Needed for a SQL Server Data Warehouse (mssqltips.com)

Create an Extended Date Dimension for a SQL Server Data Warehouse (mssqltips.com)

Loading Historical Data into a SQL Server Data Warehouse (mssqltips.com)

Things you should avoid when designing a Data Warehouse (sqlshack.com)

Implementing Slowly Changing Dimensions (SCDs) in Data Warehouses (sqlshack.com)

Use Replication to improve the ETL process in SQL Server (sqlshack.com)

Incremental Data Extraction for ETL using Database Snapshots (sqlshack.com)

Indexing : Adding jet fuel to your car

In database technology indexes used to improve the performance of select queries that can be seen from the following figure. 

This post is to show case different cases for indexes using different articles. This article SQL Server Clustered Index Behavior Explained via Execution Plans will tell you how different scenarios are for Clustered Index. 

However, indexes needed to be maintained using rebuild and reorganizing SQL Server Maintenance Plan Index Rebuild and Reorganize Tasks . 

How do we know what is the better index and where the indexes are needed. This can be done Troubleshooting using Wait Stats in SQL Server and specially How to Avoid CXPACKETs? . Another place indexes are Internals of Physical Join Operators (Nested Loops Join, Hash Match Join & Merge Join) in SQL Server .

Column indexes are another important aspect and Script to Create and Update Missing SQL Server Columnstore Indexes.

Furhter we have discussed few more cases for index before at here 

Happy reading into this. 

Article: Incremental Data Extraction for ETL using Database Snapshots

We have been discussing ETLs in detail for the data warehouse. As you know, in ETL one of the important task is extracting the incremental data extraction from sources. This article discusses a new method to extract data from operational databases and how to use ETL using Database snapshots in order to improve the ETL process.

What3Words - Expressing Your Location in Three Words

How many times that you had to wait so long for your taxi due to the wrong location? In these pandemic days, how many times has your order is not delivered to your location again due to an invalid location. What3Words is a simple way of expressing your location.

What3words has divided the world into three-meter squares, each with a unique three-word address made from three random words. Now people can refer to any precise location. for example, my location is ///quack.competent.workflow.

You can share the location and another 3-word location for navigating. 3-word addresses are appearing on contact pages, business cards, travel guides and physical signs all around the world. People are using the free what3words app to find friends faster, get accurate directions to that tucked away Airbnb and to drive, or ride, exactly where they want to go.

what3words is available in 40 languages, enabling over 4 billion people to use the system in their native tongue. It can be used via the free mobile app or online map. 

About us - what3words - YouTube 

Most Popular Software Programming Languages In Pictures

If you are a developer you might be wondering what is the best language for application development. Let us look at it from some pictures. 

Obviously, JAVA is the winner and you can see C, C++ is also in the higher ranking. However, not sure the reason for the higher ranking on Visual Basic .NET over C#.

Following is the ranking from IEEE. 

JAVA is the non-dispute leader in this ranking as well and it too has the same ranking as previous. The notable observation is the growth of R language over the years. It has jumped to rank 6 from 9. 

Another important parameter is the Salary and Job opening.

Though JAVA has a lot of openings than other programming languages, the average salary is higher in other languages such as Python, C++, Ruby etc.

Finally, let us compare the different aspects of programming languages. 

Cluster Validation - Purity Calculation

As we know, clustering is an unsupervised technique. When it comes to classification, there are a lot of evaluation techniques such as Precision, Recall, F1, MCC etc. However, what are the techniques that can be used to evaluate clustering techniques. Purity calculation is one of the simplest calculations to evaluate your clusters.

In the Purity cluster quality measure, we will analyse the cluster distribution with respect to a selected variable. Let us look at how to calculate Purity in a Text Clustering using Orange and the following is the Orage flow. 

Further, you can get the Orange flow from Github. 

First, let us look at how the Purity is calculated. 

Let us assume that following are the clusters and data distribution.

In each cluster, the maximum number of objects that are falling to each cluster is calculated. For example, in Cluster 1, X has three instances while Cluster 2 has three instances of O and Cluster 3 has four L instances. Those numbers are added up and divided by the total number of instances which is 16. 

Let us look at this example with our popular film review dataset.

After the text Preprocessing, the Loving Clustering technique is used. Following is the cluster distribution with respect to the review classification.

So the Purity is (190 + 193 + 158 + 123+ 136 + 112+124+102 +11) / 2000. Ideally, this should be close to 1 meanwhile in the case of multi-class we can calculate the Purity with a Minimum value which should be close to 0. 

Entropy is another calculation that is performed to measure the Cluster Quality which we will leave for another day.

Hasan Ali & Tweets

As they say in Cricket, "Catches win matches". It will be more relevant when you missed a catch in the WorldCup semi-final.  During the T20I world cup when Hasan Ali dropped the catch, the match turned to head to tail. As cricket is a great game of uncertainty, the crowd don't believe in that. After the dropped catch, there were a lot of allegations against Hasan Ali. It went to an extent that his wife and his religion also are part of these allegations. 

Let us analyse tweets against Hasan Ali using Tweet Sentiment Visualization App. 

Though there were a lot of hate comments against Hasan Ali on Facebook, Instagram etc, Twitter users are seems to be more professional as we see a lot of positive tweets against him. Some tweets wishing him success as well. 

When you look at the topics, catch, stay strong are the common topics. 

Then let us look at the Tag Cloud in different quadrants.