Still Unsure Which SCD Type to Use?

 One of the biggest challenges in designing a data warehouse or modern analytics platform is deciding how to preserve historical data.

Most data professionals are familiar with Slowly Changing Dimensions (SCDs) Type 1, Type 2, Type 3, Type 4, and Type 6 but choosing which type to use for a particular business scenario is often the real challenge.

Should you overwrite the existing value?
Should you maintain every historical version?
Should you store history in a separate table?
Or is a hybrid approach the better option?

The decision depends on your business requirements, reporting needs, query performance, and the frequency of data changes.

I've created the following decision diagram to help identify the most suitable SCD type based on common real-world use cases. I hope it serves as a quick reference for data engineers, BI developers, and data warehouse architects.

Which SCD type do you use most frequently in your projects?

#DataEngineering #DataWarehouse #DataModeling #DimensionalModeling #SlowlyChangingDimensions #SCD #ETL #BusinessIntelligence #Analytics #DataArchitecture #SQLServer #MicrosoftSQLServer #DataEngineeringCommunity #DataAnalytics

Geometric Mean in Power BI

 In my latest article, I walk through how to calculate the Geometric Mean in Power BI, including practical implementation steps and when you should use it instead of the traditional average.

Key takeaways:
🔹 Understand the difference between Arithmetic Mean and Geometric Mean.
🔹 Learn why growth and compounded values require a different approach.
🔹 Implement Geometric Mean calculations in Power BI with practical examples.
🔹 Improve the accuracy of your analytical dashboards and business reports.
Whether you're a Power BI developer, data analyst, or BI professional, understanding the right statistical measure can significantly improve decision-making.
📖 Read the full article here:
https://lnkd.in/gYwrP4za

I'd love to hear how you've handled compounded metrics or growth-rate calculations in your Power BI projects.
hashtag#PowerBI hashtag#BusinessIntelligence hashtag#DataAnalytics hashtag#DAX hashtag#DataVisualization hashtag#Statistics hashtag#DataScience hashtag hashtag#Analytics hashtag#BI hashtag#PowerBICommunity

How Data Science Project Works - From the Koobiyo Teledrama

About five years ago, the Koobiyo teledrama was very popular due to the uncharacteristic nature of the teledrama. It was a political teledrama, one reason the teledrama became popular. However, this post is not to discuss the political side of the tele drama but to discuss the data science side of it. 

Jehan, the main character of this teledrama, has the idea of building a tool to predict the future problems of people. He uses decision tree architecture to achieve this and explains his idea to his companion, Priyantha as shown in the below video. Watch from the 540th second.

So his idea is very clear. By using data like Gender, Marital Status, Professional Qualification, Parents' details, Friends' details, etc. 

After building the predictive software, he presents his case to his good friend Hiruni and his developer too joins the discussion in the following video. Watch from the 40th second.

After observing the work done by Jehan, Hiruni was impressed, but she had a very vital question to ask. "Why do you think people will buy this". Jehan was a little perplexed by the question, and he stressed that this is the software that everyone will have at their fingertips. Obviously, people know their problems they need solutions, not to suffer from their own problems. 

This is something important to us as data scientists. We are very good at technical details, but we need to understand the solution part of the tool. Remember, people need a solution to their problem not a technical tool. 

Then Hiruni provide a solution to Jehan and asked him to provide this to a human resource person to predict the future problems of its own employees. 

Then the solution can be viewed in the 21st second of the following video.

In the live demo, Jehan proves that it is important to select the best people for the organization from this software. It is clear how we should think from the business point of view rather than focusing on technical aspect mainly. As a data scientist, we need to look at the solution side of it.

Who is a Data Scientist? A Dragon a Pegasus or a Unicorn?

 

Unicorns, Pegasus, or dragons are mythical creatures that are created by combining the strong attributes of different animals. In Sri Lankan culture, we can see different dragons in temples. It is believed that these dragons are to protect various people and places. 

These dragons have seven parts from different people, as expressed in the following poem. 

As per the song, the dragon is a combination of

·       trunk of an elephant

·       legs of a lion

·       ears of a pig

·       teeth of a crocodile

·       eyes of a monkey

·       body of a fish

·       wings of a bird

The combination of these most strengthened parts will make the dragon a strong animal to achieve his required tasks. 

Isn't that the same with a data scientist? Don't they need different strengths, as shown below?  

Does that mean data scientists are mythical? Read the full article here E-Magazine – Magazine that bridges Management and IT

Data Mining in SQL Server

Data Mining or Prediction has become a buzz word not only in academia but also in the industry as well. SQL Server is providing a rich set of algorithms to support data Mining for a long time. However, most of these features are not used due to many reasons. The following article series which I completed at sqlshack provides details of how to use data mining in SQL Server. The major important advantage is that you can use the existing data in the SQL Server with the Data Mining itself. Further, you have to option of using MS BI family for data mining. 

Enjoy the article series here. 

Introduction to SQL Server Data Mining

Naive Bayes Prediction in SQL Server

Microsoft Decision Trees in SQL Server

Microsoft Time Series in SQL Server

Association Rule Mining in SQL Server

Microsoft Clustering in SQL Server

Microsoft Linear Regression in SQL Server

Implement Artificial Neural Networks (ANNs) in SQL Server

Implementing Sequence Clustering in SQL Server

Measuring the Accuracy in Data Mining in SQL Server

Data Mining Query in SSIS

Text Mining in SQL Server

Confused with Big Data, Data Science and Data Analytics

Often people confused with Big Data, Data Science and Data Analytics. Are they same or different.
This link will give you an idea about what are they, what skills you need and what are the salary levels.

Read more to get further details https://intellipaat.com/blog/compare-big-data-data-science-data-analytics/