Effectiveness & Efficiency in Data Analysis

The evolutionary world of Data Analysis

The evolution of humanity depends on the analysis of the past data, using techniques such as processing, evaluation and visualization. This is a fact, as we can see that the people evolve and try to improve the humanity every day through the technological achievements. What do you think? That the human beings remain the same from the beginning of time? Of Course NOT! Evolution in every aspect of the life, has dominate the purpose of every living human being. 

Every day, scientists, analysts and pretty much everybody from all around the world, tries to dig deeper and analyze the past to improve the future and avoiding the possible repetition of history, especially when we are considering the unfortune situations and topics like wars, economic crisis and technology. Everyone can think "Why humanity has breakthrough the last century?". Watching and observing the past, people tend to improve their selves and their beliefs each time that they are facing similar experiences but always remember that the equation increased exponentially. 

An example below will help you to understand:

10 * 10 = 100

100 * 100 = 1000

1000 * 1000 = 10000

10000 * 10000 = 100000

100000 * 100000 = 1000000

You can see that the 1st and the 2nd row contain small numbers when are improving their selves but after a long time, you will reach the "perfection". Now, at this point it is significant to point you that the perfection needs to has an upper and lower boundary. Let's say that you have the one very big number and multiply it by itself, so you have perfection * perfection, disaster it will follows. The word perfection has an objective meaning and need to be controlled by using its own boundaries. This means that always we need to keep the balance in order to avoid the opposite of your seeking results.

Effectiveness vs Efficiency

Regarding the mentioned examples above, two words need to be explained to get the full picture of the balance. Effectiveness and Efficiency has basically exactly the same meaning but when you are trying to bring them in life, both of them have a different target groups of data to provide you the correct result. 

Effectiveness

Effectiveness is when our analysis can have as an outcome the results that we are seeking without considering the computational time, memory footprint and other measures that everyone need to be considered. So, if you succeed to analyze data with effectiveness that means that your results are 100% correct and you trust the application that it will work in every situation.

Efficiency

On the other hand, efficiency is when our analysis can have as an outcome what we are seeking but now we are optimizing our Analysis procedure in order to avoid to waste computational power, time and memory.

The harmony of co-existence

Both of them need to co-exist in an analysis of every topic in order to avoid problems like disaster as mentioned above. Need to keep the upper and lower boundary of those two functionalities in every procedure to be able to continue to the next step.

So as a result, the perfection can be good but if you are perfect only on one of each of the mentioned words, without thinking of the consequences of the other, then, most probably your Analysis it will fail in long term and it will not provide you harmony and balance on your results.  

An example it will convince you

Let's take a road trip that you need the faster road to bring you in the destination.

If the starting point is the A and the ending point is the C then its not necessary to spend power and time to go through B first and then to C. This example illustrates the prove of effectiveness but not the efficiency.

On data analysis exactly the same rule applies especially when you have to process and produce reports and dashboards using big data. If you keep the most possible balance in your data analysis task it will benefit you from every aspect. 

However, remember that the balance of data analysis do not depend only to executional time that has been spent. If you are writing code in any programming language and try to shorten your code in less lines for increasing the efficiency, this might affect you negatively as it will raised problems like increasing of difficulty of the amend ability of your code.

Conclusion

Finally, now you will be able to recognize, that by providing balance in the effectiveness and the efficiency at your analysis procedure can produce amendable fast executions for your creation of reports and dashboards, even for the classification of your data if you are capable enough. By constructing pipelines with data processing procedures and visualize your information providing whatever is necessary for the end user, your tasks can be finilized with an easy way.

Most of times, the co-exist of these two fundamental words act as invest to your data analysis, business analysis, data engineering or data science tasks, by amplifying whatever is going to be executed.

And always do not forget....that the life it's a polite execution of an analysis procedure for each one of us.