In this week's reading by Lev-Libfled and Margolin (2019), the authors discuss that the MapReduce paradigm is based on several assumptions, including the completeness of data, independence of data set calculations, and relevancy distinguishability.
Paper:
1.) Describe what each of these assumptions means.
2.) How will this impact the MapReduce paradigm if you fail to evaluate the above-said assumptions?
3.) What will the effect of the impact on big data security be?
You will be assessed based on the following:
a.) Description of the completeness of data, independence of data set calculations, and relevancy distinguishability
b.) Explanation of the impact of failing to evaluate assumptions noted in the MapReduce paradigm and the effect on big data security.
Full Answer Section
How will this impact the MapReduce paradigm if you fail to evaluate the above-said assumptions?
If you fail to evaluate the above-said assumptions, it could have a number of negative impacts on the MapReduce paradigm, including:
- Inaccurate or incomplete results: If the data is not complete or correct, then the results of the MapReduce job will be inaccurate or incomplete. This could lead to errors in decision-making or to the loss of valuable data.
- Reduced performance: If the data sets are not independent, then the MapReduce framework will not be able to parallelize the processing of the data sets. This could lead to decreased performance.
- Increased security risks: If the data is not relevant, then the MapReduce framework may process data that it should not be processing. This could increase the risk of data breaches or other security incidents.
3. What will the effect of the impact on big data security be?
The impact of failing to evaluate the assumptions of the MapReduce paradigm on big data security could be significant. If the data is not complete or correct, then it could be used to make inaccurate or misleading decisions. This could have a negative impact on the organization's bottom line or could even lead to legal liability.
In addition, if the data is not independent, then it could be used to identify individuals or groups that should not be identified. This could lead to privacy violations or could even put individuals at risk.
Finally, if the data is not relevant, then it could be used to process data that should not be processed. This could increase the risk of data breaches or other security incidents.
Overall, it is important to evaluate the assumptions of the MapReduce paradigm before using it to process big data. By doing so, organizations can help to ensure that the data is processed correctly and that the security of the data is not compromised.