CT047-3-M: Analysis and Solutions (Proposing a Big Data Conceptual Framework): Big Data Analytics And Technology Assignment, AUTI, Malaysia

Tasks (60%)

Part 2 – Analysis and Solutions (Proposing a Big Data Conceptual Framework).

Description:
The Meteorological Department of Malaysia has appointed you recently as the person in charge of proposing a conceptual big data analytics framework to help the department extract and uncover the hidden patterns as well as applicable information that are useful to yield better weather forecast results. As we know, the analysis of weather data includes temperature, air pressure, wind speed, wind direction, humidity, rainfall, etc.

The conventional way of predicting the weather utilizing these data is a tedious and complex task, this is because these data are disorganized, and they are usually stored in multiple formats, in separate files, and in multiple computers. Hence the weather prediction is a human-intensive task since it may require workers to collect, copy, transfer, and combine a lot of heterogeneous weather data from multiple files/ sources into experimental datasets. Furthermore, the accuracy of the collected data may also be affected because errors might occur during the process.

At this point, you have found out that collecting weather data from sensors installed in multiple locations are not done in real-time but on a periodic basis. In addition, these weather data are unprocessed (raw data) and they are stored into flat files or DBMS directly, which you think this conventional way is inefficient and expensive.

This is because handling such enormous and increasing amounts of data has caused the processing infrastructure to become costly and time consuming. Besides, you also noticed that the department staff process and analyse the abundance of weather data using only basic statistic tools with less visualization techniques. As a result, the obtained results may not be analysed and interpreted in a speedy and efficient manner.

Therefore, your key task is to design and then propose a conceptual big data solution framework (ingestion, storage, processing, analysis, visualisation) for the department to develop a functional weather forecasting system with intelligent features. This framework shall be able to provide faster speed and cheaper cost than the conventional practice. Your proposed framework solution must address the followings:
• The benefits of weather forecasting towards the improvement of human life.
• Big data eco-system (diagram).
• Adoption of big data tool for each layer, with sound justifications.
• Data security & privacy.
• Citation relatedness to the stated problem.

Answer

Request Answer of this Assignment