IoT Fundamentals Big Data and Analytics End of Course Assessment Final Answers

Explanation: Electronic medical information is private personal information. The digital results of tests such as x-rays, MRIs, and ultrasounds do not have a format of fixed fields so they are considered unstructured. Because the information is transmitted from one place to another for review, it would be in motion. Data is at rest once it is stored in a data center.

Explanation: Three key words can help distinguish data from Big Data:

Volume – describes the amount of Big Data being transported and stored
Velocity – describes the rapid rate at which Big Data is moving
Variety – describes the type of Big Data, which is rarely in a state that is perfectly ready for processing and analysis

Explanation: Structured data is entered and maintained in fixed fields within a file or record, such as data found in relational databases and spreadsheets. Structured data entry requires a certain format to minimize errors and make it easier for computer interpretation.

Explanation: As data is collected from varying sources and in varying formats, it is beneficial to utilize specific file types that allow easy conversion and universal application support. CSV, JSON, and XML are plain text file types that allow for collecting and analyzing of data in a format that is easily compatible and applicable for analysis.

Explanation: Transforming data is the process of modifying data into a usable form. This includes tasks such as aggregating data and sorting it. Collecting data is the process of extracting data. Creating visual representations of data and presenting the knowledge gained from the data are examples of the final steps that are used in data analysis.

Explanation: Organizations can use virtualization to consolidate the number of required servers by running many virtual servers on a single physical server. Cloud computing allows organizations to scale their solutions as required and to pay only for the resources they require.

Explanation: NoSQL databases can use a key-value pair approach to store data. A NoSQL database can import unstructured data. Organizing data in columns, tables, and rows; establishing relationships within the stored data. NoSQL does not rely on the relational database approach of linked tables.

Explanation: Some of the benefits of using CFS over HDFS are are follows:
Better availability – CFS does not require shared storage solutions.
Basic hardware support – No special servers are needed and no special network devices are needed for CFS.
Data integration – All data that is written to CFS is replicated to both analytics and search nodes.
Automatic failover – As with availability, failover is automatic because of replication.
Easier deployment – Clusters are easy to setup and can be running in a matter of minutes. CFS does not require complicated storage requirements or master-slave configurations.
Supports multiple data centres – CFS can run a single database across multiple data centers.

Explanation: The rapid increase of devices in the IoT is one of the primary reasons for the exponential growth in data generation. With the number of sensors and other end devices growing exponentially, mobile routers are increasingly used to better manage Internet traffic for systems that are in motion.

Explanation: The two primary types of business information useful to a company are transactional information and analytical information. Transactional information is captured and stored as events happen. Transactional information can be used to analyze daily sales reports and production schedules to determine how much inventory to carry. Analytical information supports managerial analysis tasks like determining whether the organization should build a new manufacturing plant or hire additional sales personnel.

Explanation: Fog computing provides data, compute, storage, and application services to end-users. Fog characteristics include proximity to end-users, dense geographical distribution, and support for mobility. Services are hosted at the network edge or even on end devices such as set-top-boxes or access points.

Explanation: In the Big Data infrastructure, applications, logs, event data, sensor data, mobility data, social media, and stream data could all provide data into the Big Data infrastructure that could involve data centers, NoSQL, traditional database servers, storage, and Hadoop-based technology.

Explanation: Two popular relational database management systems are Oracle and SQL.

Explanation: There are two types of statistics used in the analysis of data: descriptive and inferential. Descriptive statistics are used to describe or summarize values in a data set. Inferential statistics are used to make predictions about data.

Explanation: A population shares a common set of characteristics. Because it is typically not feasible to study an entire population, a representative sample of the population, called a sample, is chosen for analysis.

Explanation: Pandas is an open source library with high-performance data structures and tools for analysis of large data sets.

Explanation: The commonly used correlation coefficient, Pearson r, (or r value), is a quantity that is expressed as a value between -1 and 1. Positive values indicate a positive relationship between the changes in two quantities. Negative values indicate an inverse relationship. The magnitude of either the positive or negative values indicates the degree of correlation. The closer the value is to 1 or -1, the stronger the relationship.

Explanation: In a pandas data set, NaN is used to indicate an undefined string, integer, or float. NaT is used to indicate a missing timestamp.

Explanation: An example of how a regression algorithm might be used is to predict the cost of a house by looking at variables such as crime rate, average income level in the neighborhood, and how far the house is from a school.

Explanation: The dependent variable is known as the target or response variable. The independent variable is also known as the predictor or explanatory variable.

Explanation: The scientific method is commonly used in scientific discovery and contains the following steps:
Step 1. Ask a question about an observation such as what, when, how, or why.
Step 2. Perform research.
Step 3. Form a hypothesis from this research.
Step 4. Test the hypothesis through experimentation.
Step 5. Analyze the data from the experiments to draw a conclusion.
Step 6. Communicate the results of the process.

Explanation: Researchers commonly perform verification tests using four types of validity:
Construct validity – Does the study really measure what it claims to measure?
Internal validity – Was the experiment actually designed correctly? Did it include all the steps of the scientific method?
External validity – Can the conclusions apply to other situations or other people in other places at other times? Are there any other casual relationships in the study that might cause the results?
Conclusion validity – Based on the relationships in the data, are the conclusions of the study reasonable?

Explanation: A scientist must calculate a decision boundary to detect anomalies. Anomalous data points are points that lie beyond the decision boundary sphere.

Explanation: Pyplot is a matplotlib module that includes a collection of style functions. It can be used to create and customize a plot.

Explanation: Plotly is an online tool that can be used to quickly generate data visualizations. Plotly offers a variety of resources for data analysts and web developers including API libraries, figure converters, apps for Google Chrome, and an open source JavaScript library.

Explanation: Large enterprises typically have their own data center to manage data storage and data processing needs. The data center can be used to serve internal IT needs. In other words, the data center becomes a private cloud, a cloud computing infrastructure just for internal services.

Explanation: Data analytics, applied to Big Data, can be classified into three major types:

• Descriptive – provides information about the past state or performance of a person or an organization
• Predictive – attempts to predict the future, based on data and analysis, or what will happen next
• Prescriptive – predicts outcomes and suggests courses of actions that will hold the greatest benefit to an organization

Explanation: The National Institute of Standards and Technology (NIST) defines three main cloud computing services, IaaS, PaaS, SaaS, in their Special Publication 800-145. Cloud service providers have extended this model to also provide IT support for each of the cloud computing services (ITaaS).

Explanation: A hypervisor is a key component of virtualization. A hypervisor is often software-based and is used to create and manage multiple VM instances.

Explanation: Maintaining availability is the primary concern for companies working with big data. Some solutions to improve the availability include the following:
Load Balancing – deploying multiple web servers and DNS servers to respond to requests simultaneously
Distributed Databases – improving database access speed and demands
Memcaching – offloading demand on database servers by keeping frequently requested data available in memory for fast access
Sharding – partitioning a large relational database across multiple servers to improve search speed

Explanation: The three basic components of the big data pipeline are data ingestion, data storage, and data processing or compute.

Explanation: Cassandra uses sequential read and writes to maintain fast speeds. Instead of appending files, when an addition to a file or a removal of data to a file occurs, a new file is created, and the old file or files are deleted.