Big data analysis differs from trditional data analysis primary because

In big data analysis and analytics, a fundamental step-by-step process is needed to organize the task involved

which are 1,2 and 3 stages of bigData analysis lifecycle

which are 4,5 and 6 stages of bigData analysis lifecycle

which are 7,8 and 9 stages of bigData analysis lifecycle

The business case evaluation stage requires that a business case be ________, __________ and ______________ prior to proceeding with the actual hands-on analysis tasks.

An evaluation of a Big Data analysis bussines case helps decision-makers undertand

The KPIs is ussefull in Business Case Evaluation

based on the business requirements documented in the _______________________________ , it can be determined whether the business problems being addresed are really Big data problems

a bussines problem needs to be directly related to one or more of the big data characteristics

Another outcome in Business case evaluation is

The invesment can be weighed against the expected benefits of achieving the goals

Initial iterations of the big data analysis lifecycle will not required more up-front invesment of Big Data tecnologies, products and training compared to later iterations

The data identification stage is dedicated to identifiying the _____________ required for the analysis project

identifying a wider variety of data sources may increase the probability of finding

Can be beneficial to identify as many types of releated data sources and insights as possible, especilly when we don´t know exactly what we're looking for.

Depending on the business scope of analysis project and nature of business problems being adressed, the required dataset and their sourcescan be

Internal dataset

external datasets

data is gathered from all of data sources that were identified during the previous stage, and is then subjected to the automated filtering of corrupt data or data that has been deemed to have no value to the analysis obvjectives

Depending on the type of data source , data may come as a dump of files or may require API integration

in many cases , especially where external, unstructured data is concerned, some or most of the acquired data may be irrelevant (noise) and can be discarded as part of the filtering process

data classified as "corrupt" can include records with missing or nonsensical values or invalid data type

Data thah is filtered out for one analysis may not be valueable for a different type of analysis

it is advisable to store a verbatim copy of the original dataset proceeding with the filtering.

to be persisted once it gets generated or enters the enterprise boundary

The data is persisted to disk prior to analysis

The data is analyzed first and then persisted to disk

Can be added via automation to data from both internal and external data sources to improve the classification an querying

Metadata example can include

it is vital that metadata be machine-readable and passed forward along subsequent analysis stages

This helps to maintain data provenance throughout the Big Data analysis lifecycle, wich helps establish and preserve data accuracy and quality

Some of the data identified as input for the analysis may arrive in a format incompatible with the big data solution

the need to address disparate types of data is more likely with data from

is dedicated to extracting disparate data and transforming it into a format that the underliying Big Data solution can use for the purpose of the data analysis

The extent of extraction and transformation required depends

Estracting the required fields from delimited textual data (such as with web server log files) may not be necessary

example of document that not need further transformation

The invalid data can

data input into Big Data analyses can be unstructured without any indication of validity

the complexity can further make it easy to arrive at a set of suitable validation constraint

Is dedicated to establishing (often complex) validation rules and removing any know invalid data

Big Data solutions often receive redundant data across different datasets, this redundancy can be exploited to explore interconnected datasets in order to assemble validation parameters and fill in missing valid data

For Batch analytics, data validation and cleansing can be achieved via offline

Data input in Big Data can be unstructured without any indication of validity

provenance can play an important role in determining the accuracy and quality of questionable data

data that appears to be invalid may still be valuable in that it may posses

Data may be spread across multiple datasets, requiring that datasets be joined together via common files (date or ID)

either way a method of data ________________ is required or the dataset representing ther correct value needs to be determined

Dedicated to integrating multiple datasets together to arrive at a unified view

Can become complicated because od differences in : although the data format may be the same, the data model may be different

Can become complicated because od differences in : A valuethat is labelled differently in two different datasets may mean the same thing (surname and last name)

In data Aggregation & Representation reconciling the differences can required complex logic that is executed ___________________.

Future data analysis requirements need to be considered during the stage ___________________ to help foster data reusability

whether ___________________ is required or not, it is important to understand that the same data can be stored in many different forms. One form may be better suited for a particular type of analysis than another

A data structured standarized by the Big Data solution can require establishing a central, standard analysis repository, such as a

the data analysis stage is dedicated to carriying out the actual analysis task, which typically involves one or more types of analytics

This stage can be iterative in nature, because repeated until appropiated pattern or correlation is uncovered

The approach taken when carrying out this stage, data analysis, an be classified as ______________________________

___________________________ adata analysis is a deductive approach where the cause of the phenomenon being investigated is proposed beforehand

the proposed cause or assumption is called a

data samples are tipically used

unexpected findings or anomalies are usually ignored since a predetermined cause was assumed

is an inductive approach that is closely associated to data mining

this analysis provides a general direction that can facilitate the discovery of patterns or annomalies

Large amounts of data and visual analysis are typically used

is dedicated to using _____________________ techniques and tools to graphically communicate the analysis results for effective interpretation by bussines users

Bussines users needs to be able to understand the results in order to obtain value from analysis and subsequently have de ability to provide feedback from_______________ back to stage __________________

the same results may be presented ina a number a number of different ways.

another aspect to keep in mind is that providing a method of drilling down to comparatively simple statistics is crucial, in order for users to understand how to statistics were generated

support businessdecission-making, there may be further opportunieties to utilize the analysis results

The utilization os analysis results is dedicated to determining how and where processed analysis data can be further leveraged

"models" that encapsulated new insights and understandings about the nature of the patterns and realationships that exist within data that was just analyzed

A "model" may look like a

the data analysis results may be automatically or manually fed directly into enterprise systems to enhace and optimize their behavior and performance

The identiffied patterns correlations and anomalies discovered during the data analysis are used to refine business process

Data analysis results can be used as input for existing events that requires them to take corrective action

use statistical methods based on mathematical formulas as means for analizing data

it can also be used to infer patterns ans relationships within the dataset, such as regression and correlation

also know as split or bucket testing, compares two versions of an element to determine wich version is superior based on a predefined metric

A/B testing: the current version of the element is called the ______________ version, whereas the modified version is called the ____________

both version, are subjected to an experiment simultaneously. The observationsare recorded to determine wich version is more sccessful

Athough ________________________can be implemented in almost domain, it is most often used in marketing

Generally, the objective is to gauge human behavior with the goal of increasing sales (as per the example below)

is the new version of a drug better than the old one?

is an analysis tecnique used to determine whether two variables are related to each other

an example of a relationship between two variables:
The value of variable A increases whenever the value of variable B increases

Helps to develop an understanding of a dataset and find relationships that can assist in explaining a phenomenon

commonly used for data mining where the identification between variables in a dataset leads to the discovery of patterns ans anomalies

When two variables are considered to be correlated they are considered to be aligned based on a linear relationship

This mean that when one variable changes, the other variable also changes proportionally and constantly

______________________ is expresed a a decimal number between -1 to 1, which is know as the correlation coeficient

Correlation +1

0 Correlation

-1 Correlation

sample: "Do students who perform well at elementary school perform equally well at high school"

explores how a dependent variable is related to an independent variable within a dataset

Helpss determine how the value od dependent variable changes in relation to changes in the value of the independent varible

what the analysts discover is that 15% of additional stock in required for enery 5-degree increase in temperature

more than one independent variable can be tested at the same time

in such cases only one independent variable may change. The others are kept constants

can help enable a better understanding of what a phenomenin is and why it ocurred

represents a constant rate of change

Represents the variable rate of change

what will be the grades of a student studying at a high school based on her primary school grades

_________________does not imply a causation. The change in the value of one variable may not be responsible for the change in the value of the second variable. although both may change at the same rate

assumes that both variables are independent

Deals with already identified dependent and independent variables

_________________ can be applied to further explore the relationship and predict the values of the dependent variable, based on the know values of the independent variable

is a form of data analysis that involves the graphic representation of data to enable or enhace its visual perception

develop a deeper understanding of the data being analyzed. Specifically, it helps identify and highlight hidden patterns, correlations and anomalies.

Are an effective visual analysis technique for expressing patterns, data compositions via part-whole relations and geographic distribution of data

They also facilitate the identification of areas of interest ans the discovery of extreme (high/low) values wihin a dataset

___________ itself is a visual, color-coded representation of data values

A _______________ can be in the form of a chart or a map, as shown in the following pages

A___________ represents a matrix of values in which each cell is color-coded according to the value

A ___________ represents a geographic measure by wich different regions are color-code according to certain theme

How can i visually identify any patterns related to carbon emission across a large number of cities around the world

____________is the analysis of data that is recorded over periodic intervals of time

Helps to uncover patterns within data that are time-dependent. Once identified, the patterns can be axtrapollated for future predictions

time series analyses are usually used for forecasting by identifiying long-term trends. seasonal periodic patterns and irregular short-term variations in the dataset

always includes time as a comparision variable

is generally expressed using a line chart, with time plotted on the x-axis and the recorded data values plotted on the y-axis

how much yield should the farmer expect based on historical yield data

is an interconected collection of entities

An entity can be a person a group or some other business domain object such as a product

some conectios may only be one-way, so that transversal in the reverse direction is nor possible

is a techniquethat focuses on analizing relationships between entities within the network

is used to find the shortest routes between the central warehouse and remote stores in order to minimize the durations of deliveries

How can identify interaction patterns among a very large number of protein-to-protein interactiona?

is focused on analizing location-based data in order to find different geographic relationships and patterns between entities

____________________________ is manipulated through a geographical information system (Gis) that plots spatial data on a map generally using its longitude and latitude coordinates

no two stores can be within a distance of 5 kilometers of each other to prevent the stores from competing with each other.

how far do customers have to commute in order to get to a supermartket?

if the human knowledge can be combined with the processing speed of machines, machines will be able to process large amounts of data without requiring much human intervention

the law _____________________________states that the confidence with wich predictions can be made increases as the size of data that is being analyzed increases

in the context of traditional data analysis, ___________________________ states that, starting with a reasonably large sample size, the value obtained from the analysis of additional data decreases as more data is successively added to the original sample

is a supervised learning technique by witch data is classified into relevant, previously learned categories

Step 1: The system is fed data that is already categorized or labeled, so that it can develop an understanding of different categories

step 2: The system is fed unknow (but similar) data for classification, based on the understanding it developed

A common application of this techniques is for the filtering of e-mail spam. Note that classification can be performed for two or more categories

Based on old data, a training dataset is compiled that contains tagged examples of customers that have or not previously defaulted

Does a fingerprint belong to a suspect based on a record of this previous fingerprints

Is an unsupervised learning technique by wich data is divided into different groups so that the data in each group has similar properties

There is no prior learning of categories required: instead categories are implicity generated based on the data groupings

Is generally used in data minig to get an understanding of properties of a given dataset. Afterdeveloping this understanding, classificatioin can be used to make better predictions about similar, but new or unseen data

In a bank each group is the introduced to one or more financial products most suitable to the characteristics of the overall profile of the group

How many different categories of elements are there in the periodic table

Detection is the process of finding data that is significantly different from or inconsistent with the rest of the data within a given dataset

this machine learning tecnique is used to identify anomalies, abnormalities and deviations that can be opportunities or risks

it can be bsaed on either supervised or unsupervised learning

include fraud detection, medical diagnosis, network data analysis and sensor data analysis

In order ti find if a transaction is likely to be fraudulent or not, the bank´s IT team builds a sustem emplying ____________________ technique that is based on supervised learning

are there any wrongly identified fruits and vegetables in the training dataset used for classification task

is the automated process of finding relevant items from a pool of items

items can be filtered either based on a users own behavior or by matching the behavior of multiple users

items can be filtered either based on a users own behavior or by matching the behavior of multiple users

A common medium by wich ________________is implemented is via the use of a recomender system

technique based on the collaboration of users past behavior

based on the similarityof users behavior, items are filtered for the target user

is solely based on the similarity between users behavior, and requires a large amount of user behavior data in order to accurately

technique focused on the similarity between users an items

A user profile is created based on the users past behavior (likes, ratings, purchase history, etc)

A recomender system predicts user preferences and generate suggestions for the user accordingly

suggestions commonly pertain to recomending items, such as movies, books, web pages, people etc

Based on matches found between financial product purchased by customers and the properties of similar financial products, the recommnder system automates seggestion for potential financial products that customers may also be interested in

Wich holiday destinations can be recommended based on the travel history of a holiday makes?

A fragment of text or speech data can carry different meanings in different contexts, whereas a complete sentence may retain its meaning, even if structured in different ways. In order for the machines to extract valuable information, text and speech data needs to be understood by the machines in the same way as humans do. Semantic analysis represents practices for extracting meaningful information from textual and speech data

Is a computers ability to comprehend human speech and text as naturally understood by humans

This allows computers to perfom a variety of useful task, such as full-text searches

instead of hard-coding the required learning rules, either supervised or unsupervised machine learning is applied to develop the computer undestanding of the natural language

in general the more learning data the computer has, the more correctly it can decipher human text and speech

For speech recognition the system attempts to comprehend the speech and then performs an action, such as transcribing text

How can grammatical mistakes be automaticalle identified?

is the specialized analysis of text through the application of data mining, machine learning and natural language processing techniques to extract value out of unstructured text. Text analytics essentially provides the ability to discover text rather than just search it

the basic tenet of text analytics is to turn unstructured text into data that can be searched and analyzed