Copiar y Editar

Module 11: Advanced Big Data Architecture

Descripción

Module 11: Fundamental Big Data Architecture
Alveiro Garcia
Test por Alveiro Garcia, actualizado hace más de 1 año
Alveiro Garcia
Creado por Alveiro Garcia hace alrededor de 7 años
115
0
0

Resumen del Recurso

Pregunta 1

Pregunta
Operational Data Store (ODS)
Respuesta
  • As an EDW contains large amounts of data, it is of particular interest when designing an architecture for a Big Data platform. It not only serves as a data source but also as the default interface through which various BI and analysis activities are carried out.
  • Although a single EDW can house multiple ODSs, because their primary role is to facilitate near-realtime reporting, their use is optional.
  • On the other hand, Big Data is mostly comprised of unstructured data that has no defined structure. Unless analyzed, the data may not have any value. Big Data analysis requires data to be stored in its raw form without being modeled first. Once collected, the exploratory phase separates signal (valuable data) from noise.
  • EDWs contain high value data that has gone through rigorous validation and quality control checks

Pregunta 2

Pregunta
Enterprise Data Warehouse & Big Data
Respuesta
  • staging area
  • operational data store (ODS)
  • data mart
  • analytical database

Pregunta 3

Pregunta
Staging Area
Respuesta
  • Although a single EDW can house multiple ODSs, because their primary role is to facilitate near-realtime reporting, their use is optional.
  • It may not be possible to extract data from all systems at the same time because of various technical or business-related issues. Due to this, a storage buffer where data extracted from different systems at varying times with differing frequencies can be stored is required
  • It is generally an insert/read-only database utilizing either shared-nothing MPP architecture or shared-everything architecture. Data is fed from the data warehouse into the analytical database on regular intervals
  • It usually includes an ETL process that ferries data from source systems into a temporary storage area. This process also contains data cleansing, validation and model transformation operations

Pregunta 4

Pregunta
Data Warehouse
Respuesta
  • generally contains recent data. However, the degree of “data freshness” depends upon the reporting requirements. As a result, the range of data stored may span from hours to months
  • a relational database that acts as the single version of truth for the enterprise by storing standardized data from across the enterprise in a denormalized form that is fit for reporting and data analysis
  • stores data related to various business entities, such as products or customers. Unlike an OLTP system, data is either inserted or retrieved but not updated in a data warehouse
  • the queries are generally more complex, involving multiple tables spanning a longer range of data.

Pregunta 5

Pregunta
Data Mart
Respuesta
  • Although the historical data can go back up to several years, the freshness of the current data depends on an enterprise’s reporting and analysis requirements
  • Some basic level of data model transformation and denormalization may also be performed in support of efficient reporting
  • provides a particular view on the data held in the data warehouse. Although makes data analysis and reporting easier and faster because the stored data is highly customized according to the specific requirements, it does result in data redundancy.
  • contains large amounts of data, it is of particular interest when designing an architecture for a Big Data platform

Pregunta 6

Pregunta
Analytical Database
Respuesta
  • It is generally an insert/read-only database utilizing either shared-nothing MPP architecture or shared-everything architecture
  • Data is highly standardized because it has gone through data cleansing, validation, quality and de-duplication processes, further suggesting that the data is of high value
  • Some basic level of data model transformation and denormalization may also be performed in support of efficient reporting
  • These are generally expensive and may come bundled with the required hardware and software in the form of an appliance

Pregunta 7

Pregunta
EDW & Big Data Comparison
Respuesta
  • Contain high value data that has gone through rigorous validation and quality control checks
  • On the other hand, Big Data datasets must be stored in their raw unstructured forms, and their values are unknown
  • Big Data requires a repository that acts as a sink for a variety of data sources where data is stored as is
  • Stores data related to various business entities, such as products or customers

Pregunta 8

Pregunta
EDW & Big Data Integration
Respuesta
  • Big Data requires a distributed and highly scalable storage and processing architecture with scale-out support
  • Most implementations of the Big Data appliance enable realtime and near-realtime analytics without the need for integrating multiple disparate technologies
  • A batch processing engine, such as MapReduce, can be used to convert semi- and unstructured data into meaningful structured data
  • The next-generation data warehouse consists of heterogeneous technologies providing support for structured as well as semi- and unstructured data storage and analysis

Pregunta 9

Pregunta
Series Approach
Respuesta
  • The introduction of the Big Data platform in this configuration is comparatively less disruptive because the Big Data platform is essentially an add-on module for processing semi- and unstructured data
  • Provides a highly scalable data storage and processing environment
  • BI tools and other analytical applications are unable to make use of the Big Data platform directly
  • The implementation and maintenance of the interconnect can become complex if it incorporates complicated data processing, such as translation between different data types

Pregunta 10

Pregunta
Big Data Appliance Approach
Respuesta
  • relational and non-relational storage
  • configuration, management and application development environments
  • an interconnect (between data storage and processing resources)
  • is analogous to the parallel approach and is also known as the logical data warehouse

Pregunta 11

Pregunta
Data Virtualization Approach
Respuesta
  • It requires complex initial configuration, which usually results in consultation costs
  • It is generally implemented as Data-as-a-Service (DaaS) by applying service-orientation principles.
  • This approach makes non-relational data (Big Data datasets) more accessible through the use of standardized interfaces
  • Is generally implemented through complex software that can be expensive to acquire

Pregunta 12

Pregunta
To reduce storage cost and speed up operational reporting, an online transaction processing system (OLTP) can be replaced with an operational data store (ODS).
Respuesta
  • True
  • False

Pregunta 13

Pregunta
In a data warehouse, data is kept in a fully normalized form for easier reporting
Respuesta
  • True
  • False

Pregunta 14

Pregunta
When compared with an ODS, a data warehouse’s queries are generally more complex, involving multiple tables spanning over a longer range of data. However, data import is less frequent because a data warehouse is not used for operational reporting
Respuesta
  • True
  • False

Pregunta 15

Pregunta
An analytical database can either be based on a columnar database or in-memory solutions for fast data access
Respuesta
  • True
  • False

Pregunta 16

Pregunta
To obtain the benefits linked with the adoption of Big Data, an EDW needs to be replaced with Big Data-specific technologies since the EDW cannot store unstructured data
Respuesta
  • True
  • False

Pregunta 17

Pregunta
The next-generation data warehouse consists of Big Data storage technologies that can store large amounts of structured as well as unstructured data
Respuesta
  • True
  • False

Pregunta 18

Pregunta
In a Big Data environment, the query workloads are generally unknown because of the adhoc nature of analytical queries
Respuesta
  • True
  • False

Pregunta 19

Pregunta
In the series approach of EDW and Big Data integration, semi-structured and unstructured data is ingested by the Big Data platform, and only structured data is ingested by the EDW
Respuesta
  • True
  • False

Pregunta 20

Pregunta
One disadvantage of the series approach is that the Big Data platform cannot be directly accessed for performing analysis on large amounts of raw data
Respuesta
  • True
  • False

Pregunta 21

Pregunta
In the parallel approach of EDW and Big Data integration, the interconnect is a one-way connector between the EDW and the Big Data platform
Respuesta
  • True
  • False

Pregunta 22

Pregunta
One of the disadvantages of the Big Data appliance is that it does not provide horizontal scalability since it is a boxed solution
Respuesta
  • True
  • False

Pregunta 23

Pregunta
The Big Data appliance approach makes on-going system maintenance easier because this approach combines the EDW and the Big Data platform into a single preconfigured system
Respuesta
  • True
  • False

Pregunta 24

Pregunta
The data virtualization approach is also known as the logical data warehouse
Respuesta
  • True
  • False

Pregunta 25

Pregunta
The data virtualization approach uses an interconnect to provide a unified view of data across multiple data sources.
Respuesta
  • True
  • False

Pregunta 26

Pregunta
One of the disadvantages of the virtualization approach is that data from all data sources still needs to be copied over into a central repository in order to create the required services
Respuesta
  • True
  • False

Pregunta 27

Pregunta
Big Data & Cloud Computing
Respuesta
  • Can be utilized as a technology-enabler for Big Data under such circumstances
  • Ingested data is stored in a distributed file
  • A single dataset may be of interest to multiple clients developed using different technologies that require data to be available in a specific format
  • Specialized form of distributed computing that introduces utilization models for remotely provisioning scalable and measured IT resources

Pregunta 28

Pregunta
Big Data and Cloud Computing
Respuesta
  • Processing and storage technologies that use cluster-based processing and storage resources
  • The on-demand and elastic nature provides the ability for a much quicker setup of a Big Data platform
  • Has the potential to provide the basic components for a Big Data solution environment, including data, storage and processing resources
  • Whether processing data in batch or realtime mode, the pay-per-use model can be fully utilized to build a cluster whose size can be regulated based on the volume and velocity characteristics of Big Data

Pregunta 29

Pregunta
Cloud Delivery Models
Respuesta
  • Infrastructure-as-a-Service (IaaS)
  • Platform-as-a-Service (PaaS)
  • Software-as-a-Service (SaaS)
  • Component-as-a-Service (CaaS)

Pregunta 30

Pregunta
Cloud Deployment Model
Respuesta
  • Heterogeneous Cloud
  • Private Cloud
  • Managed Cloud
  • Hybrid Cloud

Pregunta 31

Pregunta
Public Cloud
Respuesta
  • Is ideal for enterprises that initially built up Big Data analytics in-house but now want to scale out.
  • Can be used when input datasets are already stored in the cloud
  • Is generally less secure but more scalable due to larger pooling of storage and processing resources
  • It is also ideal when datasets reside within an enterprise’s firewall.

Pregunta 32

Pregunta
Private Cloud
Respuesta
  • It is also ideal when workloads vary
  • Is generally less secure but more scalable due to larger pooling of storage and processing resources
  • It is also ideal when datasets reside within an enterprise’s firewall
  • Can help develop low latency data analysis capabilities

Pregunta 33

Pregunta
Hybrid Cloud
Respuesta
  • It is also ideal when workloads vary
  • Can be used when input datasets are already stored in the cloud
  • Is a suitable choice when starting a Big Data project
  • is a suitable choice when using a combination of sensitive data and public datasets

Pregunta 34

Pregunta
Big Data and Cloud Computing Issues
Respuesta
  • data privacy
  • regulatory compliance
  • network connectivity
  • data virtualization

Pregunta 35

Pregunta
Cloud-Related Big Data Patterns
Respuesta
  • Cloud-based Big Data Analysis
  • Cloud-based Big Data Visualization
  • Cloud-based Big Data Storage
  • Cloud-based Big Data Processing

Pregunta 36

Pregunta
Cloud-based Big Data Storage
Respuesta
  • This pattern can also be employed when the data sources, such as the CRM system, reside in the same cloud (faster data transfer) or a proof-of-concept is being developed
  • This ability to store raw data spanning over longer periods of time increases the overall potential of finding valuable insights
  • Represents a solution environment comprised of inexpensive NoSQL storage
  • Is associated with the storage device (distributed file system/NoSQL) and data transfer engine mechanisms

Pregunta 37

Pregunta
Data Transformation Compound Pattern
Respuesta
  • This generally involves the use of NoSQL databases such that the downstream applications can communicate directly with these databases using RESTful APIs
  • The underlying idea is to be able to ingest large amounts of raw data and pre-process it in order to make it suitable for traditional enterprise systems
  • Keeping multiple copies of the same dataset in different formats is not only inefficient but also adds operational and storage overheads
  • The involved operations can include data cleansing, validation, model transformation and format transformation, as well as the joining of disparate datasets

Pregunta 38

Pregunta
Data Transformation Compound Pattern
Respuesta
  • Poly Source
  • Large-Scale Batch Processing
  • High Volume Tabular Storage
  • Large-Scale Graph Processing

Pregunta 39

Pregunta
Application Enhancement Compound Pattern
Respuesta
  • Ingesting large amounts of data in order to calculate certain statistics or execute a machine learning and then to feed results to enterprise systems
  • This generally involves the use of NoSQL databases such that the downstream applications can communicate directly with these databases using RESTful APIs
  • The underlying idea is to be able to ingest large amounts of raw data and pre-process it in order to make it suitable for traditional enterprise systems
  • A dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure

Pregunta 40

Pregunta
Application Enhancement Compound Pattern
Respuesta
  • High Volume Tabular Storage
  • Large-Scale Graph Processing
  • Canonical Data Format
  • Data Size Reduction

Pregunta 41

Pregunta
Canonical Data FormatPattern
Respuesta
  • Warrants the use of a memory-based storage device with random read and write capability.
  • Keeping multiple copies of the same dataset in different formats is not only inefficient but also adds operational and storage overheads
  • A separate connector is used to connect to a particular query engine or the storage device
  • The ingested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database

Pregunta 42

Pregunta
Realtime Access Storage Pattern
Respuesta
  • Ingested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database
  • Exporting the data in the form of a file, importing it into a database and then connecting the analytics tool to the database is not a viable option
  • Is associated with the serialization engine, data transfer engine, storage device and processing engine mechanisms
  • The use of disk-based storage devices can severely impact the processing time of data

Pregunta 43

Pregunta
Direct Data Access Pattern
Respuesta
  • Greatly helps in speeding up data analysis and reduces dependence on IT personnel for data analysis tasks
  • Incurs increased cost because memory-based storage devices are expensive when compared with disk-based storage devices
  • Keeping multiple copies of the same dataset in different formats is not only inefficient but also adds operational and storage overheads
  • Is generally employed by enterprises that have just embarked on a Big Data journey

Pregunta 44

Pregunta
Analytical Sandbox Compound Pattern
Respuesta
  • The results are fed directly to various downstream applications, such as an e-commerce application
  • Is generally employed by enterprises that have just embarked on a Big Data journey
  • Represents a standalone solution environment
  • Offloads existing databases from having to perform complex and long-running data transformation jobs on large datasets

Pregunta 45

Pregunta
Analytical Sandbox Compound Pattern
Respuesta
  • Poly Storage
  • Poly Source
  • Poly Sink
  • Confidential Data Storage

Pregunta 46

Pregunta
Confidential Data Storage Pattern
Respuesta
  • In the case of a clustering algorithm applied to a customer dataset for finding customer cohorts
  • Is generally opted for by enterprises that want to move towards predictive and prescriptive analytics by creating richer statistical and machine learning models
  • Can be applied in such a case to ensure that even if malicious users get access to sensitive data, they are unable to read and make use of it
  • This approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues

Pregunta 47

Pregunta
Large-Scale Graph ProcessingPattern
Respuesta
  • A dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure
  • It involves traversing through a large number of nodes (entities) via their defined edges (links).
  • This approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
  • Storing and analyzing very large amounts of structured, unstructured and semi-structured Big Data datasets

Pregunta 48

Pregunta
Unstructured Data Store Compound Pattern
Respuesta
  • The analytical operations performed in support of BI, data mining and creating statistical and machine learning models do not affect the performance
  • This configuration is generally opted for by enterprises that want to move towards predictive and prescriptive analytics by creating richer statistical and machine learning models
  • Capable of ingesting and storing large amounts of semi-structured and unstructured data to develop highfidelity statistical and machine learning models for performing predictive and prescriptive analytics
  • Although analogous to the use of a cloud, this approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues

Pregunta 49

Pregunta
Unstructured Data Store Compound Pattern
Respuesta
  • Random Access Storage
  • Automated Dataset Execution
  • File-based Sink
  • Big Data Processing Environment.

Pregunta 50

Pregunta
Batch Data Processing Compound Pattern
Respuesta
  • The ingested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database for performing analytical queries
  • Their current storage infrastructure does not allow them to store semi-structured and unstructured data
  • A solution environment where the sole purpose of using the Big Data platform is to offload processing of large amounts of structured data
  • This approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues

Pregunta 51

Pregunta
Batch Data Processing Compound Pattern
Respuesta
  • Canonical Data Format
  • Relational Sink
  • Automatic Data Replication and Reconstruction
  • Automatic Data Sharding

Pregunta 52

Pregunta
Dataset DenormalizationPattern
Respuesta
  • Requires exporting data via a relational data transfer engine to the data warehouse
  • Can be applied in such a case to ensure that even if malicious users get access to sensitive data
  • Is a solution environment comprised of inexpensive storage used to store large amounts of data from both internal and external data sources in an online fashion ready for consumption by any enterprise system
  • Enable the processing of datasets, which requires the use of a batch processing engine

Pregunta 53

Pregunta
Online Data Repository Compound Pattern
Respuesta
  • Storing and analyzing very large amounts of structured, unstructured and semi-structured Big Data datasets
  • A solution environment comprised of inexpensive storage used to store large amounts of data from both internal and external data sources in an online fashion ready for consumption by any enterprise system
  • Large data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
  • The sole purpose of using the Big Data platform is to offload processing of large amounts of structured data

Pregunta 54

Pregunta
Online Data Repository Compound Pattern
Respuesta
  • Automated Dataset Execution
  • Streaming Access Storage
  • Random Access Storage
  • Canonical Data Format

Pregunta 55

Pregunta
Big Data Warehouse Compound Pattern
Respuesta
  • This configuration is generally opted for by enterprises that want to move towards predictive and prescriptive analytics by creating richer statistical and machine learning models
  • Large data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
  • Storing and analyzing very large amounts of structured, unstructured and semi-structured Big Data datasets
  • Data from structured sources and from unstructured sources can first be stored on a distributed file system

Pregunta 56

Pregunta
Big Data Warehouse Compound Pattern
Respuesta
  • Automatic Data Sharding
  • Canonical Data Format
  • Random Access Storage
  • Confidential Data Storage

Pregunta 57

Pregunta
Operational Data Store Compound Pattern
Respuesta
  • a solution environment comprised of inexpensive NoSQL storage that is utilized as ___________ where large amounts of transactional data from operational systems across the enterprise are collected for operational BI and reporting
  • Data from structured sources and from unstructured sources can first be stored on a distributed file system
  • Large data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
  • Larger amounts of data that spreads over longer time periods can be stored, thereby providing the opportunity to enrich operational BI

Pregunta 58

Pregunta
Operational Data Store Compound Pattern
Respuesta
  • High Volume Tabular Storage
  • Relational Sink
  • Indirect Data Access
  • Automated Dataset Execution

Pregunta 59

Pregunta
Indirect Data Access Pattern
Respuesta
  • The data can be imported into fit-forpurpose NoSQL databases, where it can be easily accessed in support of BI, reporting and other analytical use cases
  • Enable access to pre-processed data or analysis results stored in a Big Data solution environment via existing BI tools
  • A solution environment comprised of inexpensive NoSQL storage
  • Enable the processing of such datasets, which requires the use of a batch processing engine

Pregunta 60

Pregunta
Realtime Data Processing Compound Pattern
Respuesta
  • The data can be imported into fit-forpurpose NoSQL databases, where it can be easily accessed in support of BI, reporting and other analytical use cases
  • A solution environment capable of processing streams of data in realtime or near-realtime, such as performing analytics on machine-generated or social media data
  • The streaming data can be stored in disk-based storage, such as the distributed file system, for further analysis
  • Enable the processing of such datasets, which requires the use of a batch processing engine

Pregunta 61

Pregunta
Realtime Data Processing Compound Pattern
Respuesta
  • Large-Scale Batch Processing
  • Streaming Source
  • Automatic Data Replication and Reconstruction
  • Data Size Reduction

Pregunta 62

Pregunta
High Velocity Realtime ProcessingPattern
Respuesta
  • Enable the immediate export of results
  • Scenarios where the data needs processing as it arrives to obtain immediate results
  • A solution environment capable of processing streams of data in realtime or near-realtime
  • Enable access to pre-processed data or analysis results stored in a Big Data solution environment via existing BI tools

Pregunta 63

Pregunta
Streaming Egress Pattern
Respuesta
  • Storing high-volume and high-variety data in order to perform various analytics in isolation from other enterprise systems
  • Data needs processing as it arrives to obtain immediate results
  • Provide integration with the enterprise identity and access management systems (IAMs)
  • Enable the immediate export of results

Pregunta 64

Pregunta
Additional Big Data Patterns
Respuesta
  • Centralized Dataset Governance
  • Fan-in Ingress
  • Centralized Dataset Management
  • Streaming Egress Pattern

Pregunta 65

Pregunta
Centralized Access ManagementPattern
Respuesta
  • Provides a means for performing a range of data governance tasks from a central location
  • Provide integration with the enterprise identity and access management systems (IAMs)
  • Maintain data lineage and details about operations performed on the data across multiple processing stages
  • Enable policy-based access to resources within the Big Data platform via a central interface

Pregunta 66

Pregunta
Integrated Access Pattern
Respuesta
  • Provides a means for performing a range of data governance tasks from a central location
  • Enable policy-based access to resources within the Big Data platform via a central interface
  • Can be used to provide integration with the enterprise identity and access management systems (IAMs)
  • Is associated with the processing engine, storage device, query engine and productivity portal mechanisms

Pregunta 67

Pregunta
Centralized Dataset Governance Pattern
Respuesta
  • A security engine is used to enable single sign-on (SSO) functionality that generally works on the basis of trusting the IAM system for user authentication via the use of tokens
  • Provides a means for performing a range of data governance tasks from a central location
  • In order to have maximum confidence in the processing results, there needs to be a way to retrace the processing steps that were taken
  • Data merging may be required due to reasons such as the data is too fine-grained or arrives out of order, due to network latency or due to factors that are beyond the control of the enterprise

Pregunta 68

Pregunta
Automated Processing Metadata Insertion Pattern
Respuesta
  • Data merging may be required due to reasons such as the data is too fine-grained or arrives out of order, due to network latency or due to factors that are beyond the control of the enterprise
  • Can be applied to maintain data lineage and details about operations performed on the data across multiple processing stages
  • Intermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated
  • If the final results are incorrect, the entire series of steps need to be executed from scratch even if the results halfway were correct

Pregunta 69

Pregunta
Intermediate Results Storage Pattern
Respuesta
  • Intermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated
  • Can be applied to maintain data lineage and details about operations performed on the data across multiple processing stages
  • In order to have maximum confidence in the processing results, there needs to be a way to retrace the processing steps that were taken
  • Data needs to be simultaneously processed using different sub-systems

Pregunta 70

Pregunta
Fan-in IngressPattern
Respuesta
  • The application of this design pattern requires the automated addition of metadata, based on a machine-readable standardized structure, during each stage of data processing
  • Provides scalability in the context of being able to add more data consumers via a simple configuration
  • Is applied when data needs to be simultaneously processed using different sub-system
  • Can be applied to implement logic that merges data originating from multiple sources and generally applies to situations where data is acquired in realtime

Pregunta 71

Pregunta
Fan-out Ingress Pattern
Respuesta
  • Intermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated
  • Is applied when data needs to be simultaneously processed using different sub-systems
  • Maintain data lineage and details about operations performed on the data across multiple processing stages
  • Data is copied from the source location, stored in the queue and then forwarded to the interested subscribers

Pregunta 72

Pregunta
John wants to perform predictive analytics using a variety of textual log files. However, the current data storage infrastructure consists of relational database technologies. John accomplishes his goal by storing and pre-processing the log files without affecting current storage. Which compound pattern did John apply?
Respuesta
  • Online Data Repository
  • Unstructured Data Store
  • Big Data Warehouse
  • Operational Data Store

Pregunta 73

Pregunta
Each day ABC’s head office receives a large number of reports from each of its branches across the world. Performance data is extracted from these reports and then imported into the enterprise data warehouse, from where it is used for various reporting tasks. The reports are in XML format and are currently coerced into a relational database and then a utility is run to perform data cleansing and extraction of the required data. The entire process of ingesting and loading into the data warehouse takes a long time, and with the reports getting more detailed, it is anticipated that timely processing of reports may not be possible. Which compound pattern can be applied to address the processing of the XML reports without requiring a staging database?
Respuesta
  • Analytical Sandbox
  • Unstructured Data Store
  • Data Transformation
  • Big Data Warehouse

Pregunta 74

Pregunta
XYZ is enhancing its analytical capabilities by capturing large amounts of structured and unstructured data across the enterprise and enabling its data scientists to perform advanced analytics. However, the Big Data architects have been advised that doing so should not impact the current operations of the enterprise data warehouse and that any required technology infrastructure should be kept separate with respect to the current IT environment. Which compound pattern should the Big Data architects apply for setting up the required Big Data platform?
Respuesta
  • Batch Data Processing
  • Operational Data Store
  • Analytical Sandbox
  • Online Data Repository

Pregunta 75

Pregunta
A large online bookstore currently recommends a random array of books on its website to its potential customers. However, it is planning to display personalized recommendations to its customers based on a profile match and the kinds of books they have bought in the past. This process involves ingesting a large amount of customer profile data from the CRM system, joining it with customer’s shopping history and then applying a machine learning algorithm. The generated results are then embedded on the webpage that the customer is browsing. Which compound pattern can be applied to implement the required solution?
Respuesta
  • Big Data Warehouse
  • Online Data Repository
  • Application Enhancement
  • Realtime Data Processing

Pregunta 76

Pregunta
A large cellular company is improving its monthly billing process by introducing itemized billing. However, with more than 5 million customers, it takes a long time to complete the simple process. The company anticipates that the new feature will take twice the current time. Davon, a Big Data architect, proposes a Big Data technologies-based solution that accomplishes the new itemized billing process quickly. Which compound pattern will Davon apply to complete the task?
Respuesta
  • Application Enhancement
  • Online Data Repository
  • Batch Data Processing
  • Realtime Data Processing

Pregunta 77

Pregunta
A renowned car manufacturer, XYZ, has modernized its manufacturing facility by adding a number of sensors across the assembly line. Each sensor provides a reading every 5 seconds. XYZ needs to monitor the readings transmitted by each of the sensors as soon as they are transmitted. The monitoring process involves a comparison of related groups of sensor readings to make sure that the readings fall within a predetermined range. Which compound pattern can be applied to achieve the desired result?
Respuesta
  • Realtime Data Processing
  • Batch Data Processing
  • Data Transformation
  • Analytical Sandbox

Pregunta 78

Pregunta
The data scientists at ABC often require access to historical data, going back as far ten years, in its raw form for various data analyses. Jackie, the Big Data architect, needs to provide the required data in such a way that the data can be retrieved without any delays. In which configuration should Jackie deploy the Big Data platform?
Respuesta
  • Data Transformation
  • Application Enhancement
  • Big Data Warehouse
  • Online Data Repository

Pregunta 79

Pregunta
The business intelligence team at a large retail store has been asked to integrate weekly sales figures into a dashboard that currently displays daily sales figures. The team notices that the current operational data store used for generating the daily sales figures is already operating at its maximum storage capacity. In which configuration can the team implement a solution when using a Big Data platform?
Respuesta
  • Big Data Warehouse
  • Batch Data Processing
  • Operational Data Store
  • Analytical Sandbox

Pregunta 80

Pregunta
A small toy manufacturer, ABC, has seen a steady growth in the past 5 years. ABC’s current IT landscape consists of an ERP and a CRM system. Both the systems are Open Sourcebased, as ABC can only spare a limited amount of budget for IT. Sales are monitored by generating month-end reports by executing queries again with the ERP and the CRM. However, these reports only go back as far as 6 months, as older data is archived to a tape drive. Which compound pattern can be applied that enables ABC to keep a large amount of transactional data online, from which detailed sales reports can be generated more frequently?
Respuesta
  • Online Data Repository
  • Operational Data Store
  • Big Data Warehouse
  • Unstructured Data Store

Pregunta 81

Pregunta
Lambda Architecture
Respuesta
  • The sole purpose of using this kind of platform is to offload processing of large amounts of structured data
  • Type of Big Data solution architecture that is comprised of multiple layers and forms the basis for developing highly scalable, available, eventually consistent, fault tolerant and low latency realtime Big Data solutions
  • Uses a combination of both realtime and batch components that operate in parallel to process data without any delay
  • Additional processing is generally required to put the data in the correct structure

Pregunta 82

Pregunta
Lambda Architecture Terminology
Respuesta
  • View
  • Model
  • Indexed View
  • Indexing

Pregunta 83

Pregunta
[blank_start]Normalization[blank_end] is the process of storing data in a form that removes data duplication and ensures consistency
Respuesta
  • Normalization
  • Denormalization
  • Polyglot Persistence
  • CAP

Pregunta 84

Pregunta
[blank_start]Denormalization[blank_end] is the process of storing data in a form that introduces redundancy for faster querying
Respuesta
  • Polyglot Persistence
  • CAP
  • SCV
  • Denormalization

Pregunta 85

Pregunta
[blank_start]Polyglot persistence[blank_end] is the practice of using more than one fit-for-purpose storage device for persisting data
Respuesta
  • CAP
  • Polyglot persistence
  • SCV
  • Recomputation Algorithm

Pregunta 86

Pregunta
[blank_start]CAP[blank_end] is a theorem that states a distributed storage system is only able to support two of the following constraints at any point in time: consistency, availability and partition-tolerance
Respuesta
  • Recomputation Algorithm
  • SCV
  • CAP
  • Incremental/Approximate Algorithm

Pregunta 87

Pregunta
[blank_start]SCV[blank_end] is a principle that states that a processing system is only capable of supporting two of the following: speed, consistency and volume at any point in time
Respuesta
  • SCV
  • Recomputation Algorithm
  • Incremental/Approximate Algorithm
  • Sharding

Pregunta 88

Pregunta
The [blank_start]recomputation algorithm[blank_end] is an algorithm that processes the complete dataset to generate the result
Respuesta
  • Incremental/Approximate Algorithm
  • Sharding
  • Replication
  • recomputation algorithm

Pregunta 89

Pregunta
The [blank_start]incremental algorithm[blank_end] is an algorithm that only processes new data. It may use probability-based techniques and may generate results that are not fully reliable/accurate
Respuesta
  • Replication
  • Sharding
  • Recomputation Algorithm
  • incremental algorithm

Pregunta 90

Pregunta
[blank_start]Sharding[blank_end] is a method of achieving scalability by horizontally partitioning a large dataset across multiple nodes
Respuesta
  • Replication
  • Denormalization
  • Sharding
  • Normalization

Pregunta 91

Pregunta
[blank_start]Replication[blank_end] is a method of achieving fault-tolerance by storing multiple copies of a dataset across multiple nodes
Respuesta
  • Sharding
  • SCV
  • Replication
  • Denormalization

Pregunta 92

Pregunta
Purpose of the Lambda Architecture
Respuesta
  • This not only helps process voluminous data faster but also helps cater to infrequent or ad-hoc data processing requests that require above-average storage and processing resources
  • Data architectures are becoming difficult to design and maintain due to the ever-increasing volume, velocity and variety of data.
  • Efficient data storage and efficient querying have incompatible requirements that require following different strategies
  • Data is either stored in a disk-based NoSQL or a memory-based storage device, which can be a NoSQL or some other cluster-based storage technology, that enables low latency data access to perform realtime or near-realtime analytics

Pregunta 93

Pregunta
Lambda Architecture Characteristics
Respuesta
  • Processes raw data by employing both realtime and batch data processing techniques in parallel
  • Maintain data lineage and details about operations performed on the data across multiple processing stages
  • The results generated by realtime processing are based on incremental algorithms that may not be consistent/accurate
  • Batch data processing eliminates the complexity of maintaining data consistency across nodes by storing only immutable data

Pregunta 94

Pregunta
Lambda Architecture Layers
Respuesta
  • Batch
  • Serving
  • Speed
  • Query

Pregunta 95

Pregunta
Batch Layer
Respuesta
  • Processing of raw data
  • Storage of raw data
  • Ad hoc reporting
  • Calculation of views

Pregunta 96

Pregunta
Lambda Architecture Batch Layer
Respuesta
  • Uses incremental algorithms and processes comparatively smaller amounts of data to provide low latency results
  • Consists of a storage device (distributed file system), batch processing engine and a workflow engine
  • Uses a recomputation algorithm to provide consistent accurate views and further provides fault-tolerance when compared with an incremental algorithm
  • Comprises an enhanced version of the query engine with logic that can automatically and intelligently combine serving and speed views based on the query criteria

Pregunta 97

Pregunta
Lambda Architecture Serving Layer
Respuesta
  • Although raw data is stored, for achieving consistency, some structure needs to be applied to the data before storage
  • The storage device used in this layer only needs to support batch write (no random write) with random read capabilities
  • As the layer follows the mutable storage model and the processing results are generated more frequently, the storage device that stores the views needs to support random writes with random reads
  • For keeping the complexity to a minimum and providing faster reads, normally a simple key-value NoSQL database is used

Pregunta 98

Pregunta
Lambda Architecture Speed Layer
Respuesta
  • The use of an append-only and streaming data storage device keeps complexity to a minimum
  • The views created by the batch layer are not amenable to random querying, as these are generally stored in the distributed file system
  • A memory-based storage device for the storage of raw data and a memory or disk-based NoSQL storage device for the storage of views is generally used
  • Event data is captured using the event data transfer engine and is processed in memory via the realtime processing engine to create indexed views that are generally stored inside a NoSQL database

Pregunta 99

Pregunta
Lambda Architecture Query Layer
Respuesta
  • For easier integration, the speed and serving views should be constructed in a modular manner
  • Merging the results from views residing in the speed and serving layers for successfully executing a query
  • Once the latest batch view is available via the serving layer, the corresponding results in the realtime views can be ignored or flushed
  • Is a high latency layer such that there is a time lag before the latest version of the views, based on fresher data, is available

Pregunta 100

Pregunta
Lambda Architecture Layers in Action
Respuesta
  • Raw data is fed simultaneously to the batch and speed layers, generally using the same event data transfer engine
  • The batch layer can be further used for deep analytics, as it contains complete datasets
  • The limitations of the SCV principle are also relaxed
  • Although the speed layer is responsible for processing the entire set of fresh data while the corresponding batch view is not ready, it does not process the entire set as a single job because doing so adds to the latency and results in excessive resource usage

Pregunta 101

Pregunta
Lambda Architecture Benefits
Respuesta
  • Algorithms for the speed layer can be complex or might need some time to understand, as they use incremental or approximation (probability)-based techniques that the batch equivalent may not be using
  • The complexity of the architecture is restricted to the speed layer, as that is where the incremental algorithms and read/write database are used
  • The immutable nature of the batch layer helps re-process data as a result of a data processing logic change that may occur due to new business requirements or a bug fix
  • Realtime data processing capability is required with consistent results

Pregunta 102

Pregunta
Lambda Architecture Applicability
Respuesta
  • Realtime data processing capability is required with consistent results
  • Fault-tolerance and accuracy need to be added to the existing realtime system
  • Loss of data is not acceptable
  • Polyglot persistence by employing fit-for-purpose storage devices at each layer

Pregunta 103

Pregunta
Lambda Architecture Limitations/Challenges
Respuesta
  • Configuring the batch layer to process data in small batches reduces load on the speed layer
  • Raw data is fed simultaneously to the batch and speed layers, generally using the same event data transfer engine, and each layer can be implemented via a different set of technologies
  • Complexity is greatly increased, as two separate layers need building and maintaining while ensuring that each provides the same functionality
  • Requires schema adherence in the batch layer, which adds complexity, adds another step before data can actually be persisted and requires prior knowledge about the structure of the incoming data

Pregunta 104

Pregunta
Lambda Architecture Recommendations
Respuesta
  • Employing the same processing engine for both the speed and batch layer, such as Spark, helps keep system complexity to a minimum
  • The key-value storage model employed in the serving layer may not be sufficient for all types of query requirements
  • The immutable nature of the batch layer helps re-process data as a result of a data processing logic change that may occur due to new business requirements or a bug fix
  • A balance is required based on the processing requirements, as the throughput obtained from employing small batches may be less than from larger batches and will further require frequent updates to the serving layer

Pregunta 105

Pregunta
In Lambda architecture, which layer(s) is/are responsible for creating indexed views?
Respuesta
  • batch layer
  • serving layer
  • speed layer
  • query layer

Pregunta 106

Pregunta
In Lambda CAP is a theorem that applies to
Respuesta
  • Cloud computing
  • Distributed storage system
  • Processing system
  • EDW

Pregunta 107

Pregunta
In Lambda SCV is a theorem that applies to
Respuesta
  • Storage devices
  • Distributed Storage System
  • Cloud computing
  • Processing System

Pregunta 108

Pregunta
Data Transformation Compound Pattern
Respuesta
  • A dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure
  • The underlying idea is to be able to ingest large amounts of raw data and pre-process it in order to make it suitable for traditional enterprise systems
  • Is ideal for enriching the EDW with unstructured data
  • This generally involves the use of NoSQL databases such that the downstream applications can communicate directly with these databases using RESTful APIs

Pregunta 109

Pregunta
Application Enhancement Compound Pattern
Respuesta
  • A dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure
  • Certain statistics are calculated by processing large amounts of data, or a statistical/machine learning model is run
  • Solution environment capable of storing high-volume and high-variety data in order to perform various analytics in isolation from other enterprise systems
  • Examples of functionality enhancement include personalized recommendations and discounts as well as targeted advertisements

Pregunta 110

Pregunta
Analytical Sandbox Compound Pattern
Respuesta
  • Although analogous to the use of a cloud, this approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
  • The underlying idea is to be able to ingest large amounts of raw data and pre-process it in order to make it suitable for traditional enterprise systems
  • Is not integrated with the EDW and is instead used directly to explore data and perform analytics
  • Keep the Big Data initiative separate from existing IT operations and systems

Pregunta 111

Pregunta
Unstructured Data Store Compound Pattern
Respuesta
  • This configuration is generally opted for by enterprises that want to move towards predictive and prescriptive analytics
  • Although analogous to the use of a cloud, this approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
  • A dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure
  • Generally, the ingested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database for performing analytical queries

Pregunta 112

Pregunta
Batch Data Processing Compound Pattern
Respuesta
  • Such a solution is generally employed by enterprises that have just embarked on a Big Data journey
  • Once processed, the streaming data can be stored in disk-based storage, such as the distributed file system, for further analysis
  • This not only helps process voluminous data faster but also helps cater to infrequent or ad-hoc data processing requests
  • Although analogous to the use of a cloud, this approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues

Pregunta 113

Pregunta
Operational Data Store Compound Pattern
Respuesta
  • The data can be imported into fit-forpurpose NoSQL databases, where it can be easily accessed in support of BI
  • Large data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
  • Based on the data storage requirements, a distributed file system or a NoSQL database can be used for data storage
  • Large amounts of transactional data from operational systems across the enterprise are collected

Pregunta 114

Pregunta
Cloud-based Big Data Processing
Respuesta
  • This not only helps process voluminous data faster but also helps cater to infrequent or ad-hoc data processing requests that require above-average storage and processing resources
  • Setting up a cluster in-house may result in under-utilization of processing resources, as it would not be utilized at all times
  • Is associated with the processing engine, storage device, resource manager and coordination engine mechanisms
  • Enable the processing of such datasets, which requires the use of a batch processing engine
Mostrar resumen completo Ocultar resumen completo

¿Quieres crear tus propios Tests gratis con GoConqr? Más información.

Similar

Test de Nombres de Alimentos en Inglés
maya velasquez
Primera a Segunda Guerra Mundial
jonathanbeltran1
Fórmulas Física
Diego Santos
Campo magnético
Diego Santos
Química Orgánica
pedro.casullo
la tierra
tati benitez
Bioelementos Mapa Mental
Joseline Loza Gil
PLANTILLA DE CIMENTACION
Mariana Urrutia9576
VERBOS FRANCÉS (presente)
Virginia CR
Tipos de Investigación Científica
Karen Dubón
Flujo grama de distribución de egresos.
danny guacas
Explorar la Librería