Apache Hadoop: Manipulation and Transformation of Data Performance培训

课程编码

ApHadm1

课程时长

21 小时通常来说是3天，包括中间休息。

要求

Attendees are not required to have any specific skill as the training is focused on end users skills for both the administration and the manipulation of data under Apache Hadoop

课程概览

本课程面向开发人员，架构师，数据科学家或任何需要密集或定期访问数据的配置文件。

该课程的主要重点是数据处理和转换。

在Hadoop生态系统的工具中，本课程包括Pig和Hive的使用，这两者都大量用于数据转换和操作。

此培训还涉及性能指标和性能优化。

该课程完全是实践性的，并通过理论方面的介绍来打断。

Machine Translated

课程大纲

1.1Hadoop Concepts

1.1.1HDFS

The Design of HDFS
Command line interface
Hadoop File System

1.1.2Clusters

Anatomy of a cluster
Mater Node / Slave node
Name Node / Data Node

1.2Data Manipulation

1.2.1MapReduce detailed

Map phase
Reduce phase
Shuffle

1.2.2Analytics with Map Reduce

Group-By with MapReduce
Frequency distributions and sorting with MapReduce
Plotting results (GNU Plot)
Histograms with MapReduce
Scatter plots with MapReduce
Parsing complex datasets
Counting with MapReduce and Combiners
Build reports

1.2.3Data Cleansing

Document Cleaning
Fuzzy string search
Record linkage / data deduplication
Transform and sort event dates
Validate source reliability
Trim Outliers

1.2.4Extracting and Transforming Data

Transforming logs
Using Apache Pig to filter
Using Apache Pig to sort
Using Apache Pig to sessionize

1.2.5Advanced Joins

Joining data in the Mapper using MapReduce
Joining data using Apache Pig replicated join
Joining sorted data using Apache Pig merge join
Joining skewed data using Apache Pig skewed join
Using a map-side join in Apache Hive
Using optimized full outer joins in Apache Hive
Joining data using an external key value store

1.3Performance Diagnosis and Optimization Techniques

Map
- Investigating spikes in input data
- Identifying map-side data skew problems
- Map task throughput
- Small files
- Unsplittable files
Reduce
- Too few or too many reducers
- Reduce-side data skew problems
- Reduce tasks throughput
- Slow shuffle and sort
Competing jobs and scheduler throttling
Stack dumps & unoptimized code
Hardware failures
CPU contention
Tasks
- Extracting and visualizing task execution times
- Profiling your map and reduce tasks
Avoid the reducer
Filter and project
Using the combiner
Fast sorting with comparators
Collecting skewed data
Reduce skew mitigation

客户评论

★★★★★

★★★★★

你最喜欢什么培训？：详细介绍每个工具。

Amber Mehrotra - NIIT Limited

课程:Hadoop Administration

Machine Translated

事实上，所有数据和软件都已准备好在已准备好的VM上使用，由外部磁盘中的培训师提供。

vyzVoice

课程:Hadoop for Developers and Administrators

Machine Translated

我最喜欢教练提供真实的实例。

Simon Hahn

课程:Administrator Training for Apache Hadoop

Machine Translated

我真的很享受培训师的巨大能力。

Grzegorz Gorski

课程:Administrator Training for Apache Hadoop

Machine Translated

我真的很喜欢这些实践课程。

Jacek Pieczątka

课程:Administrator Training for Apache Hadoop

Machine Translated

这是非常亲力亲为的，我们花了一半的时间实际在Clouded / Hardtop做事，运行不同的命令，检查系统，等等。额外的材料（书籍，网站等）真的很感激，我们将不得不继续学习。安装非常有趣，而且非常方便，从头开始的集群设置非常好。

Ericsson

课程:Administrator Training for Apache Hadoop

Machine Translated

很多动手练习。

Ericsson

课程:Administrator Training for Apache Hadoop

Machine Translated

Ambari管理工具。能够讨论除电信之外的其他业务案例的实际Hadoop体验。

Ericsson

课程:Administrator Training for Apache Hadoop

Machine Translated

我非常喜欢的虚拟机老师对这个话题以及其他话题都很了解, 他很好, 也很友好。我喜欢迪拜的设施。

Safar Alqahtani - Elm Information Security

课程:Big Data Analytics in Health

Machine Translated

与参加培训的人沟通。

Andrzej Szewczuk - Izba Administracji Skarbowej w Lublinie

课程:Apache NiFi for Administrators

Machine Translated

实际做的事情，理论也得到了Ajay的好评

Dominik Mazur - Capgemini Polska Sp. z o.o.

课程:Hadoop Administration on MapR

Machine Translated

演习

Capgemini Polska Sp. z o.o.

课程:Hadoop Administration on MapR

Machine Translated

练习的有用性

Algomine sp.z.o.o sp.k.

课程:Apache NiFi for Administrators

Machine Translated

我发现培训很好，信息量很大......但是可以分发4到5天，让我们可以了解不同方面的更多细节。

Veterans Affairs Canada

课程:Hadoop Administration

Machine Translated

我非常喜欢这种训练。安东拥有丰富的知识，并以非常方便的方式提出了必要的理论。很棒的是，培训是很多有趣的练习，所以我们从一开始就接触到了我们所知道的技术。

Szymon Dybczak - Algomine sp.z.o.o sp.k.

课程:Apache NiFi for Administrators

Machine Translated

我发现这个课程给了一个很好的概述，并迅速触及了一些我甚至没有考虑的领域。

Veterans Affairs Canada

课程:Hadoop Administration

Machine Translated

课程分类

相关课程

促销课程

Docker管理与开发集成

10/21/2019 - 09:30

广东国际大厦

查看更多

Automation Anywhere for End Users

10/28/2019 - 09:30

北京 - 创而新大厦

查看更多

Automation Anywhere for End Users

10/31/2019 - 09:30

北京 - 创而新大厦

查看更多

Docker和Kubernetes：构建和缩放容器化应用程序

11/04/2019 - 09:30

上海 - 上海中区广场

查看更多

Kubernetes：基础到高级

11/18/2019 - 09:30

浙江 - 外经贸广场

查看更多

Docker (introducing Kubernetes)

11/25/2019 - 09:30

深圳 - 新世界中心

查看更多

我们的客户

mizuho corporate bank

axel springer se

ibm business consulting services

european molecular biology laboratory (embl)

european organization for nuclear research (cern)

nintendo of europe

avid technology

aviva commercial union

seagate technology

embassy of the united states of america

glaxosmithkline

international air transport association (iata)

pearson education

emc corporation

fiat automobiles

royal dutch shell

emerson electric

universal mccann

mizuho corporate bank

axel springer se

ibm business consulting services

european molecular biology laboratory (embl)

european organization for nuclear research (cern)

nintendo of europe

avid technology

aviva commercial union

seagate technology

embassy of the united states of america

glaxosmithkline

international air transport association (iata)

pearson education

emc corporation

fiat automobiles

royal dutch shell

emerson electric

universal mccann

is growing fast!

We are looking to expand our presence in China!

As a Business Development Manager you will:

expand business in China
recruit local talent (sales, agents, trainers, consultants)
recruit local trainers and consultants

We offer:

Artificial Intelligence and Big Data systems to support your local operation
high-tech automation
continuously upgraded course catalogue and content
good fun in international team

If you are interested in running a high-tech, high-quality training and consulting business.