PART 1 BACKGROUND AND FUNDAMENTALS ......................1

1 Hadoop in a heartbeat 3

1.1 What is Hadoop? 4

1.2 Running Hadoop 14

1.3 Chapter summary 23

PART 2 DATA LOGISTICS.................................................25

2 Moving data in and out of Hadoop 27

2.1 Key elements of ingress and egress 29

2.2 Moving data into Hadoop 30

TECHNIQUE 1 Pushing system log messages into HDFS with

Flume 33

TECHNIQUE 2 An automated mechanism to copy files into

HDFS 43

TECHNIQUE 3 Scheduling regular ingress activities with Oozie 48

TECHNIQUE 4 Database ingress with MapReduce 53

TECHNIQUE 5 Using Sqoop to import data from MySQL 58