Big Data course
CentraleDigitalLab@LaPlateforme
Stéphane Vialle, CentraleSupélec & LISN, Stephane.Vialle@centralesupelec.fr
Gianluca Quercini, CentraleSupélec & LISN, Gianluca.Quercini@centralesupelec.fr
-
Course theme and objectives:
The
main aim of this course is to provide an understanding of algorithms
and programming in BigData paradigms (various variants of Map-Reduce,
analysis of structured documents in NoSQL databases, etc.), as well as
the technical principles underlying Big Data environments (distributed
file systems, fault tolerance and load resistance through redundancy,
etc.). A quantified presentation of the concept of Scaling completes
the course, and practical exercises on Spark and MongoDB environments
will illustrate the concepts seen in the course.
-
Course content:
- Map-Reduce programming paradigms, optimization and implementation with Spark on PC clusters
- Performance metrics and scalability
- Large-scale SQL and NoSQL databases, introduction to Spark-SQL and experimentation with MongoDB
- Course materials:
1a - Introduction to MapReduce & Spark
|
 | 1 slide per page |
| 6 slides per page |
1b - Spark application deployment (TP1 & TP2)
|
|
1 slide per page |
|
6 slides per page |
2 - Performance metrics & scalability
|
|
1 slide per page |
|
6 slides per page |
3a - NoSQL principles and emergence
|
|
1 slide per page |
|
6 slides per page |
| 3b - Distributed and NoSQL databases
|
|
1 slide per page |
|
6 slides per page |
3c - Spark-SQL
| Documentation Spark-SQL
|
| 3d - MongoDB : syntax and examples (TD3 & TP3)
|
|
1 slide per page |
|
6 slides per page |
|
MongoDB introduction notebook
|
|
MongoDB manual |
Tut1: Designing basic Map-Reduce algorithms in Spark
|
Statement
|
Tut2: Designing advanced Map-Reduce algorithms in Spark
|
Statement
|
Tut3: MongoDB syntax and examples
|
See slides 3d
|
NoSQL DBs:
- Kristina Chorodorw. MongoDB. The Definitive Guide. 2nd edition. O'Reilly. 2013.
- Rudi Bruchez. Les bases de données NoSQL et le Big Data. 2ème édition. Eyrolles. 2016.
Hadoop & Map-Reduce:
- Tom White. Hadoop. The definitive Guide. 3rd edition. O'Reilly. 2013.
- Donald Miner and Adam Shook. MapReduce Design Patterns. O'Reilly. 2013.
Spark :
- M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley,
M.J. Franklin, S. Shenker, and I. Stoica. Resilient Distributed
Datasets : A Fault-tolerant Abstraction for In-memory Cluster
Computing. In Proceedings of the 9th USENIX Conference on Networked
Systems Design and Implementation, NSDI’12, 2012.
- H. Karau, A. Konwinski, P.Wendell, and M.Zaharia. Learning Spark. O’Reilly, 1st edition, 2015.
- H. Karau and R. Warren. High Performance Spark. O’Reilly, 1st edition, 2017.