TPT-DATAAI-921 Architecture for Big Data (2022)

Section outline

• Select section Overview

  Overview

  Collapse all Expand all

  The Architectures for Big Data course (TPT-DATAAI921) presents the new needs raised by the management of very large volumes of data, the tools and techniques used to address these needs, as well as the most prominent classes of systems in the area.
  

  
  

  Instructors:

  • Lectures: Ioana Manolescu, Inria and Ecole Polytechnique ioana.manolescu@inria.fr
  • Labs: Madhulika Mohanty, Inria and Ecole Polytechnique madhulika.mohanty@inria.fr 

  
  

  • Select activity Annonces

    

    Annonces Forum
  • Select activity Annonces

    

    Annonces Forum
  • Select activity Announcements

    

    Announcements Forum
  • Select activity Past exam topic

    

    Past exam topic File

    Completion

    Students must

    

    Mark as done
• Select section Week 1 (Sep 20): Introduction, Database fundamentals (Ioana Manolescu)

  Week 1 (Sep 20): Introduction, Database fundamentals (Ioana Manolescu)

  We will start with an introduction to Big Data and its requirements.

  The file is too large for Moodle, thus I put it here: https://pages.saclay.inria.fr/ioana.manolescu/IntroductionBigData.pdf

  Then, we will recall fundamental features of a Database Management System.
  

  • Select activity DatabaseFundamentals

    

    DatabaseFundamentals File
• Select section Week 2 (Sep 27): Distributed Databases, Mediators (Ioana Manolescu)

  Week 2 (Sep 27): Distributed Databases, Mediators (Ioana Manolescu)

  Today, we will, first, finalize our recall of database system fundamentals. Then, we start the discussion of concrete architectures for Big Data, notably:

  • We introduce distribution and study distributed databases
    
  • We show how to handle heterogeneity, through mediators, data lakes and data spaces
    

  
  

  • Select activity DistributedDB-Mediators

    

    DistributedDB-Mediators File
  • Select activity Sample Java file for testing nested loops, vs. hash join performance

    

    Sample Java file for testing nested loops, vs. hash join performance

    Completion

    Students must

    

    Mark as done
  • Select activity View unfolding example

    

    View unfolding example File

    Completion

    Students must

    

    Mark as done
• Select section Week 3 (Oct 4): Redis lab (Madhulika Mohanty)

  Week 3 (Oct 4): Redis lab (Madhulika Mohanty)

  • Select activity Lab 1 - Redis

    

    Lab 1 - Redis Assignment

    Opened: Tuesday, 4 October 2022, 5:30 PM

    Due: Monday, 17 October 2022, 11:55 PM

    Please submit two files, (1) and (2), as described below. Use the specified naming convention, where LASTNAME and FIRSTNAME should consist of ASCII letters only. 
    (1) a zip archive with all the source files (including run scripts if any) called "LASTNAME_FIRSTNAME.zip";
    (2) a PDF file with your report called "LASTNAME_FIRSTNAME_report.pdf".

  • Select activity Lab 1 slides

    

    Lab 1 slides File

    Completion

    Students must

    

    Mark as done
  • Select activity Lab-1 Assignment Statement

    

    Lab-1 Assignment Statement File

    Completion

    Students must

    

    Mark as done
  • Select activity Submission Guidelines and Policies

    

    Submission Guidelines and Policies File

    Completion

    Students must

    

    Mark as done
• Select section Week 4 (Oct 11): Mediator systems (end), Peer-to-peer data management, the CAP theorem, Big Data management on top of MapReduce (Ioana Manolescu)

  Week 4 (Oct 11): Mediator systems (end), Peer-to-peer data management, the CAP theorem, Big Data management on top of MapReduce (Ioana Manolescu)

  First, we will finalize the discussion of mediator systems. See the slides "Distributed databases and mediators" in Week 2 above.

  Then, we will cover:

  • Data management on top of peer-to-peer networks
  • Transactional properties in a distributed setting
    

  Finally, if time allows, we will start a discussion of data management on top of MapReduce platforms.
  

  • Select activity Peer-to-peer data management

    

    Peer-to-peer data management File
  • Select activity The CAP theorem

    

    The CAP theorem File

    Completion

    Students must

    

    Mark as done
  • Select activity Structured data management on top of MapReduce

    

    Structured data management on top of MapReduce File

    Completion

    Students must

    

    Mark as done
• Select section Week 5 (Oct 18): JSON stores and MongoDB lab (Madhulika Mohanty)

  Week 5 (Oct 18): JSON stores and MongoDB lab (Madhulika Mohanty)

  
  

  • Select activity JSON-MongoDB

    

    JSON-MongoDB File

    Completion

    Students must

    

    Mark as done
  • Select activity Lab 2 Assignment statement

    

    Lab 2 Assignment statement File

    Completion

    Students must

    

    Mark as done
  • Select activity Lab 2 Datasets

    

    Lab 2 Datasets File
  • Select activity Lab 2 - MongoDB

    

    Lab 2 - MongoDB Assignment

    Opened: Tuesday, 18 October 2022, 4:00 PM

    Due: Monday, 31 October 2022, 11:55 PM
• Select section Week 6 (Oct 25): Cloud Data Services (Ioana Manolescu)

  Week 6 (Oct 25): Cloud Data Services (Ioana Manolescu)

  Course recording: https://drive.google.com/file/d/1mAfBmoycLlOnH6sQYRJQJRM2Nenj-ASd/view?usp=sharing
  

  • Select activity Course slides

    

    Course slides File

    Completion

    Students must

    

    Mark as done
• Select section Week 7 (Nov 8): Amazon Cloud Lab (Madhulika Mohanty)

  Week 7 (Nov 8): Amazon Cloud Lab (Madhulika Mohanty)

  • Select activity Lab-3 slides

    

    Lab-3 slides File

    Completion

    Students must

    

    Mark as done
  • Select activity Lab-3 Assignment Statement

    

    Lab-3 Assignment Statement File

    Completion

    Students must

    

    Mark as done
  • Select activity Lab-3 files

    

    Lab-3 files URL

    Completion

    Students must

    

    Mark as done
  • Select activity Lab 3 - Amazon Cloud

    

    Lab 3 - Amazon Cloud Assignment

    Opened: Tuesday, 8 November 2022, 5:00 PM

    Due: Tuesday, 29 November 2022, 11:55 PM

    Completion

    Students must

    

    Mark as done