Case Study

Education Analytics SaaS Platform

LET'S TALK
education analytics case study hero scaled

A Self-Serve SaaS Education Data Analytics Platform with focus on Performance, Extensibility, Data Privacy, Security, and Compliance.

Challenges/Objectives

  • Data Engineering
    • Extensible and pluggable data integration to support low-turn around for new data sources
    • Batch and CEP – hybrid data integration workloads
  • Security/Privacy
    • Cross cloud data pipelines to ensure privacy of school’s raw datasets
    • GDPR compliance
  • DevOps
    • Fully automated cross-cloud deployment
    • Cross-cloud upgrades and rollbacks
  • User Interface
    • Pluggable and configurable visualization/charting
    • 4-level drill down

The Solution

education analytics case study architecture
School GCP cloud account to be used to deploy part of the data pipeline and databases to adhere to the compliances. The deployment automation carries out provisioning, de-provisioning, upgrades/downgrades in the multiple tenant (school) clouds.
Data compliance micro-service which ensures the enforcement of compliance policies
Integration micro-services which make use of API integration, web-hook listeners and Google Cloud Pub-Sub based messaging integration to provide a multi-tenant integration framework
Mapping micro-services to map the incoming data with the entities queried via the LMS and user directory platforms
Infrastructure and application configuration as a code service which utilizes the cloud deployment manager and cloud build services
BigQuery Datawarehouse to store the raw data in Level 0 and mapped data in Level 1
Aggregation pipeline in Cloud Dataflow to provide daily aggregated metrics and store the same in the serving DB (Cloud SQL – Postgres)

Cross-cloud, multi-tenant platform powered by Micro-services and scalable data integration pipeline.

Solution Hightlights

  • Data Engineering
    • Google Cloud Dataflow and BigQuery based CEP and Batch Data Processing Pipeline
    • BigQuery Datawarehouse as L0 and L1 for saving the events data
    • CloudSQL (Postgres) used as the Serving DB with BI-ready Schema
  • Application & Integration Services
    • Google Cloud Functions, Google Cloud Run, and Google App Engine based micro-services architecture
    • Google Cloud Pub-Sub for micro-service communication
  • DevOps
    • Google Cloud Deployment Manager based automated Cross-Cloud Deployment of 25+ system components
    • Google Cloud Marketplace billing and subscription management integration for provisioning and de-provisioning
    • Google Cloud Build and Google Cloud Container Registry based Continuous Integration
  • Security/Privacy
    • Data Retention Policy enforcement through Google Cloud Scheduler and retention policy micro-service
    • PII Masking for identity/privacy protection
  • User Interface
    • Management and analytics portal built with Angular 8 and Highcharts
    • Declarative charting and API query for extensible reporting and dashboarding

Solution in Numbers

19

Micro-services

16

Google Cloud Platform Services Used

5

Months from Conceptualization to Delivery

5

Data Sources

30

Component Deployment Automated