QuestionsAnswered.net
What's Your Question?
What Is a Case Study?
When you’re performing research as part of your job or for a school assignment, you’ll probably come across case studies that help you to learn more about the topic at hand. But what is a case study and why are they helpful? Read on to learn all about case studies.
Deep Dive into a Topic
At face value, a case study is a deep dive into a topic. Case studies can be found in many fields, particularly across the social sciences and medicine. When you conduct a case study, you create a body of research based on an inquiry and related data from analysis of a group, individual or controlled research environment.
As a researcher, you can benefit from the analysis of case studies similar to inquiries you’re currently studying. Researchers often rely on case studies to answer questions that basic information and standard diagnostics cannot address.
Study a Pattern
One of the main objectives of a case study is to find a pattern that answers whatever the initial inquiry seeks to find. This might be a question about why college students are prone to certain eating habits or what mental health problems afflict house fire survivors. The researcher then collects data, either through observation or data research, and starts connecting the dots to find underlying behaviors or impacts of the sample group’s behavior.
Gather Evidence
During the study period, the researcher gathers evidence to back the observed patterns and future claims that’ll be derived from the data. Since case studies are usually presented in the professional environment, it’s not enough to simply have a theory and observational notes to back up a claim. Instead, the researcher must provide evidence to support the body of study and the resulting conclusions.
Present Findings
As the study progresses, the researcher develops a solid case to present to peers or a governing body. Case study presentation is important because it legitimizes the body of research and opens the findings to a broader analysis that may end up drawing a conclusion that’s more true to the data than what one or two researchers might establish. The presentation might be formal or casual, depending on the case study itself.
Draw Conclusions
Once the body of research is established, it’s time to draw conclusions from the case study. As with all social sciences studies, conclusions from one researcher shouldn’t necessarily be taken as gospel, but they’re helpful for advancing the body of knowledge in a given field. For that purpose, they’re an invaluable way of gathering new material and presenting ideas that others in the field can learn from and expand upon.
MORE FROM QUESTIONSANSWERED.NET
- Bahasa Indonesia
- Sign out of AWS Builder ID
- AWS Management Console
- Account Settings
- Billing & Cost Management
- Security Credentials
- AWS Personal Health Dashboard
- Support Center
- Knowledge Center
- AWS Support Overview
- AWS re:Post
Amazon Migrates 50 PB of Analytics Data from Oracle to AWS
Ending Support for Internet Explorer
Implementing a Large AWS Data Lake for Analysis of Heterogeneous Data
C4ADS users were finding it increasingly difficult to sift through the company’s massive database collection.
ClearScale implemented a data lake with an Amazon Virtual Private Cloud (VPC), designed a web-based user interface, and used AWS Lambda and API Gateway to ingest data.
C4ADS’ new solution can scale as needed without compromising security and is able to meet user demands more effectively.
AWS Services
Amazon Virtual Private Cloud (VPC), AWS S3, AWS Lambda, Amazon API Gateway, Amazon CloudWatch, AWS CloudTrail, Amazon DynamoDB
Executive Summary
C4ADS (Center for Advanced Defense Studies) is a nonprofit organization based in Washington DC that is dedicated to providing data-driven analysis and evidence-based reporting on global conflict and transnational security issues. In this pursuit, C4ADS focuses on a variety of issues, including threat finance, transnational organized crime, and proliferation networks.
The Challenge
The world is a complex ecosystem of people, economies, competing interests, and political ambiguity. Being able to track many different events to determine if there are patterns that would warrant a more critical look and analysis is a difficult task, even under the best conditions. With new regional or political developments each day, sometimes even hour by hour, combing through enormous sets of data is challenging; especially when that data is from different sources and in various formats.
C4ADS is tasked with just this sort of activity. Their clients require evidence-based and data-driven analysis concerning global conflict and transnational security issues. With a focus on identifying the drivers and enablers of such conflict, this organization has to be absolutely confident in the analysis and assessments they provide. However, the first step to performing any sort of review requires analysts to comb through extensive records from different sources and formats to compile a list of potential hits.
As C4ADS increased the number of datasets it ingested, new challenges arose, specifically the ability to make use of all the data at its disposal. As more and more data has become available, their analysts were finding it difficult to sift through all of the incoming information in a quick and expedient way. The company approached ClearScale, an AWS Premier Consulting Partner, and wanted to see if there was a way that they could leverage what they did currently by using AWS to assist in making the data more user-friendly.
The ClearScale Solution
The challenge put forth by C4ADS was that a solution had to be implemented quickly, provide the ability to scale as needed, and be extremely secure given the nature of the information they were reviewing. With these three criteria in mind, ClearScale reviewed various designs and approaches that they could develop and implement on AWS.
Data Storage with Data Lake Approach
The biggest challenge was finding a way to aggregate multiple different file formats (such as PDFs, emails, Microsoft Word and Excel files, logs, XML and JSON files) while still allowing C4ADS to perform easy searches within a large data repository. It rapidly became clear that to accomplish the requirements laid out by the client, ClearScale would have to implement a Data Lake approach within an AWS Virtual Private Cloud (VPC). Unlike traditional data warehouse methodologies that require data to conform to a specific set of schema, a data lake allows for any number of data types to be stored and referenced, so long as those data types have a consistent approach to querying and retrieving data.
It was immediately clear that trying to collapse or conform all the various file types that were available into a normalized format would be too resource-intensive. To overcome this, ClearScale chose instead to implement a solution that would tag all uploaded file content with consistent metadata tagging which, in turn, would allow for greater visibility and speedier search results. This automated metadata tagging for each file that was uploaded either manually or via bulk upload would mimic the client’s existing folder structure and schema that they had adopted internally. This approach would ensure that the new solution would be easily understood by analysts that were already familiar with the current operational processes.
Data Flow Model
System Architecture Diagram
Web-Based User Interface (Web UI)
To access and search these records, ClearScale designed and implemented a web-based user interface. This UI was designed to allow for complete management of the data sources — including data upload — beyond simply searching the Data Lake. From a data repository perspective, ClearScale needed to build and deploy a solution that was scalable and reactive to increased demand but also highly secure. To accomplish this, a combination of AWS S3 was used for the storage of the data uploaded, and DynamoDB for the storage of the file metadata; ElasticSearch was used for the robust search querying that was required.
In order to get the data uploaded, ClearScale leveraged AWS Lambda and API Gateway services to properly ingest the data and automate the creation of the file metadata. Both CloudWatch and CloudTrail were also put in place to monitor resource usage and serve as triggering mechanisms to scale the environment as required.
The entire solution was encased in AWS VPC for robust security and Cognito for SAML based authentication. This approach guarantees that the information was behind a robust security layer with additional work done for data to be encrypted both at rest and in transit. It also insured that administrators could grant access to specific document types based on group roles, both for internal and external role types.
UI Welcome Screen
Bulk Indexing — Add and Index an existed S3 Bucket or Folder
Bulk Indexing — Monitoring of Long Time Backend Tasks
Bulk Indexing — Login and Automatic Errors Handling
Multi-tenancy — Agile Access Setup
Metadata — Governance
Cart — Storing and Exploring Results in Personal Cart
The Benefits
The turnaround time from design to delivery to C4ADS was a mere two months, including deployment of the solution in both a Staging and Production environment as well as training for C4ADS staff on how to use the new solution. The first release provided everything that C4ADS originally asked for: it had to be deployed quickly, it had to have the ability to scale as needed, and it had to be highly secure. Launched in October 2017, the solution has already optimized the analysts’ job activities by giving them the tools necessary to do wide-ranging search profiles and aggregate disparate heterogeneous data types.
Later releases will introduce more robust security measures that will allow C4ADS to extend the service out to their partner organizations. It will also provide multi-lingual support and optical character recognition (OCR) technology to aid in identification of important data markers in the data that is uploaded.
There are plenty of challenges in the business and technology landscape. Finding ways to overcome these challenges is what ClearScale does best. By bringing our own development resources to bear on these complex problems, we can design, build, test, and implement a solution in partnership with your organization, thus allowing you to focus on more pressing matters in running your day-to-day operations.
Headquarters
50 California Street Suite 1500 San Francisco, CA 94111
O: 800-591-0442
5450 Thornwood Dr. Suite #L San Jose, CA 95123
O: 1-800-591-0442
7887 East Belleview Avenue Suite 1100 Denver, CO 80111
O: 1-303-209-9966
2942 N 24th St. Suite 114 Phoenix, AZ 85016
O: 1-602-560-1198
165 Broadway 23rd Floor New York, NY 10006
O: 1-646-759-3656
11757 Katy Fwy Suite 1300 Houston, TX 77079
O: 1-281-854-2088
100 King St. West Suite 5600 Toronto, Ontario M5X 1C9
O: 1-416-479-5447
Kraków, Poland
Kącik 4 30-549 Kraków Poland
Case Study: Enterprise data lake on cloud
1CloudHub helped India’s leading television entertainment network bring its scattered big data into a single source of truth, to make advanced analytics affordable.
Cloud Advisory Services
Dc, servers & data, project scope, — data lake architecture design — data transformation and storage in data lake — customized reports in powerbi, about the client.
The client is a leading media production and broadcasting company, subsidiary of a global media conglomerate. They have over 30 television channels, a digital business and a movie production business, reaching over 700 million viewers in India.
Business challenge
As part of their digital strategy, our client wanted to optimise user experience across channels — iOS and Android apps, Fire TV, web, and so on — based on user behaviour and preferences. This required a deeper understanding of customer behavioural patterns across platforms.
Presently, they were using Segment as the tool to collect around 6.5 billion records (20TB of raw data) of behavioural data from their 30 million online viewers every month from across sources.
In order to deliver a user-focussed digital viewing experience, the client needed
- Reliable storage, with protection against data corruption and other types of data losses
- Security against un-authorized data access
- Ease of finding a single record in billions (by efficiently indexing data)
- An advanced analytics engine that can help them derive and visualise meaningful insights from the client’s high volume and variety of data.
- All of this forming their single source of truth.
We, at 1CloudHub, enabled an enterprise data lake for all of the client’s data to reside in one place — preserving accuracy and timeliness of the data.
Leveraging our client’s existing mechanism to collect and feed data into the data lake, we created a pipeline with EMR (Elastic MapReduce) for data crunching or ETL (Extract, Transform, Load) and Power BI for self-service visualisation.
Our approach
Completion and reporting
01. understand.
- In collaboration with the client’s development team, we outlined the volume, velocity, veracity and variety of data.
- We worked with the client’s business teams and domain experts to define reports in Power BI for the 18 use cases the client had identified.
- We mapped data to corresponding reports and planned data transformation.
- Based on these, we designed and architected the data lake and pipeline necessary for Power BI.
- With the client’s sign-off, we deployed the solution on AWS cloud.
04. Transform
- Once the infrastructure was in place, our data engineering team performed the necessary ETL steps such as cleaning and consolidation to derive value from the raw data.
- We stored this in an S3 bucket as parquet formatted files.
- We imported transformed data as data-marts into AWS Redshift, to be used for Power BI reports.
05. Completion and reporting
- We delivered a summary of findings and recommendations for production deployment to bring the PoC to a meaningful closure.
We enabled advanced analytics for data from up to a year — compared to the 3 months data as per agreement — to deliver the meaningful insights the business teams sought.
We crunched over 12 million records in under an hour, running more than 100 VMs concurrently in a cluster.
We delivered each report at a cost of $70. At this cost, we delivered an excellent price-to-performance ratio, driven by the spot fleet instances we used and our on-demand or pay-as-you-use cloud model.
A similar setup on-premise in a data centre would have cost the client 12,000 times more.
Looking forward
We are delighted to have helped the client create a centralized, analytics-ready repository for their Big Data and look forward to helping them meet their strategic goals using our cloud capabilities.
Latest case studies
Migrate and Modernize the MSSQL on RDS to PostgreSQL on Aurora (Babelfish) Using DMS
We helped to Migrate and Modernize the MSSQL on RDS to PostgreSQL on Aurora (Babelfish) Using DMS
Migrate from DB on Amazon EC2 to Amazon RDS for MySQL with HA using AWS DMS for E-Commerce Platform
We helped our top E-commerce platform customer to Migrate from DB on Amazon EC2 to Amazon RDS for MySQL with HA using AWS DMS
Migrate from MySQL on Amazon EC2 to Amazon RDS for MySQL
We helped our top customer in migrating from MySQL on Amazon EC2 to Amazon RDS for MySQL
DB Modernization MS SQL on RDS > PostgreSQL on Aurora
We helped our top customer in DB Modernization MS SQL on RDS > PostgreSQL on Aurora
AWS Well-Architected Review For A Retail NBFC
1CloudHub helped a top Retail NBFC by conducting Well-Architected Review For Their Applications
Cost effective, Scalable Cloud Solution for connected vehicle telematics
1CloudHub helped the automotive manufacturer (Ashok Leyland) with a Cost-effective, Scalable Cloud Solution for Connected vehicle telematics
Migrating LOB workloads for NBFC
1CloudHub helped a leading Investment and Finance Company (Five Star) in Migration.
E-Commerce Platform Migration to AWS Cloud
1CloudHub helped an Indian wardrobe store and silk saree emporium in migration
Application Modernization and Migration of Enterprise Workloads
1CloudHub helped a leading global shipping giant (CMA CGM) to organize shadow IT systems and bring in centralized governance.
Financial Services – Application Modernization to enhance agility & scalability
1CloudHub helped a Investment and Finance Company Limited in India in DevOps Implementation.
Winning the DevOps Way
1CloudHub helped a leading assessment and learning product firm (Brainvalley) in DevOps Implementation.
Application and Database Modernization along with CI/CD
1CloudHub helped the third largest Shipping and transportation company in DevOps Implementation.
Using DevOps To Keep An Edge
1CloudHub helped an Indian wardrobe store and silk saree emporium (Nalli) in DevOps Implementation.
Migration of large Windows landscape from On-prem Data center to AWS
Windows Workload Migration from On-Prem to AWS with Zero Down Time
Migration of e-Commerce portal from On-prem Data Center to AWS
1CloudHub helped the online shopping store in migrating from on-premises to AWS, reducing downtime and ensuring end-user security.
DevOps Implementation- Automated Deployment Process
Migrated the entire IT infrastructure – ERP & Financial systems – from OnPrem to Cloud without impacting the business which needs the systems to be running 24 x 7
Data Lake on AWS For Seats and Revenue Analytics
Data lake was successfully created by ingesting data from various sources which enabled the customer to perform seat and revenue analysis
IaC Automation & Data Generation
We at 1CloudHub successfully containerized an identified application, optimizing the cloud resources used by the application, for continuous delivery.
Migration of IT Infrastructure from OnPrem to Google Cloud
Leveraging AI/ML(Personalization) to Increase Checkout Ratio & Rationalize Discount Coupons for a leading B2C E-Ticketing Platform
Increase in Checkout Ratio by 2 basis points Decrease in cost to business by categorizing into 6 categories
App Cloud Maturity Enhancement (Using Containers)
Pro-Active 24×7 Managed Services
1CloudHub manages IT assets for India’s largest and no. 1 stem banking company to support their business continuity
CI/CD Application Deployment Process using Serverless Technology
1CloudHub helped building CI/CD pipelines for 20+ active and critical applications.
Knowledge Portal on AWS for a Leading Corporate Compliance in India
We helped one of the largest Corporate Compliance in India to create knowledge portal on AWS, cost effectively.
Migration of E-commerce Portal from OnPrem to AWS Cloud
Seamlessly migrated 10+ VMs to AWS with zero downtime. 5% savings on Infra spend by Right sizing and cost optimization
Migration of DNB platform for SMEs from an existing hyper cloud platform to Azure
Seamlessly migrated the DNB platform from an existing hyper cloud platform to Azure to support a transaction of INR 75 Cr every day
Personalize fitment to determine AI/ML driven Solution Roadmap for a leading B2C E-Ticketing Platform
1CloudHub helped one of India’s largest online e-ticketing company in Data Acquisition, Data Validation, Fitment to Personalize services and Roadmap with future insights.
SIFT Customer Engagement Platform on Cloud
We envisioned, designed and implemented cloud transformation strategy for APAC’s largest Customer Engagement Product company in a span of 4 weeks. We look forward to working with the client again on more cloud transformation projects that will help them achieve their business goals.
Digital Asset Management Platform
We helped one of the pre-school in India to build a Digital Asset Management platform which enable the organization to manage video files by tagging the assets based on their metadata for efficient search and retrieval.
DataLake and Analytics for Digital Exam Platform on AWS
We were able to unify multi source(including on-premise) data sets, created future proof analytical platform for delivering a hierarchical student performance reports.
SAP S/4 HANA Functional Enhancement & Implementation
We are glad to have played a key enabling role in helping customer to build a pricing functionality within SAP S/4 to move away from excel/manual pricing process for accurate and real-time business reporting
Case Study : Migration of popular news sites to Cloud with Zero Downtime
We at 1CloudHub successfully migrated one of India’s established news site to cloud seamlessly while optimizing the infra for cost and performance.
Case Study : Hospital Information System (HIS) set-up on Cloud
We helped one of India’s largest Health care provider, to set up their HIS application in a short period of 3 weeks
Case Study: SAP ECC Migration on Azure Cloud for a Health Care Manufacturer
Fast tracked On-Premise SAP ECC Dev, QA & Prod Landscapes to Azure while ensuring a smooth cutover within an hour.
Case Study : SAP S/4 HANA Greenfield Infra Implementation
We helped one of the largest security service company in Singapore to adopt their first cloud application (SAP S/4 HANA) in Azure cloud
Case Study: Big Data on Cloud
1CloudHub helped one of the world’s largest manufacturers of commercial vehicles deploy a cost-effective, scalable cloud solution for their Big Data.
Case Study: DR for geographically diverse SAP
We helped one of the world’s largest paper, pulp, and packaging companies, a first-time cloud adaptor, to establish a unified DR site.
Case Study: RPA on cloud
We helped a global shipping leader achieve on-demand scaling through a multi-geography accessible RPA solution.
Case Study: Multi-cloud strategy
We helped India’s leading integrated healthcare delivery service provider design and implement their HIS on cloud.
Case Study: Enterprise app migration
We helped a global leader in supply chain services efficiently and effectively host their applications on the cloud during a period of business transformation.
Case Study: DC backup and DR
We helped India’s leading television entertainment network architect, deploy, and manage their data backup system.
Case Study: SAP on cloud
We helped one of the world’s largest shipping companies increase the future load-capacity of their mission-critical SAP CRM, at significantly lower costs.
Case Study: DC and app migration
We envisioned, designed and implemented an end-to-end cloud transformation strategy for a leading gaming company in Malaysia.
Sharing is caring!
Case Studies
Aws data lake project, azure data lake project, ml/ai projects.
Data Engineering
Industry: Consulting
Subject: AWS Cloud Data Lake Development; Cloud Big Data Engineering
Description:
Developing and maintaining data lakes on AWS. Data migration from RDBMS and file sources, loading data into S3, Redshift, and RDS. Designing and developing big data batch solutions using AWS Data Pipeline and AWS Glue and EMR. Developing a massive data warehouse using Redshift and Redshift Spectrum.
Project Task Summary
ETL workflows in Data Pipeline, monitoring and management of ETL pipelines .
Batch RDBMS data migration using AWS DMS .
Batch processing in EMR and Glue using Scala Spark.
Designing and developing data warehouse on Redshift.
DWH data model and table design .
Accessing and processing big data on S3 via SQL using Redshift Spectrum.
Python ML implementation with Pandas, scikit-learn using Jupyter on AWS.
CI/CD development using Gitlab and Ansible.
Technologies
- AWS CloudWatch
- AWS Data Pipeline
- RDS PostgreSQL EMR
- Redshift Spectrum
- Scikit-learn
Subject: Cloud Data Lake DevOps; AWS DevOps
Provisioning and deployment of big data solutions on AWS. Operationalize cloud data solutions, implementing infrastructure as code (IaC), using CloudFormation templates for resource management. Provisioning and deploying on-demand Redshift cluster and RDS instances using CloudFormation. Development, management, and deployment of Docker images and containers.
Provisioning resources using CloudFormation templates .
Provisioning of Redshift, Data Pipeline, and Glue ETL pipelines .
User account and access management in IAM.
Develop Docker images for batch processing applications and Python, ML models, using AWS Container Registry (AWS ECR) .
Docker container deployment using AWS ECS .
CI/CD implementation using GitLab.
- AWS CloudFormation
Subject: Azure Cloud Data Lake Development; Azure Big Data Engineering
Data lake development on Microsoft Azure. Data migration from RDBMS and file sources, data loading into Azure Blob storage and Azure SQL. Design and development of big data batch solutions using Data Factory and Databricks. Massive data warehouse development using Azure SQL Data Warehouse.
Create ETL workflows in Data Factory with data factory ETL pipeline monitoring and management .
Batch processing in Azure Databricks using Scala Spark.
Data warehouse design and development using SQL Data Warehouse.
DWH data model design, featuring index and partitioning table design .
Accessing and processing big data in Blob storage via Transact-SQL using Polybase .
CI/CD development using SBT and Gitlab.
- Azure Portal
- Transact-SQL
- Data Factory
- Azure SQL DWH
Subject: Cloud Data Lake DevOps; Azure DevOps
Data Factory and Databricks provisioning and deployment. Operationalization of cloud data solutions and infrastructure as code (IaC) implementation using ARM templates and Azure Python SDK for resource management. Azure SQL data warehouse provisioning and deployment. CI/CD implementation using Azure DevOps tools. Development, management, and deployment of Docker images and containers.
Azure resources (VM and storage account, SQL DB and network) provisioning using Azure Python SDK and ARM template .
SQL data warehouse provisioning with Databricks and Data Factory integration, using Python scripts and ARM templates, with Azure Key Vault for deployment .
User account and role-based (RBAC) access management in Azure Active Directory.
Docker image development for batch processing applications and ML model APIs, using Azure Container Registry for build, storage, and management of images.
Azure container deployment on ACI (Azure Container Instances).
CI/CD implementation via Azure Repos, Azure Artifacts, Azure Pipelines, and Azure Test Plans.
- Azure Python SDK
- Azure Repos
- Azure Artifacts
- Azure Active Directory
- Azure Pipelines
- Azure Test Plans
- Azure Container Registry
NLP Project
Industry: AI/Automatization
Subject: Implementation of supervised Machine Learning Algorithm for automatic keyphrase extraction.
Implementation of automated Context Tagger for a B2B Marketing automated AI solution. Text classification models are implemented in Python using Python Text Mining, NLP and other ML and data analysis libraries (Python Data Science and ML stack). Text mining, data processing, and feature engineering of a massive dataset in Spark.
Design and implementation of a very fast multi-threaded AKKA-based stream (SAX/Stax) processing of XML data for transforming huge XML data to CSV format.
Preprocessing of the data by filtering normalizing text content and applying Spacy and NLTK.
Text mining and data preprocessing in Spark SQL Scala on Hadoop and S3.
Training of embedding and language models using fastText, Gensim, and GPT-2.
Multi-Class multi-label text classification using CNN and word embedding models using Keras and PyTorch.
Modeling term to tag relations in massive graph networks in Tigergraph.
Keyphrase extraction (automatic tagging) using N-Grams, Word2vec scoring and PageRank algorithm on massive graphs of tag-to-tag relations .
FP-Growth association rules learning.
Distributed CNN training in Docker containers on AWS using GPU instances.
- Classification
- Apache Spark
- Apache Hadoop
- Apache Arrow
- Nvidia-docker
Recommendation Prediction Model
Industry: E-Commerce
Subject: ML Model implementation for Recommendation Models
User tracking data is used in training ML models for user-profiling, recommendation, and prediction. RNN and CNN models are developed and trained for enrichment of user-profiles. Classification GBM (Gradient Boosting Machine) on extracted and learned features. Workflow implementation for data engineering and continuous model training implementation in Airflow.
Feature engineering using Spark SQL by joining and aggregating user tracking data.
Keras and TensorFlow implementation for training RNN and CNN models.
Using Spark ML for training gradient boosting classifiers .
Cross-validation, F1-score evaluation, hyperparameter optimization .
Containerized Spark standalone cluster, using Docker Compose for local deployment and AWS container services.
- Docker Compose Keras
- GBM (Gradient Boosting Machine)
- AWS Console
- Neural Networks
- Spark Standalone Cluster
- Supervised Learning
- 214.617.1389
Data Management
- Cloud Data Management Solutions & Consulting
- Enterprise Cloud Data Management Strategy
- Big Data Consulting
- Data Lake Consulting & Development
- Data Cleansing Services
- Data Integration Services
- Master Data Management (MDM) Consulting
BI & Analytics
- Data Analytics Consulting
- Business Intelligence Consulting
- Data Visualization
- Enterprise Business Intelligence (EBI) Consulting Services
- Cloud Business Intelligence Consultants
- On-Site BI Architect Session
- Business Intelligence Audit
Custom Development Services
- Cloud Data Management
- Data Warehouse Modeling
- Enterprise Data Strategy
- Data Lake Consulting
- Data Cleansing
- Integration Services
- Master Data Management Consulting
Support & Training
- AWS Solutions
- Azure Solutions
- SAP Business Objects
- Case Studies
- Learning Center
Serverless Data Lakes on Amazon Web Services (AWS)
Loan Marketplace Harnesses Their Data Using Server-less Data Lakes
WCI Data Solutions was able to help the largest small business loan marketplace in the United States harness their data by designing and developing a serverless data lake service on Amazon Web Services (AWS) with Amazon S3 as the primary storage platform.
Our customer is the largest small business loan marketplace in the United States. Small business owners utilize their free online service to find financing by browsing multiple loan products from a network of more than 75 lenders. The platform reviews metrics including the business's financial projections, use of funds, industry, and monthly revenue to find the right loan option for borrowers, helping traditionally underserved groups like women and minority-owned small businesses, mom and pop shops, and seasonal businesses. Their matching engine allows small business borrowers the opportunity to comparison shop across a broad range of lenders and loan products, decreasing their time and effort, and optimizing which lenders to send applications to.
Our customer came to us with an interest in the development and deployment of a Data Lake solution on Amazon Web Services . They wanted a system capable of supporting both analysis and reporting across the organization but also designed to accommodate future consumers, query methods, and new data sources. They wanted to begin this development with an internally developed system we’ll refer to as their “CRM”. The CRM application is also deployed in AWS and uses Aurora RDS as a backend. This will be the first data source used by the WCI developed solution, with other sources to follow. The data will be consumed by the organization using both Domo and Power BI .
The engagement required utilizing a broad array of services on AWS, including:
Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (Amazon RDS), Amazon Glue, Amazon Athena, Amazon CloudWatch, Amazon Simple Queue Service (Amazon SQS), AWS Lambda, Amazon Virtual Private Cloud Peering (VPC-Peering), AWS Identity and Access Management (IAM) and Amazon Simple Storage Service (Amazon S3).
WCI assisted the customer in solving this problem by designing and building a serverless data lake service on AWS with Amazon S3 as the primary storage platform. Utilizing an Amazon S3-based data lake architecture allows our customer the means to have a centralized, secure, and highly durable cloud-based storage platform. AWS makes it easy to store data in any format, both securely and at a massive scale with Amazon S3. In an effort to reduce both complexity and cost of development, WCI decided to utilize Amazon Athena as a query service to produce views that could then be used, in conjunction with Domo, to provide valuable business metrics and insights from the CRM data.
With the use of WCI’s solution, our customer now has the ability to scale their data analytics platform as they continue to add data from new sources, regardless of whether it be structured or unstructured data. The new cost-efficient and cloud-based architecture provides business end-users a data analytics process that is both expedient and lends itself to being future proof. Using AWS native services like Athena, Glue and Lambda enables the ability to continue improving data analysis without the burden of maintaining infrastructure or complex ETL processes, and with WCI’s assistance, our customer now has a secured end-to-end framework for their data and analytics practice on AWS.
Our customer now has the ability to harness increased data from various sources and options for future integrations and for use in other business-critical functions. This gives them the capability to analyze their data using different analytics tools and services, which allows them to derive more insights and provide more value for their customers, suppliers, and partners. WCI developed solution allows them the ability to gain deeper insights into their data in ways that traditional data silos cannot, which in turn helps them continue to support their customers better than any other lending platform.
About WCI Data Solutions
WCI is a consulting services company focused on the application of data through the use of Business Intelligence and Data Warehouse technologies that bring increased business performance for our clients. WCI has serviced well over 300 companies through our knowledge and expertise of how to make data valuable to decision-makers.
- Founded in 1998
- Full-Time Consultants all based in the U.S. with an average tenure of 7 years.
- Because there is a significant spectrum across the 300 customers we have serviced, we pride ourselves on our flexibility and the types of services we provide necessary to help our customers succeed no matter the platform, environment or objective.
- Forbes | America’s Best Consulting Firms | Big Data & Analytics – 2016, 2017, 2018 & 2019
Related Posts
Creating a BI Strategy for An Emergency Healthcare Company
Amalgamating Disparate Systems and Creating a Long-Term BI Strategy for an Emergency Healthcare Company "We were very impressed with the…
A Fortune 500 Real Estate Investment & Management Company
The client A Fortune 500 real estate investment & management company The challenge The company wanted to simplify its IT…
Get Started
Want a FREE on-site discovery session with your team? Receive a FREE data management evaluation session with one of our veteran data architects. Reap the benefits of an expert’s outlook on taking control of your business insights and data. Stop missing out on data-driven opportunities, and start making smarter, more profitable decisions today.
IMAGES
VIDEO
COMMENTS
When you’re performing research as part of your job or for a school assignment, you’ll probably come across case studies that help you to learn more about the topic at hand. But what is a case study and why are they helpful? Read on to lear...
Case studies are important because they help make something being discussed more realistic for both teachers and learners. Case studies help students to see that what they have learned is not purely theoretical but instead can serve to crea...
Examples of a case study could be anything from researching why a single subject has nightmares when they sleep in their new apartment, to why a group of people feel uncomfortable in heavily populated areas. A case study is an in-depth anal...
One of the initiatives adopted to rise to this challenge was the development of a data lake on Amazon Web Services (AWS). By adopting storage, databases
Build a data lake on Amazon S3: Recent customer case studies · Siemens Handles 60,000 Cyber Threats per Second Using AWS Machine Learning.
CASE STUDY. Nasdaq sign. 2020. Nasdaq Uses AWS to Pioneer Stock Exchange Data Storage in the Cloud. Nasdaq uses a data lake based on Amazon S3 and Amazon
Intel Launches Geospatial Data Management and AI Framework Using AWS. 2021. Intel Geospatial, a new venture from semiconductor chip manufacturer Intel, recently
OneFootball Built a Data Lake in Days Using AWS Lake Formation to Serve 70 Million Fans · Feeding the World's Appetite for Soccer · Seamless Integration for Self-
Data Lake on AWS. Many Amazon Web Services (AWS) customers require a data storage and analytics solution that offers more
In order to meet its growing needs, Amazon's consumer business decided to migrate the Oracle data warehouse to an AWS-based solution. The new data lake
C4ADS partnered with ClearScale to develop and implement Data Lake solution for analysis of heterogeneous data on AWS.
1CloudHub helped India's leading television entertainment network bring its scattered big data into a single source
Case Studies about AWS Data Lake, Azure Data Lake and ML/AI. Click now to see project tasks, used technologies, industries and more.
See how we helped the largest small business loan marketplace in the US harness their data by developing a serverless data lakes solution using AWS.