AWS has been supporting companies across Australia and New Zealand to put their most innovative tools and technologies to work to achieve their business needs and goals. AWS and our ecosystem of partners has helped the likes of CP Mining, IntelliHQ, WesCEF, Oz Minerals, Woodside and many more to modernise their analytics and data architecture in order to successfully generate business value from their data.
This event series aimed to educate customers with a broader understanding of how to build next-gen data lakes and analytics platforms and make connections with AWS.
25. 2727
Nitric Acid Yield Optimisation
Production
Moisture
V1
V2
Operator DashboardYield Algorithm
Extract
Data for selected
sensors extracted
every 5 mins using
Macroview datapump
Ingest
Raw data ingested
by AWS, meta tags
applied and results
stored in data lake
Process
Data cleansed and
engineered using a
Spark cluster and
stored in data lake
Serve
Curated data loaded to
data mart every 15
mins and optimisation
model applied
Visualise
Dashboard performs
live query of data
mart to advise
operating conditions
Raw Data
37. Fastest way to build secure data lakes
Data Lake Storage
Data
Catalog
Access
ControlBlueprints ML-based
data prep
Lake Formation
Data Lakes AWS Glue
Amazon Redshift
Data warehousing
Amazon EMR
Hadoop + Spark
Athena
Interactive analytics
Amazon
QuickSight
Comprehensive list of integrated tools
enable every user equally
Centralized management of fine
grained permission empower security
officers
Simplified ingest and cleaning enables
data engineers to build faster
Cost effective, durable storage with
global replication capabilities
44. Workflows : Orchestrate repeatable data pipelines
Easy way to create and visualise
you business transformation
rules
Allows for parameters and
pipeline state to be shared
across stages
Dynamic views allow inspection
of current running workflows
for diagnostic and current state
information.
45. Simplified and more granular security permissions
Control data access with simple
grant and revoke permissions
Specify permissions on tables
and columns rather than on
buckets and objects
Easily view policies granted to a
particular user
Audit all data access at one
place
47. Search and collaborate across multiple teams and users
Text based search across all of
your metadata
Add attributes like Data owners,
stewards, and other as table
properties
Add data sensitivity level,
column definitions, and others
as column properties
48. AWS Lake Formation pricing
No additional charges – Only pay for the
underlying services used.
58. H O W W E C A N H E L P
• Brainstorming
• Data platform architecture
• Building of prototype within your accounts that can be brought into production
• Work side-by-side with Amazon experts
Data Lab
• Practical education on Big Data and analytics for new and experienced
practitioners
• Learn best practice solution architecture for building modern data
platforms
Data & Analytics Learning Training and Certification