AI presentation and introduction - Retrieval Augmented Generation RAG 101
AVRO to JSON Conversion
1. 1
Baltimore Meetup will start
shortly...meanwhile
@Attendees: Kindly introduce yourself in Chat
● Name
● Company
● Location
● Mule Experience
2. BALTIMORE MuleSoft
Meetup
AVRO to JSON/JSON to AVRO
Conversion using Confluent
Schema Registry – Use Case and
Demo
6th August 2022
Start
Recording...
3. ● Introduction
● Overview on AVRO and JSON
● Convert AVRO to Json and Json
to AVRO through Mule - DEMO
● Introduction to Confluent
Schema Registry
● Convert AVRO to Json and Json
to AVRO using confluent
schema registry - DEMO
● Q&A
● Trivia
● Meetup: Feedback & Upcoming
Events
Agenda
3
5. Today’s Meetup
Speaker
Shruthi R
Application Development Analyst at Accenture
● Mulesoft Certified Developer(Mule 3 & 4) with 3.5 years of experience
working in Mule 3 & Mule 4 Live Projects.
● Have good technical experience in designing & implementing various
integration solutions for Retail and Health Care Domains.
● Worked as a key resources for many projects integrating Salesforce CRM,
Workday, MDM etc.
6
6. ● Both the speaker and host are organizing this meet up in individual capacity only. We are not
representing our companies here.
● This presentation is strictly for learning purpose only. Organizer/Presenter do not hold any
responsibility that same solution will work for your business requirements also.
● This presentation is not meant for any promotional activities.
● This meeting will be recorded and shared.
7
Safe Harbour
Statement
8. 9
● Avro is used to define the data schema/structure for a record's value.
● An AVRO schema is created using JSON format and can have single or multiple
fields.
{
"type": "record",
"namespace": "com.example",
"name": "FullName",
"fields": [
{ "name": "first", "type": "string" },
{ "name": "last", "type": "string" }
]
}
What is AVRO?
Type: For Avro schemas, type is
always record. This means that there will be
multiple fields defined.
Namespace: It is used to differentiate one
schema type from another
Name: This is the schema name which when
combined with the namespace, uniquely
identifies the schema
Field: A simple data type, such as an integer or
a string, or it can be complex data
9. 9
Why AVRO?
● Embedding documentation in the schema reduces data interpretation
misunderstandings, allows other team members to know about your data
without asking you for clarification.
● It has a very compact format. The bulk of JSON, repeating every field name
with every single record, is what makes JSON inefficient for high-volume usage.
● It is very fast.
Advantages
● Helps producers or consumers of data streams know the right fields that are
needed in an event and what type each field is.
● Keeps data clean, and make everyone more agile
● They protect downstream data consumers from malformed data, as only valid
data will be permitted in the topic.
● Schemas also help solve one of the hardest problems in organization-wide data
flow modeling and handling change in data format
10. 9
● JSON is basically a combination of key/value pair and arrays.
● The data types supported by JSON are string, number, object, array, boolean and
null.
{
"id" : 100,
"name" : "John",
"subject" : [ "Maths",
"address" : {
"city" : "faridabad",
What is JSON?
11. 9
● To prevent data loss or corruption by maintaining the integrity of the data and
embedded structures.
● End Systems supports or requires data in the format
Why Data Conversion Required?
12. AVRO to JSON and JSON to AVRO
Conversion through Mule
13. 9
● Each Binary message should be embedded with AVRO schema to process via mule.
● Is feasible when the payload received from external system is in embedded
format(Binary + AVRO schema).
JSON to AVRO Conversion
● Json payload is converted into binary data embedded with AVRO schema.
● Is feasible when the external system accepts the data in embedded format(Binary
+ AVRO schema).
AVRO to JSON Conversion
16. 9
Pre-Requisites:
1. Anypoint Studio Version 7.12.1
2. Mule Runtime Version 4.4.0
3. Confluent Cloud Account
● Anypoint Connector for Confluent Schema Registry provides a mechanism to store
and retrieve AVRO and JSON Schema.
Advantages of Confluent Schema Registry
● It reduces the size of the message because the entire schema does not need to be
embedded and sent or received from the external system/APIs
● Not all systems support embedded data.
Confluent Schema Registry Connector
17. 9
Creating Confluent Cloud Account
Confluent Cloud is a resilient, scalable streaming data service based on Apache Kafka,
which is an event streaming platform used to collect, process, store, and integrate data.
Create a free account at https://www.confluent.io/get-started/
18. 9
Creating Schema Registry
● Select the Region to create the schema registry.
● Click on Add Schema.
● Copy paste you AVRO Schema and add the schema name.
20. 9
● Required AVRO Schema to be registered in Confluent Cloud.
● During conversion corresponding Avro schema is retrieved using schema ID to
convert the binary data to JSON.
JSON to AVRO Conversion
● Retrieve the AVRO schema from confluent and replace with ID to convert JSON to
Binary.
AVRO to JSON Conversion
23. 9
● Medium Blog on Confluent Schema Registry in Mule 4:
https://medium.com/@shruthi_r/confluent-schema-registry-in-mule-4-
9e4b55f495e0
● Github link to code:
https://github.com/Shruthiii960/confluentSchemaRegistry
Resources
29. ● Share:
○ Tweet using the hashtag #MuleSoftMeetups #MuleMeetup
○ Invite your network to join: https://meetups.mulesoft.com/baltimore/
● Feedback:
○ Fill out the survey feedback and suggest topics for upcoming events
○ Contact MuleSoft at meetups@mulesoft.com for ways to improve the program
● Nominate Yourself as Meetup Speaker:
○ Amazing opportunity to public speaking, broadening skills and expanding
network
31
Knowledge Shared is Knowledge
Squared!
