3. Who Am I? Why Should You Listen To Me?!
● Programming since ‘11 (not bragging)
○ Java, Python, JavaScript
● Working w/ MuleSoft products for about 3 years
● Saw no DataWeave experts at my company or in the community, I LOVE learning programming
languages, so decided to focus efforts there
● Started learning DataWeave about the same time Functional Programming started making sense
to me (more on functional programming later)
4. Disclaimer
I don’t work for MuleSoft, and I’ve never seen the source code for DataWeave. Because of this, I have no
way of knowing with 100% certainty how the language works; DataWeave is a black box to me.
What I’m going to cover today is the mental model I’ve built over the last 3 years that represents how I
believe DataWeave works. In other words, what I will describe may not be how DataWeave actually
works, but it will be an effective model for you to use when creating solutions using the language.
5. What is DataWeave?
● Domain-specific Language created specifically to optimize data transformations regardless of
source and target formats
● Fully-fledged programming language
● Adheres to the core tenets of functional programming
6. Why Should You Learn DataWeave?
● Honest answer: If you’re planning to use MuleSoft products in the future or planning to be seen as
an expert, you don’t really have a choice
● Don’t need to concern yourself with parsing and (de)serialization of data
● Less lines of code needed to perform data transformations (Less bugs)
● The programming concepts you must learn to effectively use DataWeave will make you a better
programmer. The goal of this talk is to give you an introductory understanding of these concepts.
7.
8. Why not Java?
● Scope is massive (general purpose programming language)
● Object-Oriented: great for designing large-scale solutions (like the Mule runtime), not great for
designing hundreds of tiny data transformations
● Supports Functional programming (Java 8 Streams), but rather reluctantly.
● Verbose
○ When creating small hundreds of small transformation scripts, you don’t want 50% of that code to be
boilerplate like public class and public static void
12. What is Functional Programming?
● Programming paradigm, like “procedural” or “object-oriented”
● Wikipedia says:
“... a programming paradigm … that treats computation as the evaluation of mathematical
functions and avoids changing-state and mutable data” (emphasis mine)
and
“It is a declarative programming paradigm, which means programming is done with expressions
and declarations instead of statements.” (emphasis mine, again)
13. What is meant by “mathematical functions”:
Pure functions, the same input ALWAYS yields
the same output:
fun add(x, y) = x + y
add(1, 1) // = 2
add(1, 1) // = 2
This is different from methods, which can modify
the state of the object they’re associated with:
Adder one = new Adder(1);
one.add(1) // = 2
adder.setValue(2)
one.add(1) // = 3 ???
METHODS DON’T EXIST IN DATAWEAVE
14. Core Concepts (Language Fundamentals)
“... a programming paradigm … that treats computation as the evaluation of mathematical
functions and avoids changing state and mutable data” (emphasis mine)
“It is a declarative programming paradigm, which means programming is done with expressions
and declarations instead of statements.” (emphasis mine, again)
1. Expressions Instead of Statements
2. Immutable Data
3. Mathematical Pure Functions
15. Everything is an Expression
Most programs are made up of expressions.
● An expression is something that evaluates to a value, e.g, 2 + 2
● A statement is more like a command, and represents an action, e.g., System.out.print(“Hello,
World!);
In other words: Expressions return values, statements do not.
Statements are great for doing things like printing, saving and setting values, and I/O operations.
Expressions are great for doing calculations and transformations
16. Everything is an Expression (cont.)
Java Statements
boolean even = true;
int x = 0;
if(even) {
x = 2;
} else {
x = 1;
}
DataWeave Expressions
var even = true
var x = if (even) 2 else 1
This returns
This does not return
17. Expressions Can Always Be Substituted
Example:
// Returns an array
fun complexStuff(arr) =
...
---
complexStuff(arr) map $.one
Example Pt2 (Complex Stuff is failing and
preventing you from testing the map:
// replace complexStuff with what it
// SHOULD evaluate to
...
---
[{one: 1},{one: 2}] map $.one
USE THIS WHEN YOU’RE DEBUGGING
18. Key Point 🔑
There are no statements in DataWeave, so you can’t perform an action that doesn’t return something
(e.g. simply writing to the console). Every logical grouping of DataWeave code returns something.
Expressions can always be replaced by the value they return. Use this property when you’re debugging!
19. Objects are Immutable, No Changing State
● Once an object is created, it never changes
● Typically, this is an advantage, because you have WAY less to keep track of
● How does any work get done?
21. No, so what’s happening instead?
The payload “variable” is being reassigned, like this:
// Psuedo Code
payload = http.getRequestData()
// New payload created from old payload, old payload reassigned to new payload
payload = transformMessage(payload)
And this happens to the target of every DataWeave transform. If the target is a variable, and the variable is not
assigned yet, it is assigned. If the variable is already assigned, it is reassigned to the output of the DataWeave
script.
23. Key Point 🔑
Always remember that you are never modifying anything when you use DataWeave code. DataWeave
cannot modify values, it can only create new ones. It gives the “illusion” of modification through
reassignment.
24. Functions (cont.)
There is no return keyword (it doesn’t need one, everything is an expression).
Functions return the last expression in the body that is evaluated:
fun add(n, m) = n + m
Isn’t necessarily the same expression every time:
fun addIfOdd(n, m) = if ((n mod 2) == 0) n + m else n - m
25. More Functions
DataWeave Supports Higher-Order Functions
● Fancy way of saying: functions that take other functions as inputs
● You do it all the time with `map`, `filter`, `groupBy`, etc.
payload map (employee) -> employee.id
This is a function passed to the `map` function
26. Understanding Higher-Order Functions
Higher-order functions are very good at separating concerns. Using our map example from earlier… map
takes care of looping through the input array and applying the function to each value. The function
passed to map describes how each element in the input array should be transformed.
payload map (employee) -> employee.id
Takes care of the iteration and applying the function
Describes how each element should be transformed
27. Did I Mention… Functions?
Named functions:
fun greeting(name) = “Hello, $(name)”
Unnamed functions (aka “lambdas”):
(name) -> “Hello, $(name)”
Function assignment is just syntax sugar for lambdas assigned to a variable:
var greeting = (name) -> “Hello, $(name)”
greeting(“Virtual Muleys!”)
28. Did I Mention… Functions? (cont.)
Without Lambdas
fun mapping(obj) =
{name: obj.name, age: obj.age + 1}
---
payload map mapping($)
With Lambdas
payload map (obj) -> {
name: obj.name,
age: obj.age
}
Lambdas make it so it’s really easy to use higher order functions. We don’t need to use as much code.
29. Key Point 🔑
Functions are DataWeave’s most powerful and flexible tool for building transformations. Make sure you
fully understand how they’re different from Java methods (pass functions around w/o classes, assign
them to variables), and how to effectively use them (building higher-order functions, using lambdas).
Higher-order functions are great for separating concerns. Could you imagine if you had to write the code
to loop through an array and modify every value in a specific way every time? With the higher-order
function map, you don’t have to.
31. Settle in for the journey
1. Accept that DataWeave is a complete programming language
2. Accept that DataWeave is a functional programming language, and you might not know anything
about functional programming, yet
3. Accept that there is not a lot of content out there that discusses functional programming
principles as they relate to DataWeave, or how DataWeave is a functional programming language
(except my blog 😉)
32. Functions are King
● Make sure you fully understand what “pure” (i.e. mathematical) functions are, and how they’re
beneficial to this style of programming (transformation)
● Make sure you understand that when you’re using `map`, `filter`, etc, that you’re using higher-
order functions
● Finally, try writing some higher order functions of your own
33. Don’t Give Up!
● The paradigm shift from object-oriented to functional is difficult
● Takes time and deliberate practice
● If it doesn’t work with DataWeave, try practicing with a different language:
○ Scala
○ JavaScript (some parts, search “Functional JavaScript”)
○ Clojure (if you want to get weird)
○ Haskell (if you want to get even weirder)
● Ask questions on the Mule Forums (to get help from everyone besides me), Stack Overflow (to get
help from me and some other people)
● Reach out to me if I can help
34. Additional Info:
Follow us on twitter: @MuleMadeEasy
Learn how to use reduce (good content on HOFs as well): https://www.jerney.io/dataweave-when-to-use-
reduce/
Looking for something more advanced? (pattern matching, recursion): https://www.jerney.io/dataweave-
applywhenkey-function/, https://www.jerney.io/how-to-modify-all-values-of-an-element/
36. Additional Info (Cont.)
Good Intro to Functional Programming: First 6 Chapters of “On Lisp”, by Graham:
http://ep.yimg.com/ty/cdn/paulgraham/onlisp.pdf
Another Good Intro: First 2 Chapters of “Structure and Interpretation of Computer Programs”, by
Abelson, Sussman & Sussman: http://web.mit.edu/alexmv/6.037/sicp.pdf
Something way more advanced: “Functional Programming in Scala” by Chiusana & Bjarnason:
https://github.com/awantik/scala-
programming/blob/master/Manning%20Functional%20Programming%20in%20Scala%20(2015).pdf
More esoteric languages like Clojure, Racket, Scala, Ruby
Saw no DataWeave experts: when I started using Mule dataweave was just coming out, people were skeptical to learn it because they were afraid it would get replaced again like Data Mapper did. After using it for a few months, I thought MuleSoft nailed it, didn’t see why it would ever be replaced. Dove in.
What problem is it solving?
Didn’t find any information regarding the core language concepts behindDataWeave programming, so I found languages that exhibited similar traits, and learned them. If a 9 year old can do it, so can I!
Most of what I learned about how to effectively use DataWeave came from learning the functional concepts that more prominent languages, like JavaScript, Python, Scala, and Clojure.
Once I had a handle on these concepts, I started sharing what I learned with my company, and later, the public.
What problem is it solving?
Might not get it the first time around
Jot down any questions or doubts you have! We will have plenty of time to answer them at the end
Evaluation of mathematica functions vs using statements to call methods
Functions are completely deterministic
An expression is something that always evaluates to a value. Expressions can always be replaced by the value that they evaluate to.
This means everything except variable and function declarations return something in DataWeave. This has a profound affect on the way that you write your code.
Think of logical groupings as any piece of code you could put a set of parenthesis around
Objects in the object oriented sense, not the JavaScript sense
This is great because when you call a function in dataweave you never need to worry about if that function is going to change anything you passed into it.
