The document discusses Power Query M, which is a powerful query language optimized for building data mashup queries. It is a functional, case-sensitive language. Power Query enables users to access and reshape data from hundreds of sources into a no-code experience. Some key data sources it can connect to include Excel, SQL databases, web pages and more. The document also provides an example of an Power Query M query that accesses an Excel file, promotes headers, cleans column names, unpivots columns and pivots data to transform it into a clean table.

2. Power Query M?
(Informally known as "M")
The Power Query Formula Language is a powerful query
language optimized for building queries that mashup data. It's a
functional, case sensitive language.
For computer language theorists: Power Query is a mostly pure,
higher-order, dynamically typed, partially lazy, functional
language. You can create a variety of data mashup queries from
simple to advanced scenarios.
(Source: MSDN)

3. Power Query?!
(Data Connectivity and Preparation)
Power Query enables business users to seamlessly access data stored in
hundreds of data sources and reshape it to fit their needs, with an easy to
use, engaging and no-code user experience.
Data Connectors:
(Source: Office Support)
Access Database2 Hadoop File (HDFS) Microsoft Azure Table Storage PostgreSQL Database Text
Active Directory IBM DB2 Database Microsoft Exchange SharePoint List CSV
Excel Microsoft Azure HDInsight MySQL Database SQL Server Database Web Page
Facebook Microsoft Azure Marketplace OData Feed Sybase Database XML
Folder Microsoft Azure SQL Database Oracle Database Teradata Database And More…

4. “M”
POWER QUERY
FOR POWER BI

5. Alex M PowersPremier Field Engineer – Data & AI @ Microsoft
Co-Organizer St Louis Power BI User Group #STLPBIUG
Professional:
• Microsoft Certified Solutions Expert: Data Management and Analytics
• Microsoft Certified Solutions Associate: BI Reporting (Excel, Power BI)
• Microsoft Certified Associate: Azure AI Engineer
• Microsoft Certified Trainer
• Microsoft Technology Associate: Python
• Microsoft Office Expert & Specialist: Excel, Access
• Microsoft #HowWeExcel Contest Winner
Personal:
• Met the Wife on Craigslist
• Enjoys Pineapple on his Pizza

6. Click to edit
master title
style
SESSION
OBJECTIVES
• Convince you why
Power Query (M)
should be on every
resume.
• Have fun.

7. MAKE IT FEEL FAMILIAR.
BEFORE IT FEELS FOREIGN.

8. Click to edit
master title
style
PRIMITIVE
VALUES
Type Example
binary #binary("AQID")
date #date(2018,7,10)
time #time(18,30,0)
datetime #datetime(2018,7,10,18,30,0)
datetimezone #datetimezone(2018,7,10,18,30,0,0,6)
duration #duration(1,0,0,0)
logical true and false
null null
number 0, 1, -1, 1.5
text “hello, world”

9. Click to edit
master title
style
STRUCTURED
VALUES
Type Example
list {“foo”, “bar”, 10}
record [a = 10, b = 20]
table #table({“Animal”}, {{“Cat”},{“Dog”}})

10. let
Source = “Hello, World!”
in
Source
Power Query M’s let expression stores a set of named values called variables. Each variable has a unique
name called an identifier.
In the above let expression – the variable Source stores the primitive text value “Hello, World!”. The final
expression is evaluated in the identifier titled Source.
let
in
let
Source =
in
let
Source = “Hello, World!”
in

11. let
Source = "Hello, World",
#"Something Else" = "Keepin' It",
Val = 100,
#"Struct List" = {Source, #"Something Else", Val},
#"List – Index" = #"Struct List"{2}, // 0 Based Index
#"Numbered List - 0 to 100" = {0..#"List – Index"},
#"Table With Meta" = Table.FromList(#"Numbered List - 0 to 100", Splitter.SplitByNothing(), type table [Index = Int64.Type], null, ExtraValues.Error),
#"Each Loop with ASCII Number" = Table.AddColumn(#"Table With Meta", "ASCII", each Character.FromNumber([Index]), type text),
#"ASCII Column and Row" = #"Each Loop with ASCII Number"[ASCII]{97},
#"Alphabetical List - a to z" = {#"ASCII Column and Row".."z"}
in
#"Alphabetical List - a to z"
In the above let expression – the identifiers enclosed in #” “, denote the use of a special character (space) or
number for the variables identifier. Starting from the beginning of the expression we begin with primitive
values, introduce lists { }, positional index operator {2}, list sequences for numbers {0..100}, for each list
item additional column transformations, lookup and positional index operators [Column1]{2} and the use of
alphabetical list sequences {“a”..”z”}.
Full Solution: Familiar Feeling

12. To view the available Power Query M functions use the formula =#shared to view as structured records or
=Record.ToTable(#shared) to cast the records into a structured table format.
=#shared

13. WE CAN REBUILD IT.
WE HAVE THE TECHNOLOGY.
BETTER THAN IT WAS BEFORE.
BETTER. STRONGER. FASTER.
THE SIX MILLION DOLLAR QUERY.

14. let
Source = Excel.Workbook(Web.Contents("https://www.ers.usda.gov/webdocs/DataFiles/48747/Unemployment.xls"), null, true),
#"Unemployment Med HH Inc_Sheet" = Source{[Name="Unemployment Med HH Inc"]}[Data],
// Extract the previous identifier's Column1 as a list. Find the position of the text "FIPStxt", its first occurence while ignoring case sensitivity
#"Find Position" = Table.Skip(
#"Unemployment Med HH Inc_Sheet",
List.PositionOf(
#"Unemployment Med HH Inc_Sheet"[Column1], "FIPStxt", 1, Comparer.OrdinalIgnoreCase
)
),
#"Promoted Headers" = Table.PromoteHeaders(#"Find Position", [PromoteAllScalars=true]),
/*
Clean each item in Column Names.
The use of a underscore avoids the hardcoding of individual column headers and instead transforms each item in the collection
We used a Replacer to ReplaceText of underscores to spaces in our headers, this function is wrapped with a Text.Proper to standardize all column headers
*/
#"Clean Headers" = Table.TransformColumnNames(
#"Promoted Headers",
each
Text.Proper(
Replacer.ReplaceText( _ , "_", " ")
)
),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Clean Headers", {"Fipstxt", "State", "Area Name"}, "Attribute", "Value"),
// Rename the auto generated Attribute.2 Column Header to Year. This step avoids and unnnecessary Table.RenameColumns function
#"Split Column by Delimiter" = Table.SplitColumn(#"Unpivoted Other Columns", "Attribute", Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.Csv, true), {"Attribute.1", "Year"}),
#"Pivoted Column" = Table.Pivot(#"Split Column by Delimiter", List.Distinct(#"Split Column by Delimiter"[Attribute.1]), "Attribute.1", "Value", List.Count),
#"Changed Type" = Table.TransformColumnTypes(#"Pivoted Column",{{"Fipstxt", type text}, {"State", type text}, {"Area Name", type text}, {"Year", Int64.Type}, {"Civilian Labor Force", Int64.Type}, {"Employed", Int64.Type}, {"Unemployed", Int64.Type},
{"Unemployment Rate", Int64.Type}, {"Median Household Income", Int64.Type}, {"Med Hh Income Percent Of State Total", Int64.Type}, {"Rural Urban Continuum Code", Int64.Type}, {"Urban Influence Code", Int64.Type}, {"Metro", Int64.Type}})
in
#"Changed Type"
In the above let expression – we introduce future proofing the header row indicator of "FIPStxt“ in our
dataset and ignoring case sensitivity. Transforming our column names with an each loop we utilize the
underscore character to iterate thru each item in our column headers collection.
Full Solution: Six Million Dollar Query

15. NoSQLNO. NOT THAT KIND.

16. Query Folding
One of the most powerful capabilities of Power Query and the M Language is Query Folding
(also referred to as query delegation, and predicate push-down). Query Folding allows the M
Engine to push the transformations expressed in an M query to the source, in the source's
native query language, resulting in more efficient data processing.
Data sources will support different levels of query capabilities. To provide a consistent data
transformation experience, the M engine compensates (i.e. does the processing locally) for
transformations that cannot be sent to the source. It is the Data Connector's responsibility to
report its capabilities to the engine, carving off the transformations it can handle, generating
the appropriate query syntax, and letting the M Engine handle the remaining work.
(Source: Microsoft Docs)

17. Full Solution: Query Folding PowerPlatformSummit
Custom Column Name Transition Function: fxColumnNamesTransition

18. ROADMAPS?...
WHERE WE’RE GOING,
WE DON’T NEED ROADMAPS.
THE FUTURE IS OURS TO WRITE.

19. IF YOU WANT YOUR SERVICE TO BE A HIT.
MAKE SURE IT WORKS WITH EXCEL.
Oct-11 SQL Azure Labs
Feb-13 Excel Add-In
Jul-13 Excel (Natively)
Dec-14 Power BI
Apr-17 Azure Analysis
Services
Mar-18 Flow
Aug-17 PowerApps
Nov-18 Dataflows
Feb-19 SSIS
May-19 Azure Data Factory
May-19 Excel (Mac)

20. FLOW

21. PowerApps

22. Azure Analysis Services - 1400

23. Power BI service

24. VISUAL STUDIO

25. THE FUTURE
M-DBA (Highly Specialized in Writing Optimized M Syntax and Building Custom Connectors)
Intellisense (DONE)
Seamless Integration Across All Offerings (Working Toward a Unified Experience)
SSIS and Azure Data Factory Integration (DONE… or in Preview)
Microsoft Access (WHY NOT?)
Excel for Mac (DONE)
Excel Online (Crystal Ball Prediction)
Version Control (GitHub)
Standalone Product (WHY NOT?)

26. RESOURCES
Power Query Language Specification Guide
https://docs.microsoft.com/en-us/powerquery-m/power-query-m-language-specification
Collect, Combine, and Transform Data Using Power Query in Excel and Power BI
https://www.amazon.com/Collect-Combine-Transform-Business-Skills/dp/1509307958
M Is for (Data) Monkey: A Guide to the M Language in Excel Power Query
https://www.amazon.com/Data-Monkey-Guide-Language-Excel/dp/1615470344/
Excel 2016: Get & Transform Data (Power Query)
https://www.lynda.com/Excel-tutorials/About-Get-Transform/608994/645967-4.html

27. CLOSING REMARKS
IT’S NOT ABOUT WHAT YOU KNOW.
IT’S ABOUT WHAT YOU KNOW IS POSSIBLE.

28. QUESTIONS?
SUGGESTIONS
AM I CRAZY?!

29. Click to edit
master title
style
Thank you for
attending!
Please complete your
survey

30. Alex Power[s] BI
@notaboutthecell
/in/alexmpowers