Most data is allocated to a period or to some point in time. We can gain a lot of insight by analysing what happened when. The better the quality and accuracy of our data, the better our predictions can become. Unfortunately the data we have to deal with is often aggregated for example on a monthly basis, but not all months are the same, they may have 28 days, 31 days, have four or five weekends,… It’s made fit to our calendar that was made fit to deal with the earth surrounding the sun, not to please Data Scientists. Dealing with periodical data can be a challenge. Pandas is a powerful framework for working with time series data and can make your life a lot easier. This talks will feature: how to analyse periodical data with pandas read and write data in various formats how to mangle, reshape and pivot gain insights with statsmodels (e.g. seasonality) caveats when working with timed data visualize your data on the fly

1. Time Travel and Time Series Analysis with
pandas & statsmodels
Alexander C. S. Hendorf
@hendorf
PYCON SETTE, Florence, 2016

2. Alexander C. S. Hendorf
CTO Königsweg GmbH
15+ years in software development
Always love data and new ideas
Mannheim mongoDB meet-up Organizer
mongoDB master, certified mongoDB DBA
EuroPython organizer + program chair
speaker mongoDB world NYC, CEBIT,…
Hobbies: see above
@hendorf

3. Pandas
• data analysis library
• simple interface
• making data analysis fast, efficient and easy
• started by Wes McKinney in 2008
• DataFrame object for data manipulation with integrated indexing (R)
• Reshape & pivot data
• Merge & join data
• high-level building block for doing practical, real world data analysis in Python
• database like operations
• import pandas as pd
Statsmodels
is a Python module that allows users to explore data, estimate statistical models,
and perform statistical tests

4. 1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8
9
pd.Series
index
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8
9
pd.DataFrame
1
2
3
4
5
6
7
8
9

5. pandas Q'n'D

6. Year
12 Months
February
90% of March
31 31
31 31 31
31 31
30
30
30 30
28

7. Roman year used to start in March and had 10 months
2 months there was "no" month
solar | topical year
quick & funny explanation: https://www.youtube.com/watch?v=AgKaHTh-_Gs

8. ?

9. use case

11. sales report
27.11.11 -
01.01.2012
5 weeks
4 weeks
4 weeks
typical quarter
November
January
DB
stored only as sold in
majority month

13. 2nd Call for Proposals
reserved for Hot Topics
~first week of June
ADVERTISEMENT

14. Data Mangling with mongoDB
the Right Way
tomorrow@17:15

15. Alexander C. S. Hendorf
@hendorf
LinkedIn