Model Drivers: Datapoint modelling and its opportunity 2013-09-01
•Descargar como PPTX, PDF•
1 recomendación•1,606 vistas
The datapoint model offers a solution to the problem of giving data consumers flexibility while enforcing integrity. It breaks down data structures into datapoints, maximizing flexibility and re-use. It also enables end users perspectives of the data to be wire together, giving them structure and simplicity. These slides are from our YouTube presentation.
Recomendados
Recomendados
Más contenido relacionado
Último
Último (20)
Destacado
Destacado (20)
Model Drivers: Datapoint modelling and its opportunity 2013-09-01
- 1. The Datapoint modelling breakthrough and its opportunity 2013-09
- 2. About Model Drivers Business model industrialization. Model driven products and services: • Reporting • Testing • Regulatory conformance • Business strategy and business cases • Business architecture • Systems delivery 2013-09 2 Datapoint modelling and its opportunity
- 3. Agenda A. B. C. D. 2013-09 The problem What is Datapoint Modelling? Steps to exploiting the opportunity of Data Point Modelling Summary and further reading 3 Datapoint modelling and its opportunity
- 4. A: The problem Everyone wants their cake, and to eat it too. There is only one cake. Each data management approach has weaknesses 2013-09 4 Datapoint modelling and its opportunity
- 5. One piece of data, many voices All this must be true: • There can only be one source of truth • It must be concrete and accurate • Many players must view it in their own, very differently ways Look, it … Client data We order that… I need ….. 2013-09 No way can we… I have to … 5 Accounts data I don’t care! Datapoint modelling and its opportunity
- 6. The model-less reporting architecture Business Unit A 2013-09 Business Unit B Business Unit C 6 Business Unit D No one can afford gaps, inconsistency, errors in 100’s of business reports. Not sure which is worse – wrong / late internal reports for decision making Or wrong reports to regulators Datapoint modelling and its opportunity
- 7. Will Logical Data Modelling solve the problem? LDM has • Hard coded concepts and relationships • Single set of opinions of how things are New concepts are big deal Great for transaction processing Hard coded relationships Hard coded attributes of concepts 2013-09 7 Datapoint modelling and its opportunity
- 8. Will Ontology based technologies solve the problem? Mammal Ontologies are • Generic and global • Flexible Is a species of Is a species of Carnivore Great for understanding complex worlds Is in the family of Is in the family of “Concepts “ have relationships: “Is like”, “Is A” 2013-09 Herbivore 8 Rabbit Fox eats Datapoint modelling and its opportunity
- 9. Will Data Warehouse technology solve the problem? Country Year State City Month Day Client facts table Star Schema models • Handle large volumes • Give access via many dimensions 2013-09 9 Branch Department Company Datapoint modelling and its opportunity
- 10. B: What is Data Point Modelling? Datapoint Modelling gives you both concrete data and flexibility The global business reporting language XBRL support DPM Data point model meta model has one dozen main concepts The value of DPM shows in the user experience 2013-09 10 Datapoint modelling and its opportunity
- 11. Data Point Modelling is getting global adoption • Industry leaders are implementing Data Point Modelling for reporting and big data – XBRL International: Abstract Model – EBA : CRD IV Reporting http://www.eba.europa.eu/News-Communications/Year/2013/Update-on-the-technical-standards-onsupervisory-r.aspx – OMG (CWM includes data point modelling) http://en.wikipedia.org/wiki/Common_Warehouse_Metamodel • Frsglobal – [Datapoint models] …will also give the regulatory authorities a tool to address systemic industry-wide issues, something that has been called for time and time again in light of the financial crisis. Furthermore, the regulator will be able to combine information effectively in new and innovative ways. 2013-09 11 Datapoint modelling and its opportunity
- 12. The Data Point Model Daily Sales Company Data Point Models • Every piece of data is unique, identifiable • Every piece of data has a set of dimensions (Aspects) • The aspects can be pulled together dynamically Department Branch DP 1 Country DP 1 DP 1 State City Year Month Day 2013-09 12 Datapoint modelling and its opportunity
- 13. ABOUT XBRL • • • • • eXstensible Business Reporting Language Reports business data in XML Taxonomies define valid forms e.g. SEC Quarterly filing Allows extensions by individual submitters Scope includes: – – – – Accounts Social responsibility Carbon and other emissions Many others • Mandated by – – – – – 2013-09 SEC EU Governments of Australian, Singapore etc China stock exchange Many more 13 Datapoint modelling and its opportunity
- 14. Example implantation: The SEC and regulatory reports Massive volumes of business data Every data point annotated with definition and other meta data 2013-09 14 Datapoint modelling and its opportunity
- 15. Example implementation: Arelle, open source XBRL • Taxonomy review • Basic ability with instance docs JPMorgan instance doc http://www.sec.gov/Archives/edgar/data/19617/000001961713000221/jpm-20121231.xml 2013-09 15 Datapoint modelling and its opportunity
- 16. Data point model: the business report meta model has dozen main concepts Cube Business class Table Aspect Axis Value Set Resource Axis coord. Value Resource link Data point 2013-09 16 Datapoint modelling and its opportunity
- 17. Abstract / Meta model of a report We get graphical, automated, logical report design 2013-09 17 And mapping to the business domain Datapoint modelling and its opportunity
- 18. C: Steps to exploiting the opportunity of DPM 1) 2) 3) 4) 2013-09 Adopt the mental model Architect your understanding of your data Given users both concrete and flexible data with PDM Industrialize your reporting process 18 Datapoint modelling and its opportunity
- 19. 1) Adopt the mental model • There are a number of data architecting approaches, each with strengths and weaknesses • The Data Point model supports both concrete data definitions and multiple view points • DMP therefor is ideal for supporting reporting and analytic type functions 2013-09 19 we are gearing up for a shift to polyglot persistence [1] where any decent sized enterprise will have a variety of different data storage technologies for different kinds of data. http://www.martinfow ler.com/bliki/PolyglotPersist ence.html Datapoint modelling and its opportunity
- 20. 2) Architect your understanding of your data RDF, Sparql Ontology modellDR Arelle, EDGAR Business domain LDM Business Domain Data Data point model 2013-09 Extract & format Data warehouse Oracle, NoSQL View, Publish, Analyse Taxonomy design Manage Business Objects, Cognos 20 Submission Consumer (Regulator, manager, analyst..) Datapoint modelling and its opportunity
- 21. 3) Given users both concrete and flexible data with PDM Automated Taxonomy Still un-happy, but with someone else Environment & technology can be different – the taxonomy has to be the same 2013-09 21 Datapoint modelling and its opportunity
- 22. 4) Industrialize your reporting process Implement the big ideas of Reporting Industrialization • Big idea 1: Industrialise the understanding of your data • Big idea 2: Industrialise your reporting and analytics ecosystem • Big idea 3: Map your business data understanding to your reporting and analytics requirements • Big idea 4: Drive reporting and analytics through automation and tooling See the presentation “How to industrialize business reporting” 2013-09 22 Datapoint modelling and its opportunity
- 23. D: Summary and further reading Data point modelling has strengths in reporting and analytics It supports industrialization of the reporting process 2013-09 23 Datapoint modelling and its opportunity
- 24. D: Next steps Contact me http://uk.linkedin.com/in/gregsoulsby/ www.modeldrivers.us News on twitter https://twitter.com/model_dr This slide deck http://www.slideshare.net/greg.soulsby/ Next presentation: Building data point models To be updated here. Follow me on twitter https://twitter.com/model_dr EDGAR – SEC online view of submitted http://www.sec.gov/edgar.shtml Arelle – Open source XBRL software report http://arelle.org/ XBRL Abstract model specification http://www.xbrl.org/Specification/abstractmodel-primary/PWD-201206-06/abstractmodel-primary-pwd-2012-06-06.html DATA POINT MODEL presentation, Ignacio Santos, Bank of Spain http://www.openfiling.info/wpcontent/upLoads/data/DPMvsMDM_1.pdf EBA’s Data Point Model: A reporting game changer for management information http://www.frsglobal.com/news_and_events/ebas-data-point-modela-reporting-game-changer-for-management-information.html EBA: Implementing Technical Standard (ITS) on Supervisory Reporting (Data Point Model) http://www.eba.europa.eu/regulation-and-policy/supervisoryreporting/implementing-technical-standard-on-supervisory-reportingdata-point-model-/-/regulatory-activity/consultation-paper The Data Point Model methodology in the European Supervision: COREP/FINREP 2013-09 24 http://www.eurofiling.info/documents/DataPointModelMethodologyI B.pdf Datapoint modelling and its opportunity
Notas del editor
- Hello, my name is Greg Soulsby, welcome to my presentation on datapoint modelling. I will introduce you to my mental model of datapoint modelling, with the hope that you will get the same sense of excitement that I do for the opportunity that brings.
- I am thinking the problems of information management will be well known to you, since you are listening to a conversation on datapoints. It is just that I want to contrast and compare datapoint modelling with the other information architecture tools, in terms of strengths and weakness in solving these problems. I will give an idiosyncratic metal model of datapoint modelling. As far as I know there is no formal definition, so this will be mine.Then lets explore how this can be helpful to you and or your enterprise to tackle some of these problems.At the end is a set of further resources.
- Everyone wants their cake, and to eat it too. There is only one cake.Since you are engaging in a conversation on datapoint modelling these will be familiar to you.
- Unlike, say, the Arts, in business data there can be only version of the truth.But there can, is, and should be many uses and interpretations of that. Client billing people say “Clients must have a name”. So the client on boarding people make that mandatory. Then the marketing department want to send the customer sales to the ad agency, and no way can the client name be passed over. So client is one data concept, with 2 directly opposite requirements. Obviously a trite example but that problem exists in thousands of cases, and there is no economy of scale, the problems compound and you get a bigger and broader organization.
- An example of an implication of multiple view points is in business reporting and analysis.If it was only a matter of duplication of processes and systems that would not be so bad. The real costs is in the manual, error prone, and impossible data integration. So lets look at non datapoint modelling approach to data management.
- Firstly, Logical Data Modelling. This is a large part of my career, so I a massive fan of understanding your data in as structured, concrete way as possible. But, the more concrete the harder people find to slice and dice for their own purposes. They find is hard to be agile. My experience is that the main problem is the absolutely appalling quality of their designs. But even with optimum design people still need to flexibly assemble views of the data, and an LDM is not a tool for that.
- The next approach people is, for want of a better word, Ontology based technology. The semantic web is an example – pull together data using an infinitely flexibly powerful language. But to me, we have gone too far – You can say “Foxes eat Rabbits” in your ontology, great. You can also say “Freddie Fox ate Peter Rabbit”, something more concrete. But if you Fox is a Credit Card and the Rabbit is a purchase, MasterCard cant afford flexibility in a global system like that – it has to be very concrete. But they still want analysis and slicing and dicing when they are data mining for fraud, for example.So we could also look at Data Warehouse approach.
- These Star schemas type approaches are optimised for slicing and dicing data. There are facts and they can be accessed and summarised by dimensions, like geography or time or sales channel.Of course they are massively large. And they are hard coded to their original design – you can just think of a new access path on the fly – you have to go through IT to get that implemented. And adding more data can also be anything but agile. So all these approaches have their strengths and weaknesses. As does Datapoint modelling. So the Datapoint approach is an augmentation, another tool in the kit back, and all these technology live side by side.
- Datapoint modelling shares many elements of the other approaches, they are all part of the same family, tackling the problem of data management. Datapoint modelling is related to the NoSQL movement in technology, which for me a backing away from the Oracle / relational database approach – losing the constraints which were put into Oracle etc to guarantee data integrity and passing the responsibility back the developer.But is more concrete the Ontolotgy kinds of thinking, so there are is no loser concepts like “this is like that”.
- The recent uptake of Datapoint modelling owes as lot to the GolobalFinanical Crisis of late. You can read the quote there from FrsGlobal who are a player in the reguatory space of the finance industry.I will am going to here use the XBRL Abstract Model as my Datapoint model example. I think Dave Francle and Herm Fischer and others on that team have done a fantastic job. And besides that every bank and insutrance company in the world is being fog marhing down the XBRL path, so it has real world relevance as well.A short comment on XBRL- eXstensible Business Reporting Language- Reports business data in XML- Taxonomies define valid forms e.g. SEC Quarterly filingAllows extensions by individual submittersScope includes: AccountsSocial responsibilityCarbon and other emissionsEtcMandated bySECEUGovernments of Australian, Singapore etcChina stock exchangeMany moreIn short, public company data is or will be in XBRL, globally.
- So in a datapoint model we have a few features to call out:Every piece of controlled, for want of a better work, data is a datapoint. 2) Every data point is uniquely defined and defined only once3) Every datapoint is triangulated by any number of dimensions or aspects.So you can see features shared with other approaches.
- To demonstrate the implications of this, lets look at EDGAR, the SEC system for publishing their reports. Search of JPMC ticker, filter on 10-K, an annual SEC return. 2) Select an instance document using the blue button. See how the information is highly structured, using the directory structure on the left.By clicking on a data item you get the definition and other data. So it is all highly structured, well documented and consistent with every other X-10 filing. But it gets better. Any firm can submit their return with extensions – they can also send in data that is special, differentiating facts about themselves. Go back, then into the actual sumitted files. At the bottom you will see “exteantions.xml”. Open this up to see JPMogan providing extra detail, specifical only to them. Still integrated, but different.I am still in awe of the volume of data you have at your fingertips here – every filing for every company in the US, in a well structured form, annotated with meaningSo we are seeing volume, standardisation and flexibility. So lets dive a bit deeper to see how datapoint modelling has contributed to this. We will do that using Arelle, open source XBRL tool.
- After opening Arelle, we can navigate to a similar JPMorgan document and look under the hood.We can see Facts table – a “simple” table of the reported factsPresentation: How the information is viewedCalculations: The calculations and derivations are definedDimension: How you can slide and diceIs it just me, or does this look horrendously complex? The front end looks great, but the detail of how that is done it incredible. Firstly, what you are seeing is based on a huge amount of fantastic work done by standards bodies, government departments and incredibly, volunteers. The time and effort to get all these details agreed to by so many people around world is un seen and un thanked, but incredible and inspiring.Secondly, simplicity is genius. So it takes some kind of genius to find any kind of simplicity here, you will agree. And that genius is the Datapoint model.
- So with apologies to all the people who thanklessly put in the hard work, this is my mental model of XBRL Abstract Model. With this relatively simple toolkit or point by numbers approach you can build the worlds business big data solution.The concept of a data point has a few main impacts:Firstly, Datapoints have values. Actually they have any number of values. First there is the value value –100 wigets were sold in August by the Delaware Branch. So 100 is a value of the datapoint, but there are also values for Widgets, August and Delaware Branch. Values come in Sets, like all the Months of the Year, all the Products and all the Branches. Of course the values in a set are of the same type, so that is the Aspect concept, like we saw in the Datawarehouse model, if you like. Secondly, we have to have an opinion on how we want the data presented, and for that we need tables, which have axis and the axis coordinates triangulate onto the datapoints.Other information about the datapoint might be the definition, the label, the source, could be anything. This meta data is represented by there Resources concepts. The reports obviously has to relate to the real world, and I have called this the Business Classes. This is not strictly part the XBRL Abstract model, it is something I need to be able to talk about building reports in the real world.The Cube concept is the way to express partitioning data into buckets. So that is rather abstract, deliberately so. How we be relate this to the real world?
- We have seen how Datapoint modelling is a design tool in the armoury of the data architect, and how it relates to others, and what a solution looks like.To put datapoint modelling into a context, lets look at an end to end process.
- You cant develop datapoint models without a concrete understanding of your business domainHere I have drawn a flow from business data, from transactional management domain to the consumption by, say, a manager, regulator or analyst. It shows both - the function in the process - and examples of the supporting technology or toolsA)We have an ontology here for high level, over arching concepts.B) Typically Oracle will be the technology for concrete transaction processing. C) People use a lot of Oracle for their data warehouse, with noSQL technologies are contributing here as well.D) Reporting and analtics have Business Objects as an example technology.E) For business domain modelling we have tools like MagicDraw and Enterprise Architect. F) I am developing a solution in this space, modelDR. That supports datapoint and XBRL design. G) And Arelle and EDGAR are examples of technology for managing and viewing XBRL data and taxonomiesThis is a highly subjective, simplified view, of course.But it does show the sweet spot for Datapoint modelling - between A) the concrete, transactional world of Oracle, and B) the flexible, generic world of Ontology's and less, or no, structured data with less or no rules.So in the last few minutes, lets look at how we might manage the relationship between these .