Database Refactoring With Liquibase
•
13 recomendaciones•3,721 vistas
Let's bring the best of Agile thinking from software development and apply it to our databases as well. Liquibase is a great tool to help.
Recomendados
Recomendados
Más contenido relacionado
Destacado
Destacado (20)
Similar a Database Refactoring With Liquibase
Similar a Database Refactoring With Liquibase (15)
Más de Tim Berglund
Más de Tim Berglund (9)
Último
Último (20)
Database Refactoring With Liquibase
- 1. DATABASE REFACTORING WITH Tim Berglund March, 2009 Monday, March 16, 2009
- 2. YOU HAVE A DATABASE IN YOUR LIFE Monday, March 16, 2009
- 3. YOUR DATABASE HAS A FUTURE Monday, March 16, 2009
- 4. CAN IT ADAPT? Monday, March 16, 2009
- 5. HAS IT BEEN EMBALMED? Monday, March 16, 2009
- 6. Monday, March 16, 2009
- 7. WON’T DIE Monday, March 16, 2009
- 8. NOT REALLY ALIVE Monday, March 16, 2009
- 9. CURSES THE DBA WHO CHANGES IT Monday, March 16, 2009
- 10. WE WILL BREAK THE CURSE Monday, March 16, 2009
- 11. EVOLUTIONARY DATABASE DEVELOPMENT Monday, March 16, 2009
- 12. EVOLUTIONARY DATABASE DEVELOPMENT Monday, March 16, 2009
- 13. EVOLUTIONARY DATABASE DEVELOPMENT REFACTORING Monday, March 16, 2009
- 14. EVOLUTIONARY DATABASE DEVELOPMENT REFACTORING EVOLUTIONARY DATA MODELING Monday, March 16, 2009
- 15. EVOLUTIONARY DATABASE DEVELOPMENT REFACTORING EVOLUTIONARY DATA MODELING AUTOMATED TESTS Monday, March 16, 2009
- 16. EVOLUTIONARY DATABASE DEVELOPMENT REFACTORING EVOLUTIONARY DATA MODELING AUTOMATED TESTS SOURCE CONTROL Monday, March 16, 2009
- 17. EVOLUTIONARY DATABASE DEVELOPMENT REFACTORING EVOLUTIONARY DATA MODELING AUTOMATED TESTS SOURCE CONTROL SANDBOXES Monday, March 16, 2009
- 18. REFACTORING Monday, March 16, 2009
- 19. REFACTORING Monday, March 16, 2009
- 20. REFACTORING A disciplined way to make small changes to your source code to improve its design, making it easier to work with. Monday, March 16, 2009
- 21. REFACTORING Not adding features! Monday, March 16, 2009
- 22. REFACTORING Not fixing bugs! Monday, March 16, 2009
- 23. REFACTORING Opportunistically improving design to accommodate change. Monday, March 16, 2009
- 24. DATABASE REFACTORING A simple change to a database schema that improves its design while retaining both its behavioral and informational semantics. Monday, March 16, 2009
- 25. EVOLUTIONARY DATA MODELING Monday, March 16, 2009
- 26. EVOLUTIONARY DATA MODELING NOT TRYING TO “GET IT RIGHT UP FRONT.” Monday, March 16, 2009
- 27. EVOLUTIONARY DATA MODELING MAYBE SOME BIG THOUGHTS AT FIRST Monday, March 16, 2009
- 28. EVOLUTIONARY DATA MODELING BUILD THE SIMPLEST THING THAT CAN POSSIBLY WORK...EVERY DAY Monday, March 16, 2009
- 29. DATABASE TESTS TDD IN THE DB Monday, March 16, 2009
- 30. DATABASE TESTS Monday, March 16, 2009
- 31. DATABASE TESTS STORED PROCEDURES Monday, March 16, 2009
- 32. DATABASE TESTS STORED PROCEDURES SQLUNIT Monday, March 16, 2009
- 33. DATABASE TESTS STORED PROCEDURES SQLUNIT DBUNIT Monday, March 16, 2009
- 34. DATABASE TESTS STORED PROCEDURES SQLUNIT DBUNIT CUSTOM HYBRIDS Monday, March 16, 2009
- 35. SOURCE CONTROL Monday, March 16, 2009
- 36. SOURCE CONTROL LIKE MOTHER TAUGHT YOU Monday, March 16, 2009
- 37. SOURCE CONTROL Monday, March 16, 2009
- 38. SOURCE CONTROL DDL EXTRACT/MIGRATION SCRIPTS REFERENCE DATA STORED PROCS/TRIGGERS Monday, March 16, 2009
- 39. SANDBOXES Monday, March 16, 2009
- 40. Monday, March 16, 2009
- 41. Monday, March 16, 2009
- 42. CHANGELOG Monday, March 16, 2009
- 43. CHANGELOG COMMANDS Monday, March 16, 2009
- 44. CHANGELOG COMMANDS MIGRATION SCENARIOS Monday, March 16, 2009
- 45. CHANGELOG Monday, March 16, 2009
- 46. CHANGELOG SQL IN XML Monday, March 16, 2009
- 47. CHANGELOG SQL IN XML ONE OPERATION AT A TIME Monday, March 16, 2009
- 48. CHANGELOG SQL IN XML ONE OPERATION AT A TIME TRACKED IN DATABASE Monday, March 16, 2009
- 49. CHANGELOG <changeSet author=quot;tlberglund (generated)quot; id=quot;1236660747556-11quot;> <createTable tableName=quot;scshoppingcartquot;> <column autoIncrement=quot;truequot; name=quot;idquot; type=quot;INTquot;> <constraints nullable=quot;falsequot;/> </column> <column name=quot;useridquot; type=quot;INTquot;> <constraints nullable=quot;falsequot;/> </column> <column name=quot;lastmodifiedquot; type=quot;DATETIMEquot;/> <column name=quot;namequot; type=quot;VARCHAR(50)quot;/> </createTable> </changeSet> <changeSet author=quot;tlberglundquot; id=quot;spot-1-1-2quot;> <renameTable oldTableName=quot;scscrubrecordwarehousequot; newTableName=quot;scrub_record_logquot; /> </changeSet> Monday, March 16, 2009
- 50. LIQUIBASE COMMANDS Monday, March 16, 2009
- 51. LIQUIBASE COMMANDS generateChangeLog Monday, March 16, 2009
- 52. LIQUIBASE COMMANDS changeLogSync Monday, March 16, 2009
- 53. LIQUIBASE COMMANDS update Monday, March 16, 2009
- 54. LIQUIBASE COMMANDS tag Monday, March 16, 2009
- 55. LIQUIBASE COMMANDS rollback Monday, March 16, 2009
- 56. LIQUIBASE COMMANDS diff Monday, March 16, 2009
- 57. DEMO Monday, March 16, 2009
- 58. AND WE’RE BACK Monday, March 16, 2009
- 59. CULTIVATING THE BROWNFIELD Monday, March 16, 2009
- 60. LEGACY ENTERPRISE APP Monday, March 16, 2009
- 61. NEW GRAILS HOTNESS Monday, March 16, 2009
- 62. HARD CUTOVER Monday, March 16, 2009
- 63. CREATIVE ORM MAPPINGS Monday, March 16, 2009
- 64. WITHERING ON THE VINE Monday, March 16, 2009
- 65. BROWNFIELD DATA INTEGRATION Monday, March 16, 2009
- 66. BROWNFIELD DATA INTEGRATION TIME ZERO MIGRATION Monday, March 16, 2009
- 67. BROWNFIELD DATA INTEGRATION TIME ZERO MIGRATION Monday, March 16, 2009
- 68. BROWNFIELD TESTING Monday, March 16, 2009
- 69. BROWNFIELD TESTING INTEGRATION TEST DATA Monday, March 16, 2009
- 70. BROWNFIELD TESTING Monday, March 16, 2009
- 71. BROWNFIELD TESTING T0 MIGRATION SCRIPTS Monday, March 16, 2009
- 72. BROWNFIELD TESTING Monday, March 16, 2009
- 73. THE CURSE IS BROKEN Monday, March 16, 2009
- 74. THANK YOU! TIM BERGLUND AUGUST TECHNOLOGY GROUP, LLC http://www.augusttechgroup.com tim.berglund@augusttechgroup.com @tlberglund Monday, March 16, 2009
- 75. PHOTO CREDITS DODO: HTTP://WWW.INTERNATIONALDOVESOCIETY.COM/MISCSPECIES/DODO.HTM EMBALMING: HTTP://LIBRARY.THINKQUEST.ORG/C0116982/HTML%20PAGE%20FOLDER/ HMUMMIFICATION.HTM ATTACKING MUMMY: HTTP://WWW.FLICKR.COM/PHOTOS/ROONBABOON/292393932/ CREEPY MUMMY HEAD: HTTP://WWW.FLICKR.COM/PHOTOS/DR-INI/446311713/ MARTIN FOWLER: HTTP://WWW.FLICKR.COM/PHOTOS/PRAGDAVE/173640462/ SCOTT AMBLER: HTTP://WWW.AMBYSOFT.COM/SCOTTAMBLER.HTML BRENDEN FRASER: HTTP://SCRAPETV.COM/NEWS/NEWS%20PAGES/ENTERTAINMENT/UNIVERSAL-FIRE- ACCIDENTAL-NEW-MUMMY-MOVIE-DELIBERATE-SCRAPE-TV-THE-WORLD-ON-YOUR-SIDE.HTML SAND: HTTP://WWW.FLICKR.COM/PHOTOS/LEVIATHOR/207625319/ MOTHER AND CHILD: HTTP://WWW.FLICKR.COM/PHOTOS/PATRICK_Q/268149208/ BROWN FIELD: HTTP://WWW.FLICKR.COM/PHOTOS/ARTUR02/471094378/ JALOPY: HTTP://WWW.FLICKR.COM/PHOTOS/DRAGONFLEYE/3246153/ JALOPY ON RT 66: HTTP://WWW.FLICKR.COM/PHOTOS/EVOETSCH/2436023218/ PIMPMOBILE: HTTP://WWW.FLICKR.COM/PHOTOS/AIRGAP/1053594933/ JUNKED JALOPY: HTTP://WWW.FLICKR.COM/PHOTOS/JABOOBIE/61014658/ PAINTING CAR: HTTP://WWW.FLICKR.COM/PHOTOS/GARETHJMSAUNDERS/2066690016/ WITHERED TOMATOES: HTTP://WWW.FLICKR.COM/PHOTOS/SPACESUITCATALYST/349778837/ Monday, March 16, 2009
Notas del editor
- The overwhelming likelihood is that you’re working on some kind of software that interacts with a relational database.
- It’s going to be there, it’s going to be doing something, and you’re going to have a relationship with it. The question is, will the relationship be a good one for both of you, or bad?
- Will it die? It is said of evolutionary systems that things that fail to adapt to change die out. If only this were the case with the enterprise database...
- Rather than going extinct, the production DBAs surround it like priests, carefully filling it with embalming fluid and wrapping it in linen strips. (And possibly putting some operations personnel in the tomb to serve the database in the afterlife.)
- Inability to change and overprotection lead us to this dread antipattern.
- There are long-lived Enterprise applications that rely on it, so it can’t just go away.
- It isn’t really alive either, because business needs are constantly changing, but the database can’t change with them.
- At least the production DBAs seem to think so!
- We’ll figure out how to manage changes to the database such that we can make them with confidence in a way that brings the best of developer tools to bear and is relatively friendly to DBA workflows. We’ll do this with a mindset and a tool.
- The fix is what Scott Ambler calls “evolutionary database development.” This consists of five components.
- Let’s explore each of these in turn.
- Let’s explore each of these in turn.
- Let’s explore each of these in turn.
- Let’s explore each of these in turn.
- Let’s explore each of these in turn.
- Martin sez...
- Martin sez...
- Scott sez.... Code refactorings are really only concerned about behavior. DBs have behavior (stored procs, triggers, etc.) but also information. The database must say the same thing in the same way after the refactoring.
- These are two mistakes here: one is thinking we’re smart enough to do all this designing correctly at the outset—we’re not. The other is thinking that the database’s business context is a static thing—it isn’t. There is no “right” up front, because requirements will change constantly.
- Which doesn’t mean we can’t do a couple of days of designing at first; we can. It makes sense to try to anticipate what we can and make big, hard-to-change commitments correctly. We always expect change, though.
- TDD adoption in software development is low enough, but it is virtually unheard of, as far as I know, in database development. The tooling lags behind and the expertise is singularly rare.
- There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.
- There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.
- There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.
- There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.
- Bring all the knowledge, practices, and advantages of software source control to the database. Simply control all those text files in SVN or Git like you normally would. This practice is old hat.
- Every database artifact goes in to the repository.
- Developers need a place to deploy refactorings when they’re trying to get their tests to pass. This must be a local database not used by any other team member or system.
- If we learn how to refactor databases from Ambler and Sandalage, Liquibase is the tool that makes it easy. It is an XML-based love poem to Scott Ambler.
- We’ll consider four aspects of Liquibase. How it stores the schema, how it interacts with the database, and how to use it in some real-world scenarios.
- We’ll consider four aspects of Liquibase. How it stores the schema, how it interacts with the database, and how to use it in some real-world scenarios.
- We’ll consider four aspects of Liquibase. How it stores the schema, how it interacts with the database, and how to use it in some real-world scenarios.
- Must we rewrite our SQL in XML? We must. It’s painful and unappetizing, but worth it! Also, there’s a Grails plugin called AutoBase that lets us do it in nice Groovy Builder syntax, which is preferable.
- Must we rewrite our SQL in XML? We must. It’s painful and unappetizing, but worth it! Also, there’s a Grails plugin called AutoBase that lets us do it in nice Groovy Builder syntax, which is preferable.
- Must we rewrite our SQL in XML? We must. It’s painful and unappetizing, but worth it! Also, there’s a Grails plugin called AutoBase that lets us do it in nice Groovy Builder syntax, which is preferable.
- The changelog is a script that builds your schema one DDL statement at a time. Each changeSet is converted into a dialect-specific SQL statement and is executed in the database, then marked as complete in a log table.
- Extracts the database’s metadata and generates a changelog. This is the first step is getting started on a database that already exists.
- Since Liquibase tracks the status of all changesets in the changelog, we’ll need to tell it that our newly extracted changelog is in sync with the actual database. This command usually shouldn’t be executed except as the second step getting started with the tool.
- Plays any new changeSets in the changeLog against the database. This is the statement you’ll use the most during evolutionary database development with Liquibase.
- Marks the database for a future rollback.
- Rolls back changesets to given tag.
- Supposed to document the differences between the database and the changelog. I’ve not had good luck with this so far against MySQL and SQLServer databases.
- Evolutionary Database Development is not controversial among software developers, because we are largely persuaded of the core propositions. Moreover, Liquibase seems like a reasonable tool to help. Now let’s talk about some practices to put in place in a brownfield effort.
- People are using it, it works, it has literally years of knowledge and business process encoded in it. It’s also ugly and outdated, and its schema seems to have been built by the DBA Club after school. But you’re replacing it, with Grails...!
- You want oil changes on the jalopy to take effect in the sportscar. When you put miles on the jalopy’s odometer, they have to show up on the new car. That’s tricky. How to do it? You’ve got options.
- Write a big data migration script, test it rigorously, migrate all legacy features to the new app, and schedule a hard cutover. Not only will this never happen in the present economy, but you’re a little bit crazy if you attempt it. It’s way too risky to stake your reputation on it.
- Write the new app with its ORM layer (GORM in this case) mapping the nice, clean domain model onto the legacy database. This will work, but offers no hope of refactoring the old schema, which may sorely need reform.
- Write the new app the way you want it. Design a good, expressive domain model in the language of the business. Let the ORM layer create the schema, possibly with some tweaking. Then keep the legacy data connected to the new schema. But brownfield data integration is tricky...
- Need to write data migration scripts to populate the new schema with the mapped legacy schema at T0 (with the app off). Can do in SPs, Talend, Groovy, etc. Need to keep them synced with [cue the incidental music] TRIGGERS!
- Need to write data migration scripts to populate the new schema with the mapped legacy schema at T0 (with the app off). Can do in SPs, Talend, Groovy, etc. Need to keep them synced with [cue the incidental music] TRIGGERS!
- Integration tests run against your new schema, so they’re easy. Resist the temptation to write mapping scripts to generate test data, and just hand-code it.
- It’s a tall order to write automated tests for this. The most realistic option is automated spot-checking code that randomly picks legacy records and makes assertions on the post-migration records in the new schema.
- Also somewhat labor-intensive to test, but essential. Brownfield integration with triggers is probably new to the team, and if legacy and new-product teams are different, the legacy team will tend to finger-point and engage in magical thinking about the unknown. Confidence is key!
- We’ve seen how to think about database development in the same way we think about Agile software development. We’ve reviewed Liquibase, which enforces the discipline of controlling our changes in a repository and breaking them down into single steps which are managed individually. We’ve even talked about how to migrate real systems.