JAB14 Honey I shrunk the code

Honey, I shrunk the code !
JandBeyond 2014, Königstein, 30 May 2014
2004 - 20142004 - 2014

Hi ! :-)
2
Beat  Joomler since 12 years (ok, Mambo dinosaur ;-)
 Top-20 Joomla 1.0.15 and 3.2.0 contributor
(“App-Store” install from web + JUX team)
 Membre Joomla Security JSST and JBS
 ‘beat’ on forums, otherwise in Switzerland
 PhD Swiss Federal Institute ofTech. & Bell Labs
 Datacoms, Internet, Sécurité & web-apps
 “Make it as simple as possible, but no less”
I love open-source and work with the community

30.05.14 © Joomlapolis.com 2004-2014 3
Today…
Why are…
●
Open-source…
●
Software System Architecture and Design…
●
Abstraction, Consistency, DRY…
●
Evolution of Software Engineering…
●
Methodology and specifcations before implementation…
…so important ?
●
2014: Introducing AHA!WOW?
●
Features, Examples from CB 2.0 and CBSubs GPL 3.0
●
Conclusions
3

30.05.14 © Joomlapolis.com 2004-2014 4
Why Open-source ?
●
Freedom 0. run for any purposes
●
Freedom 1. study & change to your wish
●
Freedom 2. redistribute to help your neighbor
●
Freedom. 3. redistribute modifed copies
●
You control your computer. Not the other way around!
●
You own and control your own data: Open-data
●
The SaaS problem
●
Google Reader, Posterous, Google Wave, …
●
The free SaaS problem
●
next: Google Mail ???

30.05.14 © Joomlapolis.com 2004-2014 5
Why Software System Architecture design ?
●
GoodArchitecture brings:
●
Beauty
●
Functionality
●
Consistency
●
Durability
●
Happyness !
●
Lack of architecture results in:
●
The reverse (unless you are very lucky !)
●
Good architects are hard to fnd and hard to keep.

30.05.14 © Joomlapolis.com 2004-2014 6
●
In software good architecture translates in:
●
Clean vs messy code
●
Re-usability of code and libraries vs dedicated purpose
●
Consistent genericAPI vs unconsistent dedicatedAPI
●
No copy-paste (DRY) vs copy-pasted redundant code
●
Aspect-oriented code vs mixed-aspects code
●
Independant parts vsTightly-coupled inter-dependant parts

30.05.14 © Joomlapolis.com 2004-2014 7
●
Easy ?
●
Yes, in theory!
●
In practice, no! :
●
History
●
Can't rewrite all all the time
●
Need for backwards-compatibility
●
Lack of time
●
Large changes and refactorings needed
●
To complicated to review (thus to be committed)
●
But doable!
●
Example: Observers and Mapper Pattern I introduced for Tags
●
Used without changes for Content History !

30.05.14 © Joomlapolis.com 2004-2014 8
Why Abstraction, Consistency, DRY ?
●
Abstraction is boring
●
Requires understanding benefts of modelisation
●
But allows reuse in diferent, unexpected situations
●
Consistency is not easy
●
Requires understanding of the existing and examples
●
But is a big win for the user experience and new features
●
DRY is more work at the frst copy
●
Hard to understand for graduates of the
“Copy-Paste University”
●
But brings maintenability and code-reduction
●
Less code, Simpler code, Easier to maintain.

30.05.14 © Joomlapolis.com 2004-2014 9
Evolution of Software Engineering
●
Pionneering eras
●
Permanent Software crisis since the seventies:
●
Budgets, schedule, productivity, quality, security
●
Reaching limits of current paradigms and architectures
●
Reaching limits of current software methodology
●
80's and 90's: No silver bullet
●
80's: Software maintenance = 2x development cost
●
90's: Costs raised +30%
●
95 survey: 50% of operational software not considered as successful
●
Average software project overshoots by 50%
●
75% of large software are unused or do not meet specs.

30.05.14 © Joomlapolis.com 2004-2014 10
●
Houston, we still have a problem:
A 10+ Million $ software project has
100% chances to:
●
Run late
●
Be over budget
●
Be unproductive
●
Unusable
●
Have quality issues
●
Have security (sometimes lethal) issues
IF it even succeeds one day !

30.05.14 © Joomlapolis.com 2004-2014 11
●
Many trials to circumvent the crisis:
●
Methods:
●
Structured
●
Object-oriented
●
Unit-tests
●
Output-tests
●
Tools (IDE, lint, testing)
●
Discipline and professionalism
●
Formal methods
●
Processes
●
No Silver Bullet yet found!
●
And there is no single solution to the crisis.

30.05.14 © Joomlapolis.com 2004-2014 12
●
Current trends:
●
Aspects (Aspect Oriented Programming)
●
Entity – Relation - Processor
●
Hierarchical Model – View - Controller
●
Model-driven design
●
Model transformation
●
Code generation
●
Dependency Injection Containers
●
Agile
●
Extreme Programming
●
Lightweight coding
●
Formal methods
●
Code-generators

30.05.14 © Joomlapolis.com 2004-2014 13
●
Next trends: (my own forecast)
●
Formal Description Languages
●
Specifcations-driven interpreters and code-generators
●
Automated Dependency Injection Containers
●
Lightweight coding
●
Domain-specifc Languages
●
And in Web User Experience:
●
More natural interactions
●
Simpler Uis, but more powerful
●
Workfow-oriented Uis
●
Mobile frst

30.05.14 © Joomlapolis.com 2004-2014 14
Why Methodology and specs before implementation ?
●
Well…
●
It's like building a house without drawings !
●
Will it hold tight ?
●
Will workers be happy to frst build then go through a design
review process ?
●
Will it:
●
Look beautiful ?
●
Be functional ?
●
Be consistent ?
●
Durable ?
●
Will its users be happy ?
●
With careful planning and specs:Yes !

30.05.14 © Joomlapolis.com 2004-2014 15
Enough with problems!… But is there a solution ?
●
Remember: There is no silver bullet…
●
But we can do much, much, much, much better !
●
How ?
●
Easy:
1. Specify functionality
2. Review specifcation
3. Design software
4. Review design to respect specs
3. Implement according to design
4. Verify that implementation respects design
●
Is that easy ?
●
Does it guarantee that implementation respects specs ?

More
Traditional design
●
Specify
●
Design
●
Implement
(in MVC)
16© Joomlapolis.com 2004-2014
…Unrelated !

30.05.14 © Joomlapolis – 2004 - 2014 17
AHAWOW

More
Community Builder 2.0
 5Years of research
at Joomlapolis Labs!
Related !

2012-05-20 19© Joomlapolis.com 2004-2014
NO Implementation!
No <?PHP ?> code!
No <script /> code!
No SQL queries!
Look Ma !

INTRODUCING…
AHA?
© Joomlapolis.com 2004-2014 20
Automatc
Homogenous
Applicatons
Web
Optmized
Workfows? !
WOW!™

AHA? …Compare… WOW!
Classical (H)MVC:
 Model+View+Controller
implementation
 + Helpers
 and Javascript helpers
 Nothierarchical
 Not agonistic (PHP)
 “COPY-PASTE” code
 Lot of layouting
 Notformal
AHA?WOW!™ MVC:
 Specifcation only
 Hierarchical – MVC
 Agonistic (portable)
 DRY, no specifc code
(or very little)
 Formal

AHA?WOA!
So……?
…The 2’ Example:
Currency rates browser

More
❶ Model:
<table name="#__currencies" class="Currency">
<columns>
<column name="id" type="sql:int(11)" label="Currency Id" />
<column name="base_currency" type="sql:varchar(3)" label="Base currency" />
<column name="currency" type="sql:varchar(3)" label="Target currency" />
<column name="rate" type="sql:foat" null="true" label="Exchange rate" />
</columns>
</table>
This installs / upgrades database !

More
❷ Controller:
<action name="show_currencies" request="view" requestparams="showcur" >
<showview view="showcurrencies" mode="show">
<data name="currencies" table="#__currencies">
</showview>
</action>
Controller links Model with View

More
❸ View:
<view name="showcurrencies" type="params" ui="admin" label="Currencies">
<tablebrowser name="currenciesbrowser" label="Currency exchange rates">
<listfelds>
<rows type="standard" height="auto" limit="15" >
<feld name="" type="rownumber" width="20px" align="right" label="#" />
<feld name="id" type="primarycheckbox" width="20px" />
<feld name="currency" type="string" width="10%" label="Currency" />
<feld name="base_currency" type="string" width="30%" label="Base currency" />
<feld name="rate" type="foat" width="30%" label="Exchange rate" />
<feld name="id" type="int" width="20%" label="Id" />
</rows>
</listfelds>
</tablebrowser>
</view>
Views defne Items to display & Layout

THERESULT
 The result:

CBSubs GPL 3.0
uses already
AHA?WOW!™
CB 2.0 now too !

CBSubs Administration: 0 lines of specifc code
2012-05-20 © Joomlapolis.com 2004-2014 28
 6900 lines of specifcations (core CBSubs CB plugin) only
– 41 controllers
– 50 views
– 56 models
– 397 “queries”
– 650 settings

`ONE’ MORE THING…
 AHA?WOW DOES MUCH MORE:
 includes:
– Database-agonistic query-compiler and schema
– Formal dataset query specifcations
– Inheritence
– Permissions &View Access Levels handling
 Handles 100% of the Admin interface
– Toolbar, Menus
– Online Help
– All feldTypes
– Tables, Graphs, Widgets, …
 And is event-driven and event-triggering
2012-05-20 © Joomlapolis.com 2004-2014 29

Database scheme description
●
Handles:
●
Creation
●
Checks
●
Upgrades
●
Downgrades
automaticaly
<table name="#__comprofiler_tabs" class="" strict="false" drop="never">
<columns>
<column name="tabid" type="sql:int(11)" auto_increment="23" />
<column name="title" type="sql:varchar(50)" />
<column name="description" type="sql:text" null="true" />
<column name="ordering" type="sql:int(11)" />
<column name="ordering_register" type="sql:int(11)" default="10" />
<column name="width" type="sql:varchar(10)" default=".5" />
<column name="enabled" type="sql:tinyint(1)" default="1" />
<column name="pluginclass" type="sql:varchar(255)" null="true" />
<column name="pluginid" type="sql:int(11)" null="true" />
<column name="fields" type="sql:tinyint(1)" default="1" />
<column name="params" type="sql:mediumtext" null="true" />
<column name="sys" type="sql:tinyint(4)" />
<column name="displaytype" type="sql:varchar(255)" />
<column name="position" type="sql:varchar(255)" />
<column name="viewaccesslevel" type="sql:int(10)" unsigned="true" initialvalue="1" initialvaluetype="int" />
<column name="useraccessgroupid" type="sql:int(9)" default="-2" mandatory="false" />
<column name="cssclass" type="sql:varchar(255)" null="true" />
</columns>
<indexes>
<index name="PRIMARY" type="primary">
<column name="tabid" />
</index>
<index name="enabled_position_ordering">
<column name="enabled" />
<column name="position" />
<column name="ordering" />
</index>
<index name="orderreg_enabled_pos_order">
<column name="enabled" />
<column name="ordering_register" />
<column name="position" />
<column name="ordering" />
</index>
</indexes>
</table>
© Joomlapolis.com 2004-2014

Database queries compiler
Handles automatically:
• SQL queries • Mix-ins • Searches • Filters
<filter name="tabid" value="tabid" valuetype="sql:string" type="data">
<data name="tab" type="sql:multiplerows" table="#__comprofiler_tabs" key="id">
<rows>
<field name="tabid" as="value" />
<field name="title" as="text" />
</rows>
<orderby>
<field name="position" ordering="ASC" />
<field name="ordering" ordering="ASC" />
</orderby>
<where>
<column name="fields" operator="=" value="1" type="sql:field" valuetype="const:int" />
</where>
</data>
</filter>

XML Query can be used inTableBrowser:
Example: CB 2.0 Tab Manager, with “Tools”
JDAY-FR-2014 © Joomlapolis.com 2004-2014 32

3/4/2011 © Joomlapolis.com 2004-2014 33
More
XML TableBrowser example: CBSubs™ plans
 Presentation:
– Hierarchieswithsub-plans
– Yes-noradios/clickabletoggles
– Links
– Cssclasses
– Textformatting
– Hiddencolumns
– Permissions
– Dynamicattributes
33

Extending and Inheriting
Two-way inherience: Extend and inherit:
• Explicit (<extend>) • Implicit (<inherit>)
<extend toxpathnode="/cbxml/types" fromxpathnode="/*/types/type" file="xml/models/model.blogs" />
<extend toxpathnode="../following-sibling::*[1]/params/fieldset"
fromxpathnode="/cbxml/views/view[@name="radiosettings"]/*"
file="xml/admin.lib.processors" mode="prepend">
<inherit name="editgatewayinherit" from="/*/views/view[@name='editgenericgateway']">

Extension example 1: CB Privacy on CB Field

3/4/2011 © Joomlapolis.com 2004-2014 36
More
Extension example 2: CBSubs™ integrations
 CBSubs™ plan edit is automatically extended with available
integrations. By pure XML AHAWOW magic:
36

Extension example 3: CB 2.0 plugins can extend Joomla menus

Permissions andView Access Levels
Handles:
●
Confguring
●
Enforcing
<actions ui="admin" permission="core.manage" permissionasset="com_comprofiler">
<action name="users" request="task" action="showusers" permission="core.manage"
permissionasset="com_users">
<showview view="showusers" mode="show" />
</action>
</actions>
<tablebrowser name="fieldsbrowser" table="#__comprofiler_fields"
class="CBDatabaseTableFieldsTable" permissionasset="com_comprofiler.fields">
<param name="permissions_fields" type="permissions" assetname="com_comprofiler.fields" label="" display="2lines">
<permission name="core.manage" label="Manage" description="Allows viewing of Fields Management." />
<permission name="core.delete" label="Delete" description="Allows deleting of fields." />
<permission name="core.create" label="Create" description="Allows creating new fields." />
<permission name="core.edit" label="Edit" description="Allows editing existing fields and editing the state of existing fields." />
<permission name="core.edit.state" label="Edit State" description="Allows editing the state of existing fields." />
</param>

Toolbars and online Help
<toolbarmenu>:
<toolbarmenu>
<menu name="apply" action="apply" task="act=apply" label="Save" />
<menu name="save" action="save" task="act=save" label="Save & Close" />
<menu name="cancel" action="linkAction" task="cancel" link="cb_action:fields" label="Cancel" />
<menu name="help" action="help" ref="https://www.joomlapolis.com/help/cb/2/permissions"
label="Help" />
</toolbarmenu>

Example XML toolbars: CB 2.0 Field edit

Menu buttons
<menugroup>:
<menugroup name="mailersettingsmenu" type="buttonMenu" label="Mailer menu" description="">
<menu name="mailers" action="cb_action:show_mailers" label="Mailers" description="Configure Mailers">
<fields>
<field name="mailers_count" label="active mailers" type="int">
<data name="mailers_count" type="sql:count" table="#__cbsubs_mailer_automessages" key="published" value="1" valuetype="const:int" />
</field>
</fields>
</menu>
<menu name="mailerautomation" action="cb_action:mailer_sendmail" label="Automation Settings" description="Configure Automation for Mailers">
</menu>
<menu name="mailersentmessages" action="cb_action:show_mailersentmessages" label="Sent Messages" description="Show mailer sent
messages">
<fields>
<field name="mailer_used_count" label="messages sent" type="int">
<data name="mailer_used_count" type="sql:count" table="#__cbsubs_mailer_sentmails" key="state" value="A" valuetype="const:string" />
</field>
</fields>
</menu>
<showview name="integrationsglobals" mode="edit" type="plugins" groups="user/plug_cbpaidsubscriptions/plugin,user[cbsubs.]" action=""
path="payintegrationmailersettingsmenu" />
</menugroup>

Bar graphs and graphs
<feld type=”bargraph”> and <plot>:
<field name="graph_paid_amounts" type="bargraph" key="paid_mc_gross_total" width="10%" label="Completed" description="Relative amount to other days displayed."
align="left">
<data name="0" type="const:int" />
</field>
<field name="id" type="plot" rowspan="all" class="cbregAdminTotalGraph" width="50%" label="Graph" description="This graph shows the table content in a graphical way">
<plot name="front_plot">
<series>
<serie label="New">
<data missing="0">
<y name="newregs" type="int" />
<x name="time_paid_date" type="date" />
</data>
<lines show="true" />
<points show="true" />
</serie>
<serie label="Upgrades">
<data missing="0">
<y name="upgrades" type="int" />
<x name="time_paid_date" type="date" />
</data>
<lines show="true" />
<points show="true" />
</serie>
</series>
<format>
<yaxis min="0" tickDecimals="0" />
<xaxis mode="time" timeformat="%d %b %y" />
<grid markings="cbPlotWeekends" />
<legend noColumns="4" position="nw" container="th.cbregAdminTotalGraph" />
</format>
</plot>
</field>

30.05.14 © Joomlapolis.com 2004-2014 44
Example Bargraphs and graphs: CBSubs™ panel
 Shows detailed weekly activities:
●
In numbers, bar-graph and graphs,
●
Transactions, Payments, Registrations, Upgrades, renewals
44

Events generation
<trigger>:
<trigger group="user/plug_cbpaidsubscriptions/plugin"
event="onxmlBeforeCbSubsDisplayOrSaveInvoice" results="extend" toxpathnode="../fieldset"
mode="replaceorappend" keyattribute="name" />

30.05.14 © Joomlapolis.com 2004-2014 46
Community Builder 2.0 uses AHAWOW
●
Community Builder handles site members:
100 % Libre GPL !
●
Registration process,user profles, lists, photos, videos, ...

30.05.14 © Joomlapolis.com 2004-2014 48
CB 2.0 Software Architecture
Joomla!
CBLib 2.0 = CB Framework (Joomla library)
CB 2.0 (Joomla component, modules)
CB Plugin 2.0 (Joomla plugin)

30.05.14 © Joomlapolis.com 2004-2014 49
CB 2.0 Software Architecture
Joomla!
CBLibCms
CBLibCBLib
CBLibCB
CB 2.0 (component, modules)
CBLib
Composer
CB Plugin
Libraries
AhaWow

30.05.14 © Joomlapolis.com 2004-2014 50
CBLib 2.0 Software principles
●
CBLib 2.0 has a modern software architecture:
●
Leightweight (“Small is beautifull”, completely modular
●
Uses the new PHP 5.3 functionalities, in particular:
●
Namespacing
●
Interfaces
●
Closures and function variables
●
Uses extensively:
●
Depedency Injection Containers
●
Class Interfaces
●
Inversion of Control
●
Aspect Oriented Programming
●
Entity-Relation-Processor
●
Model-View-Controller

30.05.14 © Joomlapolis.com 2004-2014 51
CBLib 2.0 namespaced packages
●
CBLib 2.0 includes mostly independent packages:
●
AhaWow™ : Specifcation-based execution
●
Dependency Injection (DI) Containers
●
Observers+Mappers
●
Languages (incl. Pluralizations + Multi-keys)
●
Auto-loaders for PHP, XML and languages
●
Permissions and Access handler
●
Database Schema checker and upgrader
●
Database Query Compiler
●
Image Library
●
LESS + Bootstrap 3
●
Legacy Compatibility Layer

30.05.14 © Joomlapolis.com 2004-2014 53
Conclusion
Using a formal approach it is possible to specify software
instead of implementing it.
This has many advantages:
●
Beautiful
●
Functional
●
Consistent
●
Durable
●
Verifable
●
Auditable
And AHA?WOW! ™ is now here after 5 years of research!
Rings a bell ?

30.05.14 © Joomlapolis.com 2004-2014 54
●
GoodArchitecture brings:
●
Beauty
●
Functionality
●
Consistency
●
Durability
●
Happyness !
●
Lack of architecture results in:
●
The reverse (unless you are very lucky !)
●
Good architects are hard to fnd and hard to keep.
Remember that slide ?

30.05.14 © Joomlapolis – 2004 - 2014 55
Thanks !
CBLibAhaWow and Community Builder 2.0:
An exciting software architecture!

JAB14 Honey I shrunk the code

Recomendados

Recomendados

Más contenido relacionado

Similar a JAB14 Honey I shrunk the code

Similar a JAB14 Honey I shrunk the code (20)

Más de CB Team @ Joomlapolis

Más de CB Team @ Joomlapolis (20)

Último

Último (20)

JAB14 Honey I shrunk the code