Delft3D Open Source Editing Workshop

1. Delft3D Open Source Workshop Delft, Nov. 07, 2014 Qinghua Ye Adri Mourits

2. Overview
•Introduction
•Code management
•Development environment Exercises
•General code structure Delft3D-FLOW, -MOR, -WAVE
•Ongoing developments OR
•Optional topics Exercises
In main building: Lunch: 12:30 – 14:00 Drinks: 17:30

3. Overview
Optional topics (Votes will be counted before the first exercise)
•Linux
•Parallel
•64-bit
•Debugging
•Domain Decomposition
•OSS site
•Mixed C++/Fortran
•Your suggestions
•…

4. Overview
Organisation of this workshop
•All information is at https://svn.oss.deltares.nl/repos/delft3d/branches/research/Deltares/20131108_workshop
•You do the work
•Qinghua and Adri give information when needed
•Work on exercises/your own stuff whenever you want
•Switch to other workshops whenever you want
•For the rest:

5. Introduction

6. Introduction
•Assumed knowledge
•What is Delft3D? http://www.deltaressystems.com/hydro/product/621497/delft3d-suite
•NOT in this workshop
•GUI usage http://oss.deltares.nl/web/delft3d/screen-casts
•Creating a model http://oss.deltares.nl/web/delft3d/modelling-guidelines http://oss.deltares.nl/web/delft3d/training-courses
Adri Mourits
Qinghua Ye

7. Introduction
Open Source Software site: www.oss.deltares.nl
•Getting started, download source code
•Screen casts
•Webinars
•Modelling Guidelines
•FAQ, release notes
•Discussion groups

8. Code management

9. Code management
Overview:
•SVN
•Setups
•FLA
•Future Open Source plans

10. Code management - SVN
SVN
•Trunk, main line:
•Fixing bugs
•New developments being merged in
•Run on testbench
=> Possibly not stable
•Tags:
•Copies of stable, fully tested Trunk-revisions => Use the latest. Currently https://svn.oss.deltares.nl/repos/delft3d/tags/3058
•Branches:
•Separate develop versions
•“Your own private version”

11. Code management - SVN
SVN
•New branch:
•For “big” changes
•When stability is harmed too much
•Private developments

12. Code management – Setups
Receiving from Deltares: Setups
•Open source installation (“Install-Shield.exe”): All User Interfaces but not the FLOW/WAVE executables
•Services installation: All user interfaces and also FLOW/WAVE executables (Delft3D service package needed, see http://oss.deltares.nl/web/delft3d/services)
•Setup variants including WAQ License File still needed when using UserInterfaces / Delft3D-menu.

13. Code management – from Deltares
Receiving from Deltares: GUI
•Quickplot source code: https://svn.oss.deltares.nl/repos/delft3d/trunk/src/tools_lgpl/matlab/ quickplot MatLab needed.

14. Code management - FLA
FLA When delivering source code changes, an FLA agreement must be signed.

15. Code management – future plans
Future Open Source plans
•Flexible mesh: currently in a restricted community, later fully open
•Delta Shell (Framework, partly): 2014

16. Development environment

17. Development environment
Overview:
•Windows
•VisualStudio 2008/2010/2012
•Build configuration (Release/Debug)
•Linux
•Automake, valgrind
•Resulting files
•Version numbers See file “….srcREADME “

18. Development environment
Windows
•Visual Studio 2008: generation as standard version -> d_hydro_open_source.sln
•Visual Studio 2010: also supported -> d_hydro_open_source_vs2010.sln -> d_hydro_open_source_vs2010_intel13.sln
•Visual Studio 2012: not supported yet, but it does compile -> d_hydro_open_source_vs2010_intel13.sln
•Build configuration “Release” version: Use by default
•Build configuration “Debug” version: Use only for tracing/solving bugs

19. Development environment
Linux
•GNU Autotools/Libtools
•2 build methods:
1.If all tools are installed on default location: ./autogen.sh ./configure --prefix=`pwd` make ds-install
2.If some tools are not on default location: Check paths/settings in script “build.sh” and execute build.sh –intel11.1 -64bit
•Debug: Add flag –debug when executing build.sh valgrind $exedir/deltares_hydro.exe $argfile TotalView Write statements

20. Development environment
Resulting release files (Windows)
•FLOW: ….binwin32flow2d3dbin: d_hydro.exe flow2d3d.dll mormerge, plugins, datsel, kubint, lint DOL dll’s MPICH dll’s more dll’s (OpenDA, compiler, xml etc.)
•Wave with SWAN: ….binwin32wavebin: wave.exe ….binwin32swanbin: swan_4072ABCDE_del_w32_i11_omp.exe Resulting files FLOW, debug (Windows) ….srcengines_gpld_hydrobinDebug
To be started Small executable XML input file
Big dll containing Delft3D-FLOW
Remote Online Visualisation
Parallel calculations

21. Development environment
Resulting release files (Linux)
•FLOW: ….binlnxflow2d3dbin: d_hydro.exe flow2d3d.so mormerge, plugins, datsel, kubint, lint DOL so’s MPICH so’s more so’s (OpenDA, compiler, xml etc.)
•Wave with SWAN: ….binlnxwavebin: wave.exe ….binlnxswanbin: swan_4072ABCDE_del_l64_i11_omp.exe Install script called via “post build event (Win)”/ “ds-install rule (Lin)”
To be started Small executable XML input file
Big dll containing Delft3D-FLOW
Remote Online Visualisation
Parallel calculations

22. Development environment
Version number 6.01.04.3058 Update_version_tools:
•Windows: pre-build commands in projects
•Linux: make rule “BUILT_SOURCES = ./always” Every module has it’s own version number!
Increased when not backwards compatible
New major functionality
Minor changes, used to distinct stable versions
SVN, Automatically generated revision number

23. Development environment
Update_version_tools:
•On every compile action, version number, date and time are generated
•Tri-diag file:
•“C:Delft3Dw32libwhat.exe flow2d3d.dll”:
Deltares, DELFTIO Version 1.09.00.4108, Sep 16 2014, 11:22:21 Deltares, DELTARES_COMMON Version 1.00.00.4108, Sep 16 2014, 11:22:06 Intel Fortran RTL Message Catalog V12.1-110 Apr 04 2012 $HeadURL: https://svn.oss.deltares.nl/repos/delft3d/trunk/src/utils_lg pl/deltares_common/packages/deltares_common/src/deltares_common_version. F90.svn $ Deltares, NEFIS Version 5.07.02.4108, Sep 16 2014, 11:22:10 Deltares, ESMFSM Version 4.06.00.4108, Sep 16 2014, 11:22:02 Deltares, FLOW2D3D Version 6.01.08.4108, Sep 16 2014, 11:26:16
*** FLOW2D3D Version 6.01.08.4108, Sep 16 2014, 11:26:16 *** built from : https://svn.oss.deltares.nl/repos/delft3d/trunk

24. Vote for optional topics
Optional topics:
•Linux
•Parallel
•64-bit
•Debugging
•Domain Decomposition
•OSS site
•Mixed C++/Fortran
•Your suggestions
•… Prepared:
•General code structure Delft3D-FLOW, -MOR, -WAVE
•Ongoing developments

25. Exercises

26. General code structure

27. General code structure
•Directory structure / VS projects
•FLOW
•Main (DD/OLV)
•Trisim (gdp)
•Tricom (time loop)
•Trisol (ADI)
•MOR
•Erosed
•Bott3d
•WAVE
•SWAN
•Doxygen
•NaN check

28. General code structure
FLOW Directory structure From top-level To …srcengines_gplflow2d3dpackageskernelsrccomputeadi.f90

29. General code structure
Directory structure

30. General code structure
Directory structure
•Bin: resulting executables
•Examples: Ready-to-run testcases (after compiling release version)
•Src: source code
•Engines_gpl: source code per kernel
•Third_party_open: source code/binaries from outside this source tree (mainly from outside Deltares)
•Tools: additional programs, Post processing
•Utils: Generic libraries, used by more than one kernel

31. General code structure
Directory structure Engines:
•d_hydro: Executable starting FLOW dynamic lib
•flow2d3d: Delft3D-FLOW kernel
•Packages: source code, separated from binaries, doc, scripts, etc.
•wave

32. General code structure
Directory structure Generic package names Package dependencies:
data
manager
kernel
io
plugin
flow2d3d
main

33. General code structure
Directory structure Each package: Separate src (,include) and binaries

34. General code structure
Directory structure Kernel/src: Traditional Functional splitting Trisim structure

35. General code structure
Directory structure Files!

36. General code structure
Directory structure Linux:
•Binaries next to source code (*.o, *.lo, *.la) default
•Libraries/executables copied to “src/lib” and “src/bin” default
•Install script copies to “bin/lnx” Deltares

37. General code structure
Visual Studio 2010

38. General code structure
FLOW - Main …srcengines_gpld_hydropackagesd_hydrosrcd_hydro.cpp main (argc, argv, envp) … DeltaresHydro * DH = new DeltaresHydro (argc, argv, envp); DH->Run (); DeltaresHydro::DeltaresHydro (argc, argv, envp) … dllhandle = LoadLibrary (“flow2d3d”); entryPoint = GetProcAddress (dllhandle, “DeltaresHydroEntry”); DeltaresHydro::~DeltaresHydro (void) DeltaresHydro::Run (void) … this->startComponent->Run ();
exe

39. General code structure
FLOW - Main …srcengines_gplflow2d3dpackagesflow2d3dsrcflow2d3d.cpp DeltaresHydroEntry (DeltaresHydro * DH) DH->startComponent = new Flow2D3D (DH); return DeltaresHydroEntry (DH); Flow2D3D::Flow2D3D (DeltaresHydro * DH) Flow2D3D::~Flow2D3D (void) Flow2D3D::Run (void) if (this->dd != NULL) this->dd->Run (); else { TRISIM (ndom, nmap, id, flags, runid); }
dll

40. General code structure
FLOW - Fortran entry for each subdomain: …srcengines_gplflow2d3dpackagesmanagersrctrisim.F90 subroutine trisim … allocate(gdp) … retval = trisim_init … retval = trisim_step … retval = trisim_finish
gdp: GlobalDataPointer Structure containing (pointers to) all arrays/parameters being global inside one subdomain. gdp must be passed through “everywhere” inside one subdomain for thread safety.
Fortran

41. General code structure
FLOW - Subdomain initialization: …srcengines_gplflow2d3dpackagesmanagersrctrisim_mod.F90 function trisim_init … call gdp_alloc(gdp) … call tdatmain … call tripoi … call tricom_init
Scan input, write time dependent data to TMP-files
Allocate arrays
Read complete input Check input Initialization

42. General code structure
FLOW - Subdomain step: …srcengines_gplflow2d3dpackagesmanagersrctrisim_mod.F90 function trisim_step call tricom_step …srcengines_gplflow2d3dpackagesmanagersrctricom_step.F90 subroutine tricom_step … do nst = itstrt, itstop - 1, 1 … call postpr … update wave info call trisol (/z_trisol/z_trisol_nhfull) enddo
Time loop
Post processing
Solver

43. General code structure
FLOW - Subdomain solver: …srcengines_gplflow2d3dpackageskernelsrcmaintrisol.f90 subroutine trisol … call f0isf1 timnow = timnow + 0.5_fp call incbc call adi(…stage1) … susp. transport solvers call f0isf1 timnow = timnow + 0.5_fp call incbc call adi(…stage2) … susp. transport solvers
First half timestep: v implicit, u explicit
Second half timestep: u implicit, v explicit

44. General code structure
Morphology …srcengines_gplflow2d3dpackageskernelsrcmaintrisol.f90 subroutine trisol … call adi(…stage1) … susp. transport solvers call erosed … call bott3d … call adi(…stage2) … susp. transport solvers call erosed … call bott3d
Sediment fluxes at bed
Bathymetry update

45. General code structure
WAVE - main …srcengines_gplwavepackageswavesrcwave.f90 program waves_main … if (not stand_alone) do while perform_step wait for FLOW signal call swan_tot enddo else call swan_tot endif

46. General code structure
WAVE – swan_tot …srcengines_gplwavepackagesmanagersrcswan_tot.f90 subroutine swan_tot
… do itide = 1, swan_run%nttide do i_swan = 1, n_swan_grids … get input (from FLOW)
… write swan input call run_swan call read_swan_output
enddo
… write wave output (to FLOW) enddo

47. General code structure
Code documentation using Doxygen: www.oss.deltares.nl -> download -> manuals Link to preprocessed Doxygen output: http://apidocs.oss.deltares.nl/delft3d Or generate it yourself:
•Download and install Doxygen from
•http://www.stack.nl/~dimitri/doxygen/
•Download and install Graphviz from
•http://www.graphviz.org/
•In src directory give command: doxygen doxyfile_delft3d
•Open the generated “delft3d-apidochtmlindex.html”
•Search for top level flow routines as a starting point: trisim, tricom, trisol, or specifically for morphology erosed.

48. General code structure
Code documentation using Doxygen

49. General code structure
Not a Number (NaN) check By default, FLOW continues calculation with NaNs:
•Full check is too time consuming
•Input is scanned on NaNs
•Waterlevel is scanned on NaNs every half time step Debugging with full NaN check (Intel compiler):
•Activate 4 lines of code in trisim.f90 (see next slide)
•Recompile

50. General code structure
NaN check
…srcengines_gplflow2d3dpackagesmanagersrctrisim.F90 subroutine trisim … use ifcore
… INTEGER*4 OLD_FPE_FLAGS, NEW_FPE_FLAGS
…
NEW_FPE_FLAGS = FPE_M_TRAP_OVF + FPE_M_TRAP_DIV0 + FPE_M_TRAP_INV
OLD_FPE_FLAGS = FOR_SET_FPE (NEW_FPE_FLAGS)

51. Ongoing developments

52. Ongoing developments
Overview of developments on next 3 slides (for completeness)
•OSS site: http://oss.deltares.nl/web/delft3d/publications
•Open source branches 16 new branches in 2013 from inside Deltares. More than 10 new branches in 2014
•Closed source branches Old (< 2011)

53. Ongoing developments – OSS site

54. Ongoing developments – Open source branches
20110201-BwN-mud_dynamics/ 20110308_Kleptsova_Pietrzak_Stelling_Coriolis/ 20110405_12987_openmi_openda_merge/ 20110408_Liang_consolidation/ 20110420_OnlineVisualisation/ 20110510_cormix/ 20110512-Rivermorphology-semi_fixed_layers/ 20110630_GrandCanyon/ 20110712_14855_flow-wave-timelag/ 20110818_nieuwboer_sediment/ 20111011_luijendijk/ 20111205_bedform_angle/ 20120109_k_tau/ 20120131-Bank_erosion/ 20120217_tangential_boundary_conditions/ 20120413_outflow-boundary/ 20120606_reorganizing_sediment_transport/ 20120817_15922_gnu_optimization_problem_wavef90/ 20121029-RMD/ 20121113-Semi_fixed_layers/ 20121130_workshop/ 20130228_16080_xml/ 20130416_PLIC_VOF_bankEROSION/ 20130603-DAD_uniform_graded/
20130620_reorganizing_sediment_transport/ 20130725_SANTOSS_sediment_transport/ 20130808_tidal_turbines/ 20130912_12819_EC-module/ 20131003_quasi3d_Bendflow/ 20131008_16582_mass_conservation_with_ztbml/ 20131021_Telemac_coupling/ 20131108_workshop/ 20131129_UNST-389_D-FLOWFM_WAVE 20131218_UNST-386_trachytopes 20140103_Galappatti_FacTsd 20140104_Delwaq_Special_KRWV_UNOPS 20140128_12924_ice_coverage 20140203_QP_for_VOF_bankEROSION 20140319_back_temp 20140322_nefis_variable_cell_dimension 20140326_Flow_netcdf 20140402_quasi_3d_bend_flow 20140416_ShinHanul 20140429_moerman 20140512_Irv 20140626_33406_Subsidence_Uplift 20140716_Delwaq_parallel_testing 20140911_broekema_sed_transport_analysis
https://svn.oss.deltares.nl/repos/delft3d/branches/research/Deltares/

55. Ongoing developments – Closed source branches
OpenMI
power station capacity steered
TRITON online
NearFarfield tk
OLV DOL
river
slib3D
streams on windows
CIW coolingwater Threetox
bubble screen sigma parallel
tangential BC
OLV DOL
TO semi fixed layers for bedcomposition
xbeach in delft3d-flow
DD multi node
……

56. Questions & answers / discussion
Questions or comments?