Réunion des développeurs EsiGate le 21/11/2013

1. EsiGate
Dev meeting #4
21/11/2013

2. Servlet filter (1/3)
Goals:
• Process ESI inside a local page
• Process ESI in a remote page to include local pages
Implementation:
• proxy() -> filterChain.filter()
• include() -> request.getRequestDispatcher().include()
Implemented as an extension.
Caching for local resources.

3. Servlet filter (2/3)
Impact:
• Used instead of proxy in simple projects or in dev
• Automatic switch with ESI capabilities
• Mappings
• More simple
• External
• Taglibs not needed anymore (7 maven modules!) -> Faster builds, easier
maintenance
• JSP
• Wicket
• JSF
• Servlet dependency extracted to a separate module -> possible use outside a
servlet engine
• Jetty
• .NET

4. Servlet filter (3/3)
Todo:
• Migrate esigate-app-master integration tests
• Improve test coverage
• Support for ESI capabilities
• Cross context includes
• Documentation (migration)

5. HttpAsyncClient
Goals:
• End to end asynchronous request/response processing.
• Parallel processing
Implementation:
• asynchronous 3.0 servlet-api
• HttpAsyncClient
• 1 thread / CPU
- Note : this leads to Async Esigate
We should be able to release the server worker when a call to a backend
is pending.
ESI now uses Future. With future, we need a running thread to wait for
Future

6. ESI fixes
- Variable issues
-> Both 4.x and 5.0
- esi:include src alt onerror
- Invalid markup
-> 5.0

7. SurrogateControl
- Allows proxy capabilities detection
- Used by frameworks (Symphony, …) to switch transparently between local
includes and ESI includes.
- Implemented on 5.0
- New event : CapabilitiesEvent
* Used to collect capabilities (eg. Surrogate/1.0 ESI/1.0 ESIInline/1.0)
* Created the following capabilities : X-ESI-Fragment/1.0, X-ESIReplace/1.0, X-ESI-XSLT/1.0, ESIGATE/4.0
* ESI and Aggregator extensions have been updated to use this event.
-> Use this event in your extensions !
- Can be used to disable Esigate filter if an esigate proxy is used in
front of the application
-> http://sourceforge.net/apps/mantisbt/webassembletool/view.php?id=265

8. X-Forwarded-*
- Fixed X-Forwarded-* behavior
4.3 and 5.0
- X-Forwarded-For :
fixed syntax, all proxies ip should be appended.
“45.12.34.5, 127.0.0.1”
- X-Forwarded-Proto :
these headers were duplicated in requests.
Now once present, this header is left untouched.
- Grails / spring security ‘s Channel-Security is now working.

9. Cookie handling
- Esigate is servlet 2.5 : does not support HttpOnly attribute
- Secure attribute + X-Forwarded-Proto : requires connector configuration
(server specific)
-
We should generate cookies ourselves
Tomcat 7 code can be reused (license is ASL 2)
Create a separate project ?
Use cookie generation in mediator ? In core ?

10. Proxy trust
-
X-Forwarded-For
X-Forwarded-Proto
Surrogate-Capabilities
Others ?
- To process these headers, we have to trust a remote proxy.
- Add a new setting to list trustable proxies.
-> http://sourceforge.net/apps/mantisbt/webassembletool/view.php?id=272

11. Default settings
- Change default settings, since they are often causing issues :
-
Cookie handling
->
Preserve host
->
Url rewriting
->
Aggregator
->
Http Authentication
Forward ?
On ?
On for html content ?
Removed ?
-> Forward ?
-> http://sourceforge.net/apps/mantisbt/webassembletool/view.php?id=258
- Forward / Discard headers -> Will be removed, if you need it, write an
extension

12. HttpClient Upgrade
- HttpClient 4.3 is released.
- Some tests are failing (11)
- Integration branch
https://ci.richeton.com/job/Esigate-5.0-HttpClient-4.3-update/
- Errors are probably caused by better request handling by HttpClient
- MISS -> VALIDATED
- We need this to release Esigate 5.0

13. Performances
- New ideas :
* In esigate-server : remove servlet dependency, use esigate code
directly in a jetty handler. Create a Jetty Mediator
-> This will remove most of the servlet logic (war, web.xml, servlet
context creation)
-> will also solve dependencies issues between jetty bootstrap
libraries and war libraries
* Correctly set inital size of all buffers.
-> StringBuilder default capacity is 16
-> Should be at least 1k, 10k ? in our case.
* StringWriter is synchronized.
-> Need a performance environnement to mesure improvements

14. Code style
- Checkstyle enabled on CI
- Rules are enforced for NEW errors
-> Do not leave the build broken by checkstyle rules
-> Improve the classes when you touch them
-> Document public methods (yes, this is useful)
Next step is PMD and sonar

15. Leaderboard
- There are a lot of things to work on.
- Need pointers ? :-)

16. Q/A ...
What about dynamic configuration load and hot reload ?
It will be possible to reload esigate.properties at runtime by doing : kill HUP ESIGATE_PID
Available in 5.0-SNAPSHOT only ; cf javadoc into ConfigReloadOnHup
Migration to Github (at least get out of sourceforge) : it will ease developer
contribution, it will also avoid sourceforge weird behavior on their
package. A maven plugin is available to deploy site on gh-pages.
There are some issues with keeping track of SVN history. A new BT has been
created : http://sourceforge.net/apps/mantisbt/webassembletool/view.
php?id=277

17. Q/A ...
It would be great to have the server.jar working with an external esigate.
properties (for now proxy are hard coded in the web.xml). useMapping
should be activated for the server feature.
Available in 5.0-SNAPSHOT only.
How are we supposed to code authentification feature with ESIgate ? What
are the good practices ? (How can we work with X-USER-LOGIN ; bypass
remote authentification with a bridge for X_USER-LOGIN ?)

18. Code demo
Here are some great example of unit test :
• SurrogateTest.testSurrogateControlWithSurrogate
• SurrogateTest.testSurrogateCapabilitiese
• UriResponse