The document discusses server log files and the forensic tools and processes used to analyze them. It describes the key steps in the forensic process as securing the scene, investigating the scene by examining log files and other system evidence, gathering and correlating evidence, and building a hypothesis. It provides examples of different types of server log files that may contain useful information, such as WebLogic server logs, HTTP access logs, and JMS message logs. It also presents two case file examples involving issues like an unbalanced load and integration failures.
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
Developer and Fusion Middleware 1 _ Kevin Powe _ Log files - a wealth of forensic evidence.pdf
1. Log
files:
A
wealth
of
forensic
evidence
Kevin
Powe
Integral
Technology
Solu6ons
The most comprehensive Oracle applications & technology content under one roof
2. More
info
at
h:p://bit.ly/kapowelogs
The most comprehensive Oracle applications & technology content under one roof
4. The Forensic Process
The most comprehensive Oracle applications & technology content under one roof
5. Step One: Secure The Scene
The most comprehensive Oracle applications & technology content under one roof
6. Operating System Evidence
netstat
for
network
issues
top
or
Windows
Task
Manager
for
CPU
issues
iostat
or
vmstat
for
I/O
issues
The most comprehensive Oracle applications & technology content under one roof
7. Rolling Log Files
The most comprehensive Oracle applications & technology content under one roof
8. Cause Symptoms
2-‐4PM
4-‐6PM
The most comprehensive Oracle applications & technology content under one roof
9. Step Two: Investigate The Scene
The most comprehensive Oracle applications & technology content under one roof
11.
‘Error’
versus
‘Warning’
‘Failing’
versus
‘Failed’
The most comprehensive Oracle applications & technology content under one roof
12. Step Three: Gather And Correlate
Evidence
The most comprehensive Oracle applications & technology content under one roof
13. Step Four: Build A Hypothesis
The most comprehensive Oracle applications & technology content under one roof
14. 1) Secure the scene
2) Investigate the scene
3) Gather and correlate evidence
4) Build a hypothesis
The most comprehensive Oracle applications & technology content under one roof
16. WebLogic
Server
Domain
AdminServer
managedServer1
Java
managedServer2
processes
The most comprehensive Oracle applications & technology content under one roof
17. HTTP Access Logs
The most comprehensive Oracle applications & technology content under one roof
18. 192.168.5.6
-‐
-‐
[19/Nov/2010:13:34:49
+0800]
"POST
/AccountServices/ProxyServices/AccountServices
HTTP/1.1"
200
29487
192.168.5.6
-‐
-‐
[19/Nov/2010:13:34:49
+0800]
"POST
/WarehousingServices/ProxyServices/RequestOrderDetails
HTTP/1.1"
200
1167
rfc931
date
Remote
host
authuser
192.168.5.6
-‐
-‐
[19/Nov/2010:13:34:49
+0800]
request
"POST
/WarehousingServices/ProxyServices/RequestOrderDetails
HTTP/1.1“
status
bytes
200
1167
The most comprehensive Oracle applications & technology content under one roof
19. ELF = Extended Logging Format
The most comprehensive Oracle applications & technology content under one roof
20. Extended Logging Format Fields
Common
format
fields
Request
fields
date
cs-‐method
6me
cs-‐uri
bytes
cs-‐uri-‐stem
sc-‐status
cs-‐uri-‐query
Network
fields
The
Good
Stuff
c-‐ip
cs-‐comment
s-‐ip
6me-‐taken
c-‐dns
custom
s-‐dns
The most comprehensive Oracle applications & technology content under one roof
21. Server log files
The most comprehensive Oracle applications & technology content under one roof
23. ####<2/08/2011
12:49:35
AM
EST>
<No6ce>
<Server>
<brother-‐eye>
<AdminServer>
<[ACTIVE]
ExecuteThread:
'0'
for
queue:
'weblogic.kernel.Default
(self-‐tuning)'>
<<WLS
Kernel>>
<>
<>
<1312210175933>
<BEA-‐002613>
<Channel
"Default"
is
now
listening
on
10.0.2.15:7001
for
protocols
iiop,
t3,
ldap,
snmp,
h:p.>
####<2/08/2011
12:49:35
AM
EST>
<No6ce>
<WebLogicServer>
<brother-‐eye>
<AdminServer>
<[ACTIVE]
ExecuteThread:
'0'
for
queue:
'weblogic.kernel.Default
(self-‐tuning)'>
<<WLS
Kernel>>
<>
<>
<1312210175933>
<BEA-‐000331>
<Started
WebLogic
Admin
Server
"AdminServer"
for
domain
"example1030Domain"
running
in
Development
Mode>
Timestamp
Severity
Subsystem
Machine
<2/08/2011
12:49:35
AM
EST>
<Nodce>
<WebLogicServer>
<brother-‐eye>
Server
Thread
ID
<AdminServer>
<[ACTIVE]
ExecuteThread:
'0'
for
queue:
'weblogic.kernel.Default
(self-‐tuning)'>
User
Txn
ID
Diagn.
Time
(msecs)
Message
ID
Text
<<WLS
Kernel>>
<>
<>
<1312210175933>
<BEA-‐002613>
<Channel
"Default"
is
The most comprehensive Oracle applications & technology content under one roof
24. Debug flags
The most comprehensive Oracle applications & technology content under one roof
25.
HTTP:
weblogic.servlet.DebugH:p
SSL:
default.DebugSSL
JDBC:
weblogic.jdbc.sql.DebugJDBCSQL
The most comprehensive Oracle applications & technology content under one roof
26. <4/08/2011
07:47:35
PM
EST>
<Warning>
<netuix>
<BEA-‐423420>
<Redirect
is
executed
in
begin
or
refresh
ac6on.
Redirect
url
is
/console/console.portal?_nfpb=true&_pageLabel=HomePage1.>
Loaded
index.jsp
page
Loaded
index.jsp
page
Loaded
index.jsp
page
<4/08/2011
23:20:34
PM
EST>
<Info>
<Health>
<brother-‐eye>
<AdminServer>
<weblogic.GCMonitor>
<<anonymous>>
<>
<>
<1311830434630>
<BEA-‐310002>
<86%
of
the
total
memory
in
the
server
is
free>
TO
<4/08/2011
07:53:38
PM
EST>
<No6ce>
<WebLogicServer>
<BEA-‐000365>
<Server
state
changed
to
RUNNING>
<4/08/2011
07:53:38
PM
EST>
<No6ce>
<WebLogicServer>
<BEA-‐000360>
<Server
started
in
RUNNING
mode>
<4/08/2011
07:53:49
PM
EST>
<Nodce>
<Stdout>
<BEA-‐000000>
<Loaded
index.jsp
page>
<4/08/2011
07:53:50
PM
EST>
<Nodce>
<Stdout>
<BEA-‐000000>
<Loaded
index.jsp
page>
<4/08/2011
07:53:51
PM
EST>
<Nodce>
<Stdout>
<BEA-‐000000>
<Loaded
index.jsp
page>
<4/08/2011
08:20:34
PM
EST>
<Info>
<Health>
<brother-‐eye>
<AdminServer>
<weblogic.GCMonitor>
<<anonymous>>
<>
<>
<1311830434630>
<BEA-‐310002>
<86%
of
the
total
memory
in
the
server
is
free>
The most comprehensive Oracle applications & technology content under one roof
27. Oracle Service Bus tracing
The most comprehensive Oracle applications & technology content under one roof
28. JMS Message Logs
The most comprehensive Oracle applications & technology content under one roof
29. SOA Suite Diagnostic Logs
The most comprehensive Oracle applications & technology content under one roof
31. Case File #1
An Unbalanced Load
The most comprehensive Oracle applications & technology content under one roof
32. Sun
Reverse
WebLogic
Server
Proxy
Load
balancer
Sun
Reverse
WebLogic
Server
Proxy
The most comprehensive Oracle applications & technology content under one roof
33. cat
access.log*
|
awk
‘{
print
$x
}’
|
sort
|
uniq
(where
x
=
posi-on
of
the
cookie
in
the
log
file)
The most comprehensive Oracle applications & technology content under one roof
34. Case File #2
Fear Of Commitment
The most comprehensive Oracle applications & technology content under one roof
35. Oracle
Service
Bus
Tuxedo
The most comprehensive Oracle applications & technology content under one roof
37. Tools
Editors
Querying
data
Analysis
The
Gun
find
Excel
vi
grep
R
sed
Splunk
awk
tail
The most comprehensive Oracle applications & technology content under one roof
38. @kapowe
kevinpowe
kapowe
The most comprehensive Oracle applications & technology content under one roof
