Jeff Longland

Relax, don't worry – have a home brew!

BbWorld 2011 – Performance Troubleshooting

leave a comment »

  • Given this is a support session, the focus will be on post-incident troubleshooting and not performance testing best practices.
  • Heap exhaustion is where performance degrades gradually.  Incidents without heap exhaustion are usually marked by a sudden crash.
  • When you report a case, support will ask if you’ve tuned the baseline JVM config – so do it before an incident.
  • See performance recommendations in release notes and implement.
  • When troubleshooting, support needs the usual stuff: incident details, environment details, Bb logs, database reports, config files, system info, recent changes, thread dumps, heap dump, etc.
  • When looking at logs (bb-services-log.txt, stdout-stderr-log.txt, catalina-log.txt, bb-sqlerr-log.txt, bb-access-log.txt) look for errors, heap state, deadlocks, SQL not completing, garbage collection anomalies, etc.
  • Config files to include in your report: bb-config.properties, wrapper.conf, bb-tasks.xml
  • Blackboard has a tool to gather Linux performance reports.
  • If logs are > 2 GB, the app will run into performance issues.  Archive your logs!
  • Samurai is a good tool for thread dump analysis.
  • IBM Thread and Monitor Dump Analyzer is good for looking at thread blocks
  • When looking at database reports – look for poorly performing queries, row lock contention, results being returned slowly, etc.
  • Blackboard uses Eclipse Memory Analyzer to review heap dumps
  • I didn’t realize that Oracle Enterprise Manager can be used to monitor JMX and WSDL web services
  • Other JMX monitoring tools – Java VisualVM
  • On the more expensive side of things, Quest FogLight is good for monitoring.
Advertisements

Written by jlongland

July 13, 2011 at 11:41 am

Posted in BbWorld 2011, Conferences

Tagged with

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: