Monday, October 04, 2010
Top 10 Java Issues in Production
10 Heap/network issues
Tools: dtrace, hprof, introscope, jconsole, visualvm, yourkit, azul zvision
Invasive tools : bci, jvmti, jvmdi
OS Tools : dtrace, osprofile, vtune
Network/DISK tools: ganglia, iostat, lsof, nagios, netstat, snoop
Gotcha:
- instrumentation is not cheap
- avoid expensive heap walks
- thread dumps
- asynchronous logging
- finish task and then increment performance counter
- jconsole is cheap
9 Leaks
symptoms :
- app consumes all memory
- heap trend is ramping saw tooth
- finally Out of Memory
Tools : yourkit, hprof, eclipse mat, jconsole, jhat, jps, visualvm, azul zvision
Theory:
- Allotted Vs Live Objects
- Finalizers, Classloaders
8 I/O
I/O Serialization
Symptoms :
- Multi-node scale-out does not scale linearly
- Spending time CPU & IO
Tools :
- cpu profiling
- io profiling
Solution
- pick a high performance serialization library
- avro, krgo
I/O Limits/Tuning
Symptoms:
- Too many file descriptors, cursors etc
- inconsistent response times
Tools:
- Nagios
- pkg
- rpm
- info
- ulimit
- yum
Solution
- check OS patches
- check user and process limits
I/O Sockets, Files, DB
Symptoms:
- socket open/close takes a long time
- JRMP timeouts, long JDBC calls
- running out of files, cursors
Tools:
- dbms tools, du, iostat, gmon, lsof, netstat
Workaround
- ping/telnet tests
7 Locks and Synchronization
Symptoms:
- Adding users/threads/CPUs causes slow down
- High lock acquire times and contention
- race conditions, dead locks
- I/O under load
Tools:
- dtrace, lockstat, azulzvision
- thread dumps
- IBM visual analyzer ( j.u.c in eclipse )
Solution:
- Use non-blocking collections
6 Endless Compilation
symptoms:
- Time in compiling
- Time in interpretor
Tools:
- -XX:+PrintCompilation
- CPU profiler
5 Endless Exceptions
symptoms:
- Application spending time filling stack trace
tools:
- CPU profiler, zvision
- thread dumps
- Track caller/callee
- repeated kill -3
Solution:
- don't throw , return
- JVM's don't optimize exception paths
4 Fragmentation
Symptoms:
- performance degrades over time
- "Full GC" makes problem go away
- Lot of free memory, but in tiny fragments
Tools;
- GC logging flags, for CMS -XX:PrintFLSStatistics=2 -XX:+PrintCMSInitializationStatistics
- Fragger
Solution:
- Upgrade to latest CMS
- azul zing and gen pausless GC
- pooling similar sized and similar aged objects together
3 GC Tuning
Symptoms:
- Entropy(GC) = number of GC flags
- Too many free parameters
- 64bit/ large heap no solution
- Constant 40%-60% CPU utilization by GC
- Scheduled reboot before full GC
- Full-time engineer working on GC flags
Workarounds:
- Ask JVM vendor for one flag solution
- G1 GC, Zing GPGC
2 Spikes in Load
Symptoms:
- Rush hour traffic, tax day, black friday
- outage under spikes, power law of networks
Solution:
- Measure
- Test with realistic load and realistic multi-node setup
- build redundancy
1 Versionitis
Symptoms:
- different nodes have different configuration, stack components, versions
- classpath dist/* , -verbose
- hard to reproduce
Solutions:
- Method
- Version control
- rigor
0 Collapsing under load
juc profiling
Tuesday, August 26, 2008
Useful commands in Linux and Solaris
- truss in solaris , strace in linux
Tuesday, May 06, 2008
Useful Maven commands
checkout a project from subversion
mvn scm:checkout -DconnectionUrl=scm:https://svn.url.project.name -DcheckoutDirectory=name
checkout a project from subversion
mvn scm:checkout -DconnectionUrl=scm:https://svn.url.project.name -DcheckoutDirectory=name
Labels: maven mvn scm
Java Classloaders
Classloaders are responsible for finding and loading a class.
Class itself is a java object.
Following statements are true,
Custom class loaders
A custom Class Loader can be created by extending java.lang.ClassLoader.
Two things are important in creating a custom classloader.
Classloaders are responsible for finding and loading a class.
Class itself is a java object.
Following statements are true,
- All classloaders except bootstrap classloaders have a parent classloader
- All class loaders are of type java.lang.ClassLoader
Custom class loaders
A custom Class Loader can be created by extending java.lang.ClassLoader.
Two things are important in creating a custom classloader.
- Setting parent classloader
- implementing findClass() method
Labels: java classloader
Friday, August 24, 2007
Monday, July 23, 2007
List of User groups in Bay area.
ACCU http://www.accu-usa.org/
Bay Area Debian http://bad.debian.net/
BAFUG S.F. http://www.bafug.org/
BALUG http://www.balug.org/
[BAWUG http://www.bawug.org/]
BayPIGgies http://www.baypiggies.net/
BayLISA http://www.baylisa.org/
BayLISA Monitoring SIG
http://www.groundworkopensource.com/community/monitoring-sig.html
BUUG http://www.buug.org/
CABAL http://linuxmafia.com/cabal/
CalLUG http://www.ocf.berkeley.edu/~linux/
[CCSF LUG mailing list only. Dead site:
http://www.ccsf.cc.ca.us/Organizations/Linux_Users_Group/]
Cocoaheads http://cocoaheads.org/us/SiliconValleyCalifornia/
CSUEB LUG http://acc.csueastbay.edu/~linuxug/kwiki/
EBLUG http://www.eblug.org/
Linux Consultants' Lunch inquired about missing mailing list, 2006-02-21.
Linux User Group Hayward (Linuxdojo) http://www.linuxdojo.net/
Linuxstammtisch http://electriclichen.com/linuxbierwanderung/local/#sanfrancisco
NBLUG http://www.nblug.org/
Oakland Perl Mongers http://oakland.pm.org/
PC Clubhouse Linux SIG http://www.pcc.org/sigs/#Linux
PenLUG http://www.penlug.org/
SFBA NetBSD UG http://www.imrryr.org/NetBSD/sfbanug.html
SF LUG http://www.sf-lug.org/
SF OpenBSD UG http://www.sfobug.org/
SF Perl UG http://sf.pm.org/
SF PostgreSQL UG http://pugs.postgresql.org/sfpug/
SVBUG http://www.svbug.com/
SVLUG http://www.svlug.org/
Silicon Valley Perl UG http://www.rf.net/~sv-pm-org/
Silicon Valley Wireless Users and Experimenters http://www.svwux.org/
SJSU LUG http://sjsulug.engr.sjsu.edu/
Smaug http://www.scruz.org/
Sonoma County Sysadmins (SoCoSA) http://socosa.org/
SULUG http://sulug.stanford.edu/
Starship Augusta Ada http://trek.starshine.org/
Linux User Group of Davis (LUGOD) http://www.lugod.org/
Linux User Group of Sacramento (SacLUG) http://www.saclug.org/
Sites of sporadic interest:
Elec. Flea Mkt. 2nd Sat. at De Anza, Mar-Oct: http://www.asvaro.org/fleamarket
Intrusion Detection UG http://idug.cryptojail.net/
Mindsource BOFs http://www.mindsrc.com/
Linux.org events http://www.linux.org/event/
Mozilla Conference
LWCE http://www.linuxworldexpo.com/
Holidays, time changes! see: http://linuxmafia.com/bale/holidays
O'Reilly Open Source Convention, http://conferences.oreillynet.com/os2006/
EFF Bay Chapter http://www.eff.org/bayff/
Bay Area Cypherpunks http://www.cryptorights.org/lists/meetingpunks/
South Bay Community Network sbay.org
Computer History Museum http://www.computerhistory.org/events/
Annual Linux Picnix http://www.linuxpicnic.org/
Stacey's Book Store Events http://www.staceys.com/sanfranciscoevents.html
USENIX Association http://www.usenix.org/events/
CodeCon http://www.codecon.org/
Computers, Freedom, and Privacy http://www.cfp.org/
IEEE Security Symposiums http://www.ieee-security.org/
May 21-24, 2006: http://www.ieee-security.org/TC/SP2006/oakland06.html
Computer Professionals for Social Responsibility http://www.cpsr.org/
Vintage Computer Festival http://www.vintage.org/events.php (November?)
Howard's Events list: http://lg-events.blogspot.com/
Dead groups:
BANG http://www.bang.org/
Internet Developer/Technology Group http://www.inetdevgrp.org/
LBNL LUG http://lug.lbl.gov/
Linux Events http://linevents.com/ (gone)
Linux Movies Group http://www.linuxmovies.org/
SCLUG http://linus.got.net/
sig-beer-west http://www.extasia.org/sig-beer-west/
SlugLUG http://sluglug.ucsc.edu/
WebGuild http://www.webguild.org/
Monday, June 04, 2007
What would happen when tried to compile and run this program?
2. Runtime Error.
3. Compiles and Runs with no errors.
Answer : 1
1. Compilation Error
public class StaticVar
{
void execute(String b){
static String str;
str = b;
}
public static void main( String[] args ) {
StaticVar var = new StaticVar();
var.execute( "blah");
}
}
2. Runtime Error.
3. Compiles and Runs with no errors.
Answer : 1
Labels: java questions
Wednesday, May 02, 2007
Nice posting for converting bytes to string and vice versa.
http://forum.java.sun.com/thread.jspa?threadID=659432&messageID=3873051
http://forum.java.sun.com/thread.jspa?threadID=659432&messageID=3873051