Tech Force - blog sobre tecnologia

Thousands concurrent parallel read write accesses over tens of millions of small files is a terrible performance tuning problem for e-mail servers. You must understand and fine tune all your infrastructure chain, following the previous articles for data storage and multipath on Debian 5.x Lenny. We reduced the CPU I/O wait from 30% to 0,3% (XFS) and 5% (EXT3) with these combined previously undocumented file system tuning tips. Mainframes have excellent parallel I/O performance. Data storage systems could deliver parallel I/O performance to Debian GNU/Linux if you configure them , your multipath connections , undestand the concepts, application behaviour and follow some hints for confguring and tuning your LVM, RAID, XFS, EXT3. The key word is  PARALLEL . Fiber channel data storage systems, multipath connections, excell at parallel I/O. These days of multicore, multi gigabyte RAM, fiber channel and many channels gigabit connec...

Small files performance is still the Achiles heel of GlusterFS. Tuning for replication (AFR, mirroring), distributed and NUFA setups is a non-trivial task, and you must know your application behaviour, your hardware and network infrastructure. For small files performance,  forget   GlusterFS  release  prior  to v. 2.0.8. Since version 2.0.8, the  bug 314  patch was introduced, and this quick-read translator patch proved to be essential. It provides up to 10 times previous small files performance. Some options at files below were undocumented, realized from gluster-users and gluster-devel mailing lists, my own experiments, and even from the source code. Use them at your risk. I did not find any stability issues at my setup once it was defined. Mistakes at configuration gave errors and crashes in few minutes under load. One of the non obvious solutions was to use io-threads at CLIENT side TOO for dealing with parallel file system request...

E-mail servers, data base servers, file servers, web servers struggle to cope with LATENCY on accessing millions of small files under heavy concurrent load. Low latency is key factor to a good user experience. With the Debian GNU / Linux 6.x kernel tuning, combined with the previous article hints about XenServer I/O latency, tuning filesystems configuring multipath, and data storage, we reduced eighty-four times our virtual block device latency at our environment. E-mail servers, data base servers, file servers, web servers struggle to cope with LATENCY on accessing millions of small files under heavy concurrent load. Low latency is key factor to a good user experience. With the Debian GNU / Linux 6.x kernel tuning, combined with the previous article hints about XenServer I/O latency, tuning filesystems configuring multipath, and data storage, we reduced eighty-four times our virtual block device latency...