xref: /leveldb-1.20/doc/benchmark.html (revision 021ee9c3)
1e301f17cS[email protected]<!DOCTYPE html>
2f122c6dfS[email protected]<html>
3f122c6dfS[email protected]<head>
4f122c6dfS[email protected]<title>LevelDB Benchmarks</title>
5f122c6dfS[email protected]<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
6f122c6dfS[email protected]<style>
7f122c6dfS[email protected]body {
8f122c6dfS[email protected]  font-family:Helvetica,sans-serif;
9f122c6dfS[email protected]  padding:20px;
10f122c6dfS[email protected]}
11f122c6dfS[email protected]
12f122c6dfS[email protected]h2 {
13f122c6dfS[email protected]  padding-top:30px;
14f122c6dfS[email protected]}
15f122c6dfS[email protected]
16f122c6dfS[email protected]table.bn {
17f122c6dfS[email protected]  width:800px;
18f122c6dfS[email protected]  border-collapse:collapse;
19f122c6dfS[email protected]  border:0;
20f122c6dfS[email protected]  padding:0;
21f122c6dfS[email protected]}
22f122c6dfS[email protected]
23f122c6dfS[email protected]table.bnbase {
24f122c6dfS[email protected]  width:650px;
25f122c6dfS[email protected]}
26f122c6dfS[email protected]
27f122c6dfS[email protected]table.bn td {
28f122c6dfS[email protected]  padding:2px 0;
29f122c6dfS[email protected]}
30f122c6dfS[email protected]
31f122c6dfS[email protected]table.bn td.c1 {
32f122c6dfS[email protected]  font-weight:bold;
33f122c6dfS[email protected]  width:150px;
34f122c6dfS[email protected]}
35f122c6dfS[email protected]
36f122c6dfS[email protected]table.bn td.c1 div.e {
37f122c6dfS[email protected]  float:right;
38f122c6dfS[email protected]  font-weight:normal;
39f122c6dfS[email protected]}
40f122c6dfS[email protected]
41f122c6dfS[email protected]table.bn td.c2 {
42f122c6dfS[email protected]  width:150px;
43f122c6dfS[email protected]  text-align:right;
44f122c6dfS[email protected]  padding:2px;
45f122c6dfS[email protected]}
46f122c6dfS[email protected]
47f122c6dfS[email protected]table.bn td.c3 {
48f122c6dfS[email protected]  width:350px;
49f122c6dfS[email protected]}
50f122c6dfS[email protected]
51f122c6dfS[email protected]table.bn td.c4 {
52f122c6dfS[email protected]  width:150px;
53f122c6dfS[email protected]  font-size:small;
54f122c6dfS[email protected]  padding-left:4px;
55f122c6dfS[email protected]}
56f122c6dfS[email protected]
57f122c6dfS[email protected]/* chart bars */
58f122c6dfS[email protected]div.bldb {
59f122c6dfS[email protected]  background-color:#0255df;
60f122c6dfS[email protected]}
61f122c6dfS[email protected]
62f122c6dfS[email protected]div.bkct {
63f122c6dfS[email protected]  background-color:#df5555;
64f122c6dfS[email protected]}
65f122c6dfS[email protected]
66f122c6dfS[email protected]div.bsql {
67f122c6dfS[email protected]  background-color:#aadf55;
68f122c6dfS[email protected]}
69f122c6dfS[email protected]
70f122c6dfS[email protected].code {
71f122c6dfS[email protected]  font-family:monospace;
72f122c6dfS[email protected]  font-size:large;
73f122c6dfS[email protected]}
74f122c6dfS[email protected]
75f122c6dfS[email protected].todo {
76f122c6dfS[email protected]  color: red;
77f122c6dfS[email protected]}
78f122c6dfS[email protected]
79f122c6dfS[email protected]</style>
80f122c6dfS[email protected]</head>
81f122c6dfS[email protected]<body>
82f122c6dfS[email protected]<h1>LevelDB Benchmarks</h1>
83f122c6dfS[email protected]<p>Google, July 2011</p>
84f122c6dfS[email protected]<hr>
85f122c6dfS[email protected]
86f122c6dfS[email protected]<p>In order to test LevelDB's performance, we benchmark it against other well-established database implementations. We compare LevelDB (revision 39) against <a href="http://www.sqlite.org/">SQLite3</a> (version 3.7.6.3) and <a href="http://fallabs.com/kyotocabinet/spex.html">Kyoto Cabinet's</a> (version 1.2.67) TreeDB (a B+Tree based key-value store). We would like to acknowledge Scott Hess and Mikio Hirabayashi for their suggestions and contributions to the SQLite3 and Kyoto Cabinet benchmarks, respectively.</p>
87f122c6dfS[email protected]
881bfbe76bS[email protected]<p>Benchmarks were all performed on a six-core Intel(R) Xeon(R) CPU X5650 @ 2.67GHz, with 12288 KB of total L3 cache and 12 GB of DDR3 RAM at 1333 MHz. (Note that LevelDB uses at most two CPUs since the benchmarks are single threaded: one to run the benchmark, and one for background compactions.) We ran the benchmarks on two machines (with identical processors), one with an Ext3 file system and one with an Ext4 file system. The machine with the Ext3 file system has a SATA Hitachi HDS721050CLA362 hard drive. The machine with the Ext4 file system has a SATA Samsung HD502HJ hard drive. Both hard drives spin at 7200 RPM and have hard drive write-caching enabled (using `hdparm -W 1 [device]`). The numbers reported below are the median of three measurements.</p>
89f122c6dfS[email protected]
90f122c6dfS[email protected]<h4>Benchmark Source Code</h4>
91f122c6dfS[email protected]<p>We wrote benchmark tools for SQLite and Kyoto TreeDB based on LevelDB's <span class="code">db_bench</span>. The code for each of the benchmarks resides here:</p>
92f122c6dfS[email protected]<ul>
93f122c6dfS[email protected]	<li> <b>LevelDB:</b> <a href="http://code.google.com/p/leveldb/source/browse/trunk/db/db_bench.cc">db/db_bench.cc</a>.</li>
94f122c6dfS[email protected]	<li> <b>SQLite:</b> <a href="http://code.google.com/p/leveldb/source/browse/#svn%2Ftrunk%2Fdoc%2Fbench%2Fdb_bench_sqlite3.cc">doc/bench/db_bench_sqlite3.cc</a>.</li>
95f122c6dfS[email protected]	<li> <b>Kyoto TreeDB:</b> <a href="http://code.google.com/p/leveldb/source/browse/#svn%2Ftrunk%2Fdoc%2Fbench%2Fdb_bench_tree_db.cc">doc/bench/db_bench_tree_db.cc</a>.</li>
96f122c6dfS[email protected]</ul>
97f122c6dfS[email protected]
98f122c6dfS[email protected]<h4>Custom Build Specifications</h4>
99f122c6dfS[email protected]<ul>
1001bfbe76bS[email protected]<li>LevelDB: LevelDB was compiled with the <a href="http://code.google.com/p/google-perftools">tcmalloc</a> library and the <a href="http://code.google.com/p/snappy/">Snappy</a> compression library (revision 33).  Assertions were disabled.</li>
1011bfbe76bS[email protected]<li>TreeDB: TreeDB was compiled using the <a href="http://www.oberhumer.com/opensource/lzo/">LZO</a> compression library (version 2.03). Furthermore, we enabled the TSMALL and TLINEAR options when opening the database in order to reduce the footprint of each record.</li>
1021bfbe76bS[email protected]<li>SQLite: We tuned SQLite's performance, by setting its locking mode to exclusive.  We also enabled SQLite's <a href="http://www.sqlite.org/draft/wal.html">write-ahead logging</a>.</li>
103f122c6dfS[email protected]</ul>
104f122c6dfS[email protected]
105f122c6dfS[email protected]<h2>1. Baseline Performance</h2>
106b9ef9141S[email protected]<p>This section gives the baseline performance of all the
107f122c6dfS[email protected]databases.  Following sections show how performance changes as various
108f122c6dfS[email protected]parameters are varied.  For the baseline:</p>
109f122c6dfS[email protected]<ul>
110f122c6dfS[email protected]	<li> Each database is allowed 4 MB of cache memory.</li>
111f122c6dfS[email protected]        <li> Databases are opened in <em>asynchronous</em> write mode.
112f122c6dfS[email protected]             (LevelDB's sync option, TreeDB's OAUTOSYNC option, and
113f122c6dfS[email protected]             SQLite3's synchronous options are all turned off).  I.e.,
114f122c6dfS[email protected]             every write is pushed to the operating system, but the
115f122c6dfS[email protected]             benchmark does not wait for the write to reach the disk.</li>
116f122c6dfS[email protected]	<li> Keys are 16 bytes each.</li>
117f122c6dfS[email protected]        <li> Value are 100 bytes each (with enough redundancy so that
118f122c6dfS[email protected]             a simple compressor shrinks them to 50% of their original
119f122c6dfS[email protected]             size).</li>
120f122c6dfS[email protected]	<li> Sequential reads/writes traverse the key space in increasing order.</li>
121f122c6dfS[email protected]	<li> Random reads/writes traverse the key space in random order.</li>
122f122c6dfS[email protected]</ul>
123f122c6dfS[email protected]
124f122c6dfS[email protected]<h3>A. Sequential Reads</h3>
125f122c6dfS[email protected]<table class="bn bnbase">
126f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
127f122c6dfS[email protected]    <td class="c2">4,030,000 ops/sec</td>
128f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
129f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
130f122c6dfS[email protected]    <td class="c2">1,010,000 ops/sec</td>
131f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:95px">&nbsp;</div></td>
132f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
133*021ee9c3S[email protected]    <td class="c2">383,000 ops/sec</td>
134*021ee9c3S[email protected]    <td class="c3"><div class="bsql" style="width:33px">&nbsp;</div></td>
135f122c6dfS[email protected]</table>
136f122c6dfS[email protected]<h3>B. Random Reads</h3>
137f122c6dfS[email protected]<table class="bn bnbase">
138f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
139f122c6dfS[email protected]    <td class="c2">129,000 ops/sec</td>
140f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:298px">&nbsp;</div></td>
141f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
142f122c6dfS[email protected]    <td class="c2">151,000 ops/sec</td>
143f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:350px">&nbsp;</div></td>
144f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
1451bfbe76bS[email protected]    <td class="c2">134,000 ops/sec</td>
1461bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:310px">&nbsp;</div></td>
147f122c6dfS[email protected]</table>
148f122c6dfS[email protected]<h3>C. Sequential Writes</h3>
149f122c6dfS[email protected]<table class="bn bnbase">
150f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
151f122c6dfS[email protected]    <td class="c2">779,000 ops/sec</td>
152f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
153f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
154f122c6dfS[email protected]    <td class="c2">342,000 ops/sec</td>
155f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:154px">&nbsp;</div></td>
156f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
1571bfbe76bS[email protected]    <td class="c2">48,600 ops/sec</td>
1581bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:22px">&nbsp;</div></td>
159f122c6dfS[email protected]</table>
160f122c6dfS[email protected]<h3>D. Random Writes</h3>
161f122c6dfS[email protected]<table class="bn bnbase">
162f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
163f122c6dfS[email protected]    <td class="c2">164,000 ops/sec</td>
164f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
165f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
166f122c6dfS[email protected]    <td class="c2">88,500 ops/sec</td>
167f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:188px">&nbsp;</div></td>
168f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
1691bfbe76bS[email protected]    <td class="c2">9,860 ops/sec</td>
1701bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:21px">&nbsp;</div></td>
171f122c6dfS[email protected]</table>
172f122c6dfS[email protected]
173f122c6dfS[email protected]<p>LevelDB outperforms both SQLite3 and TreeDB in sequential and random write operations and sequential read operations. Kyoto Cabinet has the fastest random read operations.</p>
174f122c6dfS[email protected]
175f122c6dfS[email protected]<h2>2. Write Performance under Different Configurations</h2>
176f122c6dfS[email protected]<h3>A. Large Values </h3>
177f122c6dfS[email protected]<p>For this benchmark, we start with an empty database, and write 100,000 byte values (~50% compressible). To keep the benchmark running time reasonable, we stop after writing 1000 values.</p>
178f122c6dfS[email protected]<h4>Sequential Writes</h4>
179e8dee348S[email protected]<table class="bn bnbase">
180f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
1811bfbe76bS[email protected]    <td class="c2">1,100 ops/sec</td>
1821bfbe76bS[email protected]    <td class="c3"><div class="bldb" style="width:234px">&nbsp;</div></td></tr>
183f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
1841bfbe76bS[email protected]    <td class="c2">1,000 ops/sec</td>
1851bfbe76bS[email protected]    <td class="c3"><div class="bkct" style="width:224px">&nbsp;</div></td></tr>
186f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
1871bfbe76bS[email protected]    <td class="c2">1,600 ops/sec</td>
188e8dee348S[email protected]    <td class="c3"><div class="bsql" style="width:350px">&nbsp;</div></td></tr>
189f122c6dfS[email protected]</table>
190f122c6dfS[email protected]<h4>Random Writes</h4>
191e8dee348S[email protected]<table class="bn bnbase">
192f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
193f122c6dfS[email protected]    <td class="c2">480 ops/sec</td>
1941bfbe76bS[email protected]    <td class="c3"><div class="bldb" style="width:105px">&nbsp;</div></td></tr>
195f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
196f122c6dfS[email protected]    <td class="c2">1,100 ops/sec</td>
1971bfbe76bS[email protected]    <td class="c3"><div class="bkct" style="width:240px">&nbsp;</div></td></tr>
198f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
1991bfbe76bS[email protected]    <td class="c2">1,600 ops/sec</td>
2001bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:350px">&nbsp;</div></td></tr>
201f122c6dfS[email protected]</table>
202f122c6dfS[email protected]<p>LevelDB doesn't perform as well with large values of 100,000 bytes each. This is because LevelDB writes keys and values at least twice: first time to the transaction log, and second time (during a compaction) to a sorted file.
203f122c6dfS[email protected]With larger values, LevelDB's per-operation efficiency is swamped by the
204f122c6dfS[email protected]cost of extra copies of large values.</p>
205f122c6dfS[email protected]<h3>B. Batch Writes</h3>
206f122c6dfS[email protected]<p>A batch write is a set of writes that are applied atomically to the underlying database. A single batch of N writes may be significantly faster than N individual writes. The following benchmark writes one thousand batches where each batch contains one thousand 100-byte values. TreeDB does not support batch writes and is omitted from this benchmark.</p>
207f122c6dfS[email protected]<h4>Sequential Writes</h4>
208f122c6dfS[email protected]<table class="bn">
209f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
210f122c6dfS[email protected]    <td class="c2">840,000 entries/sec</td>
211f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
212f122c6dfS[email protected]    <td class="c4">(1.08x baseline)</td></tr>
213f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
2141bfbe76bS[email protected]    <td class="c2">124,000 entries/sec</td>
2151bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:52px">&nbsp;</div></td>
2161bfbe76bS[email protected]    <td class="c4">(2.55x baseline)</td></tr>
217f122c6dfS[email protected]</table>
218f122c6dfS[email protected]<h4>Random Writes</h4>
219f122c6dfS[email protected]<table class="bn">
220f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
221f122c6dfS[email protected]    <td class="c2">221,000 entries/sec</td>
222f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
223f122c6dfS[email protected]    <td class="c4">(1.35x baseline)</td></tr>
224f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
2251bfbe76bS[email protected]    <td class="c2">22,000 entries/sec</td>
2261bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:34px">&nbsp;</div></td>
2271bfbe76bS[email protected]    <td class="c4">(2.23x baseline)</td></tr>
228f122c6dfS[email protected]</table>
229f122c6dfS[email protected]
230f122c6dfS[email protected]<p>Because of the way LevelDB persistent storage is organized, batches of
231f122c6dfS[email protected]random writes are not much slower (only a factor of 4x) than batches
2321bfbe76bS[email protected]of sequential writes.</p>
233f122c6dfS[email protected]
234b9ef9141S[email protected]<h3>C. Synchronous Writes</h3>
235f122c6dfS[email protected]<p>In the following benchmark, we enable the synchronous writing modes
236f122c6dfS[email protected]of all of the databases.  Since this change significantly slows down the
2371bfbe76bS[email protected]benchmark, we stop after 10,000 writes. For synchronous write tests, we've
2381bfbe76bS[email protected]disabled hard drive write-caching (using `hdparm -W 0 [device]`).</p>
239f122c6dfS[email protected]<ul>
240f122c6dfS[email protected]    <li>For LevelDB, we set WriteOptions.sync = true.</li>
241f122c6dfS[email protected]    <li>In TreeDB, we enabled TreeDB's OAUTOSYNC option.</li>
242f122c6dfS[email protected]    <li>For SQLite3, we set "PRAGMA synchronous = FULL".</li>
243f122c6dfS[email protected]</ul>
244f122c6dfS[email protected]<h4>Sequential Writes</h4>
245f122c6dfS[email protected]<table class="bn">
246f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
2471bfbe76bS[email protected]    <td class="c2">100 ops/sec</td>
248f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
249f122c6dfS[email protected]    <td class="c4">(0.003x baseline)</td></tr>
250f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
2511bfbe76bS[email protected]    <td class="c2">7 ops/sec</td>
2521bfbe76bS[email protected]    <td class="c3"><div class="bkct" style="width:27px">&nbsp;</div></td>
253f122c6dfS[email protected]    <td class="c4">(0.0004x baseline)</td></tr>
254f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
2551bfbe76bS[email protected]    <td class="c2">88 ops/sec</td>
2561bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:315px">&nbsp;</div></td>
2571bfbe76bS[email protected]    <td class="c4">(0.002x baseline)</td></tr>
258f122c6dfS[email protected]</table>
259f122c6dfS[email protected]<h4>Random Writes</h4>
260f122c6dfS[email protected]<table class="bn">
261f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
2621bfbe76bS[email protected]    <td class="c2">100 ops/sec</td>
263f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
264f122c6dfS[email protected]    <td class="c4">(0.015x baseline)</td></tr>
265f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
2661bfbe76bS[email protected]    <td class="c2">8 ops/sec</td>
2671bfbe76bS[email protected]    <td class="c3"><div class="bkct" style="width:29px">&nbsp;</div></td>
268f122c6dfS[email protected]    <td class="c4">(0.001x baseline)</td></tr>
269f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
2701bfbe76bS[email protected]    <td class="c2">88 ops/sec</td>
2711bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:314px">&nbsp;</div></td>
2721bfbe76bS[email protected]    <td class="c4">(0.009x baseline)</td></tr>
273f122c6dfS[email protected]</table>
274f122c6dfS[email protected]
275f122c6dfS[email protected]<p>Also see the <code>ext4</code> performance numbers below
276f122c6dfS[email protected]since synchronous writes behave significantly differently
277f122c6dfS[email protected]on <code>ext3</code> and <code>ext4</code>.</p>
278f122c6dfS[email protected]
279f122c6dfS[email protected]<h3>D. Turning Compression Off</h3>
280f122c6dfS[email protected]
281f122c6dfS[email protected]<p>In the baseline measurements, LevelDB and TreeDB were using
282f122c6dfS[email protected]light-weight compression
283f122c6dfS[email protected](<a href="http://code.google.com/p/snappy/">Snappy</a> for LevelDB,
284f122c6dfS[email protected]and <a href="http://www.oberhumer.com/opensource/lzo/">LZO</a> for
285f122c6dfS[email protected]TreeDB). SQLite3, by default does not use compression.  The
286f122c6dfS[email protected]experiments below show what happens when compression is disabled in
287f122c6dfS[email protected]all of the databases (the SQLite3 numbers are just a copy of
288f122c6dfS[email protected]its baseline measurements):</p>
289f122c6dfS[email protected]
290f122c6dfS[email protected]<h4>Sequential Writes</h4>
291f122c6dfS[email protected]<table class="bn">
292f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
293f122c6dfS[email protected]    <td class="c2">594,000 ops/sec</td>
294f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
295f122c6dfS[email protected]    <td class="c4">(0.76x baseline)</td></tr>
296f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
297f122c6dfS[email protected]    <td class="c2">485,000 ops/sec</td>
298f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:239px">&nbsp;</div></td>
299f122c6dfS[email protected]    <td class="c4">(1.42x baseline)</td></tr>
300f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
3011bfbe76bS[email protected]    <td class="c2">48,600 ops/sec</td>
3021bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:29px">&nbsp;</div></td>
303f122c6dfS[email protected]    <td class="c4">(1.00x baseline)</td></tr>
304f122c6dfS[email protected]</table>
305f122c6dfS[email protected]<h4>Random Writes</h4>
306f122c6dfS[email protected]<table class="bn">
307f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
308f122c6dfS[email protected]    <td class="c2">135,000 ops/sec</td>
309f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:296px">&nbsp;</div></td>
310f122c6dfS[email protected]    <td class="c4">(0.82x baseline)</td></tr>
311f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
312f122c6dfS[email protected]    <td class="c2">159,000 ops/sec</td>
313f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:350px">&nbsp;</div></td>
314f122c6dfS[email protected]    <td class="c4">(1.80x baseline)</td></tr>
315f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
3161bfbe76bS[email protected]    <td class="c2">9,860 ops/sec</td>
3171bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:22px">&nbsp;</div></td>
318f122c6dfS[email protected]    <td class="c4">(1.00x baseline)</td></tr>
319f122c6dfS[email protected]</table>
320f122c6dfS[email protected]
321f122c6dfS[email protected]<p>LevelDB's write performance is better with compression than without
322f122c6dfS[email protected]since compression decreases the amount of data that has to be written
323f122c6dfS[email protected]to disk.  Therefore LevelDB users can leave compression enabled in
324f122c6dfS[email protected]most scenarios without having worry about a tradeoff between space
325f122c6dfS[email protected]usage and performance.  TreeDB's performance on the other hand is
326f122c6dfS[email protected]better without compression than with compression.  Presumably this is
327f122c6dfS[email protected]because TreeDB's compression library (LZO) is more expensive than
328f122c6dfS[email protected]LevelDB's compression library (Snappy).<p>
329f122c6dfS[email protected]
330b9ef9141S[email protected]<h3>E. Using More Memory</h3>
331f122c6dfS[email protected]<p>We increased the overall cache size for each database to 128 MB. For LevelDB, we partitioned 128 MB into a 120 MB write buffer and 8 MB of cache (up from 2 MB of write buffer and 2 MB of cache). For SQLite3, we kept the page size at 1024 bytes, but increased the number of pages to 131,072 (up from 4096). For TreeDB, we also kept the page size at 1024 bytes, but increased the cache size to 128 MB (up from 4 MB).</p>
332f122c6dfS[email protected]<h4>Sequential Writes</h4>
333f122c6dfS[email protected]<table class="bn">
334f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
335f122c6dfS[email protected]    <td class="c2">812,000 ops/sec</td>
336f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
337f122c6dfS[email protected]    <td class="c4">(1.04x baseline)</td></tr>
338f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
339f122c6dfS[email protected]    <td class="c2">321,000 ops/sec</td>
340f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:138px">&nbsp;</div></td>
341f122c6dfS[email protected]    <td class="c4">(0.94x baseline)</td></tr>
342f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
3431bfbe76bS[email protected]    <td class="c2">48,500 ops/sec</td>
3441bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:21px">&nbsp;</div></td>
3451bfbe76bS[email protected]    <td class="c4">(1.00x baseline)</td></tr>
346f122c6dfS[email protected]</table>
347f122c6dfS[email protected]<h4>Random Writes</h4>
348f122c6dfS[email protected]<table class="bn">
349f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
350f122c6dfS[email protected]    <td class="c2">355,000 ops/sec</td>
351f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
352f122c6dfS[email protected]    <td class="c4">(2.16x baseline)</td></tr>
353f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
354f122c6dfS[email protected]    <td class="c2">284,000 ops/sec</td>
355f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:280px">&nbsp;</div></td>
356f122c6dfS[email protected]    <td class="c4">(3.21x baseline)</td></tr>
357f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
3581bfbe76bS[email protected]    <td class="c2">9,670 ops/sec</td>
3591bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:10px">&nbsp;</div></td>
3601bfbe76bS[email protected]    <td class="c4">(0.98x baseline)</td></tr>
361f122c6dfS[email protected]</table>
362f122c6dfS[email protected]
363f122c6dfS[email protected]<p>SQLite's performance does not change substantially when compared to
364f122c6dfS[email protected]the baseline, but the random write performance for both LevelDB and
365f122c6dfS[email protected]TreeDB increases significantly.  LevelDB's performance improves
366f122c6dfS[email protected]because a larger write buffer reduces the need to merge sorted files
367f122c6dfS[email protected](since it creates a smaller number of larger sorted files).  TreeDB's
368f122c6dfS[email protected]performance goes up because the entire database is available in memory
369f122c6dfS[email protected]for fast in-place updates.</p>
370f122c6dfS[email protected]
371b9ef9141S[email protected]  <h2>3. Read Performance under Different Configurations</h2>
372b9ef9141S[email protected]<h3>A. Larger Caches</h3>
373f122c6dfS[email protected]<p>We increased the overall memory usage to 128 MB for each database.
374f122c6dfS[email protected]For LevelDB, we allocated 8 MB to LevelDB's write buffer and 120 MB
375f122c6dfS[email protected]to LevelDB's cache. The other databases don't differentiate between a
376f122c6dfS[email protected]write buffer and a cache, so we simply set their cache size to 128
377f122c6dfS[email protected]MB.</p>
378f122c6dfS[email protected]<h4>Sequential Reads</h4>
379f122c6dfS[email protected]<table class="bn">
380f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
381f122c6dfS[email protected]    <td class="c2">5,210,000 ops/sec</td>
382f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
383f122c6dfS[email protected]    <td class="c4">(1.29x baseline)</td></tr>
384f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
385f122c6dfS[email protected]    <td class="c2">1,070,000 ops/sec</td>
386f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:72px">&nbsp;</div></td>
387f122c6dfS[email protected]    <td class="c4">(1.06x baseline)</td></tr>
388f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
389*021ee9c3S[email protected]    <td class="c2">609,000 ops/sec</td>
390*021ee9c3S[email protected]    <td class="c3"><div class="bsql" style="width:41px">&nbsp;</div></td>
391*021ee9c3S[email protected]    <td class="c4">(1.59x baseline)</td></tr>
392f122c6dfS[email protected]</table>
393f122c6dfS[email protected]
394f122c6dfS[email protected]<h4>Random Reads</h4>
395f122c6dfS[email protected]<table class="bn">
396f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
397f122c6dfS[email protected]    <td class="c2">190,000 ops/sec</td>
398f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:144px">&nbsp;</div></td>
399f122c6dfS[email protected]    <td class="c4">(1.47x baseline)</td></tr>
400f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
401f122c6dfS[email protected]    <td class="c2">463,000 ops/sec</td>
402f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:350px">&nbsp;</div></td>
403f122c6dfS[email protected]    <td class="c4">(3.07x baseline)</td></tr>
404f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
4051bfbe76bS[email protected]    <td class="c2">186,000 ops/sec</td>
4061bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:141px">&nbsp;</div></td>
4071bfbe76bS[email protected]    <td class="c4">(1.39x baseline)</td></tr>
408f122c6dfS[email protected]</table>
409f122c6dfS[email protected]
410f122c6dfS[email protected]<p>As expected, the read performance of all of the databases increases
411f122c6dfS[email protected]when the caches are enlarged.  In particular, TreeDB seems to make
412f122c6dfS[email protected]very effective use of a cache that is large enough to hold the entire
413f122c6dfS[email protected]database.</p>
414f122c6dfS[email protected]
415b9ef9141S[email protected]<h3>B. No Compression Reads </h3>
416f122c6dfS[email protected]<p>For this benchmark, we populated a database with 1 million entries consisting of 16 byte keys and 100 byte values. We compiled LevelDB and Kyoto Cabinet without compression support, so results that are read out from the database are already uncompressed. We've listed the SQLite3 baseline read performance as a point of comparison.</p>
417f122c6dfS[email protected]<h4>Sequential Reads</h4>
418f122c6dfS[email protected]<table class="bn">
419f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
420f122c6dfS[email protected]    <td class="c2">4,880,000 ops/sec</td>
421f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:350px">&nbsp;</div></td>
422f122c6dfS[email protected]    <td class="c4">(1.21x baseline)</td></tr>
423f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
424f122c6dfS[email protected]    <td class="c2">1,230,000 ops/sec</td>
425f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:88px">&nbsp;</div></td>
426f122c6dfS[email protected]    <td class="c4">(3.60x baseline)</td></tr>
427f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
428*021ee9c3S[email protected]    <td class="c2">383,000 ops/sec</td>
429*021ee9c3S[email protected]    <td class="c3"><div class="bsql" style="width:27px">&nbsp;</div></td>
430f122c6dfS[email protected]    <td class="c4">(1.00x baseline)</td></tr>
431f122c6dfS[email protected]</table>
432f122c6dfS[email protected]<h4>Random Reads</h4>
433f122c6dfS[email protected]<table class="bn">
434f122c6dfS[email protected]<tr><td class="c1">LevelDB</td>
435f122c6dfS[email protected]    <td class="c2">149,000 ops/sec</td>
436f122c6dfS[email protected]    <td class="c3"><div class="bldb" style="width:300px">&nbsp;</div></td>
437f122c6dfS[email protected]    <td class="c4">(1.16x baseline)</td></tr>
438f122c6dfS[email protected]<tr><td class="c1">Kyoto TreeDB</td>
439f122c6dfS[email protected]    <td class="c2">175,000 ops/sec</td>
440f122c6dfS[email protected]    <td class="c3"><div class="bkct" style="width:350px">&nbsp;</div></td>
441f122c6dfS[email protected]    <td class="c4">(1.16x baseline)</td></tr>
442f122c6dfS[email protected]<tr><td class="c1">SQLite3</td>
4431bfbe76bS[email protected]    <td class="c2">134,000 ops/sec</td>
4441bfbe76bS[email protected]    <td class="c3"><div class="bsql" style="width:268px">&nbsp;</div></td>
445f122c6dfS[email protected]    <td class="c4">(1.00x baseline)</td></tr>
446f122c6dfS[email protected]</table>
447f122c6dfS[email protected]
448f122c6dfS[email protected]<p>Performance of both LevelDB and TreeDB improves a small amount when
449f122c6dfS[email protected]compression is disabled.  Note however that under different workloads,
450f122c6dfS[email protected]performance may very well be better with compression if it allows more
451f122c6dfS[email protected]of the working set to fit in memory.</p>
452f122c6dfS[email protected]
453f122c6dfS[email protected]<h2>Note about Ext4 Filesystems</h2>
4541bfbe76bS[email protected]<p>The preceding numbers are for an ext3 file system. Synchronous writes are much slower under <a href="http://en.wikipedia.org/wiki/Ext4">ext4</a> (LevelDB drops to ~31 writes / second and TreeDB drops to ~5 writes / second; SQLite3's synchronous writes do not noticeably drop) due to ext4's different handling of <span class="code">fsync</span> / <span class="code">msync</span> calls. Even LevelDB's asynchronous write performance drops somewhat since it spreads its storage across multiple files and issues <span class="code">fsync</span> calls when switching to a new file.</p>
455f122c6dfS[email protected]
456f122c6dfS[email protected]<h2>Acknowledgements</h2>
457f122c6dfS[email protected]<p>Jeff Dean and Sanjay Ghemawat wrote LevelDB. Kevin Tseng wrote and compiled these benchmarks. Mikio Hirabayashi, Scott Hess, and Gabor Cselle provided help and advice.</p>
458f122c6dfS[email protected]</body>
459f122c6dfS[email protected]</html>
460