| /rocksdb-6.9/docs/_docs/ |
| D | faq.md | 20 …that it was unsuitable for our server workloads. The [benchmark results](http://leveldb.googlecode…
22 …that RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that LevelDB’s sin…
26 RocksDB can be used by applications that need low latency database accesses. Possibilities include:
28 * A user-facing application that stores the viewing history and state of users of a website.
29 * A spam detection application that needs fast access to big data sets.
30 * A graph-search query that needs to scan a data set in realtime.
32 * A message-queue that supports a high number of inserts and deletes.
36 RocksDB is an embedded storage engine that is used in a number of backend systems at Facebook. In t…
42 Our engineering team at Facebook firmly believes that RocksDB has great potential as storage engine…
48 …ocksdb) because we think it will be useful beyond Facebook. We are hoping that software programmer…
|
| /rocksdb-6.9/docs/_posts/ |
| D | 2017-12-19-write-prepared-txn.markdown | 8 …write policy in pessimistic transactions is _WriteCommitted_, which means that the data is written…
12 …that is read by other transactions can be assumed to be committed. This write policy, however, imp…
16 …that the commit phase become lightweight and fast. 2PC is composed of Write stage, where the trans…
20 These are the primary design questions that needs to be addressed:
21 1) How do we identify the key/values in the DB with transactions that wrote them?
25 …that we call _CommitCache_. When a transaction reading values from the DB (tagged with `prepare_se…
27 …that caches the recent commit entries. Looking up the entries in the cache must be enough for almo…
|
| D | 2015-07-17-spatial-indexing-in-rocksdb.markdown | 14 …we started looking at this project, the first thing that surprised us was that our planet is not t…
18 …that you don't need to reload your database when you change your map style. You can just change yo…
20 …/spatial_db.h). The API is focused on map rendering use-case, but we hope that it can also be used…
22 … API. When you create a spatial database, you specify the spatial indexes that need to be built. E…
37 …that will index the feature. In the loading phase we process the map style to determine the list o…
50 The indexing part is pretty simple. For each feature, we first find a list of index tiles that it i…
52 After you're done inserting all the features, you can call an API Compact() that will compact the d…
61 … bounding box we're interested in, and 2) a zoom level. We find all tiles that intersect with the …
|
| D | 2018-11-21-delete-range.markdown | 18 four bytes identify the table or index to which that key belongs. Thus dropping
19 a table or index involves deleting all the keys with that prefix.
21 Rockssandra is a Cassandra variant that uses RocksDB as its storage engine. One
29 When we decide what to improve, we try to find a use case that's common across
30 users, since we want to build a generally useful system, not one that has many
42 Another common pattern is using a custom compaction filter that drops keys in
49 If space reclamation time is important, or it is important that the deleted
56 as snapshot readers do not need to access them. It drops files that are
98 that newer versions of a key are always in a higher level to prevent the seqnum
129 find it for arbitrary keys that it covers.
[all …]
|
| D | 2014-03-27-how-to-backup-rocksdb.markdown | 73 …her with information on timestamp of the backup and the size (please note that sum of all backups'…
77 …ternative is `RestoreDBFromBackup()` which takes a backup ID and restores that particular backup. …
86 `BackupableDBOptions::info_log` is a Logger object that is used to print out LOG messages if not-nu…
88 …` is true, we will sync data to disk after every file write, guaranteeing that backups will be con…
92 …kup` is false, backup will not issue flush before starting the backup. In that case, the backup wi…
115 …that is already present in the backup directory. For example, if there is a file `00050.sst` alrea…
128 …that contains the ID of the latest backup. If we crash in middle of backing up, on a restart we wi…
|
| D | 2018-08-23-data-block-hash-index.markdown | 7 We've designed and implemented a _data block hash index_ in RocksDB that has the benefit of both re…
23 // The index type that will be used for the data block.
43 …ching triggers CPU cache miss, causing much CPU utilization. We have seen that this binary search …
52 … just mark the bucket as “collision”. So that when later querying on that key, the hash table look…
60 ### Things that Need Attention
77 NOTE: to use the hash index feature, one should 1) have a comparator that can never treat different…
82 … our experiment, we found util ratio = 0.5 ~ 1 is a good range to explore that brings both CPU and…
106 … we first evaluate the hash idnex in a purely cached workload. We observe that the CPU utilization…
115 We can see that if cache size is greater than 8GB, hash index can bring throughput gain. Cache size…
|
| D | 2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown | 16 … a binary search based on FileMetaData.largest to locate a candidate file that can potentially con…
18 One observation to this problem is that: after the LSM tree is built, an SST file's position in its…
24 … key on level 1, we have derived hints that target key is in range between 200 and 300. Any files …
26 Inspired by this concept, we pre-build pointers at compaction time on level 1 files that point to a…
28 Our benchmark shows that this optimization improves lookup QPS by ~5% for similar setup mentioned […
|
| D | 2017-05-12-partitioned-index-filter.markdown | 8 … negatively affects the performance by i) occupying the block cache space that could otherwise be …
16 …her) on this scarce resource. A filter of size 5MB is occupying the space that could otherwise be …
18 …t might end up also loading multiple megabytes of index/filter blocks. If that happens often then …
22 …the top-level index to load on demand into the block cache the partitions that are required to per…
28 In this example we have a DB of size 86G on HDD and emulate the small memory that is present to a n…
32 In this example we have a DB of size 300G on SSD and emulate the small memory that would be availab…
|
| D | 2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown | 16 …that the log file is obsolete). However, if your table file is stored in in-memory file system, yo…
21 …files) with a frequency of less than one day. RocksDB provides an utility that enables you to easi…
23 … up log files, since they are already stored in persistent storage. To do that, set BackupableDBOp…
25 …ry. Since we didn't include log files in the backup, we need to make sure that restoring the datab…
|
| D | 2016-03-07-rocksdb-options-file.markdown | 10 In RocksDB 4.3, we added a new set of features that makes managing RocksDB options easier. Specifi…
16 …ook/rocksdb/blob/4.3.fb/include/rocksdb/utilities/options_util.h#L48-L58) that enables developers …
20 …ook/rocksdb/blob/4.3.fb/include/rocksdb/utilities/options_util.h#L64-L77) that performs compatibil…
|
| D | 2017-12-18-17-auto-tuned-rate-limiter.markdown | 10 …that is low enough to prevent background I/O spikes, which can impact user-visible read/write late…
12 To address this, in RocksDB 5.9 we released a dynamic rate limiter that adjusts itself over time ac…
16 …hin a window bounded by the user-specified upper-bound, and a lower-bound that we derive internall…
|
| D | 2017-03-02-rocksdb-5-2-1-released.markdown | 13 * Added new overloaded function GetApproximateSizes that allows to specify if memtable stats should…
14 * Added new function GetApproximateMemTableStats that approximates both number of records and size …
15 * (Experimental) Two-level indexing that partition the index and creates a 2nd level index on the p…
|
| /rocksdb-6.9/ |
| D | LICENSE.Apache | 14 the copyright owner that is granting the License.
17 other entities that control, are controlled by, or are under common
18 control with that entity. For the purposes of this definition,
38 copyright notice that is included in or attached to the work
51 to that Work or Derivative Works thereof, that is intentionally
80 by such Contributor that are necessarily infringed by their
92 modifications, and in Source or Object form, provided that You
99 stating that You changed the files; and
102 that You distribute, all copyright, patent, trademark, and
104 excluding those notices that do not pertain to any part of
[all …]
|
| D | COPYING | 26 in new free programs; and that you know you can do these things.
28 To protect your rights, we need to make restrictions that forbid
44 that everyone understands that there is no warranty for this free
67 that is to say, a work containing the Program or a portion of it,
77 Whether that is true depends on what the Program does.
80 source code as you receive it, in any medium, provided that you
93 above, provided that you also meet all of these conditions:
98 b) You must cause any work that you distribute or publish, that in
107 notice that there is no warranty (or else, saying that you provide
221 through that system in reliance on consistent application of that
[all …]
|
| D | WINDOWS_PORT.md | 18 … best of our knowledge there is not a specific API we used during porting that is unsupported on o…
32 At the same time it generates Visual Studio projects that are both usable from a command line and I…
35 We think that it would be beneficial to merge the existing make-based build system and the new cmak…
37 …. We have not conducted any testing for 32-bit and early reports indicate that it will not run on …
44 * Certain headers that are not present and not necessary on Windows were simply `#ifndef OS_WIN` in…
52 * Union members that have non-trivial constructors were replaced to `char[]` in one place along wit…
53 * Zero-sized arrays are deemed a non-standard extension which we converted to 1 size array and that…
71 …e that functionality of `pread/pwrite` reasonably well. The only difference is that the file point…
80 …_local.cc file that injects a cleanup callback into a `"__tls"` structure within `".CRT$XLB"` data…
84 …f the C++ globals or statics. To accomplish that we injected an initialization routine into `".CRT…
[all …]
|
| D | HISTORY.md | 59 * Fix a bug that can cause unnecessary bg thread to be scheduled(#6104).
360 …ksdb.min-obsolete-sst-number-to-keep" DB property that reports the lower bound on SST file numbers…
646 * Fix the bug that Direct I/O uses direct reads for non-SST file
656 * The old API that accepts std::string, although discouraged, is still supported.
673 * Fix the bug that iterator may skip keys
699 * Fix the bug that if 2PC is enabled, checkpoints may loss some recent transactions.
912 * Add DB::SyncWAL() that does a WAL sync without blocking writers.
1037 * Add DB::GetIntProperty(), which returns DB properties that are integer as uint64_t.
1111 we will ignore it. We assume that writers of these records were interrupted
1112 and that we can safely ignore it.
[all …]
|
| D | ROCKSDB_LITE.md | 3 … and they are very sensitive to binary size. For that reason, we added a compile flag ROCKSDB_LITE…
10 * No special-purpose memtables that are highly optimized for specific use cases
|
| /rocksdb-6.9/docs/ |
| D | LICENSE-DOCUMENTATION | 18 conditions that creators and other rights holders may use to share
31 applying our licenses so that the public can reuse the
43 copyright--then that use is not regulated by the license. Our
45 other rights that a licensor has authority to grant. Use of
49 such as asking that all changes be marked or described.
100 that applies to Your use of the Licensed Material.
118 available to the public including in ways that members of the
228 iii. a notice that refers to this Public License;
230 iv. a notice that refers to the disclaimer of
247 hyperlink to a resource that includes the required
[all …]
|
| D | TEMPLATE-INFORMATION.md | 3 …kill the `jekyll serve` instance and restart it to see those changes, but that's only the case wit…
11 …ify a series of 'promotional' elements in `_data/promo.yml`. You can read that file for more infor…
13 …Checkout this folder's index.md for an example of one common template tag that we use on our sites…
15 …that is intended to show a list of companies or apps that are using your project. The `powered_by_…
|
| D | CONTRIBUTING.md | 8 …E-FOR-RUNNING-DOCS.md` to `README.md` (replacing the existing `README.md` that came with the templ…
16 Most content is written in markdown. You name the file `something.md`, then have a header that look…
74 …` in markdown (file extension `.md` or `.markdown`). See current posts in that folder or `./doc-ty…
75 …a `<!--truncate-->` tag in the middle of your post such that you show only the excerpt above that …
103 1. If you want a visible link to that file, update `_data/nav.yml` to add a link to your new top-le…
105 …> This is not necessary if you just want to have a page that is linked to from another page, but n…
|
| /rocksdb-6.9/java/src/main/java/org/rocksdb/ |
| D | ColumnFamilyHandle.java | 77 final ColumnFamilyHandle that = (ColumnFamilyHandle) o; in equals() local
79 return rocksDB_.nativeHandle_ == that.rocksDB_.nativeHandle_ && in equals()
80 getID() == that.getID() && in equals()
81 Arrays.equals(getName(), that.getName()); in equals()
|
| D | ColumnFamilyDescriptor.java | 95 final ColumnFamilyDescriptor that = (ColumnFamilyDescriptor) o; in equals() local
96 return Arrays.equals(columnFamilyName_, that.columnFamilyName_) in equals()
97 && columnFamilyOptions_.nativeHandle_ == that.columnFamilyOptions_.nativeHandle_; in equals()
|
| /rocksdb-6.9/hdfs/ |
| D | README | 6 The configuration assumes that packages libhdfs0, libhdfs0-dev are
7 installed which basically means that hdfs.h is in /usr/include and libhdfs in /usr/lib
9 The env_hdfs.h file defines the rocksdb objects that are needed to talk to an
|
| /rocksdb-6.9/third-party/folly/folly/ |
| D | Optional.h | 234 void swap(Optional& that) noexcept(IsNothrowSwappable<Value>::value) { in swap() argument
235 if (hasValue() && that.hasValue()) { in swap()
237 swap(value(), that.value()); in swap()
239 that.emplace(std::move(value())); in swap()
241 } else if (that.hasValue()) { in swap()
242 emplace(std::move(that.value())); in swap()
243 that.reset(); in swap()
|
| /rocksdb-6.9/tools/advisor/ |
| D | README.md | 16 Users provide the Rocksdb configuration that they want to improve upon (as the
49 Most important amongst all the input that the Advisor needs, are the rules
57 (increase/decrease/set to a suggested value) to certain Rocksdb options that
59 the data source that would be checked for triggering that condition.
|