Home
last modified time | relevance | path

Searched refs:filter (Results 1 – 25 of 52) sorted by relevance

123

/rocksdb-6.9/docs/_posts/
D2014-09-12-new-bloom-filter-format.markdown21 …bloom filter for each SST file. When we conduct a query for a key, we first goes to the bloom filt…
27 Here's the work flow for checking original bloom filter in block based table:
30 1. Using the "data block ID", we goes to the filter block and get the correct "offset of filter".
31 1. Using the "offset of filter", we goes to the actual filter and do the checking.
35 …filter creates filter for all keys in SST file and we name it "full filter". The data structure of…
37     [ full filter ]
39 In this way, the work flow of bloom filter checking is much simplified.
41 (1) Given the target key, we goes directly to the filter block and conduct the filter checking.
43 To be specific, there would be no checking for index block and no address jumping inside of filter …
45 Though it is a big filter, the total filter size would be the same as the original filter.
[all …]
D2017-05-12-partitioned-index-filter.markdown8 As DB/mem ratio gets larger, the memory footprint of filter/index blocks becomes non-trivial. Altho…
10 ### How large are the index/filter blocks?
12 …ne index/filter block per SST file. The size of the index/filter varies based on the configuration…
14 ### What is the big deal with large index/filter blocks?
16 …filter blocks are stored in block cache they are effectively competing with data blocks (as well a…
18 …filter, it has to be reloaded from the disk, and its large size is not helping in reducing the IO …
22 …filter of a SST file is partitioned into smaller blocks with an additional top-level index on them…
D2014-06-23-plaintable-a-new-file-format.markdown37 To make sure the format works efficiently with empty queries, we added a bloom filter check before …
41 …filter checks typically require multiple memory access. However, because they are independent, the…
/rocksdb-6.9/db/
Dmanual_compaction_test.cc207 filter->Reset(); in TEST_F()
209 ASSERT_EQ(0, filter->NumKeys()); in TEST_F()
217 filter->Reset(); in TEST_F()
219 ASSERT_EQ(2, filter->NumKeys()); in TEST_F()
229 filter->Reset(); in TEST_F()
231 ASSERT_EQ(0, filter->NumKeys()); in TEST_F()
239 filter->Reset(); in TEST_F()
250 filter->Reset(); in TEST_F()
262 filter->Reset(); in TEST_F()
275 filter->Reset(); in TEST_F()
[all …]
Dmerge_helper.cc250 CompactionFilter::Decision filter = in MergeUntil() local
254 if (filter != CompactionFilter::Decision::kRemoveAndSkipUntil && in MergeUntil()
258 filter = CompactionFilter::Decision::kRemove; in MergeUntil()
260 if (filter == CompactionFilter::Decision::kKeep || in MergeUntil()
261 filter == CompactionFilter::Decision::kChangeValue) { in MergeUntil()
273 if (filter == CompactionFilter::Decision::kKeep) { in MergeUntil()
281 } else if (filter == CompactionFilter::Decision::kRemoveAndSkipUntil) { in MergeUntil()
Ddb_compaction_filter_test.cc605 KeepFilterFactory* filter = new KeepFilterFactory(true, true); in TEST_F() local
609 options.compaction_filter_factory.reset(filter); in TEST_F()
631 filter->expect_manual_compaction_.store(true); in TEST_F()
632 filter->expect_full_compaction_.store(true); in TEST_F()
633 filter->expect_cf_id_.store(0); in TEST_F()
637 ASSERT_TRUE(filter->compaction_filter_created()); in TEST_F()
667 KeepFilterFactory* filter = new KeepFilterFactory(false, true); in TEST_F() local
668 filter->expect_cf_id_.store(1); in TEST_F()
671 options.compaction_filter_factory.reset(filter); in TEST_F()
692 ASSERT_TRUE(filter->compaction_filter_created()); in TEST_F()
Ddb_universal_compaction_test.cc142 KeepFilterFactory* filter = new KeepFilterFactory(true); in TEST_P() local
143 filter->expect_manual_compaction_.store(false); in TEST_P()
144 options.compaction_filter_factory.reset(filter); in TEST_P()
152 filter->expect_full_compaction_.store(true); in TEST_P()
244 KeepFilterFactory* filter = new KeepFilterFactory(true); in TEST_P() local
245 filter->expect_manual_compaction_.store(false); in TEST_P()
246 options.compaction_filter_factory.reset(filter); in TEST_P()
265 filter->expect_full_compaction_.store(true); in TEST_P()
290 filter->expect_full_compaction_.store(false); in TEST_P()
332 filter->expect_full_compaction_.store(true); in TEST_P()
[all …]
Dc.cc308 dst->append(filter, len); in CreateFilter()
311 (*delete_filter_)(state_, filter, len); in CreateFilter()
313 free(filter); in CreateFilter()
319 filter.data(), filter.size()); in KeyMayMatch()
2184 rocksdb_compactionfilter_t* filter) { in rocksdb_options_set_compaction_filter() argument
2185 opt->rep.compaction_filter = filter; in rocksdb_options_set_compaction_filter()
2961 unsigned char (*filter)( in rocksdb_compactionfilter_create()
2972 result->filter_ = filter; in rocksdb_compactionfilter_create()
2979 rocksdb_compactionfilter_t* filter, in rocksdb_compactionfilter_set_ignore_snapshots() argument
2985 delete filter; in rocksdb_compactionfilter_destroy()
[all …]
/rocksdb-6.9/examples/
Dcompaction_filter_example.cc61 MyFilter filter; in main() local
71 options.compaction_filter = &filter; in main()
84 fprintf(stderr, "filter.count_ = %d\n", filter.count_); in main()
85 assert(filter.count_ == 0); in main()
86 fprintf(stderr, "filter.merge_count_ = %d\n", filter.merge_count_); in main()
87 assert(filter.merge_count_ == 6); in main()
/rocksdb-6.9/db/compaction/
Dcompaction_iterator_test.cc434 Filter filter; in TEST_P() local
487 StallingFilter filter; in TEST_P() local
492 &merge_op, &filter); in TEST_P()
505 filter.WaitForStall(1); in TEST_P()
508 filter.WaitForStall(2); in TEST_P()
513 filter.stall_at.store(3); in TEST_P()
525 StallingFilter filter; in TEST_P() local
530 &merge_op, &filter); in TEST_P()
542 filter.WaitForStall(1); in TEST_P()
545 filter.WaitForStall(2); in TEST_P()
[all …]
Dcompaction_iterator.cc193 CompactionFilter::Decision filter; in InvokeFilterIfNeeded() local
204 filter = compaction_filter_->FilterV2( in InvokeFilterIfNeeded()
211 if (filter == CompactionFilter::Decision::kRemoveAndSkipUntil && in InvokeFilterIfNeeded()
216 filter = CompactionFilter::Decision::kKeep; in InvokeFilterIfNeeded()
219 if (filter == CompactionFilter::Decision::kRemove) { in InvokeFilterIfNeeded()
227 } else if (filter == CompactionFilter::Decision::kChangeValue) { in InvokeFilterIfNeeded()
229 } else if (filter == CompactionFilter::Decision::kRemoveAndSkipUntil) { in InvokeFilterIfNeeded()
/rocksdb-6.9/tools/advisor/advisor/
Drules.ini156 [Rule "bloom-filter-percent-useful"]
157 conditions=bloom-filter-percent-useful
160 [Condition "bloom-filter-percent-useful"]
162 keys=[]rocksdb.bloom.filter.useful.count:[]rocksdb.bloom.filter.full.positive.count:[]rocksdb.bloom…
173 keys=[]rocksdb.bloom.filter.useful.count:[]rocksdb.bloom.filter.full.positive.count:[]rocksdb.bloom…
/rocksdb-6.9/table/block_based/
Dblock_based_table_reader.cc992 if (filter) { in PrefetchIndexAndFilterBlocks()
998 rep_->filter = std::move(filter); in PrefetchIndexAndFilterBlocks()
1043 if (rep_->filter) { in ApproximateMemoryUsage()
1937 FilterBlockReader* const filter = rep_->filter.get(); in PrefixMayMatch() local
1939 if (filter != nullptr) { in PrefixMayMatch()
2067 if (filter == nullptr || filter->IsBlockBased()) { in FullFilterKeyMayMatch()
2102 if (filter == nullptr || filter->IsBlockBased()) { in FullFilterKeysMayMatch()
2171 filter != nullptr && filter->IsBlockBased() == true && in Get()
2282 if (matched && filter != nullptr && !filter->IsBlockBased()) { in Get()
2593 if (matched && filter != nullptr && !filter->IsBlockBased()) { in MultiGet()
[all …]
Dblock_based_filter_block.cc273 const Slice filter = Slice(data + start, limit - start); in MayMatch() local
279 const bool may_match = policy->KeyMayMatch(entry, filter); in MayMatch()
340 Slice filter = Slice(data + start, limit - start); in ToString() local
341 AppendItem(&result, start, filter.ToString(true)); in ToString()
Dfull_filter_block_test.cc86 bool KeyMayMatch(const Slice& key, const Slice& filter) const override { in KeyMayMatch()
88 for (unsigned int i = 0; i + 4 <= filter.size(); i += 4) { in KeyMayMatch()
89 if (h == DecodeFixed32(filter.data() + i)) { in KeyMayMatch()
Dpartitioned_filter_block.cc64 Slice filter = filter_bits_builder_->Finish(&filter_gc.back()); in MaybeCutAFilterBlock() local
66 filters.push_back({index_key, filter}); in MaybeCutAFilterBlock()
124 return filters.front().filter; in Finish()
Dblock_based_filter_block_test.cc34 bool KeyMayMatch(const Slice& key, const Slice& filter) const override { in KeyMayMatch()
36 for (unsigned int i = 0; i + 4 <= filter.size(); i += 4) { in KeyMayMatch()
37 if (h == DecodeFixed32(filter.data() + i)) { in KeyMayMatch()
Dblock_based_table_reader.h382 FilterBlockReader* filter, const Slice& user_key,
389 FilterBlockReader* filter, MultiGetRange* range,
512 std::unique_ptr<FilterBlockReader> filter; member
/rocksdb-6.9/java/src/main/java/org/rocksdb/
DAbstractCompactionFilterFactory.java36 final T filter = createCompactionFilter( in createCompactionFilter() local
41 filter.disOwnNativeHandle(); in createCompactionFilter()
43 return filter.nativeHandle_; in createCompactionFilter()
DBlockBasedTableConfig.java520 final Filter filter) { in setFilter() argument
521 return setFilterPolicy(filter); in setFilter()
DTableFilter.java20 boolean filter(final TableProperties tableProperties); in filter() method
/rocksdb-6.9/utilities/ttl/
Ddb_ttl_impl.cc324 std::shared_ptr<TtlCompactionFilterFactory> filter; in SetTtl() local
327 filter = std::static_pointer_cast<TtlCompactionFilterFactory>( in SetTtl()
329 if (!filter) in SetTtl()
331 filter->SetTtl(ttl); in SetTtl()
/rocksdb-6.9/buckifier/
Dbuckify_rocksdb.py65 for filename in fnmatch.filter(filenames, '*.cc'):
67 for filename in fnmatch.filter(filenames, '*.c'):
/rocksdb-6.9/
DHISTORY.md108 …filter implementation, enabled with format_version 5 (or above) because previous releases cannot r…
232 * Accessing a partition of a partitioned filter or index through a pinned reference is no longer co…
324 * Add whole key bloom filter support in memtable.
340 * Remove Lua compaction filter.
596 * Fix transient reappearance of keys covered by range deletions when memtable prefix bloom filter i…
711 * New compaction filter API: CompactionFilter::FilterV2(). Allows to drop ranges of keys.
943 * Block based table now makes use of prefix bloom filter if it is a full fulter.
1002 …filter, there is a disk format change you need to be aware of. There are three steps you need to d…
1019 … takes one parameter for block cache size. It now builds hash index, bloom filter, and block cache.
1027 * Add feature of storing plain table index and bloom filter in SST file.
[all …]
/rocksdb-6.9/java/src/test/java/org/rocksdb/
DFilterTest.java18 public void filter() { in filter() method in FilterTest

123