Skip to content
Snippets Groups Projects
Commit c27dca1f authored by Yorhel's avatar Yorhel
Browse files

Parallel scanning: early proof-of-concept implementation

And it's not looking well; this implementation seems to be 3x slower in
the hot cache scenario with -J8, which is a major regression. There's
way too much lock contention and context switching.

Haven't tested with actual disk I/O yet and I've not yet measured how
much parallelism this approach will actually get us in practice, nor
whether the disk access patterns of this approach make a whole lot of
sense.  Maybe this low-memory approach will not work out and I'll end up
rewriting this to scan disjoint subtrees after all.

TODO:
- Validate how much parallelism we can actually get with this algorithm
- Lots of benchmarking and tuning (and most likely some re-architecting)
- Re-implement exclude pattern matching
- Document -J option
- Make OOM handling thread-safe
parent 35dd631e
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment