- Aug 10, 2022
-
-
Yorhel authored
Behavioral changes: - A single wildcard ('*') does not cross directory boundary anymore. Previously 'a*b' would also match 'a/b', but no other tool that I am aware of matches paths that way. This change breaks compatibility with old exclude patterns but improves consistency with other tools. - Patterns with a trailing '/' now prevent recursing into the directory. Previously any directory excluded with such a pattern would show up as a regular directory with all its contents excluded, but now the directory entry itself shows up as excluded. - If the path given to ncdu matches one of the exclude patterns, the old implementation would exclude every file/dir being read, this new implementation would instead ignore the rule. Not quite sure how to best handle this case, perhaps just exit with an error message? Performance wise, I haven't yet found a scenario where this implementation is slower than the old one and it's *significantly* faster in some cases - in particular when using a large amount of patterns, especially with literal paths and file names. That's not to say this implementation is anywhere near optimal: - A list of relevant patterns is constructed for each directory being scanned. It may be possible to merge pattern lists that share the same prefix, which could both reduce memory use and the number of patterns that need to be matched upon entering a directory. - A hash table with dynamic arrays as values is just garbage from a memory allocation point of view. - This still uses libc fnmatch(), but there's an opportunity to precompile patterns for faster matching.
-
- Mar 24, 2022
-
-
Yorhel authored
While it's true that the root item can't be a special, the first item to be added is not necessarily the root item. In particular, it isn't when refreshing. Probably fixes #194
-
- Feb 05, 2022
-
-
Yorhel authored
That *usually* doesn't take longer than a few milliseconds, but it can take a few seconds for some extremely large dirs, on very slow computers or with optimizations disabled. Better display a message than make it seem as if ncdu has stopped doing anything.
-
- Jan 01, 2022
-
-
Yorhel authored
-
- Dec 21, 2021
-
-
Yorhel authored
-
- Nov 02, 2021
-
-
Yorhel authored
...by making sure that Context.parents is properly initialized to null when not scanning to RAM. Fixes #179.
-
- Oct 06, 2021
- Jul 28, 2021
-
-
Yorhel authored
-
Yorhel authored
As aluded to in the previous commit. This approach keeps track of hard links information much the same way as ncdu 1.16, with the main difference being that the actual /counting/ of hard link sizes is deferred until the scan is complete, thus allowing the use of a more efficient algorithm and amortizing the counting costs. As an additional benefit, the links listing in the information window now doesn't need a full scan through the in-memory tree anymore. A few memory usage benchmarks: 1.16 2.0-beta1 this commit root: 429 162 164 backup: 3969 1686 1601 many links: 155 194 106 many links2*: 155 602 106 (I'm surprised my backup dir had enough hard links for this to be an improvement) (* this is the same as the "many links" benchmarks, but with a few parent directories added to increase the tree depth. 2.0-beta1 doesn't like that at all) Performance-wise, refresh and delete operations can still be improved a bit.
-
- Jul 26, 2021
-
-
Yorhel authored
While this simplifies the code a bit, it's a regression in the sense that it increases memory use. This commit is yak shaving for another hard link counting approach I'd like to try out, which should be a *LOT* less memory hungry compared to the current approach. Even though it does, indeed, add an extra cost of these parent node pointers.
-
- Jul 19, 2021
-
-
Yorhel authored
-
- Jul 18, 2021
-
-
Yorhel authored
I had planned to checkout out async functions here so I could avoid recursing onto the stack alltogether, but it's still unclear to me how to safely call into libc from async functions so let's wait for all that to get fleshed out a bit more.
-
Yorhel authored
-
Yorhel authored
+ a failed initial attempt at producing static binaries.
-
- Jul 16, 2021
-
-
Yorhel authored
-
- Jul 13, 2021
-
-
Yorhel authored
This complicated the scan code more than I had anticipated and has a few inherent bugs with respect to calculating shared hardlink sizes. Still, the merge approach avoids creating a full copy of the subtree, so that's another memory usage related win compared to the C version. On the other hand, it does leak memory if nodes can't be reused. Not quite as well tested as I should have, so I'm sure there's bugs.
-
- Jul 06, 2021
-
-
Yorhel authored
Two differences compared to the C version: - You can now select individual paths in the listing, pressing enter will open the selected path in the browser window. - Creating this listing is much slower and requires, in the worst case, a full traversal through the in-memory tree. I've tested this without the same-dev and shared-parent optimizations (i.e. worst case) on an import with 30M files and performance was still quite acceptable - the listing completed in a second - so I didn't bother adding a loading indicator. On slower systems and even larger trees this may be a little annoying, though. (also, calling nonl() apparently breaks detection of the return key, neither \n nor KEY_ENTER are emitted for some reason)
-
- Jun 01, 2021
-
-
Yorhel authored
It still feels kind of sluggish, but not entirely sure how to improve it.
-
Yorhel authored
Under the assumption that there are no external references to files mentioned in the dump, i.e. a file's nlink count matches the number of times the file occurs in the dump. This machinery could also be used for regular scans, when you want to scan an individual directory without caring about external hard links. Maybe that should be the default, even? Not sure...
-
- May 29, 2021
-
-
Yorhel authored
-
Yorhel authored
In a similar way to the C version of ncdu: by wrapping malloc(). It's simpler to handle allocation failures at the source to allow for easy retries, pushing the retries up the stack will complicate code somewhat more. Likewise, this is a best-effort approach to handling OOM, allocation failures in ncurses aren't handled and display glitches may occur when we get an OOM inside a drawing function. This is a somewhat un-Zig-like way of handling errors and adds scary-looking 'catch unreachable's all over the code, but that's okay.
-
Yorhel authored
Performance is looking great, but the code is rather ugly and potentially buggy. Also doesn't handle hard links without an "nlink" field yet. Error handling of the import code is different from what I've been doing until now. That's intentional, I'll change error handling of other pieces to call ui.die() directly rather than propagating error enums. The approach is less testable but conceptually simpler, it's perfectly fine for a tiny application like ncdu.
-
- May 23, 2021
-
-
Yorhel authored
I plan to add more display options, but ran out of keys to bind. Probably going for a quick-select menu thingy so that we can keep the old key bindings for people accustomed to it. The graph width algorithm is slightly different, but I think this one's a minor improvement.
-
- May 12, 2021
-
- May 11, 2021
-
-
Yorhel authored
(+ 2 minor crash fixes due to out-of-bounds cursor_idx)
-
- May 09, 2021
-
-
Yorhel authored
-
- May 06, 2021
-
-
Yorhel authored
I initially wanted to keep a directory's block count and size as a separate field so that exporting an in-memory tree to a JSON dump would be easier to do, but that doesn't seem like a common operation to optimize for. We'll probably need the algorithms to subtract sub-items from directory counts anyway, so such an export can still be implemented, albeit slower.
-
- May 03, 2021
-
-
Yorhel authored
Eaiser to implement now that we're linking against libc. But exclude pattern matching is extremely slow, so that should really be rewritten with a custom fnmatch implementation. It's exactly as slow as in ncdu 1.x as well, I'm surprised nobody's complained about it yet. And while I'm at it, supporting .gitignore-style patterns would be pretty neat, too.
-
- May 01, 2021
-
-
Yorhel authored
-
- Apr 30, 2021
-
-
Yorhel authored
Supporting kernfs checking is going to be a bit more annoying. And so is exclude patterns. Ugh.
-
- Apr 29, 2021
-
-
Yorhel authored
-
Yorhel authored
The new data model is supposed to solve a few problems with ncdu 1.x's 'struct dir': - Reduce memory overhead, - Fix extremely slow counting of hard links in some scenarios (issue #121) - Add support for counting 'shared' data with other directories (issue #36) Quick memory usage comparison of my root directory with ~3.5 million files (normal / extended mode): ncdu 1.15.1: 379M / 451M new (unaligned): 145M / 178M new (aligned): 155M / 200M There's still a /lot/ of to-do's left before this is usable, however, and there's a bunch of issues I haven't really decided on yet, such as which TUI library to use. Backporting this data model to the C version of ncdu is also possible, but somewhat painful. Let's first see how far I get with Zig.
-