In addition to being really fast, it "just works". By that I mean it automatically excludes the files I want to be excluded, like `.gitignore`d files and binary files. I know I can configure ack-grep (ag) and other tools to do that, but not needing to configure it is nice.
BTW, if anyone hasn't read https://blog.burntsushi.net/ripgrep/, highly recommended. It's about how ripgrep is so fast. (Edit: Just saw another comment mentioned this, too. Goes to show that making a single high-quality blog post has a big impact.)
Now, I will do my obligatory "burntsushi needs to clag the rest of the Teddy code" post which I must do on every discussion of ripgrep. :-)
The "subtitles_alternate_casei" examples would be a good benchmark - these 48 strings should not overwhelm Teddy as they could be sensibly merged into Teddy's 8 "buckets" (in fact, they could be merged into 5 buckets) using the simple greedy merging strategy in the original Teddy implementation.
This would probably be a quite good project for someone who wants to contribute to ripgrep and could likely get some nice performance wins...
ack and ag (the_silver_searcher) are two distinct tools. The latter automatically excludes .gitignored and binary files, just like ripgrep. In fact, it seems ag inspired ripgrep's behavior; in the blog post[0] introducing Ripgrep, BurntSushi specifically highlights that rg aims for the usability of ag:
> I will introduce a new command line search tool, ripgrep, that combines the usability of The Silver Searcher [ag] …
I’ve learned a lot by reading burntsushis code, so thank you for everything you’ve written.
Separately, ripgrep is a great advertisement of what Rust is capable of. First, it shows it’s possible to write highly performant and reliable applications without resorting to non readable code. Second, it shows how simple it is to separate applications into a library + binary thanks to Rust’s package management. Ripgrep is merely a command line interface to libripgrep, which is reused in other applications like VS Code. The regex and directory walking code that was once part of the main ripgrep code base are now crates that are reused to great effect throughout the ecosystem.
If you've been holding back on Ripgrep because of its 0.x version numbering, now's the time to adopt it. Ripgrep is faster than grep, faster than ag (the Silver Searcher), and even faster than git grep.
The author breaks down all of these details in this 17,000 word blog post, full of detailed benchmarks. https://blog.burntsushi.net/ripgrep/ (It's from 2016, but it's still mostly in good shape; ripgrep has only gotten faster since then!)
The blog post is good, and fair at the time. One caution I would add is that time has not stopped for the other projects, either. I believe ag (the silver searcher) has also seen considerable performance improvements since 2016.
I cannot understand all that amount of excitement. Are we under heavy fanbot attack or am I missing something? If the whole thing is that, is faster than grep and "in Rust we Trust" then please feel free to ignore my comment.
I would use OSX's slow grep every single day to find things and I never had any of the non-problems problems which ripgrep is trying to solve. About recursive, case insensitive and search and file number just use grep with the flags "-Rni", so I don't get in which aspect is superior. I never ever had half a thought about grep's performance at the systems I use it (already faster than my cognitive bottle neck). If I ever would need something better than grep I would try to move to the next level or deal with lots of unstructured data I would perhaps move to a local search engine using indexes.
And personally I would consider an antipattern to skip the gitignore contents per default. What kind of machines or use cases are hitting the people in the comments?
I use rg on a very large project (8.5 million LOC, 5.2G of files including the build). Searching with rg takes less than 2 seconds and I don't ever have to think about it. rg automatically handles not searching the build files and autogenerated files which often contain lots of matches but are never what I want to search. Additionally, rg has an excellent --vimgrep feature which makes it easy to integrate with the vim quickfix list.
Vanilla grep, meanwhile takes 30 seconds -- too long to search while maintaining flow.
For me, the whole thing _is_ that it is faster than grep. AFAIK, a lot of its speed is due to breaking compatibility with grep to allow for more convenient defaults (skip binary files by default, skip .gitignore by default, etc) which also happen to be faster. For me this is a clear win-win.
Also, I have an 8 core machine. To be waiting for a 30s search knowing 7 cores are doing nothing makes me sad.
> I never had any of the non-problems problems which ripgrep is trying to solve
That's OK. Not everyone has the same set of problems, and not every tool that is built must be used by everybody. I document pretty clearly what ripgrep does in the README, so if it doesn't target your use cases, then obviously you just don't need to use it. But also, just as obviously, it should be clear that plenty of other people do fit into the use cases that ripgrep targets. It turns out that a lot of people work on big code bases, and as a result, running the default grep command can be quite slow compared to something that is just a little bit smarter. It gets super annoying to always remember to type out the right `--include` commands to grep. I know, because I did it for ten years before I built ripgrep. I mean, this is the same premise that drives other tools like `git grep`, ack and ag. ripgrep isn't alone here.
You're not wrong, it's a combination! Yes, faster than grep, yes "ooh shiny rust", + "ooh shiny" in general, used as the search function in the most popular editor (statistically, for users of stackoverflow) (https://insights.stackoverflow.com/survey/2019?utm_source=so...).
For me, the performance is best in class, ux is great, and has defaults that make sense.
Perhaps this helps add context. Every day there are probably thousands of new people learning to write software and learning and command line search tools. They would compare grep, and perhaps something like ripgrep on how they stack up today. They probably don't want to search their javascript dependencies by default (from gitignore, eg node_modules). They probably don't want to search their build (binary/gitignore). They would like to have an easy way to say "only search javascript files" or "only search go files". They would like the fastest tool for the job, why would you opt for the slower one? None of greps historical clout/muscle memory has an impact on them. They would probably appreciate coloured, formatted for easier human consumption output by default.
I use "ack" rather than ripgrep, but the reason I changed was that it automatically ignores a lot of files that I don't care about: VCS files largely. That's the big feature for me, because I'm usually searching in Mercurial repos and every match appears twice (real file and .hg file).
for find/grep to catch up they really should take care of ignoring .git/.cvs/.svn/.repo directories as default these days, I don't want to type --exclude etc. I'm not that sensitive to speed here, but ignoring .git is an enough reason to make me use ripgred and fd daily instead of 'grep/ack' and 'find'.
The -pre flag is rarely mentioned but was a revelation for me in for dealing with MS office files and pdf documents. I use it every single day.
alias rgpre='rg -i --max-columns=1500 --pre rgpre'
*edit
Adding my script here, and would be grateful for any improvements. Goal was to catch doc,docx,ppt,pptx,xls,xlsx, epub, mobi, and pdf and still work on termux with default packages there only.
That's a lovely script. And yeah, I should try to publicize --pre a bit more, since it is pretty handy. I don't use it a ton myself, but when it works, I'm really thankful for it. (That might sound weird since it's in my tool, but I was initially kind of opposed to adding it.)
One bummer is that it can slow ripgrep down quite a bit: one process per file is not good. As a tip, if you add
--pre-glob '*.{pdf,xlsx,docx,doc,pptx,...}'
and whatever else you need, then ripgrep will only invoke an extra process for those files specifically, which can significantly speed up ripgrep if you're searching a mixed directory (or just want to have `--pre` on by default).
That's indeed a lovely script. It makes for seamless searching my trove of PDFs and ebooks. Thanks for sharing it, and thanks BurntSushi for considering including it to rg's README :) !
I also love ripgrep and use it all the time. Thanks!
I've lately been doing some Windows programming. Ripgrep from a Git Bash command-line can generally find ALL instances of some word before Visual Studio's Find In Solution can even return one result!
The only downside is that rg does not understand namespaces when searching, so a commonly used member variable will show up all over the place. Can still usually find the right location by visual inspection before VS2017 though!
I tried ripgrep 0.10 when it came out. I did variations of "rg foo", "rg -i foo", "rg -ri foo", "rg -ri foo .". All of them returned 0 results. Where "ack foo | wc" returned 79 lines. Sooooo. I uninstalled "rg".
I just tried again and same thing. If I run with "-u" I get results.
I've just spent some time hunting and figured out that in my "projects" directory I had a ".gitignore" that had "", which I did a long time ago to stop "git status" from spamming me if I ran it in the wrong directory (base projects, a hg project which I have a lot of).
For some reason "ack" recognizes that it is in a hg repo when I run it, and "rg" also recognizes that, but it ALSO* searches up the tree looking for a ".gitignore" which ack does not. Good to know.
If you use the `--debug` flag, it will tell you which files are being ignored and why.
If ripgrep doesn't search any files at all, then it should print a warning to stdout.
ripgrep ascends your directory hierarchy for .gitignore files because that's how git works; .gitignore files in parent directories apply. ripgrep doesn't know anything about Mercurial, so it doesn't know to stop ascending your directories.
ack is different since it doesn't look at your .gitignore files, so it doesn't have the same behavior here.
I haven't tried it yet, but it would depend on what sort of features you need
Python wouldn't be a good choice for cli usage
Perl is awesome to use from cli, and it is not just simple search and replace, see my tutorial[1] if you want to see examples
sed and awk are awesome on their own for cli usage, sed is meant for line oriented tasks and awk for field oriented ones (there is overlap too) - one main difference compared to perl is that their regex is BRE/ERE which is generally faster but lacks in many features like lookarounds, non-greedy, named capture groups, etc
you could check out sd[2] for a Rust implementation of sed like search and replacement (small subset of sed features)
Is there a way to output the results in Visual Studio format, which is <filename>(<line number>)<result>? This way when I hook up ripgrep as an External Tool, the line shown in the Output window can be clicked on to open the file and go to the specific line. I see in the source code that the separator is ":"; VS expects () around the line number.
Not natively, no. You could probably write a simple script that transforms the output into what you want though. For example, on Unix at least,
rg -n pattern | sed 's/:/(/' | sed 's/:/)/'
That won't work in every case (e.g., in the case of a file path containining a `:`), but it might get you pretty far. And it could probably be made more precise.
Here I thought I was hip using ag, the announcement blog post linked in this thread is fascinating and thorough, looks like it's time for me to try out rg.
Here's a blog post from the author explaining how rg works: [1]. It's from 2016 so the benchmarks may not be up to date, including for rg's competitors.
[+] [-] sqs|7 years ago|reply
In addition to being really fast, it "just works". By that I mean it automatically excludes the files I want to be excluded, like `.gitignore`d files and binary files. I know I can configure ack-grep (ag) and other tools to do that, but not needing to configure it is nice.
BTW, if anyone hasn't read https://blog.burntsushi.net/ripgrep/, highly recommended. It's about how ripgrep is so fast. (Edit: Just saw another comment mentioned this, too. Goes to show that making a single high-quality blog post has a big impact.)
[+] [-] glangdale|7 years ago|reply
Now, I will do my obligatory "burntsushi needs to clag the rest of the Teddy code" post which I must do on every discussion of ripgrep. :-)
The "subtitles_alternate_casei" examples would be a good benchmark - these 48 strings should not overwhelm Teddy as they could be sensibly merged into Teddy's 8 "buckets" (in fact, they could be merged into 5 buckets) using the simple greedy merging strategy in the original Teddy implementation.
This would probably be a quite good project for someone who wants to contribute to ripgrep and could likely get some nice performance wins...
[+] [-] loeg|7 years ago|reply
> I will introduce a new command line search tool, ripgrep, that combines the usability of The Silver Searcher [ag] …
[0]: https://blog.burntsushi.net/ripgrep/
[+] [-] twic|7 years ago|reply
I basically never want this, so for me, ripgrep's one flaw is that it never "just works".
[+] [-] ignoramous|7 years ago|reply
https://swtch.com/~rsc/regexp/
[+] [-] michaelhoffman|7 years ago|reply
[+] [-] nindalf|7 years ago|reply
Separately, ripgrep is a great advertisement of what Rust is capable of. First, it shows it’s possible to write highly performant and reliable applications without resorting to non readable code. Second, it shows how simple it is to separate applications into a library + binary thanks to Rust’s package management. Ripgrep is merely a command line interface to libripgrep, which is reused in other applications like VS Code. The regex and directory walking code that was once part of the main ripgrep code base are now crates that are reused to great effect throughout the ecosystem.
[+] [-] ngrilly|7 years ago|reply
[+] [-] dfabulich|7 years ago|reply
The author breaks down all of these details in this 17,000 word blog post, full of detailed benchmarks. https://blog.burntsushi.net/ripgrep/ (It's from 2016, but it's still mostly in good shape; ripgrep has only gotten faster since then!)
[+] [-] loeg|7 years ago|reply
[+] [-] nojvek|7 years ago|reply
I mostly use Vscode because of its fast search, decent auto complete and simple editing experience.
Whoever made ripgrep, deserves massive kudos.
[+] [-] fiddlerwoaroof|7 years ago|reply
[+] [-] aashcan|7 years ago|reply
[0] https://github.com/atom/fuzzy-finder/pull/369
[+] [-] carlmr|7 years ago|reply
[+] [-] nudpiedo|7 years ago|reply
I would use OSX's slow grep every single day to find things and I never had any of the non-problems problems which ripgrep is trying to solve. About recursive, case insensitive and search and file number just use grep with the flags "-Rni", so I don't get in which aspect is superior. I never ever had half a thought about grep's performance at the systems I use it (already faster than my cognitive bottle neck). If I ever would need something better than grep I would try to move to the next level or deal with lots of unstructured data I would perhaps move to a local search engine using indexes.
And personally I would consider an antipattern to skip the gitignore contents per default. What kind of machines or use cases are hitting the people in the comments?
[+] [-] Shoop|7 years ago|reply
Vanilla grep, meanwhile takes 30 seconds -- too long to search while maintaining flow.
For me, the whole thing _is_ that it is faster than grep. AFAIK, a lot of its speed is due to breaking compatibility with grep to allow for more convenient defaults (skip binary files by default, skip .gitignore by default, etc) which also happen to be faster. For me this is a clear win-win.
Also, I have an 8 core machine. To be waiting for a 30s search knowing 7 cores are doing nothing makes me sad.
[+] [-] burntsushi|7 years ago|reply
That's OK. Not everyone has the same set of problems, and not every tool that is built must be used by everybody. I document pretty clearly what ripgrep does in the README, so if it doesn't target your use cases, then obviously you just don't need to use it. But also, just as obviously, it should be clear that plenty of other people do fit into the use cases that ripgrep targets. It turns out that a lot of people work on big code bases, and as a result, running the default grep command can be quite slow compared to something that is just a little bit smarter. It gets super annoying to always remember to type out the right `--include` commands to grep. I know, because I did it for ten years before I built ripgrep. I mean, this is the same premise that drives other tools like `git grep`, ack and ag. ripgrep isn't alone here.
See also: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...
> And personally I would consider an antipattern to skip the gitignore contents per default.
Then use
and you'll never have to worry about that specific anti-pattern ever again. ;-)[+] [-] PudgePacket|7 years ago|reply
More "technical" reasons here: https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr....
For me, the performance is best in class, ux is great, and has defaults that make sense.
Perhaps this helps add context. Every day there are probably thousands of new people learning to write software and learning and command line search tools. They would compare grep, and perhaps something like ripgrep on how they stack up today. They probably don't want to search their javascript dependencies by default (from gitignore, eg node_modules). They probably don't want to search their build (binary/gitignore). They would like to have an easy way to say "only search javascript files" or "only search go files". They would like the fastest tool for the job, why would you opt for the slower one? None of greps historical clout/muscle memory has an impact on them. They would probably appreciate coloured, formatted for easier human consumption output by default.
Sorry for the rant!
[+] [-] blux|7 years ago|reply
[+] [-] linsomniac|7 years ago|reply
[+] [-] rrosen326|7 years ago|reply
I just tried ripgrep and like it even more because it automatically follows .gitignore, which is huge. To me, that's the killer feature.
[+] [-] ausjke|7 years ago|reply
[+] [-] oconnor663|7 years ago|reply
[+] [-] j0e1|7 years ago|reply
Tangentially, it is because of great tools like these written in Rust that I have major respect for the language.
[+] [-] SanchoPanda|7 years ago|reply
alias rgpre='rg -i --max-columns=1500 --pre rgpre'
*edit Adding my script here, and would be grateful for any improvements. Goal was to catch doc,docx,ppt,pptx,xls,xlsx, epub, mobi, and pdf and still work on termux with default packages there only.
https://gist.github.com/ColonolBuendia/314826e37ec35c616d705...
[+] [-] burntsushi|7 years ago|reply
One bummer is that it can slow ripgrep down quite a bit: one process per file is not good. As a tip, if you add
and whatever else you need, then ripgrep will only invoke an extra process for those files specifically, which can significantly speed up ripgrep if you're searching a mixed directory (or just want to have `--pre` on by default).[+] [-] ronjouch|7 years ago|reply
[+] [-] zlynx|7 years ago|reply
I've lately been doing some Windows programming. Ripgrep from a Git Bash command-line can generally find ALL instances of some word before Visual Studio's Find In Solution can even return one result!
The only downside is that rg does not understand namespaces when searching, so a commonly used member variable will show up all over the place. Can still usually find the right location by visual inspection before VS2017 though!
[+] [-] linsomniac|7 years ago|reply
I just tried again and same thing. If I run with "-u" I get results.
I've just spent some time hunting and figured out that in my "projects" directory I had a ".gitignore" that had "", which I did a long time ago to stop "git status" from spamming me if I ran it in the wrong directory (base projects, a hg project which I have a lot of).
For some reason "ack" recognizes that it is in a hg repo when I run it, and "rg" also recognizes that, but it ALSO* searches up the tree looking for a ".gitignore" which ack does not. Good to know.
[+] [-] burntsushi|7 years ago|reply
If ripgrep doesn't search any files at all, then it should print a warning to stdout.
ripgrep ascends your directory hierarchy for .gitignore files because that's how git works; .gitignore files in parent directories apply. ripgrep doesn't know anything about Mercurial, so it doesn't know to stop ascending your directories.
ack is different since it doesn't look at your .gitignore files, so it doesn't have the same behavior here.
[+] [-] cjohansson|7 years ago|reply
[+] [-] m0zg|7 years ago|reply
[+] [-] mikojan|7 years ago|reply
[+] [-] shaklee3|7 years ago|reply
[+] [-] asicsp|7 years ago|reply
Python wouldn't be a good choice for cli usage
Perl is awesome to use from cli, and it is not just simple search and replace, see my tutorial[1] if you want to see examples
sed and awk are awesome on their own for cli usage, sed is meant for line oriented tasks and awk for field oriented ones (there is overlap too) - one main difference compared to perl is that their regex is BRE/ERE which is generally faster but lacks in many features like lookarounds, non-greedy, named capture groups, etc
you could check out sd[2] for a Rust implementation of sed like search and replacement (small subset of sed features)
[1] https://github.com/learnbyexample/Command-line-text-processi...
[2] https://github.com/chmln/sd
[+] [-] burntsushi|7 years ago|reply
[+] [-] xrouge|7 years ago|reply
[+] [-] burntsushi|7 years ago|reply
Otherwise, you could always use ripgrep's --json output mode to craft its output into whatever format you like, but that's probably more work. :-) The JSON format is documented here: https://docs.rs/grep-printer/0.1.2/grep_printer/struct.JSON....
[+] [-] VohuMana|7 years ago|reply
[+] [-] stkrzysiak|7 years ago|reply
[+] [-] stevebmark|7 years ago|reply
[+] [-] 1023bytes|7 years ago|reply
Parallelism? Some Rust-specific features?
[+] [-] Shoop|7 years ago|reply
[1] https://blog.burntsushi.net/ripgrep/
[+] [-] PudgePacket|7 years ago|reply
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] keithnz|7 years ago|reply
[+] [-] ape4|7 years ago|reply
[+] [-] ksherlock|7 years ago|reply
[+] [-] heyoni|7 years ago|reply
[+] [-] burntsushi|7 years ago|reply
It's even faster and comes with its own built in evaluation. There's also no more reliance on an external database.