[shuzchen]

I would much prefer a library such as this be done in C/C++ so it could be packaged up as a library that could be called from other languages. Pretty much every major language has some form of FFI to call out to C/C++ code. This way, you can get consistent behavior if you need to parse robots.txt in python vs ruby vs java vs etc.

[holy_city]

I know it's a meme to say "C is not C++" but in this context C is really not C++. Calling into C through FFI is significantly easier than calling into C++. Very few languages have decent FFI with C++, while many have great support for C.

[firethief]

> so it could be packaged up as a library that could be called from other languages.

What would this mean for C++, if not an `extern C` interface?