miguelraz's comments

miguelraz | 1 year ago | on: From Julia to Rust (2021)

I'll agree on several counts (as a physicist too!):

* Rust's tooling lets you have a base-floor of what is acceptable code that is much more palatable than non-top tier Julia codes. There's a world of difference opening a random crate's code in Rust that has been linted/clippy'd/check'd by the compiler rather than a random Julia package that a coworker/colleague just fired up. This tooling is getting better and better in Julia and I'd be interested to hear which Julia projects you worked on were really hampered by TTFX and refactoring woes. Hard agree on the footguns - I want the VSCode plugin to be better at detecting dead code, it cost me weeks on a project lately.

* I'm also excited for some notion of interfaces to come to Julia. Perhaps it will be a 2.0 thing, but there's still lots of design stuff to figure out.

* We have different understandings of "productive" for different focii then - I still find it painful to do the equivalent of `rand(1:10, (20, 20))` in Rust and I can get STEM people to be productive with Julia before you can even finishing saying "borrowchecker".

miguelraz | 1 year ago | on: From Julia to Rust (2021)

I didn't switch!

At least for some scientific computing codes, but I did learn a lot of Rust along the way and extended my knowledge into (some) embedded and systems programming. Learning has been fun and the Rust people have been very friendly.

miguelraz | 1 year ago | on: From Julia to Rust (2021)

Author here!

Precisely, the most natural symbiosis for a Julia + Rust combo seems to fill a hard-realtime, critical safety or embedded niche with offline simulation.

The Py03 and maturin story is something that we haven't brought to Julia yet and I'd love to have some time to work on a tighter integration story there. For example, getting a hot-reload with Revise.jl and a bindings generator that watches file changes and recompiles and the fly so that a Julia REPL gets updated with the new definitions would be so enviable.

I haven't spoken to Tim Holy to see how to string that together but many of the pieces are there.

miguelraz | 1 year ago | on: From Julia to Rust (2021)

Right, back then there weren't as many Rust-inclined people in the Julia community and I felt like writing a trip report for my Julia friends.

Now I've done a "From Julia to Rust" workshop at the latest JuliaCon and an accompanying survival guide, so that still ongoing work.

miguelraz | 1 year ago | on: From Julia to Rust (2021)

Author here, long time no see Steve!

When I wrote this, my main takeaways were numerical computations with arrays, which don't tend to use for loops in Rust, and which require grokking iterators for maximum performance and rayon-parallelism boons.

To a Julia audience, seeing this style of iterator chaining is quite a new flavor to think about composing computation, so I wanted it was with that audience in mind that I wrote this.

miguelraz | 1 year ago | on: From Julia to Rust (2021)

Author here!

I have no idea why this blog is making the rounds again and I've learned a lot of Rust since, see my (badly benchmarked!) presentation at this year's Scientific Computing in Rust 2024:

https://www.youtube.com/watch?v=bdsBhqKOe7A

and a PR I sent in for the Rust compiler to speed it up for some low hanging fruit:

https://github.com/rust-lang/rust/pull/110477

I'll have to revisit this blog at some point and see what I should update, as well as a "From Rust to Julia" post for those that want to venture into that world as well.

miguelraz | 1 year ago | on: From Julia to Rust (2021)

Author here!

Oh, that's a neat crate, I'll check it out.

I'd be interested to see if you have benchmarks of your dynamical systems solver in Rust vs the ones implemented in Julia. They've done great work!

miguelraz | 1 year ago | on: Exact binary vector search for RAG in 100 lines of Julia

I wanted to get too fancy and I tried * LoopVectorization.jl - @turbo choked on the loop * a direct llvmcall to use AVX512 pop count - I malformed the types for the instruction * Defining the `db` as

db = [rand(Int8) for _ in 1:64, j in 1:(10^6)];

to avoid the vec of vecs structure,

and then

function my_cluster!(db, query, k) db .= query .⊻ db popcounts = mapreduce(count_ones, +, db, dims = 1) results = reshape(popcounts, last(size(db))) partialsortperm!(results, results, k) @views results[begin:k] end

...which I couldn't get to be faster than your version. If you use the `partialsortperm!` and reuse the same cache array, I suspect you'll get good speedups, as you won't be sorting the array every time. This is a classic `nth_element` algorithm.

The above is not the most amazing code, but I suspect the lack of indexing will make it ridiculously friendly for a GPU (Edit: Nope, it chokes on `partialsortperm!`).

I'm guessing the manual loopy approach should be just as good but I battled hard to get it somewhat competitive here in 6 lines of code

#@be my_cluster!(X2, q1, 5) Benchmark: 3 samples with 1 evaluation 42.883 ms (17 allocs: 15.259 MiB) 45.711 ms (17 allocs: 15.259 MiB) 46.670 ms (17 allocs: 15.259 MiB)

#@be k_closest(X1, q1, 5) Benchmark: 4 samples with 1 evaluation 27.994 ms (2 allocs: 176 bytes) 28.733 ms (2 allocs: 176 bytes) 29.000 ms (2 allocs: 176 bytes) 30.709 ms (2 allocs: 176 bytes)

I also didn't try using `FixedSizedArrays.jl` as Mose Giordano recommended in my livestream chat.

miguelraz | 1 year ago | on: Exact binary vector search for RAG in 100 lines of Julia

Dom! Fellow Julian here!

I loved this post <3 The hamming distance is one of my favorite demos of the conciseness of Julia:

hamming_distance(s1, s2) = mapreduce(!=, +, s1, s2)

I'm a bit swamped at the moment but I'll a response article later - they're still some juicy perf on the table here.

Thanks for the post, such a good showcase.

miguelraz | 4 years ago | on: From Julia to Rust

Wholeheartedly agree. I've been trying to come up with a simple metric to measure the composability of a language's ecosystem, and so far the simplest that I've come up with is to find the largest 25 libraries and try to run all their "hello world's" in the same file.

Open to suggestions on a better metric.

miguelraz | 4 years ago | on: From Julia to Rust

The book arrived today! But yeah I might skim the Tour of C++ - thanks for the *ptr!
page 1