nravic's comments

nravic | 7 months ago | on: Scamlexity: When agentic AI browsers get scammed

I think even easier in fact - what's happening behind the scenes w/ an LLM is far more opaque

nravic | 9 months ago | on: Compiling LLMs into a MegaKernel: A path to low-latency inference

This is super interesting! We do something similar I think by taking a checkpoint after model initialization. I'm curious what you think about our approach, here's some benchmarks: https://docs.cedana.ai/articles/performance-of-cedanas-gpu-i...

We do some on-the-fly optimizations as well (like compiling into CUDA graphs or fusing together calls) which ends up resulting (for some inference engines) faster token throughput too.

nravic | 1 year ago | on: CRIU, a project to implement checkpoint/restore functionality for Linux

How were you handling GPU state w/ pytorch? We added some custom code around CRIU to enable GPU checkpointing fwiw: https://docs.cedana.ai/setup/gpu-checkpointing/

nravic | 1 year ago | on: CRIU, a project to implement checkpoint/restore functionality for Linux

We do this using CRIU right now! https://github.com/cedana/cedana

In fact one of our customer's use cases is exactly what you describe, allowing users to "hibernate" container workspaces.

nravic | 1 year ago | on: CRIU, a project to implement checkpoint/restore functionality for Linux

I'm probably the cofounder of the guy you spoke with! Here's our repo: https://github.com/cedana/cedana

nravic | 2 years ago | on: 280M e-bikes and mopeds are cutting demand for oil far more than electric cars

what kind of pannier bags do you use to haul groceries? All the ones I've used have been too small for my purposes.

nravic | 2 years ago | on: Kullback–Leibler divergence

IIRC (and in my experience) KL divergence doesn't account for double counting. Wrote a paper where I ended up having to use a custom metric instead: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=4...

nravic | 2 years ago | on: Show HN: Rivet – Open-source game server management with Nomad and Rust

This is so cool! Out of curiosity, could you walk through your decisionmaking around using Nix and how it fits in with Rivet?

nravic | 2 years ago | on: Show HN: Retake – Open-Source Hybrid Search for Postgres

how often are these batch jobs run? I'm curious to know what the absolute maximum sync frequency can be.

nravic | 2 years ago | on: Guide to running Llama 2 locally

Self plug: run llama.cpp as an inference server on a spot instance anywhere: https://cedana.readthedocs.io/en/latest/examples.html#runnin...

nravic | 3 years ago | on: What are some interesting uses of WebUSB you have come across?

GrapheneOS uses WebUSB to flash your phone.

Pretty wild first experience doing it for me - I'm used to the hassle of sideloading a bootloader and then flashing.

nravic | 3 years ago | on: Ask HN: Who is hiring? (July 2022)

Hey! I'd love to learn more about the position - the hook @ jpl email on your website is bouncing though. Do you mind sending me contact details so we can chat?

Thanks :)

nravic | 4 years ago | on: This career feels like a few key hours with year-long cool-down periods

Do people out of college even do Leetcode anymore? Surely there's a better use of time once you're in a paying job.

nravic | 5 years ago | on: The death of corporate research labs

If you're looking at aerospace/defense startups, this is usually the case.

It's not unheard of for them to employ people who's primary role is grant writing to try and get (for example) SBIR funding

nravic | 6 years ago | on: Ask HN: What's your cross-platform PDF / ePub reading workflow?

Zotero is my go-to

nravic | 6 years ago | on: Clojure – Fascinated, Disappointed, Astonished (2016)

Julia's REPL is beautiful

nravic | 6 years ago | on: Computational Power Found in the Arms of Neurons

Yeah, spiking neural networks are a tough nut to crack. Check out Nengo if you're interested in learning more

nravic | 6 years ago | on: Show HN: Write a private diary using good old email

What are Protonmail's problems?

nravic | 6 years ago | on: China's growing threat to academic freedom

very visceral reaction there

nravic | 6 years ago | on: The revolution of machine learning has been exaggerated

I've followed the same path, out of curiosity do you mind if I PM you asking a few questions? I'm a couple months out of undergrad and I'd like to pick your brain a bit.