WingNews logo WingNews GitHub [2]

girfan's comments

girfan | 2 years ago | on: Teddit – An alternative Reddit front-end focused on privacy

Right now it seems legal to scrape Reddit. But given their trajectory of making the API fairly expensive to use, do you think it's likely that they would also limit/prohibit scraping (assuming apps like Apollo start scraping as an alternative)?

girfan | 2 years ago | on: Had a call with Reddit to discuss pricing

Even if that is the case, it does not say much about their ability to evaluate what category of users (e.g., those coming from Apollo or their first-party clients) is generating more views/interactions and indirectly more "value" on their platform.

girfan | 3 years ago | on: Faster CPython 3.12 Plan

That's great!

There are a few cases where `dlmopen` has issues, for example, some libraries are written with the assumption that there will only be one of them in the process (their use of globals/thread local variables etc.) which may result in conflicts across namespaces.

Specifically, `libpthread` has one such issue [1] where `pthread_key_create` will create duplicate keys in separate namespaces. But these keys are later used to index into `THREAD_SELF->specific_1stblock` which is shared between all namespaces, which can cause all sorts of weird issues.

There is a (relatively old, unmerged) patch to glibc where you can specify some libraries to be shared across namespaces [2].

[1]: https://sourceware.org/bugzilla/show_bug.cgi?id=24776#c13

[2]: https://patchwork.ozlabs.org/project/glibc/patch/20211010163...

girfan | 3 years ago | on: Faster CPython 3.12 Plan

Currently it is, yes. I am not sure how fundamental it is. I tried patching glibc to support more (128 in my case) and it seemed to work fine.

girfan | 3 years ago | on: Faster CPython 3.12 Plan

> You can't load one so/dll multiple times in some sort of container

I believe you can do that with `dlmopen` in separate link maps. I have worked with multiple completely isolated Python interpreters in the same process that do not share a GIL using that approach.

girfan | 3 years ago | on: How to choose the right Python concurrency API

This is a great analysis; thanks for writing.

I have also been working on running multiple Python interpreters in the same process by isolating them in different namespaces using `dlmopen` [1]. The objective on a high level is to receive requests for some compute intensive operations from a TCP/HTTP server and dispatch them on to different workers. In this case, a thin C++ shim receives the requests and dispatches them on to one of the Python interpreters in a namespace. This eliminates contention for the GIL amongst the interpreters and can exploit parallelism by running each interpreter on a different set of cores. The data obtained from the request does not need to be copied into the interpreter because everything is in the same address space; similarly the output produced by the Python interpreter is also just passed back without any copies to the server.

[1] https://www.man7.org/linux/man-pages/man3/dlmopen.3.html

girfan | 3 years ago | on: Dragonflydb – A modern replacement for Redis and Memcached

This looks like a cool project. Is there any support (or plan to support) I/O through kernel bypass technologies like RDMA? For example, the client reads the objects using 1-sided reads from the server given it knows which address the object lives in. This could be really benefitial for reducing latency and CPU load.

girfan | 6 years ago | on: Four Years in Startups

Reading this beautifully written piece brings to mind the recent book about Elizabeth Holmes - Bad Blood. Some of the completely unacceptable behavior by tech CEOs and "higher ups" definitely rings a bell; some startups like Theranos were doing exactly that and ended up getting a lot of negative attention.

girfan | 6 years ago | on: Ask HN: What advice would you give to a computer science undergraduate?

Take courses across a breadth of areas in Computer Science, even if you think you don't (or won't) enjoy them much. Don't take up courses only in areas you enjoy (say, Networking) and have no concrete knowledge of areas you think you don't really like (say, Graphics) - Undergrad in my opinion is ideal for building up breadth of Computer Science knowledge. You can always focus on specific areas in grad school or when you start working.

girfan | 9 years ago | on: Open Letter to Tim Cook

The 15inch MBP has i7 Quad Core processor, Radeon graphics and double the RAM than the default 13inch models. This is not a minor jump in performance. I agree that they do sometimes prioritize size, thickness and beauty over performance but I don't see what else could go into the 15inch MBP that it currently lacks?
page 1
more