top | item 8758894

Introducing Libscore: A PageRank for JavaScript Libraries

118 points| applecore | 11 years ago |medium.com

32 comments

order
[+] mynegation|11 years ago|reply
It is not PageRank, it is Altavista for JavaScript libraries. PageRank would be computing PageRank on JavaScript library dependency graph.
[+] hyp0|11 years ago|reply
I was thinking actual PageRank would be interesting, ranking libraries by who uses what, transitively.

Might be nice to include weightings of app usage by users and downloads (because apps aren't used by other libraries), but just sticking to code-used-by-code could be interesting:

So, "one lib/app, one vote" (weighted by how much it is used by other libs/apps), measures coders' evaluation, not app-users'.

And not have the search part (which PageRank ranks), just the ranking - e.g. a top 10

[+] purpleturtle|11 years ago|reply
Yeah, the submitter (unaffiliated) came up with that slogan on his/her own. Happy to have admins here change the title if they're willing?
[+] egeozcan|11 years ago|reply
A limited way to achieve PageRank-like statistics could be parsing all the js package repositories (like npm, bower, atmosphere, etc.)
[+] Kiro|11 years ago|reply
Just FYI: The article doesn't even mention PageRank.
[+] seanconaty|11 years ago|reply
> The end result is that developers contribute to open source in a vacuum; they develop, hoping — but never knowing — whether their library is being used at-large.

Is popularity is the main reason behind releasing and maintaining open source software?

[+] purpleturtle|11 years ago|reply
Unlike celebrity culture, popularity in the open source world translates to actual impact on the web. As an author of a popular library, your code plays a direct part in how other developers structure their codebase, and -- depending on the library -- the end user experience.

And, yeah, impact/change/popularity (whatever you want to call it) is certainly a main reason behind releasing and maintaining open source software. Perhaps other dominant reasons include giving users differently opionionated alternatives that better suit their workflow, advancing the technical know-how of a field, and simply experimenting for expressiveness' sake.

[+] gedrap|11 years ago|reply
It's not the main reason, but it's natural to be curious if someone is using code you spent so much time on, and there's nothing wrong or anti-open source about that.
[+] pjmlp|11 years ago|reply
Didn't you get the memo?

Software development on the 21st century is pop culture driven.

What counts is whatever everyone else is using this week on "top of the charts", not how technical good it is.

[+] mikejarema|11 years ago|reply
I've been using the http://www.ruby-toolbox.com popularity score as a similar type of proxy to usefulness, reliability and desirability for ruby gems.

It attempts to sum up how widely used and active a particular gem is. Plus the site categorizes gems, so when looking for a solution to a new problem you don't have particular expertise in, you can see at a glance which gems may not only help you out, but also those you can trust will be maintained for awhile.

[+] jack_jennings|11 years ago|reply
The score simply being the number of top sites feels pretty difficult to understand in isolation. For instance, knowing that React has a libscore of 203 doesn't mean much if I don't know both that this is out of one million sites polled and the score of some other library that I might already have a vague mental picture of its popularity (i.e. compared to jQuery's 600k+).

Other than that, very interesting information to sift through.

[+] purpleturtle|11 years ago|reply
Exactly. That's why we removed any mention of a "score" from the copywriting before launching. We realized how vague a score could be. We focus on site counts now, since they're raw/unfiltered data.
[+] elvis635|11 years ago|reply
This is awesome! Thank you for your work and Stripe and DigitalOcean for the sponsorship

I've just noticed that the search is not very good, a part the requirement of having to write exactly the correct name. Even writing "Angular" (which is the name showed in the global data) doesn't work.

A fuzzy, case insensitive and autocompleted search would be much more useful

[+] purpleturtle|11 years ago|reply
Good points :)

The problem was that fuzzy search would have been technically overwhelming to implement due to the size of our data sets (1 million sites * avg. # of leaked global variables). Also, it would have resulted in a lot of confusing matches because of how arbitrary JavaScript variable names are.

Keeping it to one-for-one case sensitive lookups was the only way to clearly express searching behavior and return accurate data every time. The downside is that we force people to read our homepage how-to to learn how to use it :)

[+] mrmagooey|11 years ago|reply
Is there a way to filter out the jquery scripts?
[+] billyhoffman|11 years ago|reply
"Before Libscore, front-end developers only had Github star counts as a proxy for their library’s success."

Uhhh no. Before Libscore, we only had Wapanalyzer and BuiltWith and Alexa and Ghostry and countless other "crawl and aggregate stats data" services.

Cool idea and all, but kinda silly to make claims like this.

[+] purpleturtle|11 years ago|reply
builtwith doesn't show all js libs. it just whitelists the top 20 i believe. looks like Wapanalyzer might be similar.

alexa is also unrelated to broad js penetration detection.

not sure how ghostery is relevant.

the novelty of libscore is that it detects all js libs (even brand new ones with only 20 sites using it); doesn't use a dumb whitelist filter.

[+] danr4|11 years ago|reply
Not very functional at the moment. Especially if you don't use jQuery - the top 200 are practically useless IMO.

That being said, being able to search for a library im already using, and seeing what are the top libraries that are used in conjunction with it would probably help.

[+] brandedkeyword|11 years ago|reply
This sounds pretty similar to http://nerdydata.com which lets you search the source code of webpages. Does this only look at the window variable on webpages?
[+] akamaka|11 years ago|reply
It's not working at the moment. All of my searches ended with no results, and it took me a while to notice that the request to the web service is returning a 502. Better error messages would be helpful.
[+] owenversteeg|11 years ago|reply
FYI: Your SSL is broken. As a result, the links on the Github repo are as well.

Would it be possible to do this for CSS frameworks as well? Some CSS frameworks have no Javascript component and are thus undetectable. Thanks!

[+] thomasfromcdnjs|11 years ago|reply
SSL will be on later tonight, we didn't have the time to make the change before launching today and worried that there might be a bit of down time.

I imagine we can programatically pick up on CSS namespaces though...you should create an issue over here -> https://github.com/julianshapiro/libscore it's a good idea

[+] floatrock|11 years ago|reply
A pagerank for npm would be interesting, arguably more so than just the "most depended on" lists. anyone know if it's been done yet?
[+] purpleturtle|11 years ago|reply
npm actually publishes its own top lists.

what's interesting about the work we've done on libscore is that it shows the end result -- whether a lib was actually ultimately used on a site. npm can tell you download stats, but that's where its data ends.

[+] hokkos|11 years ago|reply
I really like the Wappalyser chrome extension, it tells you with which technologies the site you browse uses.
[+] alandarev|11 years ago|reply
I noticed that you actually had the source code in the repository, and later removed it. The commits history still contains all the code.

Question: Why did you decide to go private? You might have your reasons, care to share? Otherwise it is worrying: helping open source community, yet not being open source.

[+] purpleturtle|11 years ago|reply
Cleaning a few things up first :) Star the repo and stay tuned for an update.
[+] zackify|11 years ago|reply
Where's ReactJS?
[+] purpleturtle|11 years ago|reply
Search for "React", which is the variable name React exposes itself under. It's also on the trending data page.