top | item 13607359

Microsoft open-sources Graph Engine

616 points| dajoh | 9 years ago |graphengine.io

176 comments

order

wyldfire|9 years ago

I find that very abstract software packages like this are difficult to visualize without an example. This page does not offer one, but TFM does -- see [1]. Also note that TFM describes the fact that this is Windows-only. That makes it substantially less interesting, IMO.

[1] https://www.graphengine.io/docs/manual/index.html#what-is-ge

v-yadli|9 years ago

Dev here. Cross-platform for Linux/BSD is on the way, and it's already in the repo now.

bluejekyll|9 years ago

At this rate, are we going to see Windows open-sourced?

MS is on a roll. My bias since 1996 is being eroded with each OSS release they have, and multi-platform targeted support.

I started using VSCode regularly as my main Rust IDE, and I feel dirty for liking it. It's seemless across macOS and Linux.

digitalzombie|9 years ago

> At this rate, are we going to see Windows open-sourced?

Doubt it.

In my personal opinion, they're just playing catch up and trying to grab a piece of the pie in the Server space that opensource have been gobbling up.

Linux got web server, cloud, scientific computing, big data (hadoop, spark, etc..), etc...

I believe they're releasing these open source so they can get people on their Azure cloud and get people into their microsoft ecosystem instead. They're emulating what makes Linux so popular, a good ecosystem and also opensource software.

I do not think they will give up Window for free or even opensource at all.

They was willing to lose the internet for desktop. Their mentality was everything goes through the desktop. Google and the internet proved them wrong and made app OS agnostic via webapp. They neglected search engine for desktop and Google ate it up. That's how crazy it is.

There's also a theory of how they dominated Gaming via DirectX so they can keep their OS popular. I doubt they would give up DirectX and gaming lead via opensource.

I think it's a good strategy but I personally love open source ecosystem much more than Microsoft and have trust issues with them in the past.

They're just playing catch up just like Bing vs Google, IE vs Mozilla, etc.. There's still money to be made even though there are clear leader in each space Microsoft neglected.

jpfed|9 years ago

I am under the impression that MS has a lot of different parts, and that these parts have different cultures and goals. MS Research and their developer division seem really open-source-friendly, but that may or may not say anything about their OS team.

PaulHoule|9 years ago

It has nothing to do with Linux vs. Windows.

It does have something to do with SQL Server vs the world -- and that is life as usual at Microsoft.

Microsoft had a big leadership role in XML standardization at the W3C but has had little to do with RDF. (ex. Oracle contains a forward-chaining triple store optimized for geospatial work, SQL Server does not.)

The Microsoft SQL server team would naturally oppose any effort to make a competing database. They've tried to deep six the JET engine that powers Microsoft Access because (i) people want access, and (ii) Microsoft SQL server Express/Compact/whatever is not an effective replacement.

Thus, there is not a lot of room under Microsoft's umbrella for a competitive project, but "yet another open source graph database" is not seen as a threat.

WayneBro|9 years ago

Not yet. Their next strategy is to try and use ARM (again) to shove UWP down your throat.

I love Windows but I seriously will not touch UWP until they loosen up the sandbox restrictions so that I can do regular IPC with a win32 desktop app. You can't even send an HTTP request to a little node.js Web server running on your desktop right now. Fuck that. I'm not buying into it.

dogma1138|9 years ago

I don't see MSFT benefiting from open sourcing windows, so likely never.

runjake|9 years ago

I think that MSFT would open source the Windows OS today, if they were able, but they have some issues to deal with:

* Intellectual Property.

* The concern about opening a can of security vulnerabailities and the associated negative press, is probably on their minds.

Still, I wish they would open source the components they can, if only as a gesture.

userbinator|9 years ago

Windows open-sourced? Maybe an older version (you can already get the kernel source for XP/2K3 under a restrictive license, but nonetheless you don't have to be an MS employee for it) but I don't see MS open-sourcing anything newer for the forseeable future --- otherwise I bet someone will just make a fork with all the telemetry and other invasive features removed, which certainly doesn't help MS.

olyjohn|9 years ago

It's not like they're doing it out of the goodness of their heart. They don't have a choice, they were becoming irrelevant, and now they're playing catch up to try to bring back all the devs to their platform.

thewhitetulip|9 years ago

VSCode is amazing, not just the product, but the team is one of the best I've encountered, they are friendly and helpful

crudbug|9 years ago

Interesting - Linux Foundation with IBM, Google and others announced last week JanusGraph [0]. Janus provides optional persistent storage option also.

What are the good use cases for these ?

[0] http://janusgraph.org/

nitrogen|9 years ago

Old-school AI maybe, fraud detection, social network analysis, recommendation engines, deanonymization, surveillance and tracking...

jknoepfler|9 years ago

analyzing click sequences (website navigation)

tracking goods (amazon uses a gigantic graph database to track everything in its warehouses throughout the goods' lifecycle)

network analysis

sidcool|9 years ago

For a noob like me, what are the applications of this framework? What are the use cases? What current tools do this?

rch|9 years ago

This era of open source from MS is great, but my reaction is always "OK, here's the Microsoft version of something I've been using for a couple of years already".

I'd like to see the Spanner+Cyc GIS-capable global distributed real-time graph engine with FPGA accelerated OLAP support MS Research is probably sitting on, because we can almost / pretty-much hack that together with OSS now.

brilliantcode|9 years ago

Microsoft of 2017 makes me forget about Microsoft of 1997. It's insane how a shakeup of CEO and new cultural shift can seemingly add another major boost to it's brand.

I welcome open source, eventually, I believe that it will eat commercial software, if the right economic incentives are in place. Microsoft may be signaling this to the market.

ukyrgf|9 years ago

I started to get this warm fuzzy feeling as well. Then, I experienced a bug in OneDrive and decided to report it. A week later they responded by referring me to a site where users get 7 votes to pick which bugs you want their team to investigate. The warm fuzzies disappeared...

webmaven|9 years ago

> Microsoft of 2017 makes me forget about Microsoft of 1997.

Never Forget.

unusximmortalis|9 years ago

everything you see changed in Microsoft was done by the people, all people, those that work there and the community as well, they pushed things, the culture and all; those people working there are just people like the rest of the world; outside it was known the company official position through their executive voices but the people working there and the community in general helped tremendously steering the direction -- and of course the company desire to survive, to stay alive. it was slow, it is still slow, it is a tug war but it is happening :)

cobookman|9 years ago

Am I the only one getting SSL cert authority invalid error

mrmondo|9 years ago

I too had this, forcing the site to load also shows a number of page errors too.

*Edit: checking the site on my phone now and it seems to load, pretty poorly designed site though IMO, unbalanced layout but at least it loads now.

debatem1|9 years ago

Nope, I am as well.

tdhz77|9 years ago

Me too. Corporate Proxy sometimes acts up though.

jovdg|9 years ago

Same here

fdsak|9 years ago

same here firefox

adamnemecek|9 years ago

I've been running into the idea of computational graphs a lot recently. It's at the core of Tensorflow (and NN in general) but it also comes up for example in Apple's AVFoundation where all audio processing happens in a graph of audio units. Does anyone know what's the theoretical foundation of computational graphs?

EDIT: I've created a wiki page for computational graphs. https://en.wikipedia.org/wiki/Computational_Graph. Add your input.

mtrn|9 years ago

There is the biological analogue, which has inspired neural networks:

> Recall that in our general definition a feed-forward neural network is a computational graph whose nodes are computing units and whose directed edges transmit numerical information from node to node.

> Each computing unit is capable of evaluating a single primitive function of its input. In fact the network represents a chain of function compositions which transfor m an input to an output vector (called a pattern).

From: https://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf (1996)

Another ancestor would be the Data-Flow paradigm:

> .. programming paradigm that internally represents applications as a directed graph, similarly to a dataflow diagram. Applications are represented as a set of nodes (also called blocks) with input and/or output ports in them. These nodes can either be sources, sinks or processing blocks to the information flowing in the system. Nodes are connected by directed edges that define the flow of information between them.

From: https://paginas.fe.up.pt/~prodei/dsie12/papers/paper_17.pdf (2012)

YeGoblynQueenne|9 years ago

>> Does anyone know what's the theoretical foundation of computational graphs?

Automata can be represented as graphs- that's the main idea. When you look at the typical automaton diagram with states and transitions- that's a graph (with states as vertices and transitions as edges).

I think the confusion arises from the fact that, while automata can be represented as graphs, graphs can represent a much broader array of processes and objects (e.g. belief networks or semantic networks). I guess you can represent pretty much anything as a graph.

So "computational graph" as I understand it, just stresses the point that what is represented is a unit of computation (a.k.a. an automaton a.k.a. a grammar a.k.a. a language etc. etc.) rather than some other kind of graph.

throwaway91111|9 years ago

I think an SSA DAG might apply here as well.

ronack|9 years ago

Does Microsoft use this for anything in production?

reubenbond|9 years ago

From the Readme, it seems they do:

> LIKQ is powering Academic Graph Search API, which is part of Microsoft Cognitive Services.

VeejayRampay|9 years ago

Funny how Microsoft has been rocking hard with their OS releases lately (and good on them for doing so), but there's still a pervasive feeling in the dev community about their true eventual motives and strategy.

"Fool me once..." I guess, or as we say in France "Cold water scares the scalded cat".

vegabook|9 years ago

Is this in the same space as Storm, Flink, or Spark?

Does it do streaming data such as Flink or Storm? Or is it batch-optimized?

What languages does the compute engine support?

kensai|9 years ago

I still like the neat Pajek. Nifty little piece of software, unknown by most, but really powerful! Especially if you are into social network analysis. Who else uses it?

http://mrvar.fdv.uni-lj.si/pajek/

webmaven|9 years ago

Never heard of it before, and wow, that page is seriously old-school.

infocollector|9 years ago

Does this have a python interface?

onion2k|9 years ago

It provides a RESTful API for getting the data out.

dogma1138|9 years ago

C# Only IIRC, but you can probably use Python for .NET and since it's open source just write your own interface ;)

sullivant|9 years ago

I am wondering how it will integrate with R.

tempVariable|9 years ago

I see that the source code "os.h" file has directives to handle Linux and Apple. Has anyone managed to build it and use it in the even most trivial way?

v-yadli|9 years ago

At the current state you could link Trinity.C with a custom Mono host and then execute your GE assemblies/Trinity.Core. Check out CMakeFile.txt for details. :) We're working to interface Trinity with CoreCLR. .NET standard 2.0 would make this much easier.

purple-dragon|9 years ago

I'm a little confused. Is this competitive to something like Spark?

linux_devil|9 years ago

I am confused, how is it different from spark graphx?

zump|9 years ago

How is this different to D-Graph?

floopidydoopidy|9 years ago

Although astroturfing doesn't seem to be a problem on HN, I'd really appreciate it if you didn't do it.

dang|9 years ago

Since you've done this repeatedly and ignored our request to stop, we've banned your account.

Actual astroturfing, when it occurs, is an abuse of HN that we crack down hard on. Defending this community against gaming and abuse is a huge priority for us. Any user who thinks they might be seeing it happen on HN should email us right away (hn@ycombinator.com) so we can investigate.

Imaginary astroturfing—the bug that causes some users to be certain that those who disagree with them can only be nefarious shills because otherwise the pure reason of their own point of view would be fully accepted—is also an abuse of HN. This one is orders of magnitude more common, and it is poison. It eats away the heart of civil, substantive discourse, the assumption of good faith on the part of others.

Therefore we ban astroturfers, and we also ban users who accuse others of astroturfing or shilling without evidence. An opposing view does not count as evidence, and playing this card as a rhetorical device in an argument breaks the HN guidelines.

We detached this subthread from https://news.ycombinator.com/item?id=13607640 and marked it off-topic.

whyileft|9 years ago

This shouldn't be downvoted. There is definitely at the minimum Microsoft employee vote brigading at this point. HN needs to start making upvotes, downvotes, and time of vote public.

anon987|9 years ago

How the hell does this story have so many upvotes?

HN really needs to do something about Microsoft's vote manipulation - it's becoming quite blatant at this point.

bitwize|9 years ago

This is not a cuddly new Microsoft. First comes the embrace (look at all this stuff on our github!), then comes the extend (run your Linux stack on Windows and never have to give up Visual Studio!), I'm sure you know what comes next. Hint: PC manufacturers no longer have to give you the option to disable Secure Boot.

whodknee|9 years ago

>PC manufacturers no longer have to give you the option to disable Secure Boot.

Then don't buy from those people. Simple