Microsoft's Open-Source Graph Engine Takes On Neo4j

An anonymous reader quotes a report from InfoWorld: Sometimes the relationships between the data you've gathered are more important than the data itself. That's when a graph processing system comes in handy. It's an important but often poorly understood method for exploring how items in a data set are interrelated. Microsoft's been exploring this area since at least 2013, when it published a paper describing the Trinity project, a cloud-based, in-memory graph engine. The fruits of the effort, known as the Microsoft Graph Engine, are now available as an MIT-licensed open source project as an alternative to the likes of Neo4j or the Linux Foundation's recently announced JanusGraph. Microsoft calls Graph Engine (GE) as "both a RAM store and a computation engine." Data can be inserted into GE and retrieved at high speed since it's kept in-memory and only written back to disk as needed. It can work as a simple key-value store like Memcached, but Redis may be the better comparison, since GE stores data in strongly typed schemas (string, integer, and so on). How does all this shape up against the leading open source graph database, Neo4j? For one, Neo4j has been in the market longer and has an existing user base. It's also available in both an open source community edition and a commercial product, whereas GE is only an open source project right now.

Language Extensions

[_xanthus_47]

One of Neo4J's strength is the availability of multiple drivers to integrate with Java, PHP, .net, Python (maybe more) My own app that uses a neo4j backend connects to it directly using a java API - which is convenient because that is what the rest of the application is written in. Until I get that convenience, I won't consider trying it out.

Re:

[_xanthus_47]

I've tried using the REST API for Neo4J. The problem with using REST APIs is the time wasted in sending data over the "network". This even applies to a local development machine. The difference in time taken for making queries over a REST API vs using a driver based API is tremendous! It makes a huge difference in data load times. I really don't think the REST API is worth for anything other than testing and prototyping.

Re:

[snookiex]

True. On the other hand, I think the Neo4J's license is inconvenient (AGPL/GPLv3), business-wise*. I'm considering to switch to ArangoDB [arangodb.com] or OrientDB [orientdb.com] (both APL).

* They provide OEM licenses for startups

What a *feature*

[ptaff]

Data can be inserted into GE and retrieved at high speed since it's kept in-memory and only written back to disk as needed.

If your database system does not avoid disk I/O when it can, and does not leverage memory allocated to it, it's a pretty shitty database system, be it graph-based, relational, key-value or schema-less. This is not a feature, it's just basic design that's found in all database systems you already use every day.

Re:

[Anonymous Coward]

I think GP says this is only marketing shit, as the opposite would never make sense. "Written back to disk as needed" -- what'd be the other proposition? "write back to disk as often as possible"? And what about "kept in-memory"? "avoids staying in memory"? Like a scheduling program that'd "use algorithms" or a "pixel perfect" paint program -- that says jackfuckall, it's just meaningless crap.

Re:

[K. S. Kyosuke]

Why? AllegroGraph does no such BS and apparently it works just fine. GGP is is completely right.

Author doesn't get it

[sttlmark]

This is a hyperbolic title, driven home by the fact that the author concedes in the last paragraph that: "What Microsoft appears to be aiming for with GE isn't head-on competition with those projects."

I get the sense the author doesn't understand this space or is trying to create drama where none exists... Graph Engine is a very different animal than Neo4J. You don't want to use GE as a system of record like you would with Neo4J--it's a pure *in-memory* computation platform. For example, TFA states: "Data can be inserted into GE and retrieved at high speed since it's kept in-memory and only written back to disk as needed," but fails to mention that it's up to the client app to handle that disk IO--Graph Engine doesn't do read-through or write-through to disk for you. (Yes, it can dump all of its memory contents to disk if you need to bounce your machine, but that's a far cry from making it a full-blown persistent data store like Neo4J).

Re:

[flink]

So it sounds like GE is more like JGraphT -- a platform for applying graph algorithms without a persistence layer.

That's what you get for messing with google

[Chrisq]

The owners of these patents are Alfonso J. Cioffi and the family of Allen Rozman, who died in 2012, shortly after he filed the patent infringement lawsuit.

That's what you get for messing with google

Was excited about Neo4j until seeing license costs

[Anonymous Coward]

About six months ago, I was doing some research on graph databases. I stumbled upon neo4j and it looked promising. I was was excited about it until I realized that you have to buy a commercial license to use neo4j in a commercial product, and the license costs are pretty high. I can't find any official publication of the current license costs, but I found unofficial references which mentioned that their special "startup" licensing model was ~12k per JVM and non-startup licenses were ~25k per JVM.

That disq