Crawling APIs Using Neo4j - Building The Star Wars Graph14 Dec 2015
A common task when working with data from APIs is crawling the API and inserting the results in a database. In this example we’ll import data into Neo4j by crawling a public API, building a graph in Neo4j as we go along. We’ll use Neo4j as a queuing mechanism to store URLs for placeholder resources waiting to be fetched.
Specifically, we’ll use SWAPI.co to build a graph of information from the Star Wars universe. SWAPI contains data about characters, planets, starships (and much more!) that appear in the Star Wars universe. Much of the data is collected from Wookieepedia.
The Data Model
The graph data model we’ll use when building the graph.
To see the full example please check out the iPython Notebook here.