<!DOCTYPE HTML>
<html lang="en" class="sidebar-visible no-js light">
<head>
<!-- Book generated using mdBook -->
<meta charset="UTF-8">
<title>Databases - Warpgrapher Book</title>
<!-- Custom HTML head -->
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#ffffff" />
<link rel="icon" href="../favicon.svg">
<link rel="shortcut icon" href="../favicon.png">
<link rel="stylesheet" href="../css/variables.css">
<link rel="stylesheet" href="../css/general.css">
<link rel="stylesheet" href="../css/chrome.css">
<link rel="stylesheet" href="../css/print.css" media="print">
<!-- Fonts -->
<link rel="stylesheet" href="../FontAwesome/css/font-awesome.css">
<link rel="stylesheet" href="../fonts/fonts.css">
<!-- Highlight.js Stylesheets -->
<link rel="stylesheet" href="../highlight.css">
<link rel="stylesheet" href="../tomorrow-night.css">
<link rel="stylesheet" href="../ayu-highlight.css">
<!-- Custom theme stylesheets -->
</head>
<body>
<!-- Provide site root to javascript -->
<script type="text/javascript">
var path_to_root = "../";
var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "navy" : "light";
</script>
<!-- Work around some values being stored in localStorage wrapped in quotes -->
<script type="text/javascript">
try {
var theme = localStorage.getItem('mdbook-theme');
var sidebar = localStorage.getItem('mdbook-sidebar');
if (theme.startsWith('"') && theme.endsWith('"')) {
localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
}
if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
}
} catch (e) { }
</script>
<!-- Set the theme before any content is loaded, prevents flash -->
<script type="text/javascript">
var theme;
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
if (theme === null || theme === undefined) { theme = default_theme; }
var html = document.querySelector('html');
html.classList.remove('no-js')
html.classList.remove('light')
html.classList.add(theme);
html.classList.add('js');
</script>
<!-- Hide / unhide sidebar before it is displayed -->
<script type="text/javascript">
var html = document.querySelector('html');
var sidebar = 'hidden';
if (document.body.clientWidth >= 1080) {
try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
sidebar = sidebar || 'visible';
}
html.classList.remove('sidebar-visible');
html.classList.add("sidebar-" + sidebar);
</script>
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
<div class="sidebar-scrollbox">
<ol class="chapter"><li class="chapter-item expanded "><a href="../warpgrapher/intro.html"><strong aria-hidden="true">1.</strong> Warpgrapher</a></li><li class="chapter-item expanded "><a href="../warpgrapher/quickstart.html"><strong aria-hidden="true">2.</strong> Quickstart</a></li><li class="chapter-item expanded "><a href="../integrations/intro.html"><strong aria-hidden="true">3.</strong> Server Integration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../integrations/actix.html"><strong aria-hidden="true">3.1.</strong> Actix Web</a></li><li class="chapter-item expanded "><a href="../integrations/lambda.html"><strong aria-hidden="true">3.2.</strong> AWS Lambda</a></li></ol></li><li class="chapter-item expanded "><a href="../configuration/intro.html"><strong aria-hidden="true">4.</strong> Configuration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../configuration/databases.html" class="active"><strong aria-hidden="true">4.1.</strong> Databases</a></li><li class="chapter-item expanded "><a href="../configuration/formats.html"><strong aria-hidden="true">4.2.</strong> Formats</a></li><li class="chapter-item expanded "><a href="../configuration/types.html"><strong aria-hidden="true">4.3.</strong> Types</a></li><li class="chapter-item expanded "><a href="../configuration/relationships.html"><strong aria-hidden="true">4.4.</strong> Relationships</a></li></ol></li><li class="chapter-item expanded "><a href="../api/intro.html"><strong aria-hidden="true">5.</strong> CRUD API Usage Examples</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../api/node_create.html"><strong aria-hidden="true">5.1.</strong> Node Create</a></li><li class="chapter-item expanded "><a href="../api/node_read.html"><strong aria-hidden="true">5.2.</strong> Node Read</a></li><li class="chapter-item expanded "><a href="../api/node_update.html"><strong aria-hidden="true">5.3.</strong> Node Update</a></li><li class="chapter-item expanded "><a href="../api/node_delete.html"><strong aria-hidden="true">5.4.</strong> Node Delete</a></li><li class="chapter-item expanded "><a href="../api/rel_create.html"><strong aria-hidden="true">5.5.</strong> Relationship Create</a></li><li class="chapter-item expanded "><a href="../api/rel_read.html"><strong aria-hidden="true">5.6.</strong> Relationship Read</a></li><li class="chapter-item expanded "><a href="../api/rel_update.html"><strong aria-hidden="true">5.7.</strong> Relationship Update</a></li><li class="chapter-item expanded "><a href="../api/rel_delete.html"><strong aria-hidden="true">5.8.</strong> Relationship Delete</a></li></ol></li><li class="chapter-item expanded "><a href="../engine/intro.html"><strong aria-hidden="true">6.</strong> Engine Features</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../engine/endpoints_static.html"><strong aria-hidden="true">6.1.</strong> Static Endpoints</a></li><li class="chapter-item expanded "><a href="../engine/endpoints_defined.html"><strong aria-hidden="true">6.2.</strong> Defined Endpoints</a></li><li class="chapter-item expanded "><a href="../engine/dynamic_props.html"><strong aria-hidden="true">6.3.</strong> Dynamic Props</a></li><li class="chapter-item expanded "><a href="../engine/dynamic_rels.html"><strong aria-hidden="true">6.4.</strong> Dynamic Relationships</a></li><li class="chapter-item expanded "><a href="../engine/context_request.html"><strong aria-hidden="true">6.5.</strong> Request Context</a></li><li class="chapter-item expanded "><a href="../engine/input_validation.html"><strong aria-hidden="true">6.6.</strong> Input Validation</a></li><li class="chapter-item expanded "><a href="../engine/event_handlers.html"><strong aria-hidden="true">6.7.</strong> Event Handlers</a></li></ol></li></ol>
</div>
<div id="sidebar-resize-handle" class="sidebar-resize-handle"></div>
</nav>
<div id="page-wrapper" class="page-wrapper">
<div class="page">
<div id="menu-bar-hover-placeholder"></div>
<div id="menu-bar" class="menu-bar sticky bordered">
<div class="left-buttons">
<button id="sidebar-toggle" class="icon-button" type="button" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
<i class="fa fa-bars"></i>
</button>
<button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
<i class="fa fa-paint-brush"></i>
</button>
<ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
<li role="none"><button role="menuitem" class="theme" id="light">Light (default)</button></li>
<li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
<li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
<li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
<li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
</ul>
<button id="search-toggle" class="icon-button" type="button" title="Search. (Shortkey: s)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="S" aria-controls="searchbar">
<i class="fa fa-search"></i>
</button>
</div>
<h1 class="menu-title">Warpgrapher Book</h1>
<div class="right-buttons">
<a href="../print.html" title="Print this book" aria-label="Print this book">
<i id="print-button" class="fa fa-print"></i>
</a>
</div>
</div>
<div id="search-wrapper" class="hidden">
<form id="searchbar-outer" class="searchbar-outer">
<input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
</form>
<div id="searchresults-outer" class="searchresults-outer hidden">
<div id="searchresults-header" class="searchresults-header"></div>
<ul id="searchresults">
</ul>
</div>
</div>
<!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
<script type="text/javascript">
document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
});
</script>
<div id="content" class="content">
<main>
<h1 id="databases"><a class="header" href="#databases">Databases</a></h1>
<p>Warpgrapher translates GraphQL queries into CRUD operations against a back-end data store, based on a configuration specifying a data model. The tutorial will return to the topic of the <a href="./config.html">configuration file</a> soon, but the first step is configuring Warpgrapher to integrate with the back-end database. Without a graph database behind it, Warpgrapher's functionality is sharply limited.</p>
<p>Warppgrapher supports several database back-ends for graph data:</p>
<ol>
<li>Apache Tinkerpop</li>
<li>AWS Neptune (Cypher variant)</li>
<li>Azure Cosmos DB (Gremlin variant)</li>
<li>Neo4J</li>
</ol>
<p>It may be possible to use Warpgrapher with other graph databases. The list above is the set that the maintainers have used previosuly. Using each of the databases above requires selecting the <a href="./intro.html">appropriate crate feature</a> and setting up environment variables to provide connection information to Warpgrapher, as described below.</p>
<p>Regardless of database, export an environment variable to control the size of the database
connection pool:</p>
<pre><code class="language-bash">export WG_POOL_SIZE=8
</code></pre>
<p>If the <code>WG_POOL_SIZE</code> variable is not set, Warpgrapher defaults to a pool the same size as the
number of CPUs detected. If the number of CPUs cannot be detected, Warpgrapher defaults to a pool
of 8 connections. </p>
<h2 id="gremlin-based-databases"><a class="header" href="#gremlin-based-databases">Gremlin-Based Databases</a></h2>
<p>For all gremlin-based databases, such as Apache Tinkerpop and Azure Cosmos DB the
following environment variables control connection to the database.</p>
<ul>
<li>WG_GREMLIN_HOST is the host name for the database to which to connect.</li>
<li>WG_GREMLIN_READ_REPICA provides a separate host name for read-only replica nodes, if being
used for additional scalability. If not set, the read pool connects to the same host as the
read/write connection pool.</li>
<li>WG_GREMLIN_PORT provides the port to which Warpgrapher should connect.</li>
<li>WG_GREMLIN_USER is the username to use to authenticate to the database, if required.</li>
<li>WG_GREMLIN_PASS is the password to use to authenticate to the database, if required.</li>
<li>WG_GREMLIN_USE_TLS is set to <code>true</code> if Warpgrapher should connect to the database over a TLS
connection, and <code>false</code> if not using TLS. Defaults to <code>true</code>.</li>
<li>WG_GREMLIN_VALIDATE_CERTS is set to <code>true</code> if Warpgrapher should validate the certificate used
for a TLS connection, and <code>false</code>. Defaults to <code>true</code>. Should only be set to false in non-production
environments.</li>
<li>WG_GREMLIN_LONG_IDS is set to <code>true</code> if Warpgrapher should use long integers for vertex and edge
identifiers. If <code>false</code>, Warpgrapher uses strings. Defaults to <code>false</code>. Consult your graph database's documentation to determine what values are valid for identifiers.</li>
<li>WG_GREMLIN_SESSIONS is set to <code>true</code> if Warpgrapher mutations should be conducted within a single
Gremlin session, which in some databases provides transactional semantics, and <code>false</code> if sessions
should not be used. Defaults to <code>false</code>.</li>
<li>WG_GREMLIN_VERSION may be set to <code>1</code>, <code>2</code>, or <code>3</code>, to indicate the version of GraphSON
serialization that should be used in communicating with the database. Defaults to <code>3</code>.</li>
</ul>
<p>Example configurations for supported databases are shown below. In many cases, some environment
variables are omitted for each database where the defaults are correct.</p>
<h3 id="apache-tinkerpop"><a class="header" href="#apache-tinkerpop">Apache Tinkerpop</a></h3>
<p>Add Warpgrapher to your project config with the gremlin feature enabled.</p>
<p><code>cargo.toml</code></p>
<pre><code class="language-toml">[dependencies]
warpgrapher = { version = "0", features = ["gremlin"] }
</code></pre>
<p>Set up environment variables to contact your Gremlin-based DB:</p>
<pre><code class="language-bash">export WG_GREMLIN_HOST=localhost
export WG_GREMLIN_PORT=8182
export WG_GREMLIN_USER=username
export WG_GREMLIN_PASS=password
export WG_GREMLIN_USE_TLS=true
export WG_GREMLIN_VALIDATE_CERTS=true
export WG_GREMLIN_LONG_IDS=true
</code></pre>
<p>The <code>WG_GREMLIN_CERT</code> environment variable is true if Warpgrapher should ignore the validity of
certificates. This may be necessary in a development or test environment, but should always be set
to false in production.</p>
<p>If you do not already have a Gremlin-based database running, you can run one using Docker:</p>
<pre><code class="language-bash">docker run -it --rm -p 8182:8182 tinkerpop/gremlin-server:latest
</code></pre>
<p>To use an interactive gremlin console to manually inspect test instances, run</p>
<pre><code class="language-bash">docker build -t gremlin-console -f tests/fixtures/gremlin-console/Dockerfile tests/fixtures/gremlin-console
docker run -i --net=host --rm gremlin-console:latest
</code></pre>
<p>In the console, connect to the remote graph:</p>
<pre><code>:remote connect tinkerpop.server conf/remote.yaml
:remote console
</code></pre>
<h3 id="aws-neptune"><a class="header" href="#aws-neptune">AWS Neptune</a></h3>
<p>Add Warpgrapher to your project config:</p>
<p><code>cargo.toml</code></p>
<pre><code class="language-toml">[dependencies]
warpgrapher = { version = "0", features = ["cypher"] }
</code></pre>
<p>Then set up environment variables to contact your Neptune DB:</p>
<pre><code class="language-bash">export WG_CYPHER_HOST=127.0.0.1
export WG_CYPHER_READ_REPLICAS=127.0.0.1
export WG_CYPHER_PORT=7687
export WG_CYPHER_USER=
export WG_CYPHER_PASS=
</code></pre>
<h3 id="azure-cosmos-db"><a class="header" href="#azure-cosmos-db">Azure Cosmos DB</a></h3>
<p>Add Warpgrapher to your project config:</p>
<p><code>cargo.toml</code></p>
<pre><code class="language-toml">[dependencies]
warpgrapher = { version = "0", features = ["gremlin"] }
</code></pre>
<p>Then set up environment variables to contact your Cosmos DB:</p>
<pre><code class="language-bash">export WG_GREMLIN_HOST=*MY-COSMOS-DB*.gremlin.cosmos.azure.com
export WG_GREMLIN_PORT=443
export WG_GREMLIN_USER=/dbs/*MY-COSMOS-DB*/colls/*MY-COSMOS-COLLECTION*
export WG_GREMLIN_PASS=*MY-COSMOS-KEY*
export WG_GREMLIN_USE_TLS=true
export WG_GREMLIN_VALIDATE_CERTS=true
export WG_GREMLIN_VERSION=1
</code></pre>
<p>Note that when setting up your Cosmos database, you must configure it to offer a Gremlin graph API.</p>
<p>Note also that Warpgrapher does not automate the setting or use of a partition key. You must select the node property you wish to use as a partition key and appropriately include it in queries. When Warpgrapher loads relationships and nodes to resolve the full shape of a GraphQL query, it will query by ID, which will likely result in cross-partition queries. This should be fine for many use cases. Extending Warpgrapher to allow more control over and use of partition keys for nested relationship resolution is future work.</p>
<p>Be advised that Gremlin traversals are not executed atomically within Cosmos DB. A traversal may
fail part way through if, for example, one reaches the read unit capacity limit. See
<a href="https://medium.com/@jayanta.mondal/cosmos-db-graph-gremlin-api-how-to-executing-multiple-writes-as-a-unit-via-a-single-gremlin-2ce82d8bf365">this article</a>
for details. The workaround proposed in the article helps, but even idempotent queries do not
guarantee atomicity. Warpgrapher does not use idempotent queries with automated retries to overcome
this shortcoming of Cosmos DB, so note that if using Cosmos, there is a risk that a failed query
could leave partially applied results behind.</p>
<h2 id="neo4j"><a class="header" href="#neo4j">Neo4J</a></h2>
<p>Add Warpgrapher to your project config.</p>
<pre><code class="language-toml">[dependencies]
warpgrapher = { version = "0", features = ["cypher"] }
</code></pre>
<p>Then set up environment variables to contact your Neo4J DB.</p>
<pre><code class="language-bash">export WG_CYPHER_HOST=127.0.0.1
export WG_CYPHER_READ_REPLICAS=127.0.0.1
export WG_CYPHER_PORT=7687
export WG_CYPHER_USER=neo4j
export WG_CYPHER_PASS=*MY-DB-PASSWORD*
</code></pre>
<p>Note that the <code>WG_CYPHER_READ_REPLICAS</code> variable is optional. It is used for Neo4J cluster
configurations in which there are both read/write nodes and read-only replicas. If the
<code>WG_CYPHER_READ_REPLICAS</code> variable is set, read-only queries will be directed to the read replicas,
whereas mutations will be sent to the instance(s) at <code>WG_CYPHER_HOST</code>.</p>
<p>If you do not already have a Neo4J database running, you can run one using Docker:</p>
<pre><code class="language-bash">docker run -e NEO4J_AUTH="${WG_CYPHER_USER}/${WG_CYPHER_PASS}" neo4j:4.4
</code></pre>
</main>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="prev" href="../configuration/intro.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next" href="../configuration/formats.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
<div style="clear: both"></div>
</nav>
</div>
</div>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="prev" href="../configuration/intro.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next" href="../configuration/formats.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>
</div>
<script type="text/javascript">
window.playground_copyable = true;
</script>
<script src="../elasticlunr.min.js" type="text/javascript" charset="utf-8"></script>
<script src="../mark.min.js" type="text/javascript" charset="utf-8"></script>
<script src="../searcher.js" type="text/javascript" charset="utf-8"></script>
<script src="../clipboard.min.js" type="text/javascript" charset="utf-8"></script>
<script src="../highlight.js" type="text/javascript" charset="utf-8"></script>
<script src="../book.js" type="text/javascript" charset="utf-8"></script>
<!-- Custom JS scripts -->
</body>
</html>