Skip to content

Ingesting Knowledge Graphs to Neo4j

Learn how to load extracted data into Neo4j for querying and analysis.

What is Ingestion?

Ingestion is the process of taking extracted entities and relationships and loading them into a Neo4j graph database, where you can query and analyze them.

Quick Start: Ingest to Neo4j

Configure Neo4j Connection

First, set up your Neo4j database configuration:

from graphaide import GraphDBConfig

graphdb_config = GraphDBConfig(
    uri="neo4j://localhost:7687",      # Neo4j server address
    username="neo4j",                   # Neo4j username
    password="password"                 # Neo4j password
)

Initialize GraphAide with Neo4j

from graphaide import GraphAide, ModelConfig, GraphDBConfig

# Setup LLM
model_config = ModelConfig(
    provider="openai",
    model_name="gpt-5.6-luna",
    api_key="..."
)

# Setup Neo4j
graphdb_config = GraphDBConfig(
    uri="neo4j://localhost:7687",
    username="neo4j",
    password="password"
)

# Initialize GraphAide with both configs
graphaide = GraphAide(
    model_config=model_config,
    graphdb_config=graphdb_config
)

# Extract AND ingest to Neo4j in one call
result = graphaide.ingest(file_path="document.pdf")

print(f"✓ Ingested {result.nodes_created} entities")
print(f"✓ Ingested {result.edges_created} relationships")

Ingest with Ontology

result = graphaide.ingest(
    file_path="document.pdf",
    ontology_path="ontology.nt"
)

Ingest from Text

text = """
GraphAide is a knowledge graph system built by PNNL.
It uses LLMs to extract entities and relationships.
"""

result = graphaide.ingest(input_text=text)

Batch Ingestion: Multiple Documents

From JSONL File

Process many documents in parallel:

# documents.jsonl
# {"text": "First document content..."}
# {"text": "Second document content..."}
# {"text": "Third document content..."}

result = graphaide.ingest(
    jsonl_file_path="documents.jsonl",
    jsonl_field_name="text",
    jsonl_lines_per_batch=4  # Process 4 documents at once
)

print(f"Processed {result.segments_processed} documents")

From Pre-extracted JSON

If you already have extracted data as JSON:

result = graphaide.load_json_to_neo4j(
    json_path="GraphAide_KG_Extract_20261014_120000.json"
)

print(f"Loaded {result.nodes_created} nodes")
print(f"Loaded {result.edges_created} edges")

Ingestion Options

Clear Existing Graph

⚠️ Warning: This deletes all data in Neo4j!

result = graphaide.ingest(
    file_path="document.pdf",
    clear_existing=True  # Only use when intentional
)

Filter by Ontology

Only ingest entities and relationships matching your ontology:

result = graphaide.ingest(
    file_path="document.pdf",
    ontology_path="ontology.nt",
    filter_by_ontology=True
)

Ingestion Workflow

Here's what happens internally:

1. Extract Entities & Relationships
   ↓
2. Filter by Ontology (optional)
   ↓
3. Merge/Deduplicate (optional)
   ↓
4. Create Nodes in Neo4j
   ↓
5. Create Edges (Relationships) in Neo4j
   ↓
6. Return Statistics

Common Patterns

Build Knowledge Graph from Corpus

import os
from graphaide import GraphAide, ModelConfig

graphaide = GraphAide(
    model_config=ModelConfig(provider="openai", model_name="gpt-5.6-luna", api_key="...")
)

# Ingest all PDFs in folder
pdf_files = [f for f in os.listdir("documents/") if f.endswith(".pdf")]

for pdf_file in pdf_files:
    print(f"Ingesting {pdf_file}...")
    result = graphaide.ingest(file_path=f"documents/{pdf_file}")
    print(f"  → {result.nodes_created} entities, {result.edges_created} relationships")

Iterative Refinement

# First pass: extract all
result1 = graphaide.ingest(file_path="doc.pdf", merge_output=False)

# Second pass: with ontology filtering
result2 = graphaide.ingest(
    file_path="doc.pdf",
    ontology_path="ontology.nt",
    filter_by_ontology=True
)

# Compare
print(f"Without filtering: {result1.nodes_created}")
print(f"With filtering: {result2.nodes_created}")

Batch Processing with Error Handling

results = []

for jsonl_file in ["batch1.jsonl", "batch2.jsonl", "batch3.jsonl"]:
    try:
        result = graphaide.ingest(jsonl_file_path=jsonl_file)
        results.append({
            "file": jsonl_file,
            "nodes": result.nodes_created,
            "edges": result.edges_created
        })
    except Exception as e:
        print(f"Error ingesting {jsonl_file}: {e}")

# Summary
total_nodes = sum(r["nodes"] for r in results)
total_edges = sum(r["edges"] for r in results)
print(f"Total ingested: {total_nodes} nodes, {total_edges} edges")

Troubleshooting

Neo4j Connection Failed

# Check Neo4j is running
# Default: localhost:7687

from graphaide import GraphDBConfig

graphdb_config=GraphDBConfig(
    uri="neo4j://localhost:7687",
    username="neo4j",
    password="password"
)

Memory Issues with Large Batches

# Process smaller batches
result = graphaide.ingest(
    jsonl_file_path="large_file.jsonl",
    jsonl_lines_per_batch=2  # Reduce batch size
)

Next Steps