Quickstart Guide
Getting started with the OntoPop open-source code base.
This page is intended to act as a quick start guide for software engineers and data engineers who wish to start becoming familiar with the OntoPop open-source code base.
OntoPop is primarily written in Java 11 and heavily utilizes the Java Spring Framework. Apache Maven is used as the dependency management and build automation tool for OntoPop's Java projects. A small number of modules are written in JavaScript for the Node.js (14 LTS) runtime environment - these modules use npm for package management.
Please ensure that the following prerequisite build tools are installed in your development environment:
The source code for OntoPop may be found on GitHub in a public repository at https://github.com/hyperlearningai/ontopop.
To clone the OntoPop source code repository into your development or deployment environment, please run the following Git command via your command line (or your preferred Git GUI tool):
# Clone the OntoPop GitHub public repository
$ git clone https://github.com/hyperlearningai/ontopop.git
# Navigate into the OntoPop project folder
# This location will hereafter be referred to as $ONTOPOP_BASE
$ cd ontopop
The directory into which you cloned the OntoPop source code repository will hereafter be referred to as $ONTOPOP_BASE throughout the remaining documentation.
The following table describes the Maven profiles defined in
$ONTOPOP_BASE/pom.xml.Profile Name | Default Profile | Description |
|---|---|---|
apps | Yes | Manages the lifecycle of all OntoPop's core services as well as all OntoPop Spring Boot applications (i.e. serverless function apps and API applications) across all cloud vendors. |
The following table describes the Maven profiles defined in
$ONTOPOP_BASE/ontopop-apps/pom.xml.Profile Name | Default Profile | Description |
|---|---|---|
apps-multicloud | Yes | Manages the lifecycle of all OntoPop Spring Boot applications (i.e. serverless function apps and API applications) across all cloud vendors. |
apps-spring | No | Manages the lifecycle of Spring Boot applications designed for development and testing purposes, or for deployment to self-managed or Spring Cloud environments. |
apps-aws | No | Manages the lifecycle of AWS Spring Boot applications designed for deployment to AWS Lambda and AWS Beanstalk apps respectively. |
apps-azure | No | Manages the lifecycle of Microsoft Azure Spring Boot applications designed for deployment to Azure Function apps and Azure Web Apps respectively. |
The global configuration for the OntoPop project may be found in the
$ONTOPOP_BASE/ontopop-configuration Maven module.The OntoPop Spring bootstrap context is configured via
src/main/resources/bootstrap.yml and is responsible for loading sensitive configuration properties from external sources such as HashiCorp Vault, Azure Key Vault or AWS Secrets Manager.The OntoPop application context is configured via
src/main/resources/application.yml and is responsible for configuring application properties including secure connections to all relevant software services (including object storage, the shared messaging system, RDF triplestore, graph database, RDBMS and search engine).To compile and build OntoPop's services and applications from source, please run the following commands via your command line:
# Navigate to $ONTOPOP_BASE
$ cd $ONTOPOP_BASE
# Clean the project working directory
$ mvn clean
# Compile, build and package OntoPop
# By default this will build all services and applications
$ mvn package
# If you wish to build OntoPop's core services only,
# then disable the "apps" Maven profile as follows.
$ mvn package -P \!apps
# If you wish to build OntoPop's core services but only the
# apps intended for deployment to a self-managed environment
# or to Spring Cloud, then enable the "apps-spring" Maven
# profile as follows.
$ mvn package -P apps-spring
# If you wish to build OntoPop's core services but only the
# apps intended for deployment to AWS, then enable the
# "apps-aws" Maven profile as follows.
$ mvn package -P apps-aws
# If you wish to build OntoPop's core services but only the
# apps intended for deployment to Azure, then enable the
# "apps-azure" Maven profile as follows.
$ mvn package -P apps-azure
The best place to get started in order to start becoming familiar with the source code is OntoPop's event-driven data pipeline, as illustrated in the following diagram:
OntoPop as a single logical service
The following table maps each of the services that make up the event-driven data pipeline to their respective Maven module (relative to
$ONTOPOP_BASE/ontopop-data).Data Pipeline Service | Maven Module | Service Class |
|---|---|---|
Ontology Ingestion service | ontopop-data-ontology-ingestor | Java Service Class
OntologyIngestorService.java
High-Level Summary
1. Ingest and parse a GitHub webhook payload.
2. Download the OWL file from the GitHub repository to object storage.
3. Publish a message to the shared messaging system. |
Ontology Validation service | ontopop-data-ontology-validator | Java Service Class
OntologyValidatorService.java
High-Level Summary
1. Download the OWL file from object storage.
2. Validate the OWL file using semantic reasoners.
3. Save the validated OWL file to object storage.
4. Publish a message to the shared messaging system. |
Ontology Triplestore Loading service | ontopop-data-ontology-loader-triplestore | Java Service Class
OntologyTriplestoreLoaderService.java
High-Level Summary
1. Download the OWL file from object storage.
2. Load the OWL file directly into the RDF Triplestore.
3. Save the loaded OWL file to object storage.
4. Publish a message to the shared messaging system. |
Ontology Parsing service | ontopop-data-ontology-parser | Java Service Class
OntologyParserService.java
High-Level Summary
1. Download the OWL file from object storage.
2. Parse the OWL file into annotation properties, object properties, classes and class relationships.
3. Serialize and save the parsed POJO as a JSON file to object storage.
4. Publish a message to the shared messaging system. |
Property Graph Modelling service | ontopop-data-ontology-modeller-graph | Java Service Class
OntologyGraphModellerService.java
High-Level Summary
1. Download the parsed JSON file from object storage and deserialize to a parsed POJO.
2. Model the parsed POJO as a directed property graph of vertices, directed edges, vertex properties and edge properties.
3. Serialize and save the modelled POJO as a JSON file to object storage.
4. Publish a message to the shared messaging system. |
Property Graph Loading service | ontopop-data-ontology-loader-graph | Java Service Class
OntologyGraphLoaderService.java
High-Level Summary
1. Download the modelled JSON file from object storage and deserialize to a modelled POJO.
2. Bulk load all the vertices (OWL classes and class properties) from the modelled POJO into the graph database.
3. Bulk load all the directed edges (OWL class restrictions) from the modelled POJO into the graph database.
4. Save the modelled POJO to object storage.
5. Publish a message to the shared messaging system. |
Property Graph Indexing service | ontopop-data-ontology-indexer-graph | Java Service Class
OntologyGraphIndexerService.java
High-Level Summary
1. Download the modelled JSON file from object storage and deserialize to a modelled POJO.
2. Bulk index all the vertices (OWL classes and class properties) from the modelled POJO into the search index.
3. Save the modelled POJO to object storage.
4. Publish a message to the shared messaging system. |