Application Context
Application configuration including connectivity properties to all relevant software services such as object storage, the RDF triplestore, graph database, RDBMS and search engine.
Please note that the OntoPop backend open-source software project, which includes the event-driven data pipelines and APIs, is undergoing extensive redesign and refactoring as part of OntoPop Community 3.x in order to improve performance, security, extensibility and maintainability. As a result, the documentation on this page will be significantly updated. Please refer to the OntoPop Roadmap for further information.
Overview
The OntoPop Spring application configuration includes the definition of connectivity properties to all relevant software services such as object storage, the RDF triplestore, graph database, RDBMS, search engine and the enterprise messaging broker.
Location
The configuration for the OntoPop Spring application context may be found in the ontopop-configuration Maven module, at src/main/resources/application.yml. The complete configuration file is as follows:
spring:
application:
name: ontopop
version: @project.version@
cloud:
stream:
binders:
rabbitmq:
type: rabbit
default-candidate: true
environment:
spring:
rabbitmq:
host: ${rabbitmq-host}
port: ${rabbitmq-port}
username: ${rabbitmq-username}
password: ${rabbitmq-password}
virtual-host: /
ssl:
enabled: false
azure_service_bus:
type: servicebus-topic
default-candidate: false
environment:
spring:
cloud:
azure:
servicebus:
connection-string: ${azure-service-bus-connection-string}
bindings:
webProtegeProjectUpdatedPublicationChannel:
destination: webprotege.project.updated
binder: rabbitmq
webProtegeProjectUpdatedConsumptionChannel:
destination: webprotege.project.updated
group: ontopop
binder: rabbitmq
gitRepositoryUpdatedPublicationChannel:
destination: git.repository.updated
binder: rabbitmq
gitRepositoryUpdatedConsumptionChannel:
destination: git.repository.updated
group: ontopop
binder: rabbitmq
ingestedPublicationChannel:
destination: ontopop.data.ingested
binder: rabbitmq
ingestedConsumptionChannel:
destination: ontopop.data.ingested
group: ontopop
binder: rabbitmq
validatedPublicationChannel:
destination: ontopop.data.validated
binder: rabbitmq
validatedTriplestoreLoaderConsumptionChannel:
destination: ontopop.data.validated
group: ontopop.loaders.triplestore
binder: rabbitmq
validatedParserConsumptionChannel:
destination: ontopop.data.validated
group: ontopop.parsers
binder: rabbitmq
triplestoreLoadedPublicationChannel:
destination: ontopop.data.loaded.triplestore
binder: rabbitmq
triplestoreLoadedConsumptionChannel:
destination: ontopop.data.loaded.triplestore
group: ontopop
binder: rabbitmq
parsedPublicationChannel:
destination: ontopop.data.parsed
binder: rabbitmq
parsedConsumptionChannel:
destination: ontopop.data.parsed
group: ontopop
binder: rabbitmq
modelledPublicationChannel:
destination: ontopop.data.modelled
binder: rabbitmq
modelledGraphLoaderConsumptionChannel:
destination: ontopop.data.modelled
group: ontopop.loaders.graph
binder: rabbitmq
modelledGraphIndexerConsumptionChannel:
destination: ontopop.data.modelled
group: ontopop.indexers.graph
binder: rabbitmq
graphLoadedPublicationChannel:
destination: ontopop.data.loaded.graph
binder: rabbitmq
graphLoadedConsumptionChannel:
destination: ontopop.data.loaded.graph
group: ontopop
binder: rabbitmq
graphIndexedPublicationChannel:
destination: ontopop.data.indexed.graph
binder: rabbitmq
graphIndexedConsumptionChannel:
destination: ontopop.data.indexed.graph
group: ontopop
binder: rabbitmq
servicebus:
topic:
bindings:
webProtegeProjectUpdatedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
gitHubRepositoryUpdatedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
ingestedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
validatedTriplestoreLoaderConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
validatedParserConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
triplestoreLoadedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
parsedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
modelledGraphLoaderConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
modelledGraphIndexerConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
graphLoadedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
graphIndexedConsumptionChannel:
consumer:
checkpoint-mode: MANUAL
function:
definition: consume;supply;
poller:
fixed-delay: 1000
initial-delay: 0
default:
contentType: application/json
vault:
enabled: true
host: localhost
port: 8200
scheme: http
authentication: TOKEN
token: ${VAULT_TOKEN}
kv:
enabled: true
backend: secret
default-context: ontopop/development
datasource:
driver-class-name: ${spring-datasource-driver-class-name}
url: ${spring-datasource-url}
username: ${spring-datasource-username}
password: ${spring-datasource-password}
jackson:
serialization:
write_dates_as_timestamps: false
jpa:
hibernate:
ddl-auto: update
show-sql: false
autoconfigure:
exclude: org.springframework.boot.autoconfigure.security.servlet.UserDetailsServiceAutoConfiguration
security:
secrets:
service: hashicorp-vault
aws-secrets-manager:
access-key-id: ${aws-secrets-manager-access-key-id}
access-key-secret: ${aws-secrets-manager-access-key-secret}
region: eu-west-2
azure-key-vault:
url: ${azure-key-vault-url}
clientId: ${azure-key-vault-client-id}
clientSecret: ${azure-key-vault-client-secret}
tenantId: ${azure-key-vault-tenant-id}
hashicorp-vault:
paths:
subpaths:
ontologies: /ontologies/
authentication:
api:
enabled: true
engine: secrets
storage:
object:
service: local
local:
baseUri: ${local-storage-base-directory-path}
aws-s3:
access-key-id: ${aws-s3-access-key-id}
access-key-secret: ${aws-s3-access-key-secret}
region: eu-west-2
bucket-name: ontopop-data
azure-storage:
account-name: ${azure-storage-account-name}
account-key: ${azure-storage-account-key}
blob-endpoint: ${azure-storage-blob-endpoint}
container-name: ontopop-data
containers:
ingested: ingested
validated: validated
parsed: parsed
modelled: modelled
loaded:
triplestore: loaded/triplestore
graph: loaded/graph
indexed: indexed
webprotege:
exported: webprotege/exported
triplestore:
service: apache-jena
apache-jena:
fuseki:
url: ${jena-fuseki-url}
username: ${jena-fuseki-username}
password: ${jena-fuseki-password}
ontotext-graphdb:
url: ${ontotext-graphdb-url}
graph:
service: janusgraph-http
engine:
supportsUserDefinedIds: false
supportsNonStringIds: true
supportsSchema: true
supportsTransactions: true
supportsGeoshape: true
supportsTraversals:
by: true
gremlin-graph:
configuration-filename: gremlin-graph.properties
gremlin-server:
url:
host: ${gremlin-server-host}
port: ${gremlin-server-port}
username: ${gremlin-server-username}
password: ${gremlin-server-password}
enableSsl: false
remoteTraversalSourceName: g
serializer:
className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1
ioRegistries: org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry
serializeResultToString: false
bulkExecutor:
rateLimiter:
enabled: false
actionsPerSecond: 100
maximumAttempts: 10
search:
service: elasticsearch
indexNamePrefix: ontopop
elasticsearch:
url: ${elasticsearch-url}
username: ${elasticsearch-username}
password: ${elasticsearch-password}
ssl: true
shards: 1
replicas: 0
azure-search:
endpoint: ${azure-search-endpoint}
key: ${azure-search-key}
vcs:
git:
service: github
committer:
name: ontopop
email: service@ontopop.com
web:
client:
codecs:
maxInMemorySize: 10
azure:
webapps:
port: 80
aws:
beanstalk:
port: 5000
springdoc:
api-docs:
path: /docs
plugins:
webprotege:
exporter:
enabled: true
http:
enabled: false
scheduler:
enabled: true
cron: "0 0/10 * * * *"
Configuration
RabbitMQ
Namespace: spring.cloud.stream.binders.rabbitmq
Configure this namespace if you wish to use the RabbitMQ message broker.
| Property | Description | Example Value |
|---|---|---|
host | RabbitMQ hostname. Note that the hostname should be set as an externalized variable and NOT stored as plaintext in application.yml. | localhost |
port | RabbitMQ port. Note that the port should be set as an externalized variable and NOT stored as plaintext in application.yml. | 5672 |
username | RabbitMQ username. Note that the username should be set as an externalized variable and NOT stored as plaintext in application.yml. | guest |
password | RabbitMQ password. Note that the password should be set as an externalized variable and NOT stored as plaintext in application.yml. | guest |
virtual-host | Rabbit MQ virtual host. | / |
ssl.enabled | Whether communication is secured by SSL. | false |
Azure Service Bus
Namespace: spring.cloud.stream.binders.azure_service_bus
Configure this namespace if you wish to use the Azure Service Bus message broker.
| Property | Description | Example Value |
|---|---|---|
connection-string | The connection string to the Azure Service Bus (this can be identified and regenerated via the Azure Portal or Azure CLI). Note that the connection string should be set as an externalized variable and NOT stored as plaintext in application.yml. | Endpoint=sb://mysb.servicebus.windows.net/; SharedAccessKeyName=SendListenSharedAccessKey; SharedAccessKey=judhsd87+KJnsgvd68929M |
Spring Cloud Stream
Namespace: spring.cloud.stream.bindings
Configure this namespace if you wish to change the names of the messaging topics and subscriptions, as well as to define which message broker to use for each channel.
| Property | Description | Example Value |
|---|---|---|
*.destination | The name of the topic that this channel will publish messages to, and from which subscribers will consume messages. Note that RabbitMQ will create these topics automatically if they do not already exist. However they need to be created in advance if using Azure Service Bus (via the Azure Portal or Azure CLI). | ontopop.data.ingested |
*.group | The name of the queue or subscription that this channel will use and which is subscribed to the topic. Note that RabbitMQ will create these queues automatically if they do not already exist. However these need to be created in advance as subscriptions if using Azure Service Bus (via the Azure Portal or Azure CLI). | ontopop.data.ingested.consumers |
*.binder | The name of the binder to use for this channel. Set this to either rabbitmq or azure_service_bus. | rabbitmq |
Spring Cloud Vault
Namespace: spring.cloud.vault
Configure this namespace if you wish to use HashiCorp Vault for managing OntoPop dynamic application secrets.
| Property | Description | Example Value |
|---|---|---|
enabled | Whether to enable HashiCorp Vault for managing OntoPop dynamic application secrets. | true |
host | HashiCorp Vault hostname. The host name will be used for SSL certificate validation if required. | localhost |
port | HashiCorp Vault port | 8200 |
scheme | Whether to use http or https. | http |
authentication | Authentication mechanism to authorize client requests. | TOKEN |
token | If using token-based authentication, then the static token to use. Note that the token should be set as an externalized variable and NOT stored as plaintext in application.yml. | s.AB638dhfdnbC7dehq |
kv.enabled | Whether to enable the Key-Value secrets backend, supporting storage of arbitrary values as key-value store. | true |
kv.backend | The path of the secret mount to use. | secret |
kv.default-context | The context name used by all applications. | ontopop/development |
Spring Datasource
Namespace: spring.datasource
Configure this namespace to update connectivity details to the relational database management system (RDBMS) or "SQL" database used by OntoPop.
| Property | Description | Example Value |
|---|---|---|
driver-class-name | The JDBC driver to use when connecting to the RDBMS. | org.hsqldb.jdbc.JDBCDriver |
url | The JDBC connection string to connect to the RDBMS. Note that the URL should be set as an externalized variable and NOT stored as plaintext in application.yml. | jdbc:hsqldb:hsql:// localhost:9001/ontopop |
username | The username to authenticate the connection to the RDBMS. Note that the username should be set as an externalized variable and NOT stored as plaintext in application.yml. | sa |
password | The password to authenticate the connection to the RDBMS. Note that the password should be set as an externalized variable and NOT stored as plaintext in application.yml. | password |
Secrets
Namespace: security.secrets
Configure this namespace if you wish to use either Azure Key Vault or AWS Secrets Manager for managing OntoPop dynamic application secrets.
| Property | Description | Example Value |
|---|---|---|
service | The name of the service to use as the secrets manager for managing OntoPop dynamic application secrets. Set this to either hashicorp-vault, aws-secrets-manager or azure-key-vault. | hashicorp-vault |
aws-secrets-manager.access-key-id | If using AWS Secrets Manager, the access key of the IAM user with read and write privileges to AWS Secrets Manager. Note that the access key should be set as an externalized variable and NOT stored as plaintext in application.yml. | AKIA738HGBNX99K65 |
aws-secrets-manager.access-key-secret | If using AWS Secrets Manager, the access secret of the IAM user with read and write privileges to AWS Secrets Manager. Note that the access secret should be set as an externalized variable and NOT stored as plaintext in application.yml. | gdb67398cnmaqew982m |
aws-secrets-manager.region | If using AWS Secrets Manager, then the region of the AWS Secrets Manager. | eu-west-2 |
azure-key-vault.url | If using Azure Key Vault, the URL to the Key Vault instance. Note that the URL should be set as an externalized variable and NOT stored as plaintext in application.yml. | https://mykv.vault.azure.net |
azure-key-vault.clientId | If using Azure Key Vault, the client ID of the Key Vault instance. Note that the client ID should be set as an externalized variable and NOT stored as plaintext in application.yml. | 71hdjhd3-c9023-76fg-3939sks3w90b |
azure-key-vault.clientSecret | If using Azure Key Vault, the client secret of the Key Vault instance. Note that the client secret should be set as an externalized variable and NOT stored as plaintext in application.yml. | C-r.Jk-cBGD7398dn.SDKK |
azure-key-vault.tenantId | If using Azure Key Vault, the tenant ID in which the Key Vault resides. Note that the tenant ID should be set as an externalized variable and NOT stored as plaintext in application.yml. | 23hg783-028-3352-bdsudhg278 |
API Authentication
Namespace: security.authentication.api
Configure this namespace to change settings in relation to authenticating OntoPop API requests.
| Property | Description | Example Value |
|---|---|---|
enabled | Enable OntoPop's native API Key-based API authentication mechanism. | true |
engine | The type of engine that OntoPop will use to manage API Keys. Currently only secret engines that are configured in the security.secrets namespace are supported (for example HashiCorp Vault, AWS Secrets Manager and Azure Key Vault). | secrets |
Object Storage
Namespace: storage.object
Configure this namespace to select and configure the object storage service used by OntoPop, including Azure Storage and AWS S3.
| Property | Description | Example Value |
|---|---|---|
service | The name of the object storage service used by OntoPop. Set this to either local, aws-s3 or azure-storage. | aws-s3 |
local.baseUri | If using local storage, the absolute path to a base directory in which to serialize and persist ontology data. | /home/myuser/ontopop |
aws-s3.access-key-id | If using AWS S3, the access key of the IAM user with read and write privileges to AWS S3. Note that the access key should be set as an externalized variable and NOT stored as plaintext in application.yml. | AKIA738HGBNX99K65 |
aws-s3.access-key-secret | If using AWS S3, the access secret of the IAM user with read and write privileges to AWS S3. Note that the access secret should be set as an externalized variable and NOT stored as plaintext in application.yml. | gdb67398cnmaqew982m |
aws-s3.region | If using AWS S3, then the region of the AWS S3 instance. | eu-west-2 |
aws-s3.bucket-name | If using AWS S3, the name of the bucket to which to upload and persist ontology data. | mybucket |
azure-storage.account-name | If using Azure Storage, the name of the storage account. Note that the name of the storage account should be set as an externalized variable and NOT stored as plaintext in application.yml. | mystorage |
azure-storage.account-key | If using Azure Storage, the storage account access key. Note that the access key should be set as an externalized variable and NOT stored as plaintext in application.yml. | awv6/s/dgdyhd73hjdBHGYs// edhdbd7783== |
azure-storage.blob-endpoint | If using Azure Storage, the storage account endpoint. Note that the endpoint of the storage account should be set as an externalized variable and NOT stored as plaintext in application.yml. | https://mystorage.blob.core.windows.net/ |
azure-storage.container-name | If using Azure Storage, the name of the container to which to upload and persist ontology data. | mycontainer |
containers.* | The names of the various directories, relative to the bucket or container, into which ontology data will be persisted at various stages of the ontology data pipeline. | ingested |
Triplestore
Namespace: storage.triplestore
Configure this namespace to select and configure the RDF triplestore used by OntoPop, including Apache Jena (Fuseki).
| Property | Description | Example Value |
|---|---|---|
service | The name of the RDF triplestore service used by OntoPop. Currently only apache-jena is supported. | apache-jena |
apache-jena.fuseki.url | If using Apache Jena, the URL to the Apache Jena Fuseki server. Note that the URL should be set as an externalized variable and NOT stored as plaintext in application.yml. | http://localhost:3030 |
apache-jena.fuseki.username | The username to use as part of basic authentication requests to the Fuseki server. Note that the username should be set as an externalized variable and NOT stored as plaintext in application.yml. | guest |
apache-jena.fuseki.password | The password to use as part of basic authentication requests to the Fuseki server. Note that the password should be set as an externalized variable and NOT stored as plaintext in application.yml. | password |
Graph Database
Namespace: storage.graph
Configure this namespace to select and configure the graph database engine used by OntoPop, including TinkerGraph, JanusGraph and Azure Cosmos DB.
| Property | Description | Example Value |
|---|---|---|
service | The name of the specific graph database implementation or graph database general communication protocol used by OntoPop. OntoPop currently supports the following general communication protocols: - gremlin-graph (embedded) - gremlin-server-ws (Gremlin Server via websockets) - gremlin-server-driver (Gremlin server via remote connection driver) - gremlin-server-http (Gremlin server via HTTP) OntoPop also currently supports the following specific implementations: - janusgraph-ws (JanusGraph via websockets) - janusgraph-driver (JanusGraph Gremlin Server via remote connection driver) - janusgraph-http (JanusGraph Gremlin Server via HTTP) - azure-cosmosdb (Azure CosmosDB Gremlin Server via HTTP) - tinkergraph (Embedded Java TinkerGraph) | janusgraph-http |
engine.supportsUserDefinedIds | If using a general communication protocol, whether the graph database engine supports user defined vertex and edge IDs. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | false |
engine.supportsNonStringIds | If using a general communication protocol, whether the graph database engine supports non-String IDs. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | true |
engine.supportsSchema | If using a general communication protocol, whether the graph database engine supports the definition of a schema. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | true |
engine.supportsTransactions | If using a general communication protocol, whether the graph database engine supports transactions. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | true |
engine.supportsGeoshape | If using a general communication protocol, whether the graph database engine supports polygon objects. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | true |
engine.supportsTraversals.by | If using a general communication protocol, whether the graph database engine supports .by() traversals. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | true |
gremlin-graph.configuration-filename | If using gremlin-graph, the name of the configuration file (that must be available in the classpath) defining the graph implementation and connection properties. | gremlin-graph.properties |
gremlin-server.url | If using a gremlin-server-http or janusgraph-http, the URL to the Gremlin server. Note that the URL should be set as an externalized variable and NOT stored as plaintext in application.yml. | |
gremlin-server.host | If using | localhost |
gremlin-server.port | If using gremlin-server-ws, gremlin-server-driver, janusgraph-ws, janusgraph-driver or azure-cosmosdb, the Gremlin server port number. Note that the port number should be set as an externalized variable and NOT stored as plaintext in application.yml. | 8182 |
gremlin-server.username | If the Gremlin server requires authentication, then the username to use as part of basic authentication requests to the Gremlin server. Note that the username should be set as an externalized variable and NOT stored as plaintext in application.yml. | guest |
gremlin-server.password | If the Gremlin server requires authentication, then the password to use as part of basic authentication requests to the Gremlin server. Note that the password should be set as an externalized variable and NOT stored as plaintext in application.yml. | password |
gremlin-server.enableSsl | If the Gremlin server is configured with SSL enabled, then Gremlin clients must also be SSL enabled via this property. | false |
gremlin-server.remoteTraversalSourceName | If using gremlin-server-ws, the name assigned to the remote traversal source that is then referenced when executing graph traversals. | g |
gremlin-server.serializer.className | If using gremlin-server-ws or gremlin-server-driver, the serializer class to use when serializing the results of Gremlin traversal queries. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | org.apache.tinkerpop.gremlin.driver.ser. GraphBinaryMessageSerializerV1 |
gremlin-server.serializer.ioRegistries | If using gremlin-server-ws or gremlin-server-driver, the name of the generalized custom serializer registry for graph implementations requiring custom serialization of identifiers. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | org.janusgraph.graphdb.tinkerpop. JanusGraphIoRegistry |
gremlin-server.serializer.serializeResultToString | If using gremlin-server-ws or gremlin-server-driver, whether to serialize the results of Gremlin traversal queries as string objects. Note that for specific implementations, this property is already set and any value entered in application.yml for this property will be overridden. | false |
gremlin-server.bulkExecutor.rateLimiter.enabled | If using gremlin-server-http, gremlin-server-driver, janusgraph-http, janusgraph-driver or azure-cosmosdb, whether to enable a rate limiter when performing bulk actions such as bulk deletion or bulk insertion of vertices and/or edges. This property is useful particularly when using Azure Cosmos DB so as to not exceed the RU/s for manually provisioned throughput instances. | true |
gremlin-server.bulkExecutor.rateLimiter.actionsPerSecond | If the rate limiter is enabled, the maximum number of bulk actions to perform per second. | 100 |
gremlin-server.bulkExecutor.rateLimiter.maximumAttempts | If the rate limiter is enabled and the manually provisioned throughput is still exceeded, the maximum number of times to retry the bulk actions before throwing an exception. | 10 |
Search
Namespace: storage.search
Configure this namespace to select and configure the search engine used by OntoPop, including Elasticsearch and Azure Search.
| Property | Description | Example Value |
|---|---|---|
service | The name of the search service used by OntoPop. Set this to either elasticsearch or azure-search. | elasticsearch |
elasticsearch.url | If using Elasticsearch, the URL to Elasticsearch server. Note that the URL should be set as an externalized variable and NOT stored as plaintext in application.yml. | http://localhost:9200 |
elasticsearch.username | If using an instance of Elasticsearch that requires authentication, the username to use as part of basic authentication requests. Note that the username should be set as an externalized variable and NOT stored as plaintext in application.yml. | user |
elasticsearch.password | If using an instance of Elasticsearch that requires authentication, the password to use as part of basic authentication requests. Note that the password should be set as an externalized variable and NOT stored as plaintext in application.yml. | password |
elasticsearch.ssl | Whether communication with the Elasticsearch instance is secured by SSL. | true |
elasticsearch.shards | If using Elasticsearch, the default number of shards to create when creating a new index. | 1 |
elasticsearch.replicas | If using Elasticsearch, the default number of replicas to create when creating a new index. | 0 |
azure-search.endpoint | If using Azure Search, the endpoint of the Azure Search service. Note that the endpoint should be set as an externalized variable and NOT stored as plaintext in application.yml. | https://mysearch.search.windows.net |
azure-search.key | If using Azure Search, the access key of the Azure Search service. Note that the access key should be set as an externalized variable and NOT stored as plaintext in application.yml. | 05GFD88536BWERMAB8 |
Web Clients
Namespace: web.client
Configure this namespace to change settings in relation to HTTP requests made from OntoPop clients.
| Property | Description | Example Value |
|---|---|---|
codecs.maxInMemorySize | The maximum number of MB that can be buffered by Spring web clients instantiated by OntoPop. | 10 |