This page is part of archived documentation for openHAB 4.1. Go to the current stable version
# JDBC Persistence
This service writes and reads item states to and from a number of relational database systems that support Java Database Connectivity (JDBC) (opens new window). This service allows you to persist state updates using one of several different underlying database services. It is designed for a maximum of scalability, to store very large amounts of data and still over the years not lose its speed.
The generic design makes it relatively easy for developers to integrate other databases that have JDBC drivers. The following databases are currently supported and tested:
# Table of Contents
# Configuration
This service can be configured in the file services/jdbc.cfg
.
Property | Default | Required | Description |
---|---|---|---|
url | Yes | JDBC URL to establish a connection to your database. Examples:jdbc:derby:./testDerby;create=true jdbc:h2:./testH2 jdbc:hsqldb:./testHsqlDb jdbc:mariadb://192.168.0.1:3306/testMariadb jdbc:mysql://192.168.0.1:3306/testMysql?serverTimezone=UTC jdbc:postgresql://192.168.0.1:5432/testPostgresql jdbc:timescaledb://192.168.0.1:5432/testPostgresql jdbc:sqlite:./testSqlite.db .If no database is available it will be created; for example the url jdbc:h2:./testH2 creates a new H2 database in openHAB folder. Example to create your own MySQL database directly:CREATE DATABASE 'yourDB' CHARACTER SET utf8 COLLATE utf8_general_ci; | |
user | if needed | database user name | |
password | if needed | database user password | |
errReconnectThreshold | 0 | No | when the service is deactivated (0 means ignore) |
sqltype.CALL | VARCHAR(200) | No | All sqlType options allow you to change the SQL data type used to store values for different openHAB item states. See the following links for further information: mybatis (opens new window) H2 (opens new window) PostgresSQL (opens new window) |
sqltype.COLOR | VARCHAR(70) | No | see above |
sqltype.CONTACT | VARCHAR(6) | No | see above |
sqltype.DATETIME | DATETIME | No | see above |
sqltype.DIMMER | TINYINT | No | see above |
sqltype.IMAGE | VARCHAR(65500) | No | see above |
sqltype.LOCATION | VARCHAR(50) | No | see above |
sqltype.NUMBER | DOUBLE | No | see above |
sqltype.PLAYER | VARCHAR(20) | No | see above |
sqltype.ROLLERSHUTTER | TINYINT | No | see above |
sqltype.STRING | VARCHAR(65500) | No | see above |
sqltype.SWITCH | VARCHAR(6) | No | see above |
sqltype.tablePrimaryKey | TIMESTAMP | No | type of time column for newly created item tables |
sqltype.tablePrimaryValue | NOW() | No | value of time column for newly inserted rows |
numberDecimalcount | 3 | No | for Itemtype "Number" default decimal digit count |
itemsManageTable | items | No | items manage table. For Migration from MySQL Persistence, set to Items . |
tableNamePrefix | item | No | table name prefix. For Migration from MySQL Persistence, set to Item . |
tableUseRealItemNames | false | No | table name prefix generation. When set to true , real item names are used for table names and tableNamePrefix is ignored. When set to false , the tableNamePrefix is used to generate table names with sequential numbers. |
tableCaseSensitiveItemNames | false | No | table name case. This setting is only applicable when tableUseRealItemNames is true . When set to true , item name case is preserved in table names and no prefix or suffix is added. When set to false , table names are lower cased and a numeric suffix is added. Please read this before enabling. |
tableIdDigitCount | 4 | No | when tableUseRealItemNames is false and thus table names are generated sequentially, this controls how many zero-padded digits are used in the table name. With the default of 4, the first table name will end with 0001 . For migration from the MySQL persistence service, set this to 0. |
rebuildTableNames | false | No | rename existing tables using tableUseRealItemNames and tableIdDigitCount . USE WITH CARE! Deactivate after Renaming is done! |
jdbc.maximumPoolSize | configured per database in package org.openhab.persistence.jdbc.db.* | No | Some embedded databases can handle only one connection. See this link (opens new window) for more information |
jdbc.minimumIdle | see above | No | see above |
enableLogTime | false | No | timekeeping |
All item- and event-related configuration is done in the file persistence/jdbc.persist
.
To configure this service as the default persistence service for openHAB, add or change the line
org.openhab.core.persistence:default=jdbc
in the file services/runtime.cfg
.
# Minimal Configuration
services/jdbc.cfg
url=jdbc:postgresql://192.168.0.1:5432/testPostgresql
# Case Sensitive Item Names
To avoid numbered suffixes entirely, tableUseRealItemNames
and tableCaseSensitiveItemNames
must both be enabled.
With this configuration, tables are named exactly like their corresponding items.
In order for this to work correctly, the underlying operating system, database server and configuration must support case sensitive table names.
For MySQL, see MySQL: Identifier Case Sensitivity (opens new window) for more information.
Please make sure to have a dedicated schema when using this option, since otherwise table name collisions are more likely to happen.
# Migration from MySQL to JDBC Persistence Services
The JDBC Persistence service can act as a replacement for the MySQL Persistence service.
Here is an example of a configuration for a MySQL database named testMysql
with user test
and password test
:
services/jdbc.cfg
url=jdbc:mysql://192.168.0.1:3306/testMysql
user=test
password=test
itemsManageTable=Items
tableNamePrefix=Item
tableUseRealItemNames=false
tableIdDigitCount=0
Remember to install and uninstall the services you want, and rename persistence/mysql.persist
to persistence/jdbc.persist
.
# Technical Notes
# Database Table Schema
The table name schema can be reconfigured after creation, if needed.
The service will create a mapping table to link each item to a table, and a separate table is generated for each item. The item data tables include time and data values. The SQL data type used depends on the openHAB item type, and allows the item state to be recovered back into openHAB in the same way it was stored.
With this per-item layout, the scalability and easy maintenance of the database is ensured, even if large amounts of data must be managed.
To rename existing tables, use the parameters tableNamePrefix
, tableUseRealItemNames
, tableIdDigitCount
and tableCaseSensitiveItemNames
in the configuration.
Please be aware that changing the name of itemsManageTable
is not supported by the migration.
If this is changed, the table must be renamed manually according to new configured name.
# Number Precision
Default openHAB number items are persisted with SQL datatype double
.
Internally openHAB uses BigDecimal
.
If better numerical precision is needed, for example set sqltype.NUMBER = DECIMAL(max digits, max decimals)
, then on the Java side, the service works with BigDecimal
without type conversion.
If more come decimals as max decimals
provides, this persisted value is rounded mathematically correctly.
The SQL types DECIMAL
or NUMERIC
are precise, but to work with DOUBLE
is faster.
# Rounding results
The results of database queries of number items are rounded to three decimal places by default.
With numberDecimalcount
decimals can be changed.
Especially if sql types DECIMAL
or NUMERIC
are used for sqltype.NUMBER
, rounding can be disabled by setting numberDecimalcount=-1
.
# Maintenance
Some maintenance tools are provided as console commands.
# List Tables
Tables and corresponding items can be listed with the command jdbc tables list
.
Per default only tables with some kind of problem are listed.
To list all tables, use the command jdbc tables list all
.
The list contains table name, item name, row count and status, which can be one of:
- Valid: Table is consistent.
- Item missing: Table has no corresponding item.
- Table missing: Referenced table does not exist.
- Item and table missing: Referenced table does not exist nor has corresponding item.
- Orphan table: Mapping for table does not exist in index.
# Clean Inconsistent Items
Some issues can be fixed automatically using the command jdbc tables clean
(all items having issues) or jdbc tables clean <itemName>
(single item).
This cleanup operation will remove items from the index (table Items
) if the referenced table does not exist.
If the item does not exist, the table will be physically deleted, but only if it's empty.
This precaution is taken because items may have existed previously, and the data might still be valuable.
For example, an item for a lost or repurposed sensor could have been deleted from the system while preserving persisted data.
To skip this check for a single item, use jdbc tables clean <itemName> force
with care.
Prior to performing a jdbc tables clean
operation, it's recommended to review the result of jdbc tables list
.
Fixing integrity issues can be useful before performing a migration to another naming scheme.
For example, when migrating to tableCaseSensitiveItemNames
, an index will no longer exist after the migration:
Before migration:
Table | Row count | Item | Status |
---|---|---|---|
ActualItem | 0 | Orphan table | |
TableNotBelonging | 0 | Orphan table | |
item0077 | 0 | MyItem | Table missing |
After migration:
Table | Row count | Item | Status |
---|---|---|---|
ActualItem | 0 | ActualItem | Valid |
TableNotBelonging | 0 | TableNotBelonging | Item missing |
This happened:
ActualItem
was missing in the index and became valid because it was left untouched, not being a part of the migration. After the migration, it happened to match the name of an existing item, thus it became valid.TableNotBelonging
was also not part of the migration, but since now assumed to match an item, status changed since no item with that name exists.item0077
, being the only correct table name according to previous naming scheme, disappeared from the list since it didn't have a corresponding table, and is now no longer part of any index.
In other words, extracting this information from the index before removing it, can be beneficial in order to understand the issues and possible causes.
# Reload Index/Schema
Manual changes in the index table, Items
, will not be picked up automatically for performance reasons.
The same is true when manually adding new item tables or deleting existing ones.
After making such changes, the command jdbc reload
can be used to reload the index.
# Check/fix Schema
Use the command jdbc schema check
to perform an integrity check of the schema.
Identified issues can be fixed automatically using the command jdbc schema fix
(all items having issues) or jdbc schema fix <itemName>
(single item).
Issues than can be identified and possibly fixed:
- Wrong column name case (
time
andname
). - Wrong column type. Before fixing this, make sure that time-zone is correctly configured.
- Unexpected column (identify only).
# For Developers
- Clearly separated source files for the database-specific part of openHAB logic.
- Code duplication by similar services is prevented.
- Integrating a new SQL and JDBC enabled database is fairly simple.
# Performance Tests
Not necessarily representative of the performance you may experience.
DATABASE | FIRST RUN | AVERAGE | FASTEST | SIZE AFTER | COMMENT |
---|---|---|---|---|---|
Derby | 7.829 | 6.892 | 5.381 | 5.36 MB | local embedded |
H2 | 1.797 | 2.080 | 1.580 | 0.96 MB | local embedded |
hsqldb | 3.474 | 2.104 | 1.310 | 1.23 MB | local embedded |
mysql | 11.873 | 11.524 | 10.971 | - | ext. Server VM |
postgresql | 8.147 | 7.072 | 6.895 | - | ext. Server VM |
sqlite | 2.406 | 1.249 | 1.137 | 0.28 MB | local embedded |
- Each test ran about 20 Times every 30 seconds.
- openHAB 1.x has ready started for about a Minute.
- the data in seconds for the evaluation are from the console output.
Used a script like this:
var count = 0;
rule "DB STRESS TEST"
when
Time cron "30 * * * * ?"
then
if( count = 24) count = 0
count = count+1
if( count > 3 && count < 23){
for( var i=500; i>1; i=i-1){
postUpdate( NUMBERITEM, i)
SWITCHITEM.previousState().state
postUpdate( DIMMERITEM, OFF)
NUMBERITEM.changedSince( now().minusMinutes(1))
postUpdate( DIMMERITEM, ON)
}
}
end