height:30px PgMetadata

height:50px PgMetadata


A height:30px plugin to manage metadata
for your PostgreSQL data height:40px


Michaël Douchin

height:80px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

What is Metadata ?

Help people to understand your data

  • Identification: Title, abstract, categories, themes, keywords, data last update,
  • Spatial properties: spatial level, optimal scales,
  • Publication: date, frequency, license, confidentiality
  • Computed: feature count, geometry type, projection name & code, extent
  • Contact(s): owner, publisher, custodian, etc.
  • Link(s) to resources, web pages, documents
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Pg 🐘 Metadata

Designed for people using PostgreSQL to store their layers data.

  • Centralized: data & metadata in the same database
  • Accessible: a PostgreSQL connection to share the metadata
  • PostgreSQL rich features:
    • SQL powered: relations, constraints, views, functions, triggers
    • Rights & access control: readers VS editors
  • See & Edit with your preferred SQL client:
    • Libreoffice, PgAdmin, psql, DBeaver,
    • QGIS with its powerful forms !
  • Backup & restore metadata with your data
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

As the GIS administrator

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Create the pgmetadata schema

The plugin is using a schema pgmetadata in PostgreSQL.

A QGIS processing algorithm allows to create it in your database and fill it with the needed tables, views and data (glossary and translations)

image fit height:400px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

A QGIS project builder

A QGIS processing algorithm to create a full featured QGIS administration project with rich forms:

image contain width:500px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Prepare editing

Create the needed contextual data in the dedicated tables:

  • User-defined themes
  • Contacts: name, organisation, unit, email
  • The existing glossary can be changed
  • Translations can be added if missing
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Edit your datasets with QGIS

Choose the schema and table, then edit:

  • the main fields: title, abstract, keywords, etc.
  • the contacts and their roles
  • the dataset related links
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Admin helpers

Some data are calculated from the table content:

  • valid unique id for the dataset e0940d27-0059-4156-85e7-ef6b3cb57230
  • layer extent, feature count, geometry type, projection id & name.
  • creation and update dates, etc.

Some useful views:

  • Orphan PostgreSQL tables: no metadata exists in the dataset table for this tables
  • Orphan metadata: a line exists in your dataset table, but no table corresponds in your database
  • Flat representation of the datasets: lists the datasets with contacts and links aggregated
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

As the GIS user in

height:50px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

QGIS locator & Metadata panel

CTRL+K, type meta, find the table, add the layer & view metadata

image height:450px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Export

The user can export each dataset metadata to:

  • HTML
  • PDF
  • DCAT https://www.w3.org/TR/vocab-dcat-2/
    <dcat:dataset>
  <dcat:Dataset>
    <dct:identifier>e0940d27-0059-4156-85e7-ef6b3cb57230</dct:identifier>
    <dct:title>Trees (demo)</dct:title>
    <dct:description>Trees around the botanical garden in Montpellier.
Source: OpenStreetMap</dct:description>
    <dct:language>en</dct:language>
    <dct:license>Open Data Commons Open Database License</dct:license>
    <dct:rights>Open</dct:rights>
    <dct:accrualPeriodicity>Yearly</dct:accrualPeriodicity>
    <dct:spatial>{&quot;type&quot;:&quot;Polygon&quot;,&quot;coordinates&quot;:[]]}</dct:spatial>
    <dct:created rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-09-28T08:55:44.606067</dct:created>
    <dct:issued rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-09-28T08:55:44.606067</dct:issued>
    <dct:modified rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-09-28T08:55:44.606067</dct:modified>
    <dcat:contactPoint>
      <vcard:Organization>
        <vcard:fn>Jane Doe - ACME (SIG)</vcard:fn>
        <vcard:hasEmail rdf:resource="jane.doe@acme.corp">jane.doe@acme.corp</vcard:hasEmail>
      </vcard:Organization>
    </dcat:contactPoint>
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

More ?

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Advanced features

  • Easily change the templates for the HTML content (visible in the panel): they are stored inside the html_template table

  • Generate a dataset HTML card with SQL

    SELECT pgmetadata.get_dataset_item_html_content('demo', 'trees', 'fr');

  • Generate a DCAT representation with SQL for one or many tables

    SELECT *
FROM pgmetadata.get_datasets_as_dcat_xml('fr')
WHERE True

  • QGIS configuration file variables when deploying QGIS in you organisation (hide admin tools, auto-activate plugin)

    [pgmetadata]
auto_open_dock=true
end_user_only=true
connection_names=Connection 1;Connection 2;Connection 3

[Plugins]
pg_metadata=true
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Share

(web) Applications can use the SQL functions to show the localized metadata in HTML format or publish the full catalog in DCAT (and be harvested by Third party Metadata portals).

Example of Lizmap Web Client PgMetadata module: https://github.com/3liz/lizmap-pgmetadata-module/

image height:450px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Documentation

  • For the administrator
  • For the end user
  • For the system administrator
  • Changelogs, videos, road map, database structure, etc.

https://docs.3liz.org/qgis-pgmetadata-plugin/

image height:450px

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Conclusion

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Why another metadata tool ?

Many open-source tools already exist to store and share metadata.
Why PgMetadata ?

  • See the previous slide about PostgreSQL 🐘
  • Keep the metadata as close as possible to the data
  • Not a new application, but a set of tools for QGIS and your existing PostgreSQL database:
    • the GIS administrator already uses PostgreSQL and can understand easily how PgMetadata works,
    • the GIS users do not need to learn to use a new application
  • GIS user oriented: as a user, search & get the metadata from QGIS VERSUS browse a web page and download the data
  • It is NOT designed to replace the existing metadata web portals, but to be used as a complementary tool !
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Road map

More locales (today in English, French & German)

New features:

  • Support raster tables
  • Auto-fill the dataset table from a selection of PostgreSQL tables/views
  • Import/Export the QGIS native layer metadata properties
  • Import metadata from DCAT
height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Resources

New version 1.1.0 released today : views support, German translations, new items in the glossary, enhanced locator search, etc.

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Thanks

image width:50px Thanks to the French Gard province for funding this extension

PgMetadata already has external contributors: thanks @effjot & @tschuettenberg for testing and helping !

height:30px Foss4G - Buenos Aires 2021
height:30px PgMetadata

Thank you for your attention

image height:450px

height:30px Foss4G - Buenos Aires 2021