Develop and use the dmdScheme

dmdScheme_0.9.9

Rainer M Krug Rainer.Krug@uzh.ch

2020-06-12

Using the dmdScheme

The functionality for using the dmdScheme is available either via the online app (which is identical to being housed at an inhouse shiny server), the local running app, or via the R command line. The simplest method is using the online app, as no additional software needs to be installed locally. To be able to use the dmdScheme functionality locally, either via the app or via the R command line, it is necessary to install R [@RCoreTeam2019] and the dmdScheme package [@Krug2019a] in R.

In the following section, I will go through the different stages of using the dmdScheme web app, a local app or from the R prompt. The detailed commands which have to be used can be found in Figure 1.

**Figure 1**: Workflow of using `dmdScheme` (A) via the web app, (B) via the app locally, (C) via the R prompt. Square boxes indicate steps which are not common to all three, rounded boxes indicate steps identical to the different ways, although how they are executed can differ.**Figure 1**: Workflow of using `dmdScheme` (A) via the web app, (B) via the app locally, (C) via the R prompt. Square boxes indicate steps which are not common to all three, rounded boxes indicate steps identical to the different ways, although how they are executed can differ.**Figure 1**: Workflow of using `dmdScheme` (A) via the web app, (B) via the app locally, (C) via the R prompt. Square boxes indicate steps which are not common to all three, rounded boxes indicate steps identical to the different ways, although how they are executed can differ.

Figure 1: Workflow of using dmdScheme (A) via the web app, (B) via the app locally, (C) via the R prompt. Square boxes indicate steps which are not common to all three, rounded boxes indicate steps identical to the different ways, although how they are executed can differ.

Preparation

The app (Figure 2) can be accessed either via the internet as a web app at https://rmkrug.shinyapps.io/dmd_app/, or locally. To run it locally, you need R installed and the dmdScheme package installed, preferably from CRAN. After loading the dmdScheme package, you can start the app locally by running run_app() at the R prompt. After these steps, the usage of the two apps is identical.

When using the dmdScheme from the command line, the initial setup is the same as running the app locally, only that it is not necessary to start the app.

**Figure 2**: Screenshot of the dmdScheme app.

Figure 2: Screenshot of the dmdScheme app.

Select scheme

The package dmdScheme does not come with a specific scheme, and installs upon loading a generic dmdScheme from the dmdScheme scheme repository at https://github.com/Exp-Micro-Ecol-Hub/dmdSchemeRepository. In nearly all circumstances, a specific scheme needs to be installed, together with the accompanying R package. In the app, this is done via selecting a theme in the section “Available dmdSchemes”. This list is generated automatically upon starting of the app from the schemes available in the scheme repository. The selection of the scheme will download the scheme definition package, install any accompanying R package as specified in the scheme definition package, load the accompanying R package, and activate the scheme definition.

To do this from the R prompt, one has do these steps manually and install the scheme, install the accompanying R package, load the accompanying R package, and activate the scheme itself (see Figure 1 C for the commands).

Download new scheme

The spreadsheet to enter the metadata can be obtained from the app via the “Empty scheme spreadsheet” bottom. This will download an .xlsx spreadsheet containing the definition of the scheme and fields which need to be filled in.

In R, the spreadsheet can be obtained by using the open_new_spreadsheet() command.

Figure 3 shows two screenshots of the spreadsheet as opened in Excel.

**Figure 3**: Some example tabs from the `emeScheme` spreadsheet. The first to contains bibliometric metadata modelled along the requirements by DataCite and the authors in the second tab. The third one contains metadata about the Species used in the experiment. The complete spreadsheet can be found in the supplemental material `emeScheme.xlsx`.**Figure 3**: Some example tabs from the `emeScheme` spreadsheet. The first to contains bibliometric metadata modelled along the requirements by DataCite and the authors in the second tab. The third one contains metadata about the Species used in the experiment. The complete spreadsheet can be found in the supplemental material `emeScheme.xlsx`.**Figure 3**: Some example tabs from the `emeScheme` spreadsheet. The first to contains bibliometric metadata modelled along the requirements by DataCite and the authors in the second tab. The third one contains metadata about the Species used in the experiment. The complete spreadsheet can be found in the supplemental material `emeScheme.xlsx`.

Figure 3: Some example tabs from the emeScheme spreadsheet. The first to contains bibliometric metadata modelled along the requirements by DataCite and the authors in the second tab. The third one contains metadata about the Species used in the experiment. The complete spreadsheet can be found in the supplemental material emeScheme.xlsx.

Enter / Edit metadata on downloaded xlsx

The entering of the metadata is done in the spreadsheet downloaded before. The spreadsheet contains the definition of the scheme as well as all information needed to fill in the metadata, including “type” of the metadata (e.g. numeric, character, or boolean), “allowed values” or “suggested values”, “unit” as well as a “Description” field.

Metadata can be entered in all green cells, while all red cells are (should be) locked and are part of the scheme definition.

Here the familiarity of researchers with spreadsheets plays an important role, as the technical details of the entering itself do not need to be explained as it is a standard spreadsheet.

The metadata in the .xlsx file can be entered and edited in likely all spreadsheet programs which can read and write .xlsx files. We tested successfully Excel, Pages, Libre Office and Google Docs.

A spreadsheet as frontend for entering the metadata offers the additional possibilities to specify validation rules within the spreadsheet itself to be checked during the entering process and not only afterwards. As we wanted to maintain the flexibility to edit the spreadsheet in different spreadsheet editors in a consistent way, this is not implemented in the dmdScheme or emeScheme [@Krug2019b], but could be easily done in other schemes.

Validate metadata and create report

To be able to validate the metadata, the validation needs access to the metadata as well as (in most cases) the data files themselves, as the validation can include checking for complete definition of the column names in tables, as implemented in the @Krug2019b. Consequentially, the metadata as well as the data needs to be uploaded to the app, which is done via the “Upload spreadsheet containing metadata” and the “Upload datafiles” buttons. In the case of the web app, these are uploaded to the server running the app (in this case https://www.shinyapps.io, which is run by RStudio Inc.). If there are e.g. confidentiality or size reasons why an upload to a server is not an option, one could host an inhouse shiny server, which would run the shiny server and the app. The other option would be to use the local app, in which case the metadata and data remain on the local computer. Nevertheless, the data is copied into a different directory. If for example size constraints would prohibit that approach, one can use the R prompt. In this case, the validation function does not need to do any copying of the data, and only needs the path to the directory in which the data resides.

After completion of the validation, the app (web or local) downloads a report, in the default setting an html report. Optional, the report could be a word docx document or a pdf.

The report (see Figure 4) does show errors (which normally should be fixed), warnings, which are not necessarily errors, but can be errors (depending on the validation rules), and notes. The level of details, and aspects validated, in this report depends on the validation function, which can be changed in the accompanying R package.

The editing - validation cycle should be repeated until the report is satisfactory.

On the R prompt, the validation involve the two commands validate() and report() to validate the spreadsheet and to create the report from the object resulting from the validation.

**Figure 4**: Example validation report, as cfrom the validation included in the `emeScheme` package [@Krug2019b].

Figure 4: Example validation report, as cfrom the validation included in the emeScheme package [@Krug2019b].

Export uploaded spreadsheet to xml

In the app, the xml can be obtained by using the “Export to xml” button. Depending on the export functionality implemented for the selected scheme (as defined in the accompanying R package), the file returned is an .xml file (dmdScheme) or a compressed archive (.tar.gz) containing multiple .xml files (e.g. for the emeScheme, one per data file). On the R prompt, the command write_xml() will export to a single (dmdScheme) or multiple .xml files (emeScheme) and return the names(s) of the file(s) exported.

The R Package

The package dmdScheme is a base package for the usage and development of domain specific metadata schemes. It provides functionality to enter the metadata, validate the entered metadata, and export it to xml format for further processing by e.g. archival repositories. This document will give an overview over the dmdScheme package and what it contains. More detailed information about the development, including a suggested workflow to create a new scheme definition, can be found in the vignette Howto create a new scheme.

Installation

The recommended way is to install from CRAN the stable released version:

install.packages("dmdScheme")

To install the master branch, the stable branch which will become the new CRAN release, from the dmdScheme repository on github, run

## install the devtools package if not installed yet
# install.packages("devtools")

devtools::install_github("Exp-Micro-Ecol-Hub/dmdScheme", ref = "master", build_opts = NULL)

If you are feeling adventurous, want to live at the bleeding edge and can live with non-working features, you can install the dev branch. This branch is not stable and features and functionality can appear or be remioved without prior notice:

## install the devtools package if not installed yet
# install.packages("devtools")

devtools::install_github("Exp-Micro-Ecol-Hub/dmdScheme", ref = "dev", build_opts = NULL)

Other branches are not generally recommended for installation unless you are involved in dmdScheme package development.

Loading the package

When you load the package, the definition of the scheme is downloaded from the dmdScheme repository installed to a temporary scheme library in a temporary directory for usage in this R session. As this scheme library is styored in a temporary directory, it will be deleted when youu quit your R session and re-downloaded each time you start a new session and load the package. To create a permanent package library you have to create a cache in the user directory. To do this, run

cache(createPermanent = TRUE)

and restart your R session. Now the definitions of the installed dmdSchemes will be installed in this user cache and be available permanently. For further info, see the documenatation of the command cache().

Managing installed schemes

Once loaded, the default scheme definition from the package is, when necessary, installed and used.

There are several commands to manage installed schemes. All these commands start with scheme_.

These are in detail:

scheme_default()

returns the default scheme and version, i.e. the one with which the dmdScheme package is based. This can be the dmdScheme package itself, or a package crewated with make_new_package().

scheme_default()
#>        name version
#> 1 dmdScheme   0.9.9

scheme_repo()

Get or set scheme repository. If repo is specified, the scheme repository to be used is set. Otherwise, the scheme repository used is only returned.

scheme_repo()
#> [1] "https://github.com/Exp-Micro-Ecol-Hub/dmdSchemeRepository/raw/master/"

scheme_list_in_repo()

Show the schemes available in a repo. The defaut repo is . The function reads and simly returns the file SCHEME_DEFINITIONS.yaml in the folder schemes in the repository.

scheme_list_in_repo()
#> $dmdScheme_0.9.5
#> $dmdScheme_0.9.5$name
#> [1] "dmdScheme"
#> 
#> $dmdScheme_0.9.5$version
#> [1] "0.9.5"
#> 
#> $dmdScheme_0.9.5$description
#> [1] "Demo scheme from the dmdScheme package in R"
#> 
#> 
#> $dmdScheme_0.9.9
#> $dmdScheme_0.9.9$name
#> [1] "dmdScheme"
#> 
#> $dmdScheme_0.9.9$version
#> [1] "0.9.9"
#> 
#> $dmdScheme_0.9.9$description
#> [1] "Demo scheme from the dmdScheme package in R including bibliometric metadata"
#> 
#> 
#> $emeScheme_0.9.5
#> $emeScheme_0.9.5$name
#> [1] "emeScheme"
#> 
#> $emeScheme_0.9.5$version
#> [1] "0.9.5"
#> 
#> $emeScheme_0.9.5$description
#> [1] "Scheme for Ecological Microcosom Experiments"
#> 
#> 
#> $emeScheme_0.9.9
#> $emeScheme_0.9.9$name
#> [1] "emeScheme"
#> 
#> $emeScheme_0.9.9$version
#> [1] "0.9.5"
#> 
#> $emeScheme_0.9.9$description
#> [1] "Scheme for Ecological Microcosom Experiments"

scheme_download()

Download a scheme definition from the repo set and stores in the destfile. The function returns the fully qualified file name to the downloaded file invisibly.

scheme_download(name = "emeScheme", version = "0.9.5", destfile = tempfile())
#> Warning in normalizePath(destfile): path[1]="/var/folders/50/
#> wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/file64a17edfd388": No such file or
#> directory

scheme_install()

Install a new scheme definition. In the normal usecase, this function uses scheme_download_url() to download the scheme defintion from the default github repository and installs it. The usage is

scheme_install(name = "emeScheme", version = "0.9.5")
#> Scheme definition `/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/file64a1b03461b/emeScheme_0.9.5.tar.gz, installed with
#> name:    emeScheme
#> version: 0.9.5

scheme_install_r_package()

Installs the accompanying R package of the same name as the scheme. This does only install the package - it still b=needs to be loaded to be used!

scheme_install_r_package(name = "emeScheme", version = "0.9.5")
#> Package emeScheme is already installed!
#> To re-install or update, please specify `reinstall = TRUE`

scheme_list()

Shows the installed schemes.

scheme_list()
#>        name version
#> 1 dmdScheme   0.9.9
#> 2 emeScheme   0.9.5

scheme_use()

Activate the scheme NAME with version VERSION

scheme_use(name = "emeScheme", version = "0.9.5")
#> Theme switched to emeScheme_0.9.5

scheme_active()

Shows the currently active scheme.

scheme_active()
#>        name version
#> 1 emeScheme   0.9.5
scheme_use(name = "dmdScheme", version = "0.9.9")
#> Theme switched to dmdScheme_0.9.9
scheme_active()
#>        name version
#> 1 dmdScheme   0.9.9

scheme_path_index_template()

Returns the path to the index template. The search order is: 1. the index file included in the scheme package 2. the scheme package included in the dmdScheme package (not the acompanying package!).

scheme_path_index_template()
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/dmdScheme_64a145f69398/installedSchemes/dmdScheme_0.9.9/index.md"

scheme_path_xlsx()

Returns the path to the xlsx file included in the scheme package.

scheme_path_xlsx()
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/dmdScheme_64a145f69398/installedSchemes/dmdScheme_0.9.9/dmdScheme_0.9.9.xlsx"

scheme_path_xml()

Returns the path to the xml file included in the scheme package.

scheme_path_xml()
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/dmdScheme_64a145f69398/installedSchemes/dmdScheme_0.9.9/dmdScheme_0.9.9.xml"

scheme_uninstall()

Uninstall an installed scheme. The scheme definition, is deleted from the scheme library, and moved to a temporary flder which wil be deleted at the end of the R session.

scheme_uninstall(name = "emeScheme", version = "0.9.5")
#> Scheme emeScheme_0.9.5 deleted and moved to
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/file64a120d66d73"

scheme_installed()

Return `TRUE, if the scheme is installed

scheme_installed(name = "emeScheme", version = "0.9.5")
#> [1] FALSE

scheme_path_xlsx()

Returns the path to the .xlsx representation of the currently active scheme.

scheme_path_xlsx()
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/dmdScheme_64a145f69398/installedSchemes/dmdScheme_0.9.9/dmdScheme_0.9.9.xlsx"

scheme_path_xml()

Returns the path to the .xml representation of the currently active scheme.

scheme_path_xml()
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/dmdScheme_64a145f69398/installedSchemes/dmdScheme_0.9.9/dmdScheme_0.9.9.xml"

scheme_make()

This function is used to package a new scheme. For details see the vignette Howto Create a new scheme.

Entering new Metadata

To enter new data to the dmdScheme, you have to run the command

open_new_spreadsheet()

This will open Excel and the file should look similar to this, when looking at the second tab as in Figure 3.

The following points are important to remember:

  1. The file is saved in a temporary directory. It needs to be saved at a different location, if you want to keep the changes.
  2. Data can only be entered in the green cells with. All other cells are write protected.

After entering the data, save it to a location for further processing.

Importing Data from Excel Sheet

Next, you have to import the data entered in the Excel sheet into R. For simplicity, we use here a file included in the package. If you want to load your own file, replace scheme_path_xlsx() with the file name and path to that file.

x <- read_excel(
      file = scheme_path_xlsx(),
      verbose = TRUE
)
#> Transposing MdBibliometric...
#> Processing propertySet : MdBibliometric
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'uploadType'...
#>    Apply type 'character' to 'doi'...
#>    Apply type 'character' to 'publicationDate'...
#>    Apply type 'character' to 'title'...
#>    Apply type 'character' to 'description'...
#>    Apply type 'character' to 'version'...
#>    Apply type 'character' to 'language'...
#>    Apply type 'character' to 'keywords'...
#>    Apply type 'character' to 'additionalNotes'...
#>    Apply type 'character' to 'accessRights'...
#>    Apply type 'character' to 'accessRightsInfo'...
#>    Apply type 'character' to 'license'...
#> Set class...
#> Done
#> Processing propertySet : MdAuthors
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'authorID'...
#>    Apply type 'character' to 'familyName'...
#>    Apply type 'character' to 'givenName'...
#>    Apply type 'character' to 'affiliation'...
#>    Apply type 'character' to 'orcid'...
#>    Apply type 'character' to 'role'...
#> Set class...
#> Done
#> Transposing Experiment...
#> Processing propertySet : Experiment
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'name'...
#>    Apply type 'character' to 'temperature'...
#>    Apply type 'character' to 'light'...
#>    Apply type 'character' to 'humidity'...
#>    Apply type 'character' to 'incubator'...
#>    Apply type 'character' to 'container'...
#>    Apply type 'numeric' to 'microcosmVolume'...
#>    Apply type 'character' to 'mediaType'...
#>    Apply type 'numeric' to 'mediaConcentration'...
#>    Apply type 'character' to 'cultureConditions'...
#>    Apply type 'character' to 'comunityType'...
#>    Apply type 'character' to 'mediaAdditions'...
#>    Apply type 'integer' to 'duration'...
#>    Apply type 'character' to 'comment'...
#> Set class...
#> Done
#> Processing propertySet : Genus
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'speciesID'...
#>    Apply type 'character' to 'colour'...
#>    Apply type 'character' to 'density'...
#>    Apply type 'character' to 'functionalGroup'...
#>    Apply type 'character' to 'comment'...
#> Set class...
#> Done
#> Processing propertySet : Treatments
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'treatmentID'...
#>    Apply type 'character' to 'treatmentLevelHeight'...
#>    Apply type 'character' to 'comment'...
#> Set class...
#> Done
#> Processing propertySet : Measurement
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'measurementID'...
#>    Apply type 'character' to 'variable'...
#>    Apply type 'character' to 'method'...
#>    Apply type 'character' to 'unit'...
#>    Apply type 'character' to 'object'...
#>    Apply type 'integer' to 'noOfSamplesInTimeSeries'...
#>    Apply type 'numeric' to 'samplingVolume'...
#>    Apply type 'character' to 'dataExtractionID'...
#>    Apply type 'character' to 'measuredFrom'...
#>    Apply type 'character' to 'comment'...
#> Set class...
#> Done
#> Processing propertySet : DataExtraction
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'dataExtractionID'...
#>    Apply type 'character' to 'method'...
#>    Apply type 'character' to 'parameter'...
#>    Apply type 'character' to 'value'...
#>    Apply type 'character' to 'comment'...
#> Set class...
#> Done
#> Processing propertySet : DataFileMetaData
#> Set names...
#> Set attributes...
#> Apply types...
#>    Apply type 'character' to 'dataFileName'...
#>    Apply type 'character' to 'columnName'...
#>    Apply type 'character' to 'columnData'...
#>    Apply type 'character' to 'mappingColumn'...
#>    Apply type 'character' to 'type'...
#>    Apply type 'character' to 'description'...
#>    Apply type 'character' to 'comment'...
#> Set class...
#> Done

The verbose = TRUE argument will produce messages which will show you what is happening and will help to identify problems.

Validating your metadata

The metadata in the spreadsheet can be validated by using the following command and which results in an object of class dmdScheme_validation.

validate( scheme_path_xlsx() )
#> Validating Experiment
#> Validating MdBibliometric
#> Validating MdAuthors
#> Validating Genus
#> Validating Treatments
#> Validating Measurement
#> Validating DataExtraction
#> Validating DataFileMetaData
#> $error
#> [1] 3
#> 
#> $details
#> [1] NA
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">Overall MetaData - error</span>**"
#> 
#> $description
#> [1] "The result of the overall validation of the data."
#> 
#> $descriptionDetails
#> [1] "The details contain the different validations of the metadata as a hierarchical list. errors propagate towards the root, i.e., if the 'worst' is a 'warning' in a validation in `details` the error here will be a 'warning' as well."
#> 
#> $comment
#> [1] ""
#> 
#> $structure
#> $error
#> [1] 0
#> 
#> $details
#> [1] TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">Structural / Formal validity - OK</span>**"
#> 
#> $description
#> [1] "Test if the structure of the metadata is correct.  This includes column names, required info, ...  Should normally be OK, if no modification has been done."
#> 
#> $descriptionDetails
#> [1] ""
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $Experiment
#> $error
#> [1] 0
#> 
#> $details
#>   Module errorCode isOK
#> 1  types        OK TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">Experiment - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning **Experiment** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   name temperature light humidity incubator container microcosmVolume mediaType
#> 1 TRUE        TRUE  TRUE     TRUE      TRUE      TRUE            TRUE      TRUE
#>   mediaConcentration cultureConditions comunityType mediaAdditions duration
#> 1               TRUE              TRUE         TRUE           TRUE     TRUE
#>   comment
#> 1    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $MdBibliometric
#> $error
#> [1] 3
#> 
#> $details
#>            Module errorCode  isOK
#> 1           types        OK  TRUE
#> 2 suggestedValues        OK  TRUE
#> 3   allowedValues        OK  TRUE
#> 4         IDField     error FALSE
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">MdBibliometric - error</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning ** MdBibliometric ** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   uploadType  doi publicationDate title description version language keywords
#> 1       TRUE TRUE            TRUE  TRUE        TRUE    <NA>     TRUE     TRUE
#>   additionalNotes accessRights accessRightsInfo license
#> 1            <NA>         TRUE             <NA>    TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $suggestedValues
#> $error
#> [1] 0
#> 
#> $details
#>   uploadType  doi publicationDate title description version language keywords
#> 1       <NA> <NA>            <NA>  <NA>        <NA>    <NA>     <NA>     <NA>
#>   additionalNotes accessRights accessRightsInfo license
#> 1            <NA>         <NA>             <NA>    TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in suggestedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the suggestedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the suggestedValues list.\n    TRUE : If the cell value is contained in the suggestedValues list.\n    NA   : empty cell or no suggested values specified\n \n One or more FALSE values will result in a WARNING."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 0
#> 
#> $details
#>   uploadType  doi publicationDate title description version language keywords
#> 1       TRUE <NA>            <NA>  <NA>        <NA>    <NA>     <NA>     <NA>
#>   additionalNotes accessRights accessRightsInfo license
#> 1            <NA>         TRUE             <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in allowedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $IDField
#> $error
#> [1] 3
#> 
#> $details
#>                         hasIDField  isOK
#> 1 tab has ID field in first column FALSE
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">ID Field present and in the first column - error</span>**"
#> 
#> $description
#> [1] "Check if the tab's first column contains an ID field, named as `...ID`. This function does not check for uniqueness of this ID field!"
#> 
#> $descriptionDetails
#> [1] "Returns a boolean value, with the following possible values:\n \n    TRUE  : The tab's first column is an ID field\n    FALSE : The tab's first column is not an ID field\n \n FALSE will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $MdAuthors
#> $error
#> [1] 3
#> 
#> $details
#>            Module errorCode  isOK
#> 1           types        OK  TRUE
#> 2 suggestedValues        OK  TRUE
#> 3   allowedValues     error FALSE
#> 4         IDField        OK  TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">MdAuthors - error</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning ** MdAuthors ** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   authorID familyName givenName affiliation orcid role
#> 1     TRUE       TRUE      TRUE        TRUE  <NA> TRUE
#> 2     TRUE       TRUE      TRUE        TRUE  <NA> TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $suggestedValues
#> $error
#> [1] 0
#> 
#> $details
#>   authorID familyName givenName affiliation orcid role
#> 1     <NA>       <NA>      <NA>        <NA>  <NA> <NA>
#> 2     <NA>       <NA>      <NA>        <NA>  <NA> <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in suggestedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the suggestedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the suggestedValues list.\n    TRUE : If the cell value is contained in the suggestedValues list.\n    NA   : empty cell or no suggested values specified\n \n One or more FALSE values will result in a WARNING."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 3
#> 
#> $details
#>   authorID familyName givenName affiliation orcid  role
#> 1     <NA>       <NA>      <NA>        <NA>  <NA> FALSE
#> 2     <NA>       <NA>      <NA>        <NA>  <NA>  TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">values in allowedValues - error</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $IDField
#> $error
#> [1] 0
#> 
#> $details
#>                         hasIDField isOK
#> 1 tab has ID field in first column TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">ID Field present and in the first column - OK</span>**"
#> 
#> $description
#> [1] "Check if the tab's first column contains an ID field, named as `...ID`. This function does not check for uniqueness of this ID field!"
#> 
#> $descriptionDetails
#> [1] "Returns a boolean value, with the following possible values:\n \n    TRUE  : The tab's first column is an ID field\n    FALSE : The tab's first column is not an ID field\n \n FALSE will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $Genus
#> $error
#> [1] 1
#> 
#> $details
#>            Module errorCode  isOK
#> 1           types        OK  TRUE
#> 2 suggestedValues      note FALSE
#> 3   allowedValues        OK  TRUE
#> 4         IDField        OK  TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#55AA00\">Genus - note</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning ** Genus ** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   speciesID colour density functionalGroup comment
#> 1      TRUE   TRUE    TRUE            TRUE    TRUE
#> 2      TRUE   TRUE    <NA>            TRUE    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $suggestedValues
#> $error
#> [1] 1
#> 
#> $details
#>   speciesID colour density functionalGroup comment
#> 1      <NA>   <NA>   FALSE            TRUE    <NA>
#> 2      <NA>   <NA>    TRUE           FALSE    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#55AA00\">values in suggestedValues - note</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the suggestedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the suggestedValues list.\n    TRUE : If the cell value is contained in the suggestedValues list.\n    NA   : empty cell or no suggested values specified\n \n One or more FALSE values will result in a WARNING."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 0
#> 
#> $details
#>   speciesID colour density functionalGroup comment
#> 1      <NA>   <NA>    <NA>            <NA>    <NA>
#> 2      <NA>   <NA>    <NA>            <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in allowedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $IDField
#> $error
#> [1] 0
#> 
#> $details
#>                         hasIDField isOK
#> 1 tab has ID field in first column TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">ID Field present and in the first column - OK</span>**"
#> 
#> $description
#> [1] "Check if the tab's first column contains an ID field, named as `...ID`. This function does not check for uniqueness of this ID field!"
#> 
#> $descriptionDetails
#> [1] "Returns a boolean value, with the following possible values:\n \n    TRUE  : The tab's first column is an ID field\n    FALSE : The tab's first column is not an ID field\n \n FALSE will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $Treatments
#> $error
#> [1] 1
#> 
#> $details
#>            Module errorCode  isOK
#> 1           types        OK  TRUE
#> 2 suggestedValues      note FALSE
#> 3   allowedValues        OK  TRUE
#> 4         IDField        OK  TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#55AA00\">Treatments - note</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning ** Treatments ** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   treatmentID treatmentLevelHeight comment
#> 1        TRUE                 TRUE    <NA>
#> 2        TRUE                 TRUE    <NA>
#> 3        TRUE                 TRUE    <NA>
#> 4        TRUE                 TRUE    <NA>
#> 5        TRUE                 TRUE    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $suggestedValues
#> $error
#> [1] 1
#> 
#> $details
#>   treatmentID treatmentLevelHeight comment
#> 1       FALSE                FALSE    <NA>
#> 2       FALSE                FALSE    <NA>
#> 3       FALSE                FALSE    <NA>
#> 4       FALSE                FALSE    <NA>
#> 5       FALSE                FALSE    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#55AA00\">values in suggestedValues - note</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the suggestedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the suggestedValues list.\n    TRUE : If the cell value is contained in the suggestedValues list.\n    NA   : empty cell or no suggested values specified\n \n One or more FALSE values will result in a WARNING."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 0
#> 
#> $details
#>   treatmentID treatmentLevelHeight comment
#> 1        <NA>                 <NA>    <NA>
#> 2        <NA>                 <NA>    <NA>
#> 3        <NA>                 <NA>    <NA>
#> 4        <NA>                 <NA>    <NA>
#> 5        <NA>                 <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in allowedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $IDField
#> $error
#> [1] 0
#> 
#> $details
#>                         hasIDField isOK
#> 1 tab has ID field in first column TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">ID Field present and in the first column - OK</span>**"
#> 
#> $description
#> [1] "Check if the tab's first column contains an ID field, named as `...ID`. This function does not check for uniqueness of this ID field!"
#> 
#> $descriptionDetails
#> [1] "Returns a boolean value, with the following possible values:\n \n    TRUE  : The tab's first column is an ID field\n    FALSE : The tab's first column is not an ID field\n \n FALSE will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $Measurement
#> $error
#> [1] 1
#> 
#> $details
#>            Module errorCode  isOK
#> 1           types        OK  TRUE
#> 2 suggestedValues      note FALSE
#> 3   allowedValues        OK  TRUE
#> 4         IDField        OK  TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#55AA00\">Measurement - note</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning ** Measurement ** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   measurementID variable method unit object noOfSamplesInTimeSeries
#> 1          TRUE     TRUE   TRUE TRUE   TRUE                    TRUE
#> 2          TRUE     TRUE   TRUE TRUE   TRUE                    TRUE
#> 3          TRUE     TRUE   TRUE TRUE   TRUE                    TRUE
#> 4          TRUE     TRUE   TRUE TRUE   TRUE                    TRUE
#>   samplingVolume dataExtractionID measuredFrom comment
#> 1           TRUE             TRUE         TRUE    <NA>
#> 2           TRUE             TRUE         TRUE    <NA>
#> 3           TRUE             TRUE         TRUE    <NA>
#> 4           TRUE             TRUE         TRUE    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $suggestedValues
#> $error
#> [1] 1
#> 
#> $details
#>   measurementID variable method  unit object noOfSamplesInTimeSeries
#> 1          <NA>    FALSE   TRUE  TRUE   TRUE                    <NA>
#> 2          <NA>     TRUE  FALSE  TRUE   TRUE                    <NA>
#> 3          <NA>    FALSE  FALSE FALSE   TRUE                    <NA>
#> 4          <NA>     TRUE  FALSE FALSE  FALSE                    <NA>
#>   samplingVolume dataExtractionID measuredFrom comment
#> 1           <NA>             <NA>         <NA>    <NA>
#> 2           <NA>             <NA>         <NA>    <NA>
#> 3           <NA>             <NA>         <NA>    <NA>
#> 4           <NA>             <NA>         <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#55AA00\">values in suggestedValues - note</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the suggestedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the suggestedValues list.\n    TRUE : If the cell value is contained in the suggestedValues list.\n    NA   : empty cell or no suggested values specified\n \n One or more FALSE values will result in a WARNING."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 0
#> 
#> $details
#>   measurementID variable method unit object noOfSamplesInTimeSeries
#> 1          <NA>     <NA>   <NA> <NA>   <NA>                    <NA>
#> 2          <NA>     <NA>   <NA> <NA>   <NA>                    <NA>
#> 3          <NA>     <NA>   <NA> <NA>   <NA>                    <NA>
#> 4          <NA>     <NA>   <NA> <NA>   <NA>                    <NA>
#>   samplingVolume dataExtractionID measuredFrom comment
#> 1           <NA>             <NA>         <NA>    <NA>
#> 2           <NA>             <NA>         <NA>    <NA>
#> 3           <NA>             <NA>         <NA>    <NA>
#> 4           <NA>             <NA>         <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in allowedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $IDField
#> $error
#> [1] 0
#> 
#> $details
#>                         hasIDField isOK
#> 1 tab has ID field in first column TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">ID Field present and in the first column - OK</span>**"
#> 
#> $description
#> [1] "Check if the tab's first column contains an ID field, named as `...ID`. This function does not check for uniqueness of this ID field!"
#> 
#> $descriptionDetails
#> [1] "Returns a boolean value, with the following possible values:\n \n    TRUE  : The tab's first column is an ID field\n    FALSE : The tab's first column is not an ID field\n \n FALSE will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $DataExtraction
#> $error
#> [1] 0
#> 
#> $details
#>            Module errorCode isOK
#> 1           types        OK TRUE
#> 2 suggestedValues        OK TRUE
#> 3   allowedValues        OK TRUE
#> 4         IDField        OK TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">DataExtraction - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning ** DataExtraction ** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>   dataExtractionID method parameter value comment
#> 1             TRUE   <NA>      <NA>  <NA>    TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $suggestedValues
#> $error
#> [1] 0
#> 
#> $details
#>   dataExtractionID method parameter value comment
#> 1             <NA>   TRUE      <NA>  <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in suggestedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the suggestedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the suggestedValues list.\n    TRUE : If the cell value is contained in the suggestedValues list.\n    NA   : empty cell or no suggested values specified\n \n One or more FALSE values will result in a WARNING."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 0
#> 
#> $details
#>   dataExtractionID method parameter value comment
#> 1             <NA>   <NA>      <NA>  <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in allowedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $IDField
#> $error
#> [1] 0
#> 
#> $details
#>                         hasIDField isOK
#> 1 tab has ID field in first column TRUE
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">ID Field present and in the first column - OK</span>**"
#> 
#> $description
#> [1] "Check if the tab's first column contains an ID field, named as `...ID`. This function does not check for uniqueness of this ID field!"
#> 
#> $descriptionDetails
#> [1] "Returns a boolean value, with the following possible values:\n \n    TRUE  : The tab's first column is an ID field\n    FALSE : The tab's first column is not an ID field\n \n FALSE will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $DataFileMetaData
#> $error
#> [1] 3
#> 
#> $details
#>           Module errorCode  isOK
#> 1          types        OK  TRUE
#> 2  allowedValues        OK  TRUE
#> 3 dataFilesExist     error FALSE
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">DataFileMetaData - error</span>**"
#> 
#> $description
#> [1] "Test if the metadata concerning **DataExtraction** is correct.  This includes column names, required info, ... "
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique validation.\n The column `Module` contains the name of the validation,\n The column `error` contains the actual error of the validation.\n The following values are possible for the column `isTRUE`:\n \n    TRUE : If the validation was `OK`.\n    FALSE: If the validation was an `error`, `warning` or `note`.\n    NA   : If at least one v alidation resulted in `NA\n \n One or more FALSE or missing values values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> $types
#> $error
#> [1] 0
#> 
#> $details
#>    dataFileName columnName columnData mappingColumn type description comment
#> 1          TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 2          TRUE       TRUE       TRUE          TRUE TRUE        <NA>    <NA>
#> 3          TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 4          TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 5          TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 6          TRUE       TRUE       TRUE          <NA> TRUE        TRUE    <NA>
#> 7          TRUE       TRUE       TRUE          TRUE TRUE        <NA>    <NA>
#> 8          TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 9          TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 10         TRUE       <NA>       TRUE          TRUE TRUE        <NA>    <NA>
#> 11         TRUE       TRUE       TRUE          TRUE TRUE        <NA>    <NA>
#> 12         TRUE       TRUE       TRUE          <NA> TRUE        TRUE    <NA>
#> 13         TRUE       TRUE       TRUE          TRUE TRUE        <NA>    <NA>
#> 14         TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 15         TRUE       <NA>       TRUE          TRUE TRUE        <NA>    <NA>
#> 16         TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 17         TRUE       TRUE       TRUE          <NA> TRUE        TRUE    <NA>
#> 18         TRUE       TRUE       TRUE          TRUE TRUE        <NA>    <NA>
#> 19         TRUE       TRUE       TRUE          <NA> TRUE        <NA>    <NA>
#> 20         TRUE       TRUE       TRUE          TRUE TRUE        <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">conversion of values into specified type lossless possible - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered follows the type for the column, i.e. integer, characterd, .... The validation is done by verifying if the column can be losslessly converted from character to the columnb type specified. the value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell contains an error, i.e. can not be losslessly converted.\n    TRUE : If the cell can be losslessly converted and is OK.\n    NA   : empty cell\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $allowedValues
#> $error
#> [1] 0
#> 
#> $details
#>    dataFileName columnName columnData mappingColumn type description comment
#> 1          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 2          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 3          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 4          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 5          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 6          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 7          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 8          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 9          <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 10         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 11         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 12         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 13         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 14         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 15         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 16         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 17         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 18         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 19         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 20         <NA>       <NA>       TRUE          <NA> TRUE        <NA>    <NA>
#> 
#> $header
#> [1] "**<span style=\"color:#00FF00\">values in allowedValues - OK</span>**"
#> 
#> $description
#> [1] "Test if the metadata entered is ion the allowedValues list. The value NA is allowed in all column types, empty cells should be avoided."
#> 
#> $descriptionDetails
#> [1] "The details are a table of the same dimension as the input (green) area of the meatadata sheet. The following values are possible:\n \n    FALSE: If the cell value is not contained in the allowedValues list.\n    TRUE : If the cell value is contained in the allowedValues list.\n    NA   : empty cell or no allowed values specified\n \n One or more FALSE values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> $dataFilesExist
#> $error
#> [1] 3
#> 
#> $details
#>                    dataFileName  IsOK
#> 1 dissolved_oxygen_measures.csv FALSE
#> 2                     smell.csv FALSE
#> 3                abundances.csv FALSE
#> 
#> $header
#> [1] "**<span style=\"color:#FF0000\">`dataFile` exists in path - error</span>**"
#> 
#> $description
#> [1] "Test if all `dataFile` exist in the given `path`. The `error` can have the following values apart from `OK`:\n \n    error   : If not all `dataFile` exist in the given `path`\n \n"
#> 
#> $descriptionDetails
#> [1] "The details are a table with one row per unique `variable` The following values are possible for the column `isTRUE`:\n \n    TRUE : If `dataFile` exist in the given `path`\n    FALSE: If `dataFile` does not exist in the given `path`\n    NA   : empty cell\n \n One or more FALSE or missing values will result in an ERROR."
#> 
#> $comment
#> [1] ""
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"                
#> 
#> attr(,"class")
#> [1] "dmdScheme_validation" "list"

To create a report (html, pdf or docx) yo can use the report() function:

report( scheme_path_xlsx() )

which will open a html report in your browser.

Converting Metadata to xml

You can export the data to an an xml_document object as defined in the xml2 package to do further processing:

x <- read_excel( scheme_path_xlsx() )
xml <- as_xml( x, file = xmlFile)
xml
#> {xml_document}
#> <dmdScheme fileName="dmdScheme_0.9.9.xlsx" dmdSchemeName="dmdScheme" dmdSchemeVersion="0.9.9" propertyName="dmdScheme" output="metadata">
#> [1] <MdBibliometricList>\n  <MdBibliometric>\n    <uploadType>Dataset</upload ...
#> [2] <MdAuthorsList>\n  <MdAuthors authorID="1">\n    <authorID>1</authorID>\n ...
#> [3] <ExperimentList>\n  <Experiment>\n    <name>ASR-expt1</name>\n    <temper ...
#> [4] <GenusList>\n  <Genus speciesID="tt_1">\n    <speciesID>tt_1</speciesID>\ ...
#> [5] <TreatmentsList>\n  <Treatments treatmentID="Lid_treatment">\n    <treatm ...
#> [6] <MeasurementList>\n  <Measurement measurementID="oxygen concentration">\n ...
#> [7] <DataExtractionList>\n  <DataExtraction dataExtractionID="Mol_Analy_pipel ...
#> [8] <DataFileMetaDataList>\n  <DataFileMetaData>\n    <dataFileName>dissolved ...

You can save the xml to a file by using

xmlFile <-  tempfile(fileext = ".xml")

xml2::write_xml(x = xml, file = xmlFile)
#> NULL

or directly in one command

xmlFile <-  tempfile(fileext = ".xml")

write_xml( x = x, file = xmlFile )
#> [1] "/var/folders/50/wcr5bjwn75q595n6x82gxj280000gq/T//RtmpehUPzm/file64a17c77f2c8.xml"

Re-import from xml into R

x <- read_xml( xmlFile )
x
#>  dmdScheme - dmdSchemeSet
#>      MdBibliometric - dmdSchemeData
#> A       Names : uploadType | doi | publicationDate | title | description | version | language | keywords | additionalNotes | accessRights | accessRightsInfo | license
#> A       Units : 
#> A       Type  : character | character | character | character | character | character | character | character | character | character | character | character
#> 
#>           uploadType               doi publicationDate                   title
#> D       1    Dataset 110.1234/ffoo.bar      2020-05-26 Microbes under Pressure
#>                                                                                                                                                 description
#> D       1 The dataset was generated by putting microbes under pressure through temperatue and oxygen changes and their changes ion abundance were observed.
#>           version language
#> D       1    <NA>      eng
#>                                                         keywords
#> D       1 microbes, temperatur change, oxygen changes, abundance
#>           additionalNotes accessRights accessRightsInfo
#> D       1            <NA>         Open             <NA>
#>                                                  license
#> D       1 Creative Commons Attribution 4.0 International
#> 
#>      MdAuthors - dmdSchemeData
#> A       Names : authorID | familyName | givenName | affiliation | orcid | role
#> A       Units : NA | NA | NA | NA | NA | NA
#> A       Type  : character | character | character | character | character | character
#> 
#>           authorID familyName givenName                       affiliation orcid
#> D       1        1    Petchey   Owen L. University of Zurich, Switzerland  <NA>
#> D       2        2       Krug Rainer M. University of Zurich, Switzerland  <NA>
#>                                       role
#> D       1 Conceptualization, Investigation
#> D       2                    Data curation
#> 
#>      Experiment - dmdSchemeData
#> A       Names : name | temperature | light | humidity | incubator | container | microcosmVolume | mediaType | mediaConcentration | cultureConditions | comunityType | mediaAdditions | duration | comment
#> A       Units : NA | NA | NA | NA | NA | NA | ml | NA | g/l | NA | NA | NA | days | NA
#> A       Type  : character | character | character | character | character | character | numeric | character | numeric | character | character | character | integer | character
#> 
#>                name temperature        light humidity      incubator
#> D       1 ASR-expt1          20 semi-ambient  ambient not given here
#>                                    container microcosmVolume mediaType
#> D       1 Duran type bottle, red lids, 250ml             100       PPM
#>           mediaConcentration cultureConditions      comunityType
#> D       1               0.55             dirty initially unknown
#>                                                                   mediaAdditions
#> D       1 Wheat seeds added on specific dates, see file wheat_seed_additions.csv
#>           duration comment
#> D       1      100    <NA>
#> 
#>      Genus - dmdSchemeData
#> A       Names : speciesID | colour | density | functionalGroup | comment
#> A       Units : NA | NA | cells / ml | NA | NA
#> A       Type  : character | character | character | character | character
#> 
#>           speciesID  colour density functionalGroup
#> D       1      tt_1    ATCC       1     bacterivore
#> D       2   unknown unknown    <NA>         unknown
#>                                                            comment
#> D       1 http://www.lgcstandards-atcc.org/products/all/30007.aspx
#> D       2                                                     <NA>
#> 
#>      Treatments - dmdSchemeData
#> A       Names : treatmentID | treatmentLevelHeight | comment
#> A       Units : NA | NA | NA
#> A       Type  : character | character | character
#> 
#>             treatmentID treatmentLevelHeight comment
#> D       1 Lid_treatment                Loose    <NA>
#> D       2 Lid_treatment                Tight    <NA>
#> D       3     species_1        tt_1, unknown    <NA>
#> D       4     species_2              unknown    <NA>
#> D       5     species_3                 tt_1    <NA>
#> 
#>      Measurement - dmdSchemeData
#> A       Names : measurementID | variable | method | unit | object | noOfSamplesInTimeSeries | samplingVolume | dataExtractionID | measuredFrom | comment
#> A       Units : NA | NA | NA | NA | NA | NA | ml | NA | NA | NA
#> A       Type  : character | character | character | character | character | integer | numeric | character | character | character
#> 
#>                  measurementID  variable         method               unit
#> D       1 oxygen concentration        DO presens Optode                  %
#> D       2            abundance abundance      molecular              count
#> D       3                smell     smell           nose rotten eggs or not
#> D       4         sequenceData       DNA            NGS         Nucleotide
#>                 object noOfSamplesInTimeSeries samplingVolume
#> D       1    community                      50            0.0
#> D       2      species                       6            0.5
#> D       3    community                       6            0.0
#> D       4 DNA fragment                       6            0.0
#>              dataExtractionID measuredFrom comment
#> D       1                none          raw    <NA>
#> D       2 Mol_Analy_pipeline1 sequenceData    <NA>
#> D       3                none          raw    <NA>
#> D       4                none          raw    <NA>
#> 
#>      DataExtraction - dmdSchemeData
#> A       Names : dataExtractionID | method | parameter | value | comment
#> A       Units : NA | NA | NA | NA | NA
#> A       Type  : character | character | character | character | character
#> 
#>              dataExtractionID method parameter value
#> D       1 Mol_Analy_pipeline1   <NA>      <NA>  <NA>
#>                                   comment
#> D       1 See description in file xxx.yyy
#> 
#>      DataFileMetaData - dmdSchemeData
#> A       Names : dataFileName | columnName | columnData | mappingColumn | type | description | comment
#> A       Units : NA | NA | NA | NA | NA | NA | NA
#> A       Type  : character | character | character | character | character | character | character
#> 
#>                             dataFileName    columnName  columnData
#> D       1  dissolved_oxygen_measures.csv        Jar_ID          ID
#> D       2  dissolved_oxygen_measures.csv            DO Measurement
#> D       3  dissolved_oxygen_measures.csv        Unit_1       other
#> D       4  dissolved_oxygen_measures.csv          Mode       other
#> D       5  dissolved_oxygen_measures.csv      Location       other
#> D       6  dissolved_oxygen_measures.csv     Date_time       other
#> D       7  dissolved_oxygen_measures.csv Lid_treatment   Treatment
#> D       8  dissolved_oxygen_measures.csv      Jar_type       other
#> D       9  dissolved_oxygen_measures.csv        Jar_ID          ID
#> D       10                     smell.csv          <NA>     Species
#> D       11                     smell.csv         smell Measurement
#> D       12                     smell.csv          Date       other
#> D       13                     smell.csv Lid_treatment   Treatment
#> D       14                     smell.csv      Jar_type       other
#> D       15                abundances.csv          <NA>     Species
#> D       16                abundances.csv        Jar_ID          ID
#> D       17                abundances.csv     Date_time       other
#> D       18                abundances.csv Lid_treatment   Treatment
#> D       19                abundances.csv      Jar_type       other
#> D       20                abundances.csv  count_number Measurement
#>                   mappingColumn      type description comment
#> D       1                  <NA> character        <NA>    <NA>
#> D       2  oxygen concentration   numeric        <NA>    <NA>
#> D       3                  <NA> character        <NA>    <NA>
#> D       4                  <NA> character        <NA>    <NA>
#> D       5                  <NA> character        <NA>    <NA>
#> D       6                  <NA>  datetime      ymdhms    <NA>
#> D       7         Lid_treatment character        <NA>    <NA>
#> D       8                  <NA> character        <NA>    <NA>
#> D       9                  <NA> character        <NA>    <NA>
#> D       10            species_1 character        <NA>    <NA>
#> D       11                smell character        <NA>    <NA>
#> D       12                 <NA>  datetime      ymdhms    <NA>
#> D       13        Lid_treatment character        <NA>    <NA>
#> D       14                 <NA> character        <NA>    <NA>
#> D       15            species_3 character        <NA>    <NA>
#> D       16                 <NA> character        <NA>    <NA>
#> D       17                 <NA>  datetime      ymdhms    <NA>
#> D       18        Lid_treatment character        <NA>    <NA>
#> D       19                 <NA> character        <NA>    <NA>
#> D       20            abundance   numeric        <NA>    <NA>

Developing new dmdScheme derived scheme definitions

This package contains all the functionality to easily create a new metadata scheme. The resulting scheme package will contain all files needed so that it can be easily be distributed and used, and can be either be distributed directly or uploaded to an online repository, e.g. the default dmdScheme repository.

A new scheme definition only has to include the scheme definition and example data. It can also contain additional examples and an R script to install the accompanying R package, but these are not required.

Create and edit new definition

To create the scheme itself (as shown in Figure 5), it is recommended to begin with an existing scheme. It is possible to edit most aspects in the scheme definition xlsx file, including adding or deleting tabs (only the Experiment and DataFileProperties tabs are needed), adding or removing rows (Experiment tab) or columns (other tabs), changing the types, suggested values, allowed values and units. If the .xlsx file is locked, you can unlock it by using the password test.

It is important that the final scheme contains example data as - internal processing of the .xlsx file require example data and will fail if a scheme without example data is provided, and - it is much more user friendly if the user of a package not only sees the empty scheme, but also a filled in scheme with example data.

This .xlsx file can be edited following the limitations as specified below. After modifications have been done and the spreadsheet has been saved, the scheme package can be created by using

scheme_make( "TheFile.xlsx" )

which will create a file named NAME_VERSION.tar.gz in the directory path based on the scheme definition in TheFile.xlsx. NAME and VERSION are the scheme name and scheme version as specified in the cell (H:1) in sheet Experiment.

The following two commands will install the new scheme and use it:

scheme_install(file = "NAME_VERSION.tar.gz")
scheme_use("NAME", "VERSION")

To upload the scheme to the main repository on github, please either clone the repo and send a pull request or file an issue with the scheme package as an attachment.

Minimum requirements for metadata schemes derived from dmdScheme

There are a few minimum requirements for dmdScheme derived metadata schemes, so that all functions in the dmdScheme package will work.

These are:

  1. Tab name has to be identical to propertySet value (Cell A:2 in the tab Experiment, B:1 in other tabs)

  2. One tab named Experiment is required. This tab is different to the others as it has

    1. horizontal layout
    2. only one value per property allowed
    3. the top-right cell, i.e. the one above the values to be entered, contains the word DATA, name of the scheme and version of the scheme separated by a space, e.g. DATA emeScheme v0.9.5

  3. One tab named DataFileMetaData is required

    • one column named dataFileName

  4. Each tab except Experiment and DataFileMetaData needs an ID field
    Applies to all tabs except Experiment and `DataFileMetaData

    • The first column in the tab must be a column named ...ID.

  5. The spreadsheet can contain a tab DOCUMENTATION This tab can contain information for the user and will not be imported.

**Figure 5**: Create a new domain specific scheme based on dmdScheme.

Figure 5: Create a new domain specific scheme based on dmdScheme.

Developing accompanying R packages

The package dmdScheme package is only providing the base functionality for working with dmdSchemes. In many cases, the functionality needs to be extended to be able to work with other schemes. This can be done by creating an accompanying R package. It is a standard R package which should

It is not necessary to build an accompanying R package, but doing so will make it possible to adapt many aspects of the validation and export process to the needs of a specific domain.

The easiest to develop an accompanying R package is to - use scheme_use() to use the new scheme for which the accompanying package should be developed - use the convenience function make_new_package() to create a package skeleton which fulfils these requirements

Now, functions can be added. For an example see https://github.com/Exp-Micro-Ecol-Hub/emeScheme, in which the methods as_xml_list.emeSchemeSet.R and validate.emeSchemeSet_raw.R extend the generic functions as_xml_list() and validate() when the the scheme is an emeScheme.

Linking derived scheme to other metadata scheme

TODO

To illustrate ho a dmdScheme derived metadata scheme can be mapped (and exported) to another metadata scheme, we look at the example of the EML scheme as it is widely used in ecology and the structural functionality is included in the dmdScheme package.

The actual mapping and development of the function to do the actual export, as well as this document, is work in progress and details can (will) change in further releases of dmdScheme.

Approach and Mapping

???OP: Explain why the following is necessary.?? It is essential that only one data file is described and only one method is used in the data file. An emeScheme object containing more than one data file must be split into multiple metadata files with one data file per metadata file by using dmdScheme:::as_xml.dmdSchemeSet() to export and split the file - will be done automatically eventually.

At the moment, only a emeScheme -> dmdScheme EML is implemented. It is done by using the EML package in R.

The following section gives the mapping between emeScheme and EML.

Experiment

  • name ->
  • temperature ->
  • light ->
  • humidity ->
  • incubator ->
  • container ->
  • microcosmVolume ->
  • mediaType ->
  • mediaConcentration ->
  • cultureConditions ->
  • comunityType ->
  • mediaAdditions ->
  • duration ->
  • comment ->

Species

  • speciesID ->
  • name ->
  • strain ->
  • source ->
  • density ->
  • functionalGroup ->
  • comment ->

Treatment

  • treatmentID -> factors$attributeName
  • unit -> factors$definition
  • treatmentLevel ->
    • unit != "" -> different factor levels / codes, will be split into factors$code
  • comment -> factors$definition

Measurement

  • measurementID ->
  • variable ->
  • method ->
  • unit -> attributes$unit
  • object ->
  • noOfSamplesInTimeSeries ->
  • samplingVolume ->
  • dataExtractionID ->
  • measuredFrom ->
  • comment ->

DataExtraction

  • dataExtractionID
  • method ->
  • parameter ->
  • value ->
  • comment ->

DataFileMetaData

  • dataFileName ->
  • columnName -> attributes$attributeName
  • columnData -> defines meaning of mappingColumn (see below)
  • mappingColumn ->
    • columnData = “ID” ->
    • columnData = “Treatment” ->
    • columnData = “Measurement” ->
    • columnData = “Species” ->
    • columnData = “other” ->
  • type -> attributes$numberType
  • description ->
    • type = “datatime” -> attributes$formatString
    • else -> attributes$attributeDefinition
  • comment -> attributes$attributeDefinition

Customize the index template

The index template is a text file, in which tokens are replaced by values from the a dmdScheme when calling the function make_index( x = A_dmdScheme, template = "the/index/template/file.txt). Tokens are enclosed by%%...%% and must not contain any white spaces (tab, space, …).

The following tokens are allowed at the moment:

Tokens which are replaced with values from a dmdScheme

If the token starts with an !fun!, the R function fun is executed using the result. For example %%!unique!DataFileMetaData.dataFileName%% would only return a vector containing the unique dataFileName.

Special tokens