scipamato Scientific Paper Management Tool @codeKK AndroidOpen Source Website

scipamato

Introduction: Scientific Paper Management Tool

Tags:

image:https://img.shields.io/github/license/ursjoss/scipamato.svg[LICENSE, link=https://github.com/ursjoss/scipamato/blob/main/LICENSE.adoc] image:https://github.com/ursjoss/scipamato/workflows/Check/badge.svg?branch=main[Build Status, link=https://github.com/ursjoss/scipamato/actions] image:https://sonarcloud.io/api/project_badges/measure?project=ursjoss_scipamato&metric=alert_status[Quality Gate, link=https://sonarcloud.io/dashboard?id=ursjoss_scipamato] image:https://sonarcloud.io/api/project_badges/measure?project=ursjoss_scipamato&metric=coverage[Coverage, link=https://sonarcloud.io/dashboard?id=ursjoss_scipamato] image:https://github.com/ursjoss/scipamato/actions/workflows/codeql.yml/badge.svg?branch=main&event=push[CodeQL, link=https://github.com/ursjoss/scipamato/actions/workflows/codeql.yml]

[[scipamato]] == SciPaMaTo

SciPaMaTo (Scientific Paper Management Tool) is a database centric application that was developed for and with the LUDOK team of the Swiss Tropical and Public Health Institute (https://www.swisstph.ch/[Swiss TPH]) located in Basel, Switzerland.

https://www.swisstph.ch/en/projects/ludok/[LUDOK] (in German: "Dokumentationsstelle Luftverschmutzung und Gesundheit") is a documentation office that has been collecting and cataloguing scientific literature regarding the effects of air pollution on human health for over 30 years. This is a service LUDOK provides on behalf of the Swiss Federal Office for the Environment (https://www.bafu.admin.ch/bafu/en/home.html[FOEN]).

SciPaMaTo was developed under the BSD-3 license to replace the former Filemaker Database that has been serving its purpose for many years.

SciPaMaTo consists of two main components:

SciPaMaTo-Core is used by the LUDOK team internally. The team, upon scanning new literature from various sources, adds the relevant items of information about relevant studies into the application, supplemented with keywords, codes, PDFs and with a short summary in German. The tools assists the process of selecting relevant studies for the bi-monthly newsletter addressing newest findings regarding air pollution and health.
SciPaMaTo-Public is part of the public web-page of SwissTPH and provides a public https://www.swisstph.ch/en/projects/ludok/datenbanksuche/[database search] facility on the LUDOK database and displays a German summary. In addition, it also provides an overview on the studies that were selected for the various newsletters (the https://www.swisstph.ch/en/projects/ludok/neue-studien/[New Study page]).

SciPaMaTo is a Spring Boot web-application using Apache Wicket as front-end and stores the data in PostgreSQL-Databases. It offers simplified ways of importing relevant items of information from Pubmed. SciPaMaTo offers quite extensive search facilities that allow scrutinizing the database. Parts of the data managed within ScipMaTo-Core can be synchronized into SciPaMaTo-Public, which is accessible for the interested public over the internet.

=== C4-Diagrams

==== Context Diagram

flowchart TD
  PM["PubMed<br/>[Software System]<br/><br/>National Library of Medicine"]
  click PM href "https://pubmed.ncbi.nlm.nih.gov" _blank
  PublicViewer["Public Viewer<br/>[Person]<br/><br/>Anonymous Reader<br/>on the Internet"]
  subgraph dmz[DMZ]
    SP["SciPaMaTo Public<br/>[Software System]<br/><br/>Public System,<br/>reachable anonymously<br/>from public internet"]
    click SP href "https://www.swisstph.ch/en/projects/ludok/datenbanksuche" _blank
    SPDB("fas:fa-database Public Database<br/>[System DB]")
  end
  subgraph internal[Internal Network]
    Editor["Editor<br/>[Person]<br/><br/>Maintaining<br/>data in Core"]
    Viewer["Read only user<br/>[Person]<br/><br/>Read only viewer<br/>in SciPaMaTo-Core"]
    Admin["Admin user<br/>[Person]<br/><br/>User maintaining<br/>user accounts"]
    SC["SciPaMaTo Core<br/>[Software System]<br/><br/>Core System,<br/>only reachable inside<br/>the private network"]
    SCDB("fas:fa-database Core Database<br/>[System DB]")
  end
  Editor<==>|"Enters and<br/>maintains data"|SC
  Viewer<-->|"Views Data"|SC
  Admin<-->|"Maintains user<br/>accounts in"|SC
  SC<--->|"imports from"|PM
  SC<-.->|"read/write"|SCDB
  SC-.->|"synchronizes"|SPDB
  SP-.-|"read only"|SPDB

  PublicViewer-->|"searches database,<br/>views newsletters"|SP

  classDef focusSystem fill:#1168bd,stroke:#0b4884,color:#ffffff
  classDef supportingSystem fill:#666,stroke:#0b4884,color:#ffffff
  classDef person fill:#08427b,stroke:#052e56,color:#ffffff
  classDef boundary fill:none,stroke:#CCC,stroke-width:2px,stroke-dasharray: 5 5

  class Editor,Viewer,Admin,PublicViewer person
  class SC,SP focusSystem
  class PM supportingSystem
  class dmz,internal boundary

==== Container Diagram SciPaMaTo-Core

flowchart TD
  PM["PubMed<br/>[Software System]<br/><br/>National Library of Medicine"]
  subgraph dmz[DMZ]
    subgraph dbserver2[DB Server]
      SPDB("fas:fa-database Public Database<br/>[PostgreSQL]")
    end
  end
  subgraph internal[Internal Network]
    Editor["Editor<br/>[Person]<br/><br/>Maintaining<br/>data in Core"]
    Viewer["Read only user<br/>[Person]<br/><br/>Read only viewer<br/>in SciPaMaTo-Core"]
    Admin["Admin user<br/>[Person]<br/><br/>User maintaining<br/>user accounts"]
    subgraph webserver[Web Server]
      RP["Reverse Proxy<br/>[Apache HTTPD]"]
      SC["Web Application<br/>[Spring Boot Application]"]
    end
    subgraph dbserver1[DB Server]
      SCDB("fas:fa-database Core Database<br/>[PostgreSQL]")
    end

    Editor<==>|"enters and<br/>maintains data<br/>[HTTPS]"|RP
    Editor<-->|"triggers<br/>sync to public<br/>[HTTPS]"|RP
    Editor<-->|"triggers<br/>import from PubMed<br/>[HTTPS]"|RP
    Viewer<-->|"Views Data<br/>[HTTPS]"|RP
    Admin<-->|"Maintains user<br/>accounts in<br/>[HTTPS]"|RP
  end

  RP<-->|"relays<br/>[HTTP]"|SC
  SC-.-|"reads/writes<br/>[JDBC]"|SCDB
  SC-.->|"synchronizes<br/>[JDBC]"|SPDB
  SC<--->|"imports<br/>from<br/>[HTTPS]"|PM

  classDef container fill:#1168bd,stroke:#0b4884,color:#ffffff
  classDef supportingSystem fill:#666,stroke:#0b4884,color:#ffffff
  classDef person fill:#08427b,stroke:#052e56,color:#ffffff

  class Editor,Viewer,Admin person
  class SC,RP,SCDB container
  class PM,SPDB supportingSystem

  classDef boundary fill:none,stroke:#CCC,stroke-width:2px,stroke-dasharray: 5 5
  class dmz,internal boundary

  classDef server fill:none,stroke:#CCC,stroke-width:2px,stroke-dasharray: 5 5
  class dbserver1,dbserver2,webserver server

==== Container Diagram SciPaMaTo-Public

flowchart TD
  Searcher["Database Searcher<br/>[Person]<br/><br/>Anonymous Reader<br/>on the Internet"]
  Viewer["Newsletter Viewer<br/>[Person]<br/><br/>Anonymous Reader<br/>on the Internet"]
  subgraph dmz[DMZ]
    subgraph webserverp[Web Server]
      RP["Reverse Proxy<br/>[Apache HTTPD]"]
      SP["Web Application<br/>[Spring Boot Application]"]
      click SP href "https://www.swisstph.ch/en/projects/ludok/datenbanksuche" _blank
    end
    subgraph dbserverp[DB Server]
      SPDB("fas:fa-database Public Database<br/>[PostgreSQL]")
    end
  end

  subgraph internal[Internal Network]
    subgraph webserverc[Web Server]
      SC["Web Application<br/>[Spring Boot Application]"]
    end
    subgraph dbserverc[DB Server]
      SCDB("fas:fa-database Core Database<br/>[PostgreSQL]")
    end
    SC<-.->|"[JDBC]"|SCDB
  end

  Searcher<-->|"searches database<br/>[HTTPS]"|RP
  Viewer<-->|"views newsletters<br/>[HTTPS]"|RP

  RP<-->|"relays<br/>[HTTP]"|SP
  SP-.-|"read only<br/>[JDBC]"|SPDB
  SC-.->|"synchronizes<br/>[JDBC]"|SPDB

  classDef container fill:#1168bd,stroke:#0b4884,color:#ffffff
  classDef supportingSystem fill:#666,stroke:#0b4884,color:#ffffff
  classDef person fill:#08427b,stroke:#052e56,color:#ffffff

  class Searcher,Viewer,CoreUser person
  class SP,RP,SPDB container
  class SC,SCDB supportingSystem

  classDef boundary fill:none,stroke:#CCC,stroke-width:2px,stroke-dasharray: 5 5
  class dmz,internal boundary

  classDef server fill:none,stroke:#CCC,stroke-width:2px,stroke-dasharray: 5 5
  class dbserverp,dbserverc,webserverp,webserverc server

[[technology-stack]] == Technology Stack

Kotlin & Java 21
https://projects.spring.io/spring-boot/[Spring Boot] with https://projects.spring.io/spring-security/[Spring Security]
https://projectlombok.org/[Lombok]
Presentation Layer https://wicket.apache.org/[Apache Wicket] with https://getbootstrap.com/[Bootstrap] https://github.com/MarcGiffing/wicket-spring-boot[wicket-spring-boot-starter] https://github.com/l0rdn1kk0n/wicket-bootstrap[wicket-bootstrap] https://community.jaspersoft.com/[JasperReport] for PDF generation https://github.com/OpenFeign/feign[Feign] as HTTP client to PubMed jaxb for XML parsing
Persistence Layer https://www.jooq.org/[jOOQ] https://github.com/brettwooldridge/HikariCP[HikariCP] https://www.postgresql.org/[PostgreSQL] https://flywaydb.org/[Flyway] for database migrations https://github.com/ayedo/jooq-modelator[jOOQ modelator] gradle plugin (running https://www.jooq.org/doc/3.0/manual/code-generation/[jOOQ code generation] against a flyway-migrated dockerized db) https://www.testcontainers.org/[Testcontainers] (running integration tests against the dockerized db) ** https://projects.spring.io/spring-batch/[Spring Batch] for pushing data from the core to the public database
Caching ** JCache with https://www.ehcache.org/[Ehcache3]

[[architecture-documentation]] == Architecture Documentation

I started using https://structure101.com/[Structure101] to assess and track the architecture of SciPaMaTo. Many thanks to the friendly people of Structure101 to provide a free license for SciPaMaTo as an open source project!

image:https://structure101.com/images/s101_170.png[Structure101]

[[user-documentation]] == User Documentation

[[user-docu-scipamato-core]] === SciPaMaTo-Core Documentation