klibs-io

Project Url: JetBrains/klibs-io
Introduction: Search Kotlin Multiplatform projects and packages
More: Author   ReportBugs   OfficialWebsite   
Tags:

A service to search and discover Kotlin Multiplatform libraries. It aggregates data from Maven Central and GitHub, enriched with AI-powered metadata generation.

Note: The frontend code will be added to this repository soon.

Build & Run

Prerequisites

  • JDK 21+
  • Docker (for local PostgreSQL via docker-compose and LocalStack for S3)

Build & test

./gradlew build          # build + tests
./gradlew build -x test  # build without tests
./gradlew test           # tests only

Boot Jar

./gradlew bootJar

Output: app/build/libs/app.jar

Run locally

From CLI:

./gradlew :app:bootRun

Or run the main function from Application in IntelliJ — both use the local profile.

Configuration

Spring profiles are used to run the app in different environments. Profile-specific files are in app/src/main/resources.

Indexing of packages can be enabled/disabled via klibs.indexing. Fine-grained indexing sources (Central Sonatype, Google Maven) and executor settings are under klibs.indexing-configuration.

Files on disk

  • GitHub API request cache (managed by OkHttp). Property: klibs.integration.github.cache.request-cache-path.
  • README files (Markdown + HTML) stored in S3 with local disk cache. Properties: klibs.readme.cache-dir, klibs.readme.s3.*.

Endpoints

  • Swagger UI: /api-docs/swagger-ui.html
  • Actuator: /actuator/health, /actuator/info

Troubleshooting

See troubleshooting.md.

Modules

The project follows a "module by feature" approach:

  • app — main Spring Boot module. Configurations, scheduled jobs, glue for all other modules. Runnable.
  • core/* — domain modules (package, project, scm-owner, scm-repository, readme, search, storage).
  • integrations/* — third-party integrations (ai, github, maven).

Architecture

Indexing logic

The general flow:

  1. Check for new artifacts (published since the last check) using Maven Central's API.
  2. If new artifacts are available, add them to the processing queue (table package_index_request).
  3. Process the queue in a separate thread, one by one. If indexing of a package fails, increment its failed_attempts. Try to process each package up to N times. Projects and SCM owner/info are created in the process of indexing packages.

AI descriptions are generated by a separate scheduled task because the rate limits of OpenAI are much lower than of GitHub and Maven Central, so it's significantly slower.

Information taken from GitHub (repository/owner) is updated by a separate scheduled task too, based on github_repo.updated_at.

PostgreSQL's Full Text Search is used for FTS. Relevant data is aggregated in two materialized views — project_index and package_index — which are updated periodically and used for search queries. Known tech debt: at some point this might need to be replaced with Solr / ElasticSearch. All search-related logic is contained in the search module (ProjectSearchRepository, PackageSearchRepository), so the migration surface is limited.

Development

Development workflow: workflow.md.

How to update JVM version

There are 3 places to update:

  1. Build logic module toolchain version: build.gradle.kts
  2. Toolchain version in base jvm convention plugin: klibs.kotlin-jvm.gradle.kts
  3. Gradle daemon jvm version. Update jvm version in task updateDaemonJvm: build.gradle.kts and run updateDaemonJvm task:
./gradlew updateDaemonJvm

Gradle Build Scans

Gradle Build Scans can provide insights into an klibs.io backend Build. JetBrains runs a Gradle Develocity server that can be used to automatically upload reports.

To automatically opt in add the following to $GRADLE_USER_HOME/gradle.properties.

io.klibs.build.scan.enabled=true
# optionally provide a username that will be attached to each report
io.klibs.build.scan.username=John Wick

Also, you need to create an access key:

./gradlew provisionDevelocityAccessKey

A Build Scan may contain identifiable information. See the Terms of Use https://gradle.com/legal/terms-of-use/.

Apps
About Me
GitHub: Trinea
Facebook: Dev Tools