JLS-01#

The nlohmann/json library project CI executes on each pull request (opened, reopened, synchronized) the integration test suite, and failures in these runs are investigated by contributors.

Supported Requests:

TA-VALIDATION

Supporting Items:

None

References:

.github/workflows/parent-workflow.yml

name: Parent Workflow

on: 
  pull_request:
  push: 
    branches:
      - main
  schedule:
    - cron: '0 0 * * *' # Runs daily at midnight UTC

permissions:
  contents: read

jobs:
  labeler:
    permissions:
      contents: read
      pages: write
      pull-requests: write
    name: Run Labeler Workflow
    uses: ./.github/workflows/labeler.yml
    with:
      artifact_id: "labeler-${{ github.sha }}"

  check_amalgamation:
    name: Run Amalgamation Workflow
    if: ${{ github.event_name == 'pull_request' }} # only run check_amalgamation for PRs
    uses: ./.github/workflows/check_amalgamation.yml
    with:
      artifact_id: "check_amalgamation-${{ github.sha }}"
      
  test_trudag_extensions:
    name: Run Test Trudag Extensions Workflow
    uses: ./.github/workflows/test_trudag_extensions.yml
    with:
      artifact_id: "test_trudag_extensions-${{ github.sha }}"

  codeql:
    permissions:
      contents: read
      security-events: write
    name: Run Codeql analysis Workflow
    uses: ./.github/workflows/codeql-analysis.yml
    with:
      artifact_id: "codeql-${{ github.sha }}"

  ubuntu:
    name: Run Ubuntu Workflow
    permissions:
      contents: write
    needs: [codeql] # Error if CodeQL and Ubuntu triggered at the same time due to conflicting priorities
    uses: ./.github/workflows/ubuntu.yml
    with:
      artifact_id: "ubuntu-${{ github.sha }}"

  dependency_review:
    name: Run dependency_review Workflow
    if: ${{ github.event_name == 'pull_request' }}  # only run dependency_review for PRs
    uses: ./.github/workflows/dependency-review.yml
    with:
      artifact_id: "dependency_review-${{ github.sha }}"

  collect_artifacts_pr:
    name: "Collect Results & Deploy (PR)"
    if: github.event_name == 'pull_request'
    needs: [labeler, check_amalgamation, test_trudag_extensions, dependency_review, codeql, ubuntu]
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target: [labeler, check_amalgamation, test_trudag_extensions, dependency_review, codeql, ubuntu]

    steps:
      - name: Checkout code
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Check child workflow results 
        run: |
          echo "=== Checking Child Workflow Results ==="
          result="${{ needs[matrix.target].result }}"
          echo "${{ matrix.target }} workflow result: $result"

          if [[ "$result" != "success" ]]; then
            echo "❌ ${{ matrix.target }} workflow failed! Exiting..."
            exit 1
          fi
          echo "✅ Child workflows completed successfully!" 
        env:
          current_workflow: ${{ matrix.target }}

      - name: Download artifacts
        uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
        with:
          name: "${{ matrix.target }}-${{ github.sha }}"
          path: artifacts/

  collect_artifacts_non_pr:
    name: "Collect Results & Deploy (Non-PR)"
    if: github.event_name != 'pull_request'
    needs: [labeler, test_trudag_extensions, codeql, ubuntu]  # no check_amalgamation or dependency_review if non PR
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target: [labeler, test_trudag_extensions, codeql, ubuntu]

    steps:
      - name: Checkout code
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Check child workflow results 
        run: |
          echo "=== Checking Child Workflow Results ==="
          result="${{ needs[matrix.target].result }}"
          echo "${{ matrix.target }} workflow result: $result"

          if [[ "$result" != "success" ]]; then
            echo "❌ ${{ matrix.target }} workflow failed! Exiting..."
            exit 1
          fi
          echo "✅ Child workflows completed successfully!" 
        env:
          current_workflow: ${{ matrix.target }}

      - name: Download artifacts
        uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
        with:
          name: "${{ matrix.target }}-${{ github.sha }}"
          path: artifacts/

  test_publish_documentation:
    permissions:
      contents: write
      pages: write
      pull-requests: write
      id-token: write
      security-events: read
    name: Test publish_documentation Workflow
    if: github.event_name == 'pull_request'            # Whenever on Pull-request, test publication
    needs: [collect_artifacts_pr]
    uses: ./.github/workflows/test_publication.yml
    with:
      artifact_id: "ubuntu-${{ github.sha }}"

  publish_documentation:
    permissions:
      contents: write
      pages: write
      pull-requests: write
      id-token: write
      security-events: read
    name: Run publish_documentation Workflow
    if: github.event_name == 'push'            # Publish documentation should only run on push to main
    needs: [collect_artifacts_non_pr]
    uses: ./.github/workflows/publish_documentation.yml
    with:
      artifact_id: "ubuntu-${{ github.sha }}"

website: https://github.com/score-json/json/settings/branch_protection_rules/65227858

https://github.com/score-json/json/settings/branch_protection_rules/65227858 - branch protection rule for main branch specifying that failures of tests prevent merge.

Fallacies:

None

Graph:

No Historic Data Found

JLS-02#

Fuzz testing is used in the original nlohmann/json repository (https://github.com/nlohmann/json) to uncover edge cases and failure modes throughout development. (https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md)

Supported Requests:

TA-MISBEHAVIOURS

Supporting Items:

None

References:

website: https://introspector.oss-fuzz.com/project-profile?project=json

https://introspector.oss-fuzz.com/project-profile?project=json - most recent report for fuzzing introspection of nlohmann/json with historical plots

website: https://storage.googleapis.com/oss-fuzz-introspector/json/inspector-report/20250824/fuzz_report.html

https://storage.googleapis.com/oss-fuzz-introspector/json/inspector-report/20250824/fuzz_report.html - persistent storage of fuzz-testing-report for nlohmann/json version 3.12.0 on 24.08.2025

website: https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/.github/workflows/cifuzz.yml

https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/.github/workflows/cifuzz.yml - Configuration file for Fuzz-Testing pipeline in the original nlohmann/json repository

Fallacies:

None

Graph:

No Historic Data Found

JLS-03#

Automated tests are reviewed by a Subject Matter Expert to verify they test the properties they claim to.

Supported Requests:

TA-BEHAVIOURS

Supporting Items:

None

References:

None

Fallacies:

None

Graph:

No Historic Data Found

JLS-04#

External dependencies are checked for potential security vulnerabilities with each pull request to main. Merging is blocked until all warnings are resolved.

Supported Requests:

Supporting Items:

None

References:

.github/workflows/dependency-review.yml

# Dependency Review Action
#
# This Action will scan dependency manifest files that change as part of a Pull Request,
# surfacing known-vulnerable versions of the packages declared or updated in the PR.
# Once installed, if the workflow run is marked as required,
# PRs introducing known-vulnerable packages will be blocked from merging.
#
# Source repository: https://github.com/actions/dependency-review-action
name: 'Dependency Review'

on:  
  workflow_call:
    inputs:
      artifact_id:
        description: 'Unique identifier for artifacts'
        required: true
        type: string

permissions:
  contents: read

jobs:
  dependency-review:
    runs-on: ubuntu-latest
    steps:
      - name: Harden Runner
        uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
        with:
          egress-policy: audit

      - name: 'Checkout Repository'
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
      - name: 'Dependency Review'
        uses: actions/dependency-review-action@ce3cf9537a52e8119d91fd484ab5b8a807627bf8 # v4.6.0


      - name: Generate dependency_review artifact
        run: |
          echo "Generating Dependency Review artifact..."
          mkdir -p dependency_review
          echo "dependency review processed for ${{ inputs.artifact_id }}" &gt; dependency_review/dependency_review.txt

      - name: Upload dependency_review artifact
        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
        with:
          name: ${{ inputs.artifact_id }}
          path: dependency_review/

Description: The workflow scans PRs for dependency changes and vulnerabilities.

Fallacies:

None

Graph:

No Historic Data Found

JLS-05#

The OSS nlohmann/json is widely used, actively maintained and uses github issues to track bugs and misbehaviours.

Supported Requests:

TA-FIXES

Supporting Items:

None

References:

website: https://github.com/nlohmann/json/issues

https://github.com/nlohmann/json/issues - contains the collected github-issues for nlohmann/json

website: https://github.com/nlohmann/json/graphs/commit-activity

https://github.com/nlohmann/json/graphs/commit-activity - presents the commit activity of the past year

website: https://github.com/nlohmann/json/graphs/contributors

https://github.com/nlohmann/json/graphs/contributors - presents commits over time and per contributor

website: https://github.com/nlohmann/json/forks?include=active&page=1&period=&sort_by=last_updated

https://github.com/nlohmann/json/forks?include=active&page=1&period=&sort_by=last_updated - lists all forks of nlohmann/json by last updated

website: https://github.com/nlohmann/json/pulse

https://github.com/nlohmann/json/pulse - presents activity over the past week

Fallacies:

None

Graph:

No Historic Data Found

JLS-06#

Changes to the code (main branch) are applied only after code review and passing of all pipelines.

Supported Requests:

TA-UPDATES

Supporting Items:

None

References:

website: https://github.com/score-json/json/settings/branches

https://github.com/score-json/json/settings/branches - Branch protection policies

Fallacies:

None

Graph:

No Historic Data Found

JLS-07#

Main branch is protected, i.e. no direct commits are possible.

Supported Requests:

TA-UPDATES

Supporting Items:

None

References:

website: https://github.com/score-json/json/settings/branch_protection_rules/65227858

https://github.com/score-json/json/settings/branch_protection_rules/65227858 - branch protection rule for main branch specifying that failures of tests prevent merge.

Fallacies:

None

Graph:

No Historic Data Found

JLS-08#

Each statement is scored based on SME reviews or automatic validation functions. (TODO)

Supported Requests:

TA-CONFIDENCE

Supporting Items:

None

References:

None

Fallacies:

None

Graph:

No Historic Data Found

JLS-09#

Scores are reasonably, systematically and repeatably accumulated. (TODO)

Supported Requests:

TA-CONFIDENCE

Supporting Items:

None

References:

None

Fallacies:

None

Graph:

No Historic Data Found

JLS-10#

Every release includes source code, build instructions, tests and attestations. (TODO: Test result summary)

Supported Requests:

TA-ITERATIONS

Supporting Items:

None

References:

None

Fallacies:

None

Graph:

No Historic Data Found

JLS-11#

A score based on outstanding, fixed and mitigated faults is calculated based on github issues in nlohmann/json. (TODO)

Supported Requests:

TA-FIXES

Supporting Items:

None

References:

None

Fallacies:

None

Graph:

No Historic Data Found

JLS-12#

The S-Core change process management is followed.

Supported Requests:

TA-UPDATES

Supporting Items:

None

References:

website: https://eclipse-score.github.io/process_description/main/process_areas/change_management/index.html

https://eclipse-score.github.io/process_description/main/process_areas/change_management/index.html - Documentation of S-CORE change process management

Fallacies:

None

Graph:

No Historic Data Found

JLS-13#

The S-Core methodologies are followed.

Supported Requests:

TA-METHODOLOGIES

Supporting Items:

None

References:

website: https://eclipse-score.github.io/process_description/main/general_concepts/score_review_concept.html

https://eclipse-score.github.io/process_description/main/general_concepts/score_review_concept.html - Documentation of S-CORE methodologies

Fallacies:

None

Graph:

No Historic Data Found

JLS-14#

The builds are repeatable (i.e. different builds lead to the same SHA value). (TODO)

Supported Requests:

TA-RELEASES

Supporting Items:

None

References:

None

Fallacies:

None

Graph:

No Historic Data Found

JLS-16#

A list of tests, which is extracted from the test execution, is provided, along with a list of test environments. (TODO)

Supported Requests:

TA-TESTS

Supporting Items:

None

References:

List of all unit-tests

List of all unit-tests with test environments#

This list contains all unit-tests possibly running in this project. These tests are compiled from the source-code, where the individual unit-tests are arranged in TEST_CASEs containing possibly nested SECTIONs. To reflect the structure of the nested sections, nested lists are utilised, where the top-level list represents the list of TEST_CASEs.

It should be noted that not all unit-tests in a test-file are executed with every compiler-configuration.