The Problem: Zero CVEs on Every RPM Host

After wiring up security-exporter to the vuls-dictionary Helm chart, Debian-based hosts reported CVEs correctly. CentOS and RHEL hosts reported exactly zero. Every time.

No errors. No warnings. The vuls server returned HTTP 200 with a valid JSON response. The scannedCves map was just empty.

This is the worst kind of bug: everything appears to succeed, but the result is silently wrong.

Diagnosing the Silent Drop

The security-exporter daemon collects the list of installed packages, serialises them into a string, and POSTs them to the vuls server’s /vuls endpoint. The server passes that string through ParsePackages to build its internal package map, then looks up CVEs for each package.

Here is ParsePackages from our pkgscanner package, which mirrors the contract the vuls server expects:

// ParsePackages splits tab-separated "name\tstatus\tversion" lines into a Packages map.
// Only packages with status starting with "ii" (fully installed) are included.
func ParsePackages(raw string) Packages {
    pkgs := make(Packages)
    for _, line := range strings.Split(raw, "\n") {
        line = strings.TrimSpace(line)
        if line == "" {
            continue
        }
        const expectedFields = 3
        parts := strings.SplitN(line, "\t", expectedFields)
        if len(parts) < expectedFields {
            continue  // <-- silently skipped, no log, no error
        }
        name    := parts[0]
        status  := parts[1]
        version := parts[2]
        if name == "" || !strings.HasPrefix(status, "ii") {
            continue  // <-- also silently skipped
        }
        pkgs[name] = Package{Name: name, Version: version}
    }
    return pkgs
}

The contract is strict: three tab-separated fields, second field must start with "ii".

The original RPM collector was running a custom rpm -qa --queryformat that produced space-separated output with no status prefix at all. Every single line failed the field-count check and was silently dropped. The result was an empty Packages map, which caused vuls to return an empty scannedCves — no error, no log line, just no results.

Fix Part 1: The 6-Field RPM Query Format

The vuls project documents the exact rpm -qa --queryformat string it expects. The collector was updated to produce exactly that:

func (*rpmCollector) CollectPackages(ctx context.Context) (string, string, error) {
    cmd := exec.CommandContext(ctx, "rpm", "-qa", "--queryformat",
        "%{NAME} %{EPOCHNUM} %{VERSION} %{RELEASE} %{ARCH} %{SOURCERPM}\n")
    out, err := cmd.Output()
    if err != nil {
        return "", "", err
    }
    pkgs, srcPkgs, err := parseRpmOutput(string(out))
    if err != nil {
        return "", "", err
    }
    return pkgs, srcPkgs, nil
}

parseRpmOutput then reformats each 6-field line into the two formats that ParsePackages and ParseSrcPackages expect:

const rpmFieldCount = 6 // NAME EPOCHNUM VERSION RELEASE ARCH SOURCERPM

// pkgs line:    name\tii\tepoch:version-release
pkgLines = append(pkgLines,
    fmt.Sprintf("%s\tii\t%s:%s-%s", name, epoch, version, release))

// srcPkgs line: srcName\tsrcVersion\tbinaryName\tii
if srcName, srcVer, ok := parseSourceRPM(sourceRPM); ok {
    srcLines = append(srcLines,
        fmt.Sprintf("%s\t%s\t%s\tii", srcName, srcVer, name))
}

The status field is hardcoded to "ii" — RPM has no equivalent of dpkg’s per-package install status in rpm -qa output; if the package appears, it is installed. The version string is assembled as epoch:version-release, the standard RPM EVR format (e.g. 0:5.1.8-9.el9).

Fix Part 2: Source Package Extraction with the NVR Algorithm

Sending Packages alone was enough to get some CVE results back, but vuls2 uses source packages (SrcPackages) for more accurate detection on RPM-based distros. The advisory databases for RHEL and CentOS track many CVEs against source package names, not binary package names. Without SrcPackages, vuls silently misses those CVEs.

The SOURCERPM field from rpm -qa looks like:

bash-5.1.8-9.el9.src.rpm
openssl-3.0.7-27.el9.src.rpm
perl-IO-Socket-SSL-2.074-2.el9.src.rpm
gpg-pubkey-(none)

The challenge: package names can contain hyphens. perl-IO-Socket-SSL-2.074-2.el9.src.rpm has four hyphens before the version segment begins. A naive strings.Split(s, "-") and taking the first element gives the wrong answer.

The correct approach is the last-two-hyphens algorithm. In NVR (Name-Version-Release) format, release is always the last hyphen-delimited segment and version is always the second-to-last. The name is everything before those two:

// parseSourceRPM extracts name and version from a SOURCERPM field like
// "bash-5.1.8-9.el9.src.rpm". Returns false for "(none)" or empty.
func parseSourceRPM(s string) (name, version string, ok bool) {
    if s == "" || s == "(none)" {
        return "", "", false
    }

    s = strings.TrimSuffix(s, ".src.rpm")

    // NVR: name can contain hyphens, so find the last two hyphens.
    releaseIdx := strings.LastIndex(s, "-")
    if releaseIdx <= 0 {
        return "", "", false
    }
    nameVersion := s[:releaseIdx]
    release     := s[releaseIdx+1:]

    versionIdx := strings.LastIndex(nameVersion, "-")
    if versionIdx <= 0 {
        return "", "", false
    }
    name = nameVersion[:versionIdx]
    ver  := nameVersion[versionIdx+1:]

    return name, ver + "-" + release, true
}

For bash-5.1.8-9.el9.src.rpm: name="bash", version="5.1.8-9.el9".

For perl-IO-Socket-SSL-2.074-2.el9.src.rpm: name="perl-IO-Socket-SSL", version="2.074-2.el9". Correct, despite the four hyphens in the name.

Fix Part 3: Filtering Source Packages Before Sending

One more edge case. The vuls server validates that every binary name listed under a SrcPackages entry actually exists in the Packages map. If any binary name is absent, the server errors out for that source entry.

On RPM systems, SOURCERPM=(none) is already handled by parseSourceRPM returning false. But there are other situations where a derived binary name does not appear in Packages — for instance, debuginfo packages or sub-packages not present in the specific environment being scanned.

A filterSrcPackages pass removes binary names absent from Packages, then drops any source package entry that ends up with no valid binaries:

func filterSrcPackages(srcPkgs SrcPackages, packages Packages) SrcPackages {
    for srcName, sp := range srcPkgs {
        var filtered []string
        for _, bn := range sp.BinaryNames {
            if _, ok := packages[bn]; ok {
                filtered = append(filtered, bn)
            }
        }
        if len(filtered) == 0 {
            delete(srcPkgs, srcName)
        } else {
            sp.BinaryNames = filtered
            srcPkgs[srcName] = sp
        }
    }
    return srcPkgs
}

The Helm Chart: Removing the Legacy CVE Pipeline

Separately, the vuls-dictionary Helm chart was carrying a multi-component CVE ingestion pipeline inherited from an earlier iteration of the setup:

  • go-cve-dictionary fetched NVD and other CVE sources into PostgreSQL on a schedule
  • CNPG (CloudNativePG) managed the PostgreSQL cluster
  • dict-server served CVE data over a gRPC interface for the vuls server to query at scan time

This was a significant operational burden. PostgreSQL required persistent storage and CNPG cluster configuration. The fetch jobs had to run reliably on schedule; any failure produced stale or missing CVE data, often without any visible alert.

With vuls2, this entire stack is replaced by a single pre-built SQLite database: vuls-nightly-db. The database (~7 GB uncompressed) contains CVE data, advisory data, and detection logic for all supported distributions. It is published as an OCI image and updated nightly by the vuls project.

The chart now works like this:

Init containers (run once at pod start, skip if db >5 GB already on PVC):
  1. fetch-vuls2-db      -- pulls the OCI image via oras
  2. decompress-vuls2-db -- decompresses with zstd into /vuls/vuls.db

Vuls server:
  - Starts on port 5515
  - Reads /vuls/vuls.db at startup
  - No outbound network access needed at scan time

The resulting values.yaml is much simpler:

vuls2:
  enabled: true
  image:
    repository: ghcr.io/vulsio/vuls-nightly-db
    tag: "0"

vulsServer:
  enabled: true
  image:
    repository: vuls/vuls
    tag: "v0.38.6"
  port: 5515
  resultsStorage:
    size: 15Gi

No PostgreSQL. No CNPG. No dict-server. No fetch jobs to babysit.

Lessons

Silent data drops are the worst class of bug. ParsePackages skips malformed lines without logging anything. From the caller’s perspective the function succeeded — it returned a valid, non-nil map. The map was just empty. There was no error to catch, no warning to grep for. Diagnosing it required reading the vuls source code to understand the exact format contract, which is not documented outside the code itself.

The actionable rule: when integrating with an upstream tool’s parsing function, verify your output format against the parser’s source, not just documentation. A one-field format mismatch can produce a completely silent failure.

Always use the upstream tool’s recommended query format. The vuls project documents the rpm -qa --queryformat string it expects. Using a custom format that “looks similar” is not enough — field count, separator character, and field ordering all matter exactly.

Source packages are load-bearing on RPM-based distros. On Debian, SrcPackages is an enhancement that improves detection precision. On RHEL and CentOS, many CVEs in the advisory databases are indexed against source package names only. Omitting SrcPackages means vuls silently misses a meaningful fraction of its results.

Replacing a complex stateful pipeline with a static artefact is almost always worth it. PostgreSQL + go-cve-dictionary + dict-server was three components, each with its own failure modes and operational requirements. A nightly SQLite OCI image is one component pulled by an init container. The data freshness model is identical (both update on a schedule), but the failure surface is far smaller and operational overhead drops to nearly zero.