Overview

Output Format

Heimdall uses JSON as its default output format (unless defined otherwise by a command), but offers other formats.

Use the --output (or -o) parameter to format CLI output. The argument values and types of output are:

  • json - JSON string. This setting is the default for non-terminals.
  • jsonc - Colorized JSON. This setting is the default for interactive terminals.
  • table - Presents the information in a “human-friendly” format that is much easier to read than the others, but not as programmatically useful.
  • text - Plain text output without special formatting.
  • tsv - Tab-separated key-value pairs (useful for grep, sed, or awk).
  • yaml - YAML, a machine-readable alternative to JSON.
  • yamlc - Colorized YAML.

Filter Output

The Heimdall CLI has two built-in JSON-based client-side filtering capabilities. The optional parameters are a powerful tool you can use to customize the content and style of your output. They take the JSON representation of the internal command-specific data structures and filter the results before displaying them.

  • The --query parameter uses JMESPath syntax to create expressions for filtering your output. To learn JMESPath syntax, see Tutorial on the JMESPath website.
  • The --jq parameter uses JQ syntax for general purpose JSON processing. The JQ manual provides a comprehensive overview.

The following code listing show examples of what the --query and --jq parameter can produce.

$ env | grep -E "^USER=" | heimdall format --output json

{"USER":"me"}

$ env | heimdall format --output text --query "USER"

me

$ env | heimdall format --output text --jq ".USER"

me

Heimdall Scripts

A Heimdall Script is a declarative definition of multiple Heimdall commands grouped into Tasks. It can be executed by using the run command.

version: '3'

env:
  HEIMDALL_FILE: '{{.USER_WORKING_DIR}}/tmp/*'

tasks:
  default:
    interpreter: expr
    cmds:
      - task: Load JaCoCo report
        cmd: heimdall java jacoco -f reports/jacoco/test/jacocoTestReport.csv -s >tmp/jacoco.json
        interpreter: sh
      - task: Line coverage (70%)
        cmd: line_covered / (line_covered + line_missed) > 0.7

A comprehensive usage guide can be found at Taskfile Usage.

Examples

$ heimdall run -t ~/heimdall/docs/examples/java.yaml default

task: [gradle] Check whether the executable flag is set.
task: [gradle] Check whether Gradle 6 or newer is used.
task: [gradle] Check whether outdated GWT plugins are used.
task: Task "jacoco-single-report" is up to date
task: [jacoco] Check JaCoCo code coverage threshold.
task: [jacoco] line_covered / (line_covered + line_missed) >= 0.60
task: Task "log4j-config" is up to date
task: [web.xml-file] Check that 'WebContent/WEB-INF/web.xml' uses Servlet spec 3.1 or later.
task: [web.xml-file] Check that 'WebContent/WEB-INF/web.xml' does not contain synchronous servlet filters for asynchronous servlets.
javamelody
12:34:56 ERR error="task: Failed to run task \"default\": exit status 1"

Commands

Command Structure

The CLI uses a multipart structure on the command line that must be specified in this order:

  1. The base call to the heimdall program.
  2. The top-level command, which typically corresponds to a system or technology.
  3. The subcommand that specifies which operation to perform.
  4. General CLI options or parameters required by the operation. You can specify these in any order. If an exclusive parameter is specified multiple times, only the last value applies.
$ heimdall <command> <subcommand> [options and parameters]

Parameters can take various options. What is supported is dependent upon the command and subcommand you specify.

The examples in this guide are formatted using the following conventions:

  • Prompt – The command prompt uses the Linux prompt and is displayed as ($ ). Do not include the prompt when you type commands.
  • Directory – Paths to files or directories use the Unix file separator / e.g., ./app/Dockerfile.
  • Output – Output returned by Heimdall is shown under user input, and is formatted as monospace output.

Docker

The docker command can process Dockerfile and .dockerignore files to list matching files or print instructions.

There are three subcommands:

  • context: Parses a .dockerignore file and lists matching files.
  • file: Parses a Dockerfile and prints its instructions.
  • ignore: Parses a .dockerignore file and prints its patterns.

Examples

List all Docker Instructions

To parse a Dockerfile and print its instructions in a structured format, the docker file subcommand can be used.

$ heimdall docker file -f app/Dockerfile --output yamlc

- cmd: FROM
  line: FROM scratch
  start_line: 1
  end_line: 1
  value:
    - scratch
- cmd: COPY
  line: COPY hello /
  start_line: 2
  end_line: 2
  value:
    - hello
    - /
- cmd: CMD
  json: true
  line: CMD ["/hello"]
  start_line: 3
  end_line: 3
  value:
    - /hello

List Base Images

The built-in filter capabilities enable to query for certain values, like the base images used to build an image:

$ heimdall docker file --output text --query "[?cmd=='FROM'] | [-1:].value"
$ # or
$ heimdall docker file --output text --jq '[.[] | select(.cmd=="FROM")][-1].value'

buildpack-deps:bullseye-scm

Assert non-root User

Similarly, the following command checks whether the Dockerfile contains any USER instructions, and whether the final user is not the root user.

$ heimdall docker file --output text \
  --query "[?cmd=='USER'] | [-1:].value[0] | [0]!='root'"
$ # or
$ heimdall docker file --output text \
  --jq '[.[] | select(.cmd=="USER")] | last | .value[0] != "root"'

true

List Docker Build Context

The docker context subcommand parses the .dockerignore file and recursively lists files, which will be sent to the build context. The root directory is the directory containing the ignore-file. If the .dockerignore file does not exist, it is treated as empty and all files are listed.

$ heimdall docker context -f ./bin/.dockerignore --output text

heimdall

List ignore-file Patterns

To parse a .dockerignore file and list its patterns in any structured format, use the ignore subcommand.

$ heimdall docker ignore -f testdata/.dockerignore --output json

[".dockerignore",".git","Dockerfile"]

Echo

The echo command applies coloring, borders, and spacing to text.

Examples

$ heimdall echo --foreground '#FFFFFF' --border line --padding "1 4" Error

┌─────────────┐
│             │
│    Error    │
│             │
└─────────────┘

Eval

The eval command provides a way to evaluate complex expressions on structured input data. It supports different evaluation engines:

  • expr: Expr is a safe, fast, and intuitive expression evaluator optimized for the Go language.
  • javascript: JavaScript interpreter, which implements ECMAScript 5.1.
  • template: Standard Go Template engine.

Expr

Heimdall supports the same expression language, which is also used by Argo Rollouts and Argo Workflows for Kubernetes. The official Language Definition provides a good overview.

Custom Functions

Heimdall registers more than 120 functions, which can be used by any evaluation engine, which supports custom functions.

Examples

Check that URL matches Regular Expression

The following command checks that the Gradle Wrapper defines Gradle 7 or newer:

$ heimdall eval -e 'base(distributionUrl) matches "gradle-[7-9][.]"' \
  gradle/wrapper/gradle-wrapper.properties

true

Process Multiple Files

Heimdall accepts any number of input files. Non-existing files are treated as fatal errors, unless the --ignore-missing flag is set. This enables Heimdall to evaluate expression on a set of input files. If a variable is defined multiple times among any input files, the last definition takes precedence.

The following command reads multiple files in properties format and prints all distinct variables in lexical order.

$ heimdall eval --ignore-missing -e 'join("\n", sortAlpha(keys(_)))' \
  ${GRADLE_USER_HOME:-~/.gradle}/gradle.properties gradle.properties

org.gradle.cache.cleanup
org.gradle.caching
org.gradle.configureondemand
org.gradle.daemon.idletimeout
org.gradle.parallel
org.gradle.vfs.watch

Override File Format

The eval command determines the file format solely based on the file extension. For files, which have an unknown file extension, or when reading from standard input, the type needs to be defined. This is done with the following syntax:

$ heimdall eval -e <expr> <file1>:<prefix1>:<type1> <file2>:<prefix2>:<type2> ...

Both, prefix and type are optional, i.e., file is equivalent to file::. For example, the argument app/Dockerfile::properties instructs Heimdall to read the file as a .properties file. As a consequence, it defines variables, with the Docker instructions being the keys and the arguments being the values.

The following command reads two Dockerfiles and assigns Docker instructions to variables prefixed with a and w, respectively. It uses the default evaluation engine Expr to check whether both Dockerfile use the same base image, and the web container exposes port 8080.

$ heimdall eval -e "a.FROM == w.FROM" -e "int(w.EXPOSE) == 8080" \
  testdata/Dockerfile:a:properties testdata/Dockerfile:w:properties

true
true

Read from Standard Input

$ heimdall java jacoco --summary jacoco.csv |
  heimdall eval -E javascript \
  -e 'line_covered / (line_covered + line_missed)' -- -::json

true

First, Heimdall reads a JaCoCo code coverage report in CSV format and produces a summary in JSON format. The output is fed into the JavaScript interpreter to calculate coverage ratio. Note the -::json, which means: read standard input (-), use no variable prefix, and treat the input as JSON.

Example

$ heimdall example java --no-pager

version: '3'

set: [ errexit, nounset, pipefail ]
env:
  EVIDENCE_DIR: '{{.USER_WORKING_DIR}}/'
                                        
tasks:
...

Format

The format command is a convenient option to convert JSON or key-value pairs to various output formats.

Examples

Format Properties as Table with Fixed-Width Columns

$ heimdall format --output table <./gradle/wrapper/gradle-wrapper.properties

org.gradle.parallel           true
org.gradle.daemon.idletimeout 1800000

Format Key-Value Pairs as JSON/YAML/etc.

$ env | grep -E '^HOMEBREW_' | heimdall format --output jsonc

{
  "HOMEBREW_CELLAR": "/opt/homebrew/Cellar",
  "HOMEBREW_PREFIX": "/opt/homebrew",
  "HOMEBREW_REPOSITORY": "/opt/homebrew/Homebrew"
}

Git

The git commits subcommand lists the commits between two arbitrary commits specified by --start-ref and --end-ref, respectively.

Refs can be

  • HEAD, refs/remotes/origin/HEAD
  • branch, heads/branch, refs/heads/branch, refs/remotes/origin/branch
  • tag, refs/tags/tag
  • tilde and caret (HEAD~1, master~^, tag~2, ref/heads/master~1, …)
  • selection by text (HEAD^{/fix nasty bug})
  • hash (prefix and full)

The optional --merge-base flag finds the best common ancestor between those two commits and adjusts the starting ref accordingly.

Examples

$ heimdall git commits --merge-base --start-ref 7b2b24a --end-ref eb742d2 \
  ~/Documents/musikcube/ --output text --jq \
  '.[] | [.Author.When, .Author.Name, .Message | sub("\n.*"; ""; "g") | sub("(?<x>.{70}).+"; "\(.x)...")] | join("\t")'

2021-12-27T15:11:05-08:00       casey langen    CHANGELOG update.
2021-12-27T15:10:36-08:00       casey langen    Bump commit hash for release.
2021-12-27T13:21:17-08:00       casey langen    Refactored and added logging to SystemService to improve notification ...
2021-12-27T13:20:56-08:00       casey langen    Update Android Studio and dependencies.
2021-12-22T10:24:37-08:00       casey langen    Merge pull request #479 from ravensiris/master
2021-12-22T08:45:46+01:00       Maksymilian Jodłowski   Add missing header.
2021-12-18T21:09:37-08:00       casey langen    Merge pull request #475 from gschauer/master
2021-12-18T21:08:00-08:00       casey langen    Fixed notification tray icon for windows builds.
2021-12-08T14:43:49Z    gschauer        Add CircleCI build for Debian Bullseye
2021-12-07T22:47:28Z    gschauer        Fix dependencies for Ubuntu Impish
2021-11-18T13:15:44-08:00       casey langen    Fix mint build in circle ci script.

GitHub

The github command access the GitHub API to query information about repositories - in particular

  • code scanning results,
  • Dependabot alerts, and
  • secret scanning alerts.

Like the gh CLI, it supports environment variables, such as GH_HOST, GH_OWNER, GH_REPO, etc.

Note that the GitHub command is considered experimental and currently supports approximately 50 API endpoints. In case you encounter any issues, please report a bug report.

Examples

$ heimdall github code-scanning alerts-for-repo --repo java-playground

[                                                                                                                                                                                            
  {                                                                                                                                                                                          
    "number": 136,                                                                                                                                                                           
    "rule": {                                                                                                                                                                                
      "id": "java/call-to-object-tostring",                                                                                                                                                  
      "severity": "note",                                                                                                                                                                    
      "description": "Use of default toString()",                                                                                                                                            
      "name": "java/call-to-object-tostring",                                                                                                                                                
      "tags": [                                                                                                                                                                              
        "maintainability",                                                                                                                                                                   
        "reliability"                                                                                                                                                                        
      ]                                                                                                                                                                                      
    },                                                                                                                                                                                       
    "tool": {                                                                                                                                                                                
      "name": "CodeQL",                                                                                                                                                                      
      "version": "2.13.4"                                                                                                                                                                    
    },                                                                                                                                                                                       
    "state": "open",
    /* ... */
  }
]

Go

The go mod subcommand processes Go module files.

The output returned represents an abstract go.mod file.

{
  "Exclude": [],
  "Go": {
    "Version": "1.21"
  },
  "Module": {
    "Deprecated": "",
    "Mod": {
      "Path": "github.com/abc-inc/heimdall"
    }
  },
  "Replace": [
    {
      "New": {
        "Path": "github.com/cli/shurcooL-graphql",
        "Version": "v0.0.2"
      },
      "Old": {
        "Path": "github.com/shurcooL/graphql",
        "Version": "v0.0.0-20181231061246-d48a9a75455f"
      }
    }
  ],
  "Require": [
    {
      "Indirect": true,
      "Mod": {
        "Path": "github.com/shurcooL/graphql",
        "Version": "v0.0.0-20220606043923-3cf50f8a0a29"
      }
    }
    /* ... */
  ],
  "Retract": null,
  "Toolchain": {
    "Name": "go1.21.0"
  }
}

Examples

$ heimdall go mod --output text --query "Go.Version"

1.21

List Required Dependencies and Version

$ heimdall go mod --output text --jq \
  '[.Require[].Mod | select(.Path | test("/charmbracelet/")) | flatten | join("@")] | sort | .[]'

github.com/charmbracelet/bubbles@v0.16.1
github.com/charmbracelet/bubbletea@v0.24.2
github.com/charmbracelet/glamour@v0.6.1-0.20230531150759-6d5b52861a9d
github.com/charmbracelet/gum@v0.11.0
github.com/charmbracelet/harmonica@v0.2.0
github.com/charmbracelet/lipgloss@v0.8.0

Java

The java command can process all kinds of Java-related files. It can parse and aggregate Java code coverage reports, JUnit test reports, scan the filesystem for log4j vulnerabilities, parse a Maven POM file, and process web.xml files.

There are five subcommands:

  • jacoco: Parses and aggregates Java code coverage reports.
  • junit: Parses and aggregates JUnit test reports.
  • log4j: Scans the filesystem for log4j vulnerabilities.
  • maven: Parses a Maven POM file.
  • webxml: Processes web.xml files.

Examples

To parse and aggregate JUnit test reports, use the junit subcommand. For example:

$ heimdall java junit

This command will parse and aggregate JUnit test reports in the current directory.

Total Code Coverage

$ heimdall java jacoco -f jacoco.csv --summary --output text \
  --jq '.line_covered / (.line_covered + .line_missed)'

0.94

Line Statistics of Untested Methods

First, heimdall parses a JaCoCo code coverage report and outputs the result in CSV format. xsv - a fast CSV command line toolkit - to search within the CSV output. It selects rows where the method_covered column matches the regular expression ^0$, which means it selects rows where no methods are covered. Then xsv selects only the pkg, class, and line_missed columns from the previous output. Finally, sed effectively changes the CSV format to a tab-separated format with the pkg and class columns combined into one, separated by a dot.

$ heimdall java jacoco -f jacoco.csv --output csv |
  xsv search --select method_covered '^0$' |
  xsv select pkg,class,line_missed |
  sed -e 's/,/./' -e 's/,/\t/'

pkg.class       line_missed
org.jacoco.core.runtime.InjectedClassRuntime.Lookup     6
org.jacoco.core.runtime.InjectedClassRuntime    16
org.jacoco.agent.rt.internal.Agent.new Thread() {...}   3
org.jacoco.agent.rt.internal.IExceptionLogger   1
org.jacoco.agent.rt.internal.Offline    8
org.jacoco.agent.rt.internal.PreMain    12
org.jacoco.agent.rt.internal.IExceptionLogger.new IExceptionLogger() {...}      3
org.jacoco.agent.rt.RT  1
com.vladium.emma.rt.RT  6
org.jacoco.examples.ExecutionDataServer 5
org.jacoco.examples.MBeanClient 12
org.jacoco.examples.ExecutionDataServer.Handler 26
org.jacoco.examples.ReportGenerator     28
org.jacoco.examples.ExecutionDataClient 13
org.jacoco.examples.CoreTutorial.TestTarget     7

Find log4j Vulnerabilities

To scan the filesystem for log4j vulnerabilities, use the log4j subcommand. To get a list of files, use the text output mode.

$ heimdall java log4j --output text

12:34:56 WRN error="scanning jar: failed to check JAR: opening file decoy/JndiManager.class: zip: unsupported compression algorithm" file=corrupt.jar
12:34:56 WRN error="scanning jar: failed to check JAR: checking sub jar corrupt.jar: opening file decoy/JndiManager.class: zip: unsupported compression algorithm" file=corrupt_jar_in_jar.jar
Path
arara.jar
arara.signed.jar
bad_jar_in_jar.jar
bad_jar_in_jar_in_jar.jar
bad_jar_with_invalid_jar.jar
emptydirs.zip
log4j-core-2.0-beta9.jar
log4j-core-2.1.jar
log4j-core-2.12.1.jar
log4j-core-2.14.0.jar
log4j-core-2.15.0.jar
shadow-6.1.0.jar
vuln-class.jar

Other output modes, like json or yaml provide all details of the scanned files.

$ heimdall java log4j ~/go/pkg/mod/github.com/google/log4jscanner\@v0.5.0/jar/testdata/ \
  --output jsonc --jq '.[] | select(.Path | test("shadow"))' 2>/dev/null

{
  "MainClass": "",
  "Path": "shadow-6.1.0.jar",
  "Version": "",
  "Vulnerable": true,
  "Vulns": [
    {
      "CVE": "CVE-2021-44228"
    },
    {
      "CVE": "CVE-2021-45046"
    }
  ]
}

Check Servlet Specification in web.xml

The java webxml subcommand processes a web.xml file and outputs . The --mode raw flag indicates that the file should be read without further processing.

The expression following command checks whether the web application deployment descriptor uses the servlet 3.0 specification (or newer).

$ heimdall java webxml --mode raw ./web.xml --output text \
  --query 'to_number("web-app"."-version") >= `3.1`'
$ # or
$ heimdall java webxml --mode raw ./web.xml --output text \
  --jq '.["web-app"]["-version"] | tonumber >= 3'

true

Find Misconfigured Servlet Filters

The servlet-mappings mode lists all servlets and their configuration. For each servlet, all matching servlet filters are listed in declaration order. This comes in handy to determine potential configuration issues. For example, in order to handle WebSocket requests, all matching servlet filters must support asynchronous processing. Misconfigured servlet filters can be listed as follows:

$ heimdall java webxml --mode servlet-mappings ./web.xml --output text --jq \
  '.["servlet-mappings"][] | select(.servlet["async-supported"] == "true") | '\
  '.["filter-mappings"][].filters[] | select(["async-supported"] != "true")["filter-name"]'

javamelody

Jira

Examples

Top 10 Open Bugs

List the top 10 open bugs and return all details (including nested fields) about assignee, priority and summary.

$ heimdall jira issue -q "project = ABC AND type = Bug AND resolution = Unresolved ORDER BY priority DESC, updated DESC" \
  --fields 'assignee,priority,summary' --max-results 10

Release Notes as Custom JSON

List all stories and output a custom formatted JSON for summarizing the release notes.

$ heimdall jira issue -q "project = ABC AND type = Story AND fixVersion='ABC 1.2' ORDER BY id" \
  --output jsonc --jq '[.[] | {id: .key, summary: .fields.summary, status: .fields.status.name}]'

Keyring

The keyring command can fetch secrets with the system keyring. It supports OS X, Linux/BSD (dbus) and Windows.

Dependencies

OS X

The OS X implementation depends on the /usr/bin/security binary for interfacing with the OS X keychain. It should be available by default.

Linux and *BSD

The Linux and *BSD implementation depends on the Secret Service dbus interface, which is provided by GNOME Keyring.

It’s expected that the default collection login exists in the keyring, because it’s the default in most distros. If it doesn’t exist, you can create it through the keyring frontend program Seahorse:

  • Open seahorse
  • Go to File > New > Password Keyring
  • Click Continue
  • When asked for a name, use: login

Examples

$ heimdall keyring get --system github.com --user me

ghp_0123456789

Properties

Examples

Customize Output

By default, the properties command lists all properties. Command line flags like --get can be used to retrieve only the desired values.

$ heimdall properties -f .git/config --get name,email --output text --jq '.[0]+" <"+.[1]+">"'

gschauer <gschauer.abc.inc@gmail.com>

Modify Properties File

The properties command even allows to override individual key-value pairs.

$ heimdall properties -f ~/.gradle/gradle.properties --output text \
  --set org.gradle.caching=true | sponge ~/.gradle/gradle.properties

SSH

The ssh config command parses SSH configuration files and outputs the values for a certain host.

Examples

To get the default values for unspecified parameters in your SSH configuration file, use the --defaults flag.

For example:

Host web?? www??.company.corp
  User me

Host *
  HostName gateway.company.corp

AddressFamily inet

The command ssh config subcommand parses the SSH configuration file located at ~/.ssh/config. The configuration files contain sections separated by Host specifications, and that section is only applied for hosts that match one of the patterns given in the specification.

Thus, this command checks each Host section, which match the host named web00. For each parameter HostName, IdentityFile, and User, the first obtained value will be used. Since the first obtained value for each parameter is used, more host-specific declarations should be given near the beginning of the file, and general defaults at the end. The --defaults flag outputs the default values for all undefined parameters. Finally, the output is formatted as a table (--output table) with fixed-width columns.

$ heimdall ssh config -f ~/.ssh/config --defaults --hostname web00 \
  --keys HostName,IdentityFile,User --output table | sort

HostName     gateway.company.corp
IdentityFile ~/.ssh/identity
User         me