Skip to content

Notice

This guide is for the original release of AQtive Guard. For the current AQG SaaS platform, go to the AQtive Guard User Guide.

PKCS#11 Fuzzer reference↑

AQtive Guard works with traces of exchanges between an application and a cryptographic library. The SandboxAQ PKCS#11 Fuzzer is used to send commands to a device’s PKCS#11 interface and log the responses.

Usage↑

Prerequisites↑

A standard test session uses about 1 GB of free memory, and memory requirements increase with the session duration. Test results are stored in AQtive Guard trace (CST) files. The file size increases with the level set for each command in the test.

Before you start, make sure the device you want to test is connected. For hardware PKCS#11 devices, you’ll need the location of the PKCS#11 DLL and the user PIN.

Important

Make sure that you also have backups for key materials before running the fuzzer on a device. While the fuzzer typically doesn’t delete existing keys, it may reveal firmware and driver bugs, which may require a device reset.

Overview↑

The PKCS#11 Fuzzer is a command-line tool. While it’s normally executed from a terminal, it can also be called from scripts. In this reference, it’s assumed that the PATH environment variable includes the path to the cs-fuzzer executable.

CST Files↑

CST files contain the tests that were performed on a device, as well as the results. You need to upload a CST to AQtive Guard to obtain compliance and vulnerability results. These files, generated by cs-fuzzer, are sequences of JSON objects.

Command-line Options↑

To run cs-fuzzer on a device:

cs-fuzzer --dll DLL --pin PIN --output myfile.cst [OPTIONS]

For more information about the options, run:

cs-fuzzer --help

The parameters and options are:

  • --dll (required) specifies the location to the PKCS#11 driver DLL for the device.
  • -p PIN, --pin PIN (required) is the PIN used to open a session.
  • --output OUTPUT (required) saves the CST file as OUTPUT
  • --commands COMMANDS, --only-commands COMMANDS: Only test the following commands. This option accepts a comma-separated command list and can be used multiple times. It only sets commands that use cryptographic objects and excludes commands that create them. It can’t be used with --except-commands.

An example command that uses the above:

$ cs-fuzzer --commands C_Sign,C_Encrypt --pin PIN \
--dll DLL --output myfile.cst
  • --direct Do not use C_GetFunctionList – used when the DLL is buggy and gives the wrong output from this call, causing a later crash.
  • --duration=DURATION Maximum duration in seconds to run the fuzzer.
  • --except-commands COMMANDS : Test all but the following commands. This option accepts a comma-separated command list and may appear multiple times. This can’t be used with --commands.
  • --except-generation-commands=EXCEPT_GENERATION_COMMANDS Use all but the following EXCEPT_GENERATION_COMMANDS. This option accepts a comma-separated command list and may appear multiple times. This can’t be used with --only-generation-commands.
  • --expanded Display an expanded view of PKCS#11 return values.
  • --filter-config Only test mechanisms the token reports as supporting.
  • --help[=FMT] (default= auto) Show the help in FMT format. The FMT value must be auto, pager, groff, or plain. With auto, the format is pager or plain whenever the TERM environment variable is dumb or undefined.
  • --indirect_or_direct Try to use C_GetFunctionList, and if it fails, try again without using it. This is the default behaviour.
  • --level LEVEL: The level of tests to be completed. The more tests are executed the higher the level is. This overrides the default of 2 that most commands have. Default levels are specified in a profile.
  • --log-calls Causes all API calls to be written to a log file. The log filename is api-calls.log.
  • --nss "configDir=sql:/path/to/NSS/database" Use this NSS configuration when using the NSS library.
  • --only-generation-commands COMMANDS: Only use the following functions to create cryptographic objects.
  • -q, --quiet Print less information. This can be given several times. The opposite of -v.
  • --resume-from Resume testing from an existing trace file.
  • --unsafe Avoid running the fuzzer in an isolated process. This improves performance, but increases the likelihood of an application crash in the event of an error.
  • -s INDEX, --slot-index INDEX, --slot INDEX Test the token in the INDEX slot. The --slot-index, --slot-id, – -slot-description, and --token-label options are mutually exclusive.
  • --slot-description DESCRIPTION Test the slot with the description DESCRIPTION. DESCRIPTION can’t exceed 64 characters. The --slot-index, --slot-id, --slot-description, and --token-label options are mutually exclusive.
  • --slot-id ID Test the token in the ID slot. The --slot-index, --slot-id, --slot-description, and --token-label options are mutually exclusive.
  • --stop-on-key-regeneration-failure Stop testing if an error occurs while regenerating a key on the hardware security model (HSM) using a known valid template.
  • --token-label LABEL Test the token with label LABEL. LABEL can’t exceed 32 characters. The --slot-index, --slot-id, --slot-description, and --token-label options are mutually exclusive.
  • --user-type USER TYPE Define the USER TYPE to use for the fuzzing session. Make sure to input the correct PIN for the given user type.
  • -v, --verbose Print more information. This can be given several times. The opposite of -q.
  • --version Show version information.

Note that long-form options can also be given with the equals sign (=) instead of a space, i.e. --output=file.cst is the same as --output file.cst.

Troubleshooting↑

Device or Driver Crashes↑

An occasional issue with cs-fuzzer, particularly with a new PKCS#11 DLL, is that the fuzzer causes a segmentation fault in the PKCS#11 driver DLL. Detecting these bugs in the driver is a feature of the fuzzer.

To find the call that caused the fault, run:

cs-fuzzer --dll DLL -p PIN --output myfile.cst --log-calls

This creates a file called api-calls.log. The command inputs that caused the error can be found at the end of this file,

Below is an example extract of a log file for a PKCS#11 device:

cs-test: C_Initialize ((nil)) // -> 0
cs-test: C_GetSlotList (0, (nil), 0x1ed0010) // -> 0
cs-test: C_GetSlotList (0, 0x1ecfec0, 0x1ed0010) // -> 0
cs-test: C_GetSlotInfo (0, 0x1ecfee0) // -> 0
cs-test: C_OpenSession (0, 6, (nil), (nil), 0x1ecff60) // -> 0
cs-test: C_Login (1, 1, 0x1ecdce0, 4) // -> 0

Each call to a function of the PKCS#11 is written to the file. The first part of the log shows which Analyzer command you logged the calls of and the function that’s logged. The next part (0,0x1ecfee0) shows the parameters used.

The final part // -> 0 is the return code given by the function. In this case, it’s 0 which means there was no error.

An example of a failed call is:

cs-test: C_Login (1, 1, 0x1ecdce0, 4) // -> 1

When the PKCS#11 driver DLL crashes, the last line of the log typically looks like:

cs-test: C_Login (1, 1, 0x1ecdce0, 4)

Note that the // -> 0 part is missing, which means that the trace stopped before getting to the end of the call.

For further debugging information, the fuzzer can be run under a debugger like gdb to get the stack trace details that triggered the error.

A CST file resulting from a full fuzzing test run will contain a final comment line stating “Fuzzing end”. If the DLL crashed during the run, this entry will be absent, making it simple to identify CSTs corresponding to crashed runs.

A trace file that terminates correctly will contain the string “Fuzzing End” in the last object.

How Fuzzing Tests are Generated↑

The PKCS#11 Fuzzer is a mutation-based fuzzing engine. It starts with a set of standard calls and adds or removes parameters to generate edge-case queries. The parameters that are added and removed are tailored to the compliance and vulnerability tests that’ll be executed on the CST file by the Analyzer.

The number of tests executed in a fuzzing test run for a given --level parameter will vary considerably depending on the mechanisms and commands supported by the device under test. For instance, a successful call to C_Encrypt generates a ciphertext, which becomes input for subsequent fuzzer operations. If the call fails and no ciphertext is created, fewer tests will be executed.

Runs using --level 3 can take several days to complete on typical devices. Higher levels are typically only useful when combined with other restrictions such as --only-commands and --only-generation-commands.

Resuming from a Partial Trace↑

A fuzzer run consists of two parts:

  • Testing generation commands (C_CreateObject, C_GenerateKey, C_GenerateKeyPair)
  • Testing cryptographic commands (C_Encrypt, etc.).

Cryptographic commands require key objects to be created, and the fuzzer uses information gathered in the initial analysis to create these objects. However, this can be time-consuming since that step needs to be repeated for every run. Instead, it’s possible to create a trace that only contains information about generation commands, and start fuzzing from information in that trace.

To do this, first generate a trace containing only generation commands. This can be done by using --only-commands C_Digest:

cs-fuzzer --dll DLL -p PIN --output generation.cst --only-commands C_Digest

For subsequent traces, pass --resume-from with the first trace to bypass the testing of generation commands:

cs-fuzzer --dll DLL -p PIN --output commands.cst --resume-from generation.cst

There are some caveats with this approach:

  • Producing a trace using one set of parameters, and then resuming from it with different parameters (e.g. a different --level ) may not create a meaningful trace.
  • Traces produced using --resume only contain fuzzing information for the cryptographic commands, not the generation commands. As a result, analysis findings that depend on generation commands may not be detected in the resumed trace. For full results, you should also analyze the original generation trace.

Because of these limitations, it’s preferable to work with a single trace. The --resume option is typically helpful if the device under test is slow and/or crashes often.