PKCS#11 Fuzzer reference↑
AQtive Guard works with traces of exchanges between an application and a cryptographic library. The SandboxAQ PKCS#11 Fuzzer is used to send commands to a device’s PKCS#11 interface and log the responses.
Usage↑
Prerequisites↑
A standard test session uses about 1 GB of free memory, and memory requirements increase with the session duration. Test results are stored in AQtive Guard trace (CST) files. The file size increases with the level set for each command in the test.
Before you start, make sure the device you want to test is connected. For hardware PKCS#11 devices, you’ll need the location of the PKCS#11 DLL and the user PIN.
Important
Make sure that you also have backups for key materials before running the fuzzer on a device. While the fuzzer typically doesn’t delete existing keys, it may reveal firmware and driver bugs, which may require a device reset.
Overview↑
The PKCS#11 Fuzzer is a command-line tool. While it’s normally
executed from a terminal, it can also be called from scripts. In this reference,
it’s assumed that the PATH
environment variable includes the path to the cs-fuzzer
executable.
CST Files↑
CST files contain the tests that were performed on a device, as well as the results. You need to upload a CST to AQtive Guard to obtain
compliance and vulnerability results. These files, generated by cs-fuzzer
, are sequences of JSON objects.
Command-line Options↑
To run cs-fuzzer
on a device:
For more information about the options, run:
The parameters and options are:
--dll
(required) specifies the location to the PKCS#11 driver DLL for the device.-p PIN
,--pin
PIN (required) is the PIN used to open a session.--output OUTPUT
(required) saves the CST file asOUTPUT
--commands COMMANDS
,--only-commands COMMANDS
: Only test the following commands. This option accepts a comma-separated command list and can be used multiple times. It only sets commands that use cryptographic objects and excludes commands that create them. It can’t be used with--except-commands
.
An example command that uses the above:
--direct
Do not useC_GetFunctionList
– used when the DLL is buggy and gives the wrong output from this call, causing a later crash.--duration=DURATION
Maximum duration in seconds to run the fuzzer.--except-commands COMMANDS
: Test all but the following commands. This option accepts a comma-separated command list and may appear multiple times. This can’t be used with--commands
.--except-generation-commands=EXCEPT_GENERATION_COMMANDS
Use all but the followingEXCEPT_GENERATION_COMMANDS
. This option accepts a comma-separated command list and may appear multiple times. This can’t be used with--only-generation-commands
.--expanded
Display an expanded view of PKCS#11 return values.--filter-config
Only test mechanisms the token reports as supporting.--help[=FMT]
(default=auto
) Show the help inFMT
format. The FMT value must beauto
,pager
,groff
, orplain
. Withauto
, the format ispager
orplain
whenever theTERM
environment variable isdumb
orundefined
.--indirect_or_direct
Try to useC_GetFunctionList
, and if it fails, try again without using it. This is the default behaviour.--level LEVEL
: The level of tests to be completed. The more tests are executed the higher the level is. This overrides the default of2
that most commands have. Default levels are specified in aprofile
.--log-calls
Causes all API calls to be written to a log file. The log filename isapi-calls.log
.--nss "configDir=sql:/path/to/NSS/database"
Use this NSS configuration when using the NSS library.--only-generation-commands COMMANDS
: Only use the following functions to create cryptographic objects.-q
,--quiet
Print less information. This can be given several times. The opposite of-v
.--resume-from
Resume testing from an existing trace file.--unsafe
Avoid running the fuzzer in an isolated process. This improves performance, but increases the likelihood of an application crash in the event of an error.-s INDEX
,--slot-index INDEX
,--slot INDEX
Test the token in theINDEX
slot. The--slot-index
,--slot-id
, –-slot-description
, and--token-label
options are mutually exclusive.--slot-description DESCRIPTION
Test the slot with the descriptionDESCRIPTION
.DESCRIPTION
can’t exceed 64 characters. The--slot-index
,--slot-id
,--slot-description
, and--token-label
options are mutually exclusive.--slot-id ID
Test the token in theID
slot. The--slot-index
,--slot-id
,--slot-description
, and--token-label
options are mutually exclusive.--stop-on-key-regeneration-failure
Stop testing if an error occurs while regenerating a key on the hardware security model (HSM) using a known valid template.--token-label LABEL
Test the token with labelLABEL
.LABEL
can’t exceed 32 characters. The--slot-index
,--slot-id
,--slot-description
, and--token-label
options are mutually exclusive.--user-type USER TYPE
Define theUSER TYPE
to use for the fuzzing session. Make sure to input the correctPIN
for the given user type.-v
,--verbose
Print more information. This can be given several times. The opposite of-q
.--version
Show version information.
Note that long-form options can also be given with the equals sign (=) instead of a space,
i.e. --output=file.cst
is the same as --output file.cst
.
Troubleshooting↑
Device or Driver Crashes↑
An occasional issue with cs-fuzzer
, particularly with a new PKCS#11 DLL, is that the
fuzzer causes a segmentation fault in the PKCS#11 driver DLL. Detecting these bugs in the
driver is a feature of the fuzzer.
To find the call that caused the fault, run:
This creates a file called api-calls.log
. The command inputs that caused the error can be found at the end of this file,
Below is an example extract of a log file for a PKCS#11 device:
cs-test: C_Initialize ((nil)) // -> 0
cs-test: C_GetSlotList (0, (nil), 0x1ed0010) // -> 0
cs-test: C_GetSlotList (0, 0x1ecfec0, 0x1ed0010) // -> 0
cs-test: C_GetSlotInfo (0, 0x1ecfee0) // -> 0
cs-test: C_OpenSession (0, 6, (nil), (nil), 0x1ecff60) // -> 0
cs-test: C_Login (1, 1, 0x1ecdce0, 4) // -> 0
Each call to a function of the PKCS#11 is written to the file. The first part of the log shows which Analyzer
command you logged the calls of and the function that’s logged. The next part
(0,0x1ecfee0)
shows the parameters used.
The final part // -> 0
is the return code given by the function. In this case, it’s 0
which means there was no error.
An example of a failed call is:
When the PKCS#11 driver DLL crashes, the last line of the log typically looks like:
Note that the // -> 0
part is missing, which means that the trace stopped before getting to the end of the call.
For further debugging information, the fuzzer can be run under a debugger like gdb
to get the stack trace details that triggered the error.
A CST file resulting from a full fuzzing test run will contain a final comment line stating “Fuzzing end”. If the DLL crashed during the run, this entry will be absent, making it simple to identify CSTs corresponding to crashed runs.
A trace file that terminates correctly will contain the string “Fuzzing End” in the last object.
How Fuzzing Tests are Generated↑
The PKCS#11 Fuzzer is a mutation-based fuzzing engine. It starts with a set of standard calls and adds or removes parameters to generate edge-case queries. The parameters that are added and removed are tailored to the compliance and vulnerability tests that’ll be executed on the CST file by the Analyzer.
The number of tests executed in a fuzzing test run for a given --level
parameter will
vary considerably depending on the mechanisms and commands supported by the device under
test. For instance, a successful call to C_Encrypt
generates a ciphertext, which becomes input for
subsequent fuzzer operations. If the call fails and no ciphertext is created, fewer
tests will be executed.
Runs using --level 3
can take several days to complete on typical devices.
Higher levels are typically only useful when combined with other restrictions such as
--only-commands
and --only-generation-commands
.
Resuming from a Partial Trace↑
A fuzzer run consists of two parts:
- Testing generation commands (
C_CreateObject
,C_GenerateKey
,C_GenerateKeyPair
) - Testing cryptographic commands (
C_Encrypt
, etc.).
Cryptographic commands require key objects to be created, and the fuzzer uses information gathered in the initial analysis to create these objects. However, this can be time-consuming since that step needs to be repeated for every run. Instead, it’s possible to create a trace that only contains information about generation commands, and start fuzzing from information in that trace.
To do this, first generate a trace containing only generation commands. This can be done
by using --only-commands C_Digest
:
For subsequent traces, pass --resume-from
with the first trace to bypass
the testing of generation commands:
There are some caveats with this approach:
- Producing a trace using one set of parameters, and then resuming from it with different
parameters (e.g. a different
--level
) may not create a meaningful trace. - Traces produced using
--resume
only contain fuzzing information for the cryptographic commands, not the generation commands. As a result, analysis findings that depend on generation commands may not be detected in the resumed trace. For full results, you should also analyze the original generation trace.
Because of these limitations, it’s preferable to work with a single trace. The
--resume
option is typically helpful if the device under test is slow and/or crashes
often.