Task Enclave Development

1. Task Enclave Development

This page provides a small overview about how task enclave development is done. However it does not cover deploying enclaves in a production environment. For more information about release builds and deploying enclaves in a production setting see the Release Builds page.

You need the following things on the machine where you develop task enclaves:

The Intel® SGX SDK (https://github.com/intel/linux-sgx)
An Intel CPU.
The Sharemind HI Development Bundle, which comes in two flavors:
- SIM & Prerelease: Does not communicate with the IAS.
- HW & Prerelease: Communicates with the IAS.

To use the HW mode you also need:

CPU with support for Intel® SGX.
The Intel® SGX PSW (https://github.com/intel/linux-sgx)

A new solution can be created with the sharemind-hi-create-task-enclave-project script. It provides a series of options which you can explore through the --help flag. The script will setup a template CMake project for developing a task enclave. On success the script outputs a brief message with instructions on how to build, test, and modify the solution. A more thorough tutorial on setting up your first Sharemind HI solution can be found on the Sharemind HI As Data Analysis Platform page.

2. Build modes

Task enclaves in Sharemind HI can be built in five different modes depending on the available machines and the development stage.

To accommodate for developer machines without SGX support it is possible to develop enclaves in either a simulation mode (SIM) or hardware mode (HW). While both modes require an Intel® CPU (at least for Sharemind HI development), the SIM mode only simulates the enclave environment and as such does not require a CPU with SGX support, making it useful during development. The HW mode creates actual enclaves and to do so also requires a CPU with Intel® SGX support. It is important to note that the SIM mode offers no protection at all and building final production enclaves is only possible in HW mode.

Regardless of whether SIM or HW mode was used, additional Debug and Prerelease modes are available to simplify development. The Debug mode performs additional checks and logging, while the Prerelease mode acts as similarly to a production environment as possible. It is recommended to use the Debug mode during task enclave development and the Prerelease mode should be used for benchmarking and deployment testing. Once the task enclaves are ready for deployment, they have to be built in Release mode, which enables full memory protection in the built enclaves.

The build mode combinations and key points which apply to Sharemind HI development are listed below:

	SIM Mode	HW Mode
Debug Mode	No memory protection Requires Intel® CPU Performs additional checks Skips remote attestation	No memory protection Requires Intel® CPU with Intel® SGX support Performs additional checks Skips remote attestation
Prerelease Mode	No memory protection Requires Intel® CPU Skips remote attestation	No memory protection Requires Intel® CPU with SGX support Suitable for benchmarking and deployment testing. Performs remote attestation (but can be skipped)
Release Mode	(cannot be run)	Memory protection Requires Intel® CPU with SGX support Requires commercial license with Intel® to perform remote attestation. Performs remote attestation

SIM Mode

HW Mode

Debug Mode

No memory protection
Requires Intel® CPU
Performs additional checks
Skips remote attestation

No memory protection
Requires Intel® CPU with Intel® SGX support
Performs additional checks
Skips remote attestation

Prerelease Mode

No memory protection
Requires Intel® CPU
Skips remote attestation

No memory protection
Requires Intel® CPU with SGX support
Suitable for benchmarking and deployment testing.
Performs remote attestation (but can be skipped)

Release Mode

(cannot be run)

Memory protection
Requires Intel® CPU with SGX support
Requires commercial license with Intel® to perform remote attestation.
Performs remote attestation

3. Notes for Task Enclave Development

Development Environment: In Sharemind HI, algorithms for task enclaves are written with C++ using the implementation of the standard library provided by the Intel® SGX SDK. Most of the C++17 language is supported, but functionality around IO and other OS reliant features (like signals) are not available.
In practice this means that if you want to use existing C++ code, like third party libraries, in the task enclaves, that you need to manually port them into the restricted Intel® SGX environment.
Performance: The code in Intel® SGX enclaves can run at near native speed. However, for older CPUs with ⇐ 256 MiB EPC (Enclave Page Cache) (approximately all CPUs before 3rd Generation Intel® Xeon® Scalable Processors), due to the way how the working memory of an enclave was protected, algorithms need to optimize memory access patterns and the size of the working set or should expect a significant performance slowdown. This was alleviated in newer CPUs with larger EPC.
The EPC is the amount of memory which an enclave can directly access. If the size of the working set of the enclave exceeds the size of the EPC, then older memory needs to be encrypted and swapped out of the EPC, and other memory regions swapped into the EPC and decrypted. The swapping and encryption operations are very costly performance wise.
A natural way to process large amounts of data without storing everything at once in memory is by utilizing streams. The Sharemind HI SDK offers a library for creating such streams (contained in the Streams.h header) with implementations for a number of operations common in data processing workflows.

4. Building and Signing the Enclave `.so` Files

In order to include the enclave code in a Sharemind HI deployment it has to be compiled and built into a .so file and signed. Enclave development in Sharemind HI is commonly done in the CMake project created by the sharemind-hi-create-task-enclave-project script. The CMake project includes all of the tools required to build the enclaves and run them with a Sharemind HI server.

To build enclave packages for release builds, see the Release Builds page. To build enclaves for use with Sharemind HI in Debug or Prerelease modes, run cmake ..; make in the build/ directory of the Task Enclave Project. This will instruct CMake to compile the enclaves using the information from the Sharemind HI Development Bundle as configured in the config.local file. The enclaves will be built using the same modes as the development bundle. The built .so files should be located in build/src/{your_enclave_name}/ and named as lib{you_project_name}_{your_enclave_name}.so. The plain .so files are however not directly usable with Sharemind HI as they have to first be signed.

When the enclaves are built using the Debug or Prerelease modes, the CMake project automatically signs the enclave .so files using a default testing key, or, when defined in the config.local file, a custom private key. The signed .so files are located next to the regular .so files and named lib{you_project_name}_{your_enclave_name}.signed.so. These signed files are now ready to use with a with a Sharemind HI test deployment.

You can also use cpack, which is a part CMake, to create a tarball or .deb package for every enclave.

# Use either one of these:
cpack -G DEB
cpack -G TGZ

5. Measuring the enclaves

To let the Sharemind HI server know the exact identity of the your enclave, it needs to be provided with the enclave’s fingerprint. The fingerprint, or measurement, of the enclave is derived from the signed .so file using a special cryptographic protocol.

The testing scripts in the task enclave project can automatically measure the enclaves in the project and fill the test DFC and server configuration files. For use with custom projects and DFCs the measurements can be retrieved using the enclave-info.sh script located in the Sharemind HI Development bundle under hi-bundle/lib/cmake/sharemind-hi/. The script takes the signed .so file as input and displays information about the enclave. The relevant fields for configuration are MRENCLAVE and MRSIGNER, which contain the enclave and signer fingerprints respectively.

6. Testing

When using the Task Enclave Project tool, a roundtrip test can be run using CMake’s ctest command. The test scrips (located under the test/ directory) automatically fill all configuration files with the enclave fingerprints, starts the Sharemind HI Server including the enclave, and runs through a simple test scenario. The test and how to modify it is further described under the data analysis tutorial page.

To test the enclaves with custom projects that are not built using the Task Enclave Project tool, you need to configure the Sharemind HI server to use the newly built and signed task enclave, as well as update the DFC with the new fingerprints.