Release Notes

Java client library is now packaged as a single JAR file containing both Java class files as well as native code (.so file).

Added exponent and logarithm (exp, log, ln) operations for fixed-point numbers to SecreC Standard Library.

Added oblivious RAM (ORAM) functionality to SecreC Standard Library. This can be used to read from and write to secret-shared vectors by secret indices. See shared3p_oblivious_ram module for more details.

Javascript Client Library now supports application-level arguments that can be used by Sharemind Web Gateway applications. These arguments are separate from already existing SecreC bytecode arguments and can be set independently for each of the three hosts.

We have completely reimplemented the Java bindings for the Sharemind Web Gateway (unavailable for Ubuntu 18.04 because of missing dependencies on that distribution).

The sharemind-web-gateway Debian package was renamed to sharemind-nodejs-web-gateway.

Optimised matrixMultiplication for fixed point data types. It now uses a dedicated MPC protocol.

Added decision tree (ctree) and random forest (randomForest) algorithms to Rmind. Also available as part of SecreC Analytics Library (SAL) in analytics_trees module. Visualization of the result is not available.

Added xgboost algorithm for regression and classification to Rmind and SecreC Analytics Library (analytics_xgboost). Visualization of the result is not available.

Added glmnet (GLM + elastic net) algorithm to Rmind and SAL (analytics_linear_model).

Added Fast Fourier Transform (fft) and autocorrelation (acf) algorithms to Rmind and SAL (analytics_time_series).

Added Kalman and Liu-West filters to Rmind and SAL (analytics_kalman_filter). Related functions (dlm*) and its usage are based on the dlm package in R.

Added metrics (accuracy, precision, recall, f1, mse, rmse) to Rmind and SAL (analytics_metrics).

Added is.binary function to Rmind and SAL (analytics_metrics).

Added complex field operations (4 basic operations + exp, conjugate) to SecreC Analytics Library (see the analytics_complex module).

Added utilities for array computation to SAL module analytics_array. Notably argmax, argmin, matTimesVec, addOneColumn, unique, getTiles.

Added utilities for machine learning computation to SAL module analytics_ml. Notably, intoBinary, oneHotLabeling, selectMaxEachRow.

Added other utilities for datasets handling to SAL module analytics_datasets. The main ones are shiftLabel, scale, scaleMinMax, extractSubsets, trainTestSplit.

Added random value generators (randomUniform and randomNormal) to SAL module analytics_generate.

Added castToFloat and other wrapper functions for fix point data type to SAL module analytics_shared3p_fix.

Several internal optimisations in the SecreC compiler.

Added weighted linear regression, LOESS regression, quantiles function to SecreC Standard Library.

Added bl_strFind that returns the index of the first occurrence of needle in haystack.

Optimised the Gauss method of solving systems of linear equations.

GLM now by default uses Gauss method for solving systems of linear equations.

Added prefixSum and invPrefixSum functions for vectors.

Added shared3p_permutation module for generating public and private permutations and permuting vectors and matrices by rows or columns.

Added fast matrix transpose that uses gather and scatter syscalls.

Added clientAuth function that returns the authentication string (name) of client application running the current SecreC program. Note that the client authentication string may be different on different computation parties.

Added functions for computing quantiles and LOESS regression.

Removed unnecessary meanOp from the SecreC Analytics Library in favour of the ordinary mean operation.

Added a new analytics_date module to SecreC Analytics Library with conversion functions between Gregorian calendar and an internal Julian day format which allows calculating with dates (e.g. number of days between two dates). These functions were previously only in the Sharemind Analytics Engine.

The libsharemind-mod-shared3p, libsharemind-mod-shared3pdev and libsharemind-mod-shared3p-emu Debian packages are marked as conflicting so only one of them can be installed at any time. This is because they all provide the shared3p protection domain kind.

The sharemind-meta-shared3p, sharemind-meta-shared3pdev and sharemind-meta-shared3p-emu Debian meta-packages install SecreC compiler (scc) and Standard Library (secrec-stdlib), Sharemind Application Server (sharemind-server), HDF5 storage back-end (libsharemind-mod-tabledb-hdf5) and libsharemind-mod-algorithms modules and the respective shared3p protection domain kind module. The sharemind-meta-clientdev meta-package includes everything for custom client application development and local testing: sharemind-meta-shared3p-emu, libsharemind-controller-dev, libloghard-dev and libsharemind-mod-passthrough.

The Debian package for Sharemind Application Server (sharemind-server) now generates a server key pair on installation.

The self-hosted web front-ends for Rmind (Rmind Studio, rmind-service) and CSV Importer (sharemind-csv-importer-service) now generate a client application key pair when installed from a Debian package. This speeds up their set up process.

The Sharemind Web Gateway (sharemind-web-gateway) and JavaScript Client Library (sharemind-web-client) NodeJS modules are now installed into /usr/lib/nodejs/ on Debian, instead of /usr/local.

Sharemind configuration files can now be combined from multiple files using the @include directive, which takes a globbing pathname argument. For example, @include %{CurrentFileDirectory}/conf.d/*.conf takes all the files ending with .conf from the conf.d subdirectory and loads them as they were part of the current file.

Added support for the get_fpu_state and set_fpu_state(v) expressions to the SecreC language for changing the floating point unit state.

SecreC language now supports deprecation messages for deprecated functions via the @deprecated("message") annotation.

SecreC language gained support for the comma operator, for example to be used in cycles: for (uint i = acount - 1, j = acount; j != 0; --i, --j).

Functions for working with bounded length strings (bl_string) were moved from Sharemind Analytics Engine (Rmind back-end) to SecreC Standard Library. See the shared3p_string and shared3p_table_database modules.

Rmind gained support for factors, that allows to use string literals instead of their generated classifier values in expressions. See the documentation for factor and levels.

Rmind now supports binary operations on combinations of private and public string arguments.

Added support for ?function apropos in Rmind Studio for displaying information about function. Rmind already had this support.

Improved user exception reporting by introducing a new VM opcode (common.user_except). This statement was included in the 2018.09 changelog by mistake.

Ported libsortnetwork to C++.

Replaced OpenSSL with Crypto++ in Sharemind randomness library librandom.

Changed execution profiling to be per SecreC process instead of being per Application Server process until now. In Application Server configuration file, the ProfileLogFile option is replaced with ProfileLogPrefix that points to a directory where the profiling information of each SecreC process is written to a separate file.

SecreC Standard Library includes a new trace module that enables SecreC programs to log well-structured information about the process itself and data the process loads or saves. This module is used by the Sharemind Analytics Engine (Rmind back-end) and may also be used by custom SecreC programs.

Sharemind CSV Importer back-end logs the SHA-256 hash value of the user-uploaded data to allow re-producible analysis.

Released Sharemind MPC Tracer (sharemind-trace), a stand-alone command-line tool that extracts structured information from Application Server logs and shows data flow between SecreC processes. Its output uses Business Process Model and Notation (BPMN), where SecreC processes are modelled as BPMN Tasks and data tables as BPMN DataObjects. Sharemind MPC Tracer output can be visualised with many software supporting the BPMN 2.0 language. When given logs from more than one Application Server, Sharemind MPC Tracer also brings out discrepancies in SecreC processes between the different logs.

SecreC compiler now warns about usage of deprecated SecreC functions.

Added fixed point versions of abs, choose and shuffle to the Standard Library.

Optimised solving systems of linear equations using the Gauss method. An inner loop was transformed into a data parallel computation.

Made sharemind-runscript command-line interface more similar to that of sharemind-emulator. The --log-to-file command-line option was renamed to --logFile, the --run argument was removed. Additionally, it is now possible to write the bytecode output argument stream to file with --outFile, instead of pretty-printing it.

Improved user exception reporting by introducing a new VM opcode (common.user_except).

The mod_shared3p_emu module gained client application support (cmod_passthrough). Now it is possible to run existing client applications (e.g. Rmind) with a single Sharemind Application Server running the mod_shared3p_emu module.

Carter-Wegman hash system call (shared3p::cw128_xor_uint8_vec syscall) was added to the mod_shared3p_emu module.

SecreC bytecode compiled with scc v2.2.1+ takes advantage of the new common.user_except syscall for improved reporting of error conditions, including user assertions.

The quicksort function now works on Booleans. Boolean values are internally converted to xor_uint8 and the result back to bool.

Added floating point data type support to cut, min, max, variance, standardDev, MAD, fiveNumberSummary, covariance, t-tests, linear regression, quicksort, outlier detection, histogram, heatmap, Wilcoxon signed rank, Wilcoxon rank sum and Mann-Whitney U.

The deprecated quickquicksort function was removed from the SecreC standard library.

Added Carter-Wegman hash functionality in Standard Library shared3p_string module.

Added fixed point data type support (fix32, fix64) to shared3p module.

Added tdbVmapGetVlenValue function to table_database module to get a variable length value from a vector in a vector map.

GLM and linear regression functionality now allows to pass an empty matrix as the variables argument to specify a model with just the intercept.

Added float support to mean, median, min, max, mad, stdDev, variance, summary, heatmap, histogram, outlier detection, t-tests and linear regression.

Changed the output format of freq in Rmind to include a list of values and frequencies vectors that can be written to a file.

Added support for Hosmer–Lemeshow test.

Added support for Fisher’s exact test. Currently the functionality is limited to 2x2 contingency tables. It leaks one cell of the contingency table but without prior information it is not known which cell it is.

Added support for ROC AUC (area under the receiver operating characteristic curve). Possible privacy leak must be taken into consideration when visualising the ROC curve. If necessary, disable the ROC functionality in the Sharemind Analytics Engine configuration.

Added support for GLM and linear regression without independent variables.

In order to improve traceability of SecreC bytecode executions, Sharemind Application Server logs now contain user ID, SecreC process ID and hash on the same line.

JavaScript client library now uses the AES implementation provided by the aes-js library (MIT license) in its pseudo-random number generator (PRNG). This replaces the previously used jsaes AES implementation (GNU GPL license).

Sharemind Web Application Gateway and JavaScript client library support intercept handlers in order to allow modification of individual shares before they are sent to or received from another component. Sharemind Web Application gateway now emits the beforeSendResults and afterSendResults messages before and after sending the result shares back to the JavaScript library. The previously available beforeStartMpcProcess handler together with the added beforeSendResults handler can be used to modify individual input or output shares, respectively. The JavaScript client library provides a way to pass optional beforeResults and afterResults handlers to the runMpcComputation call, the first of which can be used in order to modify individual result shares before they are reconstructed by the library.

Disabled TLS session resumption by default in both the Sharemind Application Server and client applications as storing TLS session resumption tickets is known to defeat Perfect Forward Secrecy. The enabled cipher suites can be modified by changing the IncomingTlsPriorities and OutgoingTlsPriorities properties in Application Server and client application main configurations.

SecreC now allows numeric literals to be annotated by type,

Improved leakage information in the documentation of several SecreC standard library functions.

Significantly improved the performance of SecreC assembler when compiling large programs. As a trade-off, memory usage is increased.

The rm.missing function in Rmind now supports values of type bl_string (bounded-length strings).

The whitelisting based access control system is replaced by new access control system that manages access on three levels: authorized users, which users can run which SecreC programs and, as a new level, which SecreC programs have access to which data resources. All three levels of this access policy are configured in a single access policy configuration file, referenced by the AccessControl.PoliciesFile property in the Sharemind Application Server main configuration file. This replaces the Client.WhiteListEnabled and Client.WhiteListFile properties. Look for new syntax and examples in /usr/share/doc/sharemind/examples/server-access-control.conf.

The Server.ScriptPrefix property for setting SecreC bytecode search path is replaced by the Server.ProgramPaths property that also accepts a list of multiple search paths separated with semicolons. SecreC bytecode lookup on server side now also supports subdirectories, i.e. the file path passed to the Sharemind client library is appended to each of the paths listed in Server.ProgramPaths. The file path components passed to client library may contain ASCII alphanumeric characters, dashes (-), underscores (_), space characters and periods (.). The only exception is that no such component can consist only of periods (e.g. . or .. or …​ etc are not allowed as components). The default value of Server.ProgramPaths is changed from /var/lib/sharemind/scripts/ to /var/lib/sharemind/programs/.

The mod_keydb key-value storage now exclusively depends on hiredis library for Redis server support and dropped the cpp_redis as a dependency.

Log verbosity of sharemind-runscript and hdf5csvexporter client applications is configurable by the --logLevel command-line argument. Allowed log levels are fatal, error, warn(ing), info (normal), debug and fulldebug.

If the file ~/.rmind_profile exists, it is executed every time Rmind is started. This can be used to add convenience functions to Rmind environment.

Rmind and CSV Importer also support the above mentioned --logLevel command-line argument. Furthermore, it is possible to save Sharemind client library log to a separate file by passing the log filename with the --logFile (for Rmind) or with the --clientLog (for CSV Importer) command-line argument.

The ScanCursor struct and keydb_scan_next function interfaces have changed in the keydb module.

On platforms, where hiredis is available from a package, the mod_keydb key-value storage engine now uses hiredis C client library for connecting to the Redis backend. This replaces the cpp_redis dependency on these platforms.

Added principal component analysis (PCA) to Rmind. Available with commands prcomp and summary.prcomp. You can use screeplot to plot the variance (or cumulative proportion of total variance) of each principal component computed by prcomp.

Rmind and CSV Importer now use standard Sharemind configuration parser. They look for their configuration according to the XDG Base Directory Specification, i.e. from $XDG_CONFIG_HOME/sharemind/client.conf. If not defined in the environment, $XDG_CONFIG_HOME defaults to $HOME/.config:/etc/xdg. The configuration file location can be overridden by the --conf command-line parameter.

Added tdbVmapSetValueAsColumn method in the table_database module to insert vectors as database table columns. This can be used to insert multiple rows at a time by providing a vector for each column in the table.

It is now possible to set log verbosity in sharemind-server configuration file. Look for Server.LogLevel and comments in example file.

Implemented batching for the declassify protocol.

Created Java (JNI) bindings for building stand-alone Sharemind MPC client applications with Java.

Sharemind Analytics Engine now also supports binary operations on strings where one of the operands is public.

Reduced memory use of Sharemind Analytics Engine when working with strings.

Added support for sqrt, ln, log, log10, exp, erf, sin in Rmind.

Rmind GLM output (summary.glm) now includes Wald test for testing statistical significance of the model coefficients.

Rmind GLM output (summary.glm) now includes Akaike information criterion (AIC) for comparing models.

Added lists (dictionaries) with named elements support to Rmind.

Rmind tries to load its documentation from $XDG_DATA_DIRS/doc/rmind/man if $RMIND_DOCS_PATH is not defined.

Created web-based user interfaces for Rmind (called Rmind Studio) and CSV Importer. These are only alternative user interfaces where secret sharing is done in the backend service. Therefore, they must be deployed on premises for the end user. All Rmind statistical functionality is also available in Rmind Studio.

Log errors for client application are now also logged on the gateway.

Different Sharemind components no longer search for their configuration from the shell’s working directory. Sharemind Application Server looks for its default configuration from /etc/sharemind/server.conf. Client applications look for their configuration according to the XDG Base Directory Specification, i.e. from $XDG_CONFIG_HOME/sharemind/applicationname.conf, where applicationname is client for general purpose client applications, proxy for proxies (e.g. used with Sharemind Web Application Gateway) and emulator for Sharemind stand-alone Emulator. If not defined in the environment, $XDG_CONFIG_HOME defaults to $HOME/.config:/etc/xdg. Both Sharemind Application Server and client applications support the --conf command-line parameter override these defaults.

Files referenced in Sharemind configuration files are loaded relative the shell’s working directory. Use the %{CurrentFileDirectory} placeholder that is replaced with the absolute path of the current configuration file.

Several Sharemind binaries have been renamed:

It is possible to authorise Sharemind clients by the granularity of deployed SecreC programs. Look for examples in the Sharemind Application Server whitelist file. It is possible to reload this configuration without restarting the whole Sharemind Application Server, by sending the USR1 signal to the running sharemind-server process.

Sharemind Application Server now requires a valid license file to run. The license limits the number of SecreC programs that can run in parallel. The path to license file can be configured with the LicenseFile option in the main configuration file.

Reduced communication in integer division protocol by public divisor by 30% and in floating point protocols by up to 10%.

Added isSuffixOf function to SecreC standard library.

Rmind Read–Eval–Print Loop (REPL) supports partial input, i.e. the whole expression does not have to be on a single line.

Rmind supports commands to work with private dates: difftime, add.days.

Rmind supports casting private vectors to another type with cast.

Rmind supports concatenation of values to string with cat.

The subset command now discards non-matching rows instead of just marking them as unavailable in the private mask vector.

Added default value parameter to rm.missing in Rmind.

Rmind saves command history between sessions.

Added ls and rm for listing and removing variables from the Rmind environment.

Added ls.tables to Rmind for listing available data tables in a data store.

Rmind’s unique function now supports strings.

Sharemind Analytics Engine has a configurable threshold that sets the smallest dataset size allowed as input to procedures that publish aggregate results to the client. Look for details in the provided ae_configuration.sc file.

Sharemind Analytics Engine (SAE) allows Sharemind Application Server hosts to enable or disable individual SAE functions. Look for details in the provided ae_configuration.sc file.

Fixed several issues in Sharemind Web Application Gateway that occurred under high load.

Fixed a rare case, where Sharemind Application Server got segmentation fault after failing to open a PDPI stream for new client.

Fixed several network-layer issues.

Method quicksort now uses C++ syscall instead of SecreC-only implementation for better performance. Deprecated method quickquicksort.

Standard library functions now use quicksort in place of slower sorting networks.

Added a sort order flag for stable sorting algorithms quicksort and unsafeSort.

Added log, log10 and pow functions for shared3p protection domain.

Reduced assembler memory footprint.