Category Archives: test tool

First results of eMRTD Interoperability Test 2017 in Ispra

The European Commission (EC) organised another eMRTD interoperability test. This time the event took place in Ispra at the Joint Research Center of the European Union. The objective of this interoperability test was to assure that countries and companies have established a stable PACE protocol in their eMRTD, respective ePassports, and ID cards.

Test setup of eMRTD interoperability test

Every participant had the chance to submit up to two different sets of documents with different implementations. Altogether there were 42 different samples available at the beginning of the event. 2 samples didn’t pass the smoke test and one sample was not suitable to be tested by the labs. All remaining samples were tested in two different test procedures: crossover test and conformity test. Twelve document verification system providers with 16 different solutions took part in the crossover test. And 23 document providers threw 42 sets in the ring (28 countries, 14 industries).

In this blog post the conformity test is focused on, because protocols are in foreground in this kind of test. There were three test labs (Keolabs, UL and secunet) taking part in the interoperability test with their test tools to perform a subset of “ICAO TR RF Protocol and Application Test Standard for e-Passports, Part 3” (Version 2.10). The subset includes the following test suites and test cases:

  • ISO7816_O: Security conditions for PACE protected eMRTDs
  • ISO7816_P: Password Authenticated Connection Establishment (PACEv2)
  • ISO7816_Q: Command READ and SELECT for file EF.CardAccess
  • LDS_E: Matching between EF.DG14 and EF.CardAccess
  • LDS_I: Structure of EF.CardAccess
  • LDS_K: Structure of EF.CardSecurity
  • LDS_D_06: Test case to perform Passive Authentication

Information concerning documents

The document providers describe in the implementation conformance statement (ICS) the features of their chips. Not all ICS were fulfilled consistently, so the following information concerning the documents should be read carefully. Concerning the LDS version 16 providers reported version 1.7 to be used in their documents. And three reported version 1.8, while all others don’t deliver any information concerning the version number.

The following diagram describes the relation between DH and ECDH in PACE:

PACE algorithms

PACE algorithms

The following diagram describes the relation of the mapping protocols in PACE:

Mapping protocols in PACE

Mapping protocols

16 documents supported besides the MRZ also a CAN as a password to get access to the stored data.

Again, the number of PACEInfos store in EF.CardAccess varied:

  • 28 documents stored one PACEInfo,
  • Eight documents stored two PACEInfos,
  • One stored each three, seven or ten PACEInfos.

Investigations concerning EF.ATR/INFO in documents

The file EF.ATR/INFO allows storing some information about the chip, that allows the reader to handle the chip optimally. On this way the chip can offer the ideal buffer size used with extended length during reading and writing. In context of the event I had a closer look at EF.ATR/INFO. 26 documents of 42 stored an EF.ATR/INFO but 5 of them don’t offer information concerning extended length and buffer sizes there. So at the end I’ve investigated the reading buffer sizes of 21 documents of the eMRTD interoperability test with the following result:

  • Seven documents support buffer sizes between 1 and 1.5 KByte,
  • Three documents support buffer sizes between 1.5 and 2 KByte,
  • Eleven documents support buffer sizes of ~64 KByte.
Buffer sizes for reading in EF.ATR/INFO

Buffer sizes in EF.ATR/INFO (reading)

With these large buffer sizes data groups like DG2 storing the facial image or DG3 storing the finger prints can be read in only one command. This allows the inspection system to read the content of the chip faster and improves the reading time at eGates.

Results of conformity testing

During the conformity test, all three test labs performed 18.135 test cases altogether. Less than 1 percent of these test cases failed during the conformity test.

Overall results (layer 6 and layer 7):

  • Passed: 11.563
  • Failed: 155 (0,86%)
  • Not applicable: 6.417

Layer 6 (16.614 test cases performed):

  • Passed: 10.885
  • Failed: 124 (0,74%)

Layer 7 (1.521 test cases performed):

  • Passed: 679
  • Failed: 32 (2,10%)

The following diagram shows failed test cases per document during the eMRTD interoperability test:

Number of failures per document

Number of failures per document

The diagram below shows the number of failure per test case during the eMRTD interoperability test:

Failures per test case

Failures per test case

Observations during conformity testing

  • There are minor differences between implementation conformance statement (ICS) and chip.
  • Test results differ between test labs in some test cases.
  • There are differences in handling errors at the test tools and labs (e.g. no CAN causes a failure at the one lab and a ‘not applicable’ at the other lab).
  • Relatively more failures on layer 7 (personalisation) than on layer 6 (COS).
  • Very good quality of chip and personalisation.
  • Improvements during the last interoperability tests in London 2016 and Madrid 2014.
  • Stable specifications (BSI TR-03110, ICAO Doc 9303) and test specifications (BSI TR-03105, ICAO TR Part 3).

Call for Participation: Interoperability Test 2017 in Ispra

The European Commission will organize conformity and interoperability tests for eMRTDs together with a conference on 25th and 26th September 2017. It will be held in the European Commission Joint Research Centre (JRC) premises in Ispra, Italy. The tests will focus on the latest access control specifications (e.g. the operation of the PACE protocol with Chip Authentication Mapping). This security mechanism, known as “Password Authenticated Connection Establishment with Chip Authentication Mapping” (PACE-CAM) is specified in Technical Report BSI TR-03110 “Advanced Security Mechanisms for Machine Readable Travel Documents and eIDAS token” and it combines PACE and Chip Authentication (CA) into one protocol leading to faster ID document verification.

Logo JRC

The conference will take place on the second day (26/09/2017) and will include speakers from the EU Commission, ICAO (requested) and Member States (requested). At its end, the high-level aggregate results of the tests will be presented.

The main beneficiaries of these tests are EU Member States. Depending on the number of EU Member States that will participate in the event, and provided that it is possible from an organisational perspective, a limited number of non-EU ICAO Member States and private sector travel document manufacturers will be allowed to participate in the test (on a first come first serve basis).

The test will focus on the implementation of PACE as specified in the Technical Report “Radio Frequency Protocol and Application Test Standard for eMRTD Part 3 Tests for Application Protocol and Logical Data Structure“, Version: 2.10, July 2016.

You can find the Call for Participation for the interoperability test here with more information concerning preregistration etc. See you in Ispra!

First results of eMRTD Interoperability Test 2016

During 10th Security Document World 2016 an additional Interoperability Test for eMRTD with PACE took place in London. In context of ePassports this was the 14th event starting 2003 on Canberra. This time there were two test labs involved, 17 document providers and twelve inspection system providers. Here I will focus on the conformity test including test labs and document providers and the InteropTest results. The event was organised by the colleagues of secunet. The following document providers delivered in sum 27 samples:

Logo of SDW InteropTest

  • Arjo Systems
  • Atos IT Solutions and Services
  • Bundesdruckerei
  • Canadian Banknote Company
  • cryptovision
  • De La Rue
  • Gemalto
  • ID&Trust
  • Imprimerie Nationale Group
  • Iris Corporation Berhad
  • MaskTech
  • Morpho
  • NXP Semiconductors
  • Oberthur Technologies
  • PAV Card
  • Polygraph combine ‘Ukraina’
  • PWPW

And the following test laboratories performed a subset of tests focusing on PACE (and of course PACE-CAM):

  • Keolabs (France)
  • HJP Consulting / TÜViT (Germany)

The test cases performed during the event based on ICAO’s test specification ‘RF Protocol and Application Test Standard for eMRTD – Part 3‘ Version 2.08 RC2 including some minor adaptions based on the last WG3TF4 meeting in Berlin. The final version 2.08 of this test specification will be released soon and deltas will be listed in an additional blog post here. With focus on PACE-CAM the following test suites were performed by both test labs:

  • Test Unit 7816_O (Security Conditions for PACE-protected eMRTDs)
  • Test Unit 7816_P (Password Authenticated Connection Establishment)
  • Test Unit 7816_Q (Select and Read EF.CardAccess)
  • Test Unit 7816_S (Select and Read EF.CardSecurity)
  • Test Unit LDS_E (Data Group 14)
  • Test Unit LDS_I (EF.CardAccess)
  • Test Unit LDS_K (EF.CardSecurity)

Some statistics concerning the samples:

  • PACE-CAM was supported in the following types:
    • Generic Mapping (GM), Chip Authentication Mapping (CAM): 18 samples
    • Integrated Mapping (IM), CAM: 4 samples
    • GM, IM and CAM: 4 samples
  • LDS:
    • 25 samples used LDS1.7
    • 1 sample used LDS1.8
    • 1 sample used LDS2.0 (with backward compatibility to LDS1.7)

In the preliminary InteropTest results presented by Michael Schlüter during the SDW he mentioned, that 8502 test cases were performed during conformity testing by the test labs and 98% of the relevant test cases were passed by the samples. Additionally, the test results of both labs were fairly consistent. There was one test case that causes the most failures and this test case verifies ChipAuthenticationPublicKey in EF.CardSecurity (LDS_K_2). Here we need some clarification in the specification Doc9303 and finally in the test specification.

During the crossover test there were three problems detected: At first the sequence of PACE, CA and TA was performed correctly while the sequence of PACE-CAM and TA causes some problems during the inspection procedure of the readers. This might be based in the fact, that PACE-CAM is specified in an ICAO document and TA in a BSI document. Some inspection systems had also problems with alternative File IDs for EF.CVCA. The alternative FID can be defined in TerminalAuthenticationInfo (see A.1.1.3 in TR-03110 Part 3) and must be used by the inspection systems to address and read EF.CVCA. But a bad surprise in the InteropTest results was, that around 50% of the inspection systems don’t perform Passive Authentication (PA) correctly. During the preparation of the InteropTest a wrong CSCA certificate was distributed and 50% of the systems have not detected this wrong certificate, this means: 50% of the inspection systems failed in Passive Authentication! During the conference Dr. Uwe Seidel (Bundeskriminalamt, BKA) noticed, that this number mirrors the real world and that in fact PA is currently a real problem in border control.

The InteropTest results can be summed up in two statements:

  1. There is a very good quality of the eMRTD samples.
  2. Reader vendors have still some work to do, especially to implement Passive Authentication correctly.

A detailed test report of this event and the InteropTest results will be published by secunet in June 2016.

Update: The final test report can now be downloaded here (after a short registration at the SDW website).

New GlobalTester Release 3.1

GlobalTester is an Open Source tool to test (not only) smart cards. It’s developed with Eclipse. You can use the tool as an Eclipse plugin or standalone as Eclipse RCP. With the new release GlobalTester is not reduced to chip cards anymore. From now on you can test various protocols e.g. in context of Smart Home or IoT.

Here is a subset of the benefits of the new version:

  • Supports XML scheme according to BSI TR-03105 for test cases
  • Sample Configuration of  test objects allows switching between test objects and persistence of test information
  • Easy sharing with test houses and certification authorities
  • New, intuitive user interface, and handling
  • Extensive cheat sheets and new user guidance
  • Test Campaign allows easy reproduction of a test run and persistence of test results

The following video clip gives a first impression of the new user interface and the functionality (Video concept and recording by Anke Larkworthy).

If you are interested in the source code: we host the free available basic version of GlobalTester on the versioning system GitHub. Please feel free to download the code and to join the community. You are always welcome :)

 

 

Interoperability Test during SDW in May 2016

puzzle - interoperability test

Puzzle of InteropTest

Another interoperability test in context of ePassports (eMRTD) and inspection systems will be performed during SecurityDocumentWorld 2016 in London. The test will be focused on Supplemental Access Control (SAC) respective PACEv2, a security protocol to protect personal data stored in electronic ID documents.

An interoperability test is similar to a plugtest performed e.g. by ETSI. It’s an event during which devices (ePassport, inspection systems and test tools) are tested for interoperability with emerging standards by physically connecting them. This procedure is called crossover testing and allows all vendors to test their devices against other devices. The efforts to perform this kind of test increases very strongly with every ePassport and inspection system. Therefore these kind of tests can be performed only with a small number of devices under test.

Crossover Testing

Crossover Testing

Additionally, there is the opportunity besides this crossover tests to test the devices against conformity test suites implemented in test tools like open source tool GlobalTester. This procedure reduces efforts and allows comprehensive failure analyses of the devices like ePassports or inspection systems. To assure interoperability it is state of the art to set up test specifications. These specifications are implemented by the test labs respectively in the test tools they use.

Conformity Testing

Conformity Testing

There are well established test specifications available, both for ePassports and for inspection systems. Previous interoperability tests took place in Madrid (2014) and London (2013). Both events focused also on SAC/PACE.

If you are interested as a document provider, as a vendor of an inspection system, as a test lab or as an observer, you can register here.

Looking forward to seeing you in London during the InteropTest!

BTW: The EU article 6 group is preparing a document describing how to process an interoperability test and how to prepare such an event.

Testing ePassport Readers using TTCN-3

Currently you can find the well-known test tool Titan under the patronage of the Eclipse Foundation (proposal). This tool was developed by Ericsson several years ago to the test internet protocol (IP). Titan bases on TTCN-3, a test language focusing on communication protocols. This keeps me in mind an old project with ETSI where we used TTCN-3 to test ePassport readers concerning BSI TR-3105 part 5.1.

From end of 2009 to middle of 2011 ETSI has conducted a project to develop a test system prototype for conformance testing of ePassport readers. The objective of this project was to design, build and test a TTCN-3 (Testing and Test Control Notation, Version 3) based test system prototype for ePassport reader conformance testing. This project was a joint effort between the European Joint Research Centre (JRC) in Ispra (Italy) and ETSI in Sophia Antipolis (France). The test language TTCN-3 has already been widely used in many testing projects across different protocols and technologies. However, until this project TTCN-3 has not been applied in the area of contactless smart card testing.

The ETSI Specialists Task Force (STF) 400 with experts from the companies / organisations AMB Consulting, ARH, Comprion, ETSI, FSCOM, HJP Consulting, Masaryk University, Soliatis and Testing Technologies operated this project. The work of this STF has been split into three main phases:

  1. Design, implementation, and use of ePassport test system
  2. Development of ePassport testing framework
  3. Writing of the documentation and dissemination material

Scope of this project was to build a test system to test an inspection system typically used to read ePassports. To demonstrate the basic functionality and the feasibility, a subset of BSI TR-03105 Part 5.1 was specified and implemented in the test system.

The following image describes the architecture of the ePassport reader test system developed during this project:

System architecture of prototype to test ePassport Reader with TTCN-3

Architecture of test system based on TTCN-3 for ePassport readers (Source: ETSI White Paper No.7)

The most significant part in the architecture is “TTCN-3 Test Component”. This module simulates the ePassort behaviour  by receiving APDUs, react to these commands and send result in APDUs back to the SUT (here the ePassport reader).

The successful implementation of a TTCN-3 based test system shows that the use of TTCN-3 fits the requirements of conformance testing of eMRTD or other eID systems. The prototype demonstrates the feasibility of using such formal techniques for protocols which would improve the quality and repeatability of tests and reduce room for interpretation.

An overview of this project and the results were summarized by the colleagues Jean-Marc Chareau, Laurent Velez and Zdenek Riha in ETSI White Paper No 7.

 

 

Test tool overview

The colleagues of Imbus started a new platform with a list of several test tools in various categories. You find there both commercial and open source solutions of test software. The list is available in German and also in English.  All listed tools are classified as following:

  • Code and coverage analysis
  • Continuous integration
  • Defect and change management
  • Load and performance test
  • Test automation
  • Test management
  • Test specification and generators

You are invited to submit your tool and to interchange experiences and tips on the platform. Thanks to Imbus for this great and helpful overview!