- 1 Introduction to CAMSS
- 2 Common Assessment Method for Standards and Specifications (CAMSS)
- 3 Guidelines on the Use of CAMSS
- 4 CAMSS Sustainability and Future Objectives
Introduction to CAMSS
Background of CAMSS Project
While public administrations have specific characteristics at the political, legal, and organisational levels, semantic and technical interoperability is based mostly on formal specifications. Within the context of the elaboration of their National Interoperability Frameworks, Member States are currently organising the assessment of formal specifications, in order to establish their national recommendations. Deciding on the recommended formal specifications often calls for a resource intensive and time consuming assessment.
Within the Interoperability Solutions for European Public Administrations (ISA) programme, an action is defined on the development of a Common Assessment Method for Standards and Specifications (CAMSS). This common assessment method would provide guidance on the assessment of ICT standards and specifications when defining ICT architectures and establishing European public services. The assessment method would also ensure transparency and openness of the assessment process which would lead to better decisions.
As defined in the CAMSS action, the purpose of the common assessment method will be:
- - To ensure that assessments of formal ICT specifications and interoperability profiles are performed to high and consistent standards;
- - To ensure that assessments will contribute significantly to confidence in the interoperability of systems implementing these specifications and profiles;
- - To enable the re-use, in whole or in part, of such assessments;
- - To continuously improve the efficiency and effectiveness of the assessment process for ICT formal specifications and interoperability profiles.
The main objectives of the second iteration of CAMSS were to revise the CAMSS process and criteria in order to launch subsequently an official version, and to forster its adoption by the various stakeholders. Regarding the method, this entailed a neutral and unbiased method comprising a reference process, a set of quality criteria and a plan for an assessment library to assist Member States and EU institutions in their assessments of formal specifications needed in the development of interoperable national and cross-border eGovernment services.
The detailed objectives of the CAMSS project can be formulated as:
- - To assess the needs for the revision and improvement of the previous CAMSS
- - To study and analyse relevant ongoing actions in the European Commission, Member States and other relevant countries and organisations outside of the EU
- - To define the development objectives for the improvement of the previous CAMSS
- - To develop the process, the set of criteria and the plan for an assessment library constituting the revised CAMSS
- - To define the sustainability and future objectives for CAMSS
Recently, the European Commission has proposed a regulation on European standardisation and the recognition of technical specifications in the field of ICT . This regulation stipulates that the Commission may decide to recognise technical specifications as ICT standards, that are not national, European or international standards, which meet the requirements set out in Annex II of the proposed regulation. It will allow the use, in the field of public procurement, of ICT standards developed by fora and consortia or by organisations other than the European standardisation bodies. The requirements which are set out in Annex II of the proposed regulation on European standardisation should be used to align the criteria revision and development of CAMSS.
Furthermore, the planned actions for the Digital Agenda include a specific action (Action 23) to provide guidance on the link between ICT standardisation and public procurement in order to help public authorities in using standards. In this action, the Commission will draw up detailed guidelines on how to make best use of ICT standards in tender specifications. A revised CAMSS method could assist in this selection of possible ICT standards to be used. Finally, the European Interoperability Framework (EIF) defines the underlying principles of European public services and the guidelines on the assessment and selection of formal specifications which should be taken into account in the development of a revised CAMSS.
Status of CAMSS
As communicated by the Commission in 2004, it was recommended to continue to make more extensive use of European standardisation in European policies and legislation, to improve the efficiency, coherence, visibility of European standardisation and of its institutional framework. Within the ICT Standardisation Work Programme of the Commission, different action plans and domains are defined to cover for the standardisation needs in the following years.
During 2008, Phase 1 of the project produced a proposal for a CAMSS. This proposal for a CAMSS focused principally on defining assessment criteria, based on Member States best practices and agreement among Member State experts. Following a presentation on CAMSS Info Day (28 May 2008) and public consultation during summer 2008, the Proposal for a CAMSS received positive and constructive comments from industry, standardisation bodies and public administrations.
Phase 2, initiated beginning 2009, aims at executing pilot assessments based on the CAMSS proposed end of Phase 1. Through lessons learned, Phase 2 further shapes the CAMSS by updating the CAMSS criteria and defining the CAMSS process. Phase 1 produced criteria with different levels of detail, some are very precise (standardisation criteria), others are needing further defining. A first pilot was run during spring 2009, which provides a "proof-of-concept" by answering a business need with low complexity and narrow scope.
This first unofficial version of CAMSS was developed by the Commission and Member States under the IDABC Programme, the predecessor of the ISA Programme. Through discussions with Member States, standardisation bodies and other stakeholders the Commission has established a need for revising the first version of CAMSS and developing a widely adoptable official version.
The main objectives of the second iteration of CAMSS were to revise the CAMSS process and criteria in order to launch subsequently an official version, and to foster its adoption by the various stakeholders. In May 2012, the revised version of CAMSS was finalised and validated by the Commission and Member States.
Positioning of CAMSS
The objective of the common assessment method for standards and specifications (CAMSS) is to establish a neutral and unbiased method for the assessment of formal specifications and standards in the field of ICT. The expected beneficiaries of the CAMSS are the public administrations in the Member States and EU institutions, standardisation bodies and IT services industry. For these beneficiaries, the CAMSS aims at providing a best practice (common) assessment method and guidance for the assessment of ICT standards and formal specifications when defining ICT architectures and establishing European public services.
The CAMSS consists of a documented reference assessment process, a set of quality requirements (criteria), and an assessment library and tools. For CAMSS, the assessment process and set of quality requirements are developed using best practice assessment methods at national level and by aligning with related initiatives at European level (e.g. proposed regulation on European standardisation). In order to provide a clear positioning of the CAMSS, the following figure provides the relationships between the assessment methods at national level and the initiatives at European level.
Already existing (national) assessment methods can be adapted and aligned with the CAMSS, in order to ensure the sharing and reuse of the assessment results. If no formal assessment method is available or defined, the public administrations in the Member States and EU institutions should preferably adopt the CAMSS as their (national) assessment method. However, the CAMSS could be implemented and adapted based on the specific needs and requirements of the administrations.
Using the CAMSS at Member States and EU institutions ensures that the assessments of ICT standards and formal specifications are performed to high and consistent standards. The assessments will contribute significantly to confidence in the interoperability of systems implementing these standards and formal specifications. It brings transparency to the selection of standards in the context of ICT strategies, architectures and interoperability frameworks. Furthermore, it enables the reuse and sharing (in whole or in part) of already performed assessments and the continuous improvement of the efficiency and effectiveness of the assessment process.
The CAMSS focuses on the assessment of standards and specifications for the adoption by public administrations. It does not focus on the evaluation of the actual content of the standards or specifications. The CAMSS is not proposed as a mandatory assessment method to be implemented at Member State level. Also, it is provided as a best practice method at European level for Member States. It should not be considered as an approved or regulated European assessment method for the assessment of standards and specifications at European level.
Common Assessment Method for Standards and Specifications (CAMSS)
Scenarios for Assessment of Formal Specifications
From the analysis of the current situation and the discussions during the first workshop, concrete scenarios were discussed and described for the assessment of formal specifications in public administrations. From these discussions, the following assessment scenarios were identified and selected as relevant for the revision of CAMSS.
- Assessment scenario 1 – An assessment of a standardisation organisation
- A standardisation organisation is assessed in order to evaluate certain criteria that are relevant for the adoption of formal specifications that are put forward by these standardisation organisations. This assessment scenario is mainly triggered by the public administrations. The outcome for this assessment scenario will be a reusable assessment and evaluation of a standardisation organisation.
- Assessment scenario 2 – An assessment of a formal specification for adoption by public administrations
- A formal specification is assessed in order to evaluate and provide a recommendation on the possible adoption of the formal specification by the public administrations. This assessment scenario can be triggered by a public administration or related external stakeholders. The outcome for this assessment scenario will be the adoption of a certain formal specification by the public administrations.
- Assessment scenario 3 – An assessment and selection of formal specifications for specific business needs and requirements
- For specific business needs and requirements, a proposed set of formal specifications are assessed and evaluated in order to select and adopt the most relevant formal specification. If a certain business need arises and requires the adoption of a relevant formal specification, the business need should be examined to list the relevant requirements, and based upon those requirements, a selection of relevant formal specifications may be established. This assessment scenario is mainly triggered by the public administrations. The outcome for this assessment scenario will be the selection and adoption of relevant formal specifications for the specific business needs and requirements.
From the analysis of the current situation and the descriptions of the different assessment scenarios, common steps were identified for an assessment process. These common steps are selected and described for the high-level CAMSS process and are shown in the following figure.
From the analysis of the current CAMSS and relevant ongoing actions, a common set of roles and responsibilities involved in the assessment of formal specifications was identified and described.
The detailed description of the CAMSS process provides the objective, description, roles, input and output for each of the process steps (including the proposal step, consideration step, assessment step, recommendation step, authorisation step and adoption step). For the assessment step, a detailed set of activities and roles are provided.
For the different steps in the revised CAMSS process, specific quality criteria are used to evaluate and select formal specifications. In the following sections, the specific CAMSS criteria for the proposal, consideration, assessment and recommendation step are provided and described. The criteria for the proposal, consideration and recommendation step are provided as guidelines to facilitate the CAMSS process and could be adapted at Member State level. The criteria for the assessment step are considered to be neutral and objective, and should be provided in the CAMSS library for possible reuse. The CAMSS assessment log criteria are described that provide the general information of the organisation performing the assessment process. The specific assessment criteria for CAMSS are aligned with the proposed regulation on European standardisation (see Annex II) in order to increase consistency between European assessment methods.
As described for the CAMSS process, a specific scenario (assessment scenario 1) is defined for the assessment of a standardisation organisation. However, the assessment of a standardisation organisation would not include all the criteria applicable for the assessment of a formal specification. Therefore, the relevant criteria for the assessment of a standardisation organisation are marked in the following section with an ‘O’ in the column ‘Org.’.
- Details of CAMSS Proposal Criteria
- Details of CAMSS Consideration Criteria
- Details of CAMSS Assessment Criteria
- Details of CAMSS Recommendation Criteria
- Details of CAMSS Assessment Log
CAMSS Assessment Library and Tools
The CAMSS assessment library and tools are provided in order to facilitate the assessment process, in sharing and reusing finalised assessments and in providing ready-to-use templates. In the first section, the possible involvement of the assessment library in the assessment process is described. Furthermore, the set-up, governance and guidelines for the use of the assessment library are documented. In the second section, the proposed assessment tools are described which are provided as separate templates in order to facilitate the assessment process and the completion of the criteria in the revised CAMSS.
Please note that the CAMSS community on the Joinup platform is not established at this moment. Furthermore, the proposed assessment tools are provided as an example spreadsheet and should not be considered as officially approved.
Guidelines on the Use of CAMSS
In this chapter, concrete and practical guidelines are described for the use of CAMSS by public administrations. The guidelines are provided in order to answer the questions of why you should use CAMSS, who should use it and how you should use it.
Why Should You Use CAMSS?
Within the scope of interoperability and ICT systems, public administrations need to decide on the specific standards and specifications to be used and accepted for the implementation of these systems. As a best practice, public administrations could assess the different standards and specifications according to specific criteria in order to make an informed and objective decision on their acceptance or rejection. A method for assessing standards and specifications should be defined and agreed within the public administrations which takes into account their specific needs, requirements and priorities.
Based on the different assessment methods that are already established in Member States and the relevant regulations at European level, a common assessment method is developed at European level. The Common Assessment Method for Standards and Specifications (CAMSS) is described in detail in this document and aims at improving and aligning the different assessment methods that are (or will be) established in Member States. The CAMSS method defines a specific set of quality criteria which largely align with the general European requirements for the recognition of technical specifications as ICT standards. However, the CAMSS method does not provide criteria to assess the technical requirements of a standard or formal specification.
From the identified assessment scenarios, the CAMSS method could be used by public administrations (or other interested parties) in the following situations:
- - The assessment of a standardisation organisation in order to adopt standards and formal specifications that are put forward by these organisations
- - The assessment of a standard or formal specification in order to adopt it within public administrations
- - The assessment of standards or formal specifications for specific business needs and requirements in order to select and adopt the most relevant standard or formal specification
A practical example could be the selection of standards for the implementation of web services, such as the SOAP protocol or REST specifications. Public administrations can evaluate and select the SOAP or REST specification based on the results of a CAMSS assessment. For both SOAP and REST, the set of quality criteria are completed in order to recommend the best option for the implementation of web services.
Who Should Use CAMSS?
The CAMSS method is mainly aimed at public administrations working in the field of ICT and interoperability. For these public administrations, a best practice process is described, including a set of quality criteria, for the assessment of standards and formal specifications. At European level, an assessment library will be established to share best practices and reuse already available assessment results between public administrations. Furthermore, standardisation organisations or public communities could use the CAMSS method in order to evaluate their performance on the set of quality criteria.
How Should You Use CAMSS?
As described in detail in this document, the CAMSS process consists of a collection of process steps, starting from the proposal step to the consideration step, the assessment step, the recommendation step, the authorisation step, and finally the adoption step. These process steps need to be implemented and adapted by the public administration, taking into account the specific needs and requirements.
For the assessment of the SOAP and REST specifications, the public administration will initiate the first step in the CAMSS process which is the proposal step. In this first step, the business needs are defined for the implementation of web services and the relevant standards and specifications are identified (e.g. SOAP and REST). Furthermore, the necessary roles and responsibilities are assigned within the public administration in order to perform the process steps. These roles include the following:
- - An initiating administration
- - An assessment panel
- - A decision making body
- - External stakeholders (e.g. general public, standardisation organisations, interest groups or communities)
The European Commission is involved in the assessment process in supporting and providing the assessment library.
Next, the public administration will shift its responsibility to the assessment panel for performing the actual assessment of the SOAP and REST specifications. Relevant documentation needs to be gathered in order to evaluate and answer the set of quality criteria. This documentation could be publicly available (e.g. on the W3C website or Wikipedia) or should be requested from the specific standardisation organisation itself (e.g. W3C).
The standardisation organisation W3C could be involved in this process in providing additional input or validation of the answers to the criteria. Organisations that already used or implemented the SOAP or REST specifications could provide useful input on the availability of the documentation, the difficulties of implementing the technical specifications, or dependencies on specific technologies, etc. A public consultation could be organised to collect feedback on the performed assessments of both the SOAP and REST specification.
Based on the results of the assessments, a comparable score will be generated for the different categories of the quality criteria (including applicability, maturity, openness, intellectual property rights, market support and potential). As already pointed out, the CAMSS assessment will not evaluate the technical requirements or performance criteria for the SOAP and REST specification (e.g. processing lead times). In taking the CAMSS assessment results, the assessment panel will produce a recommendation for the SOAP and/or REST specification for its adoption by the public administration in the context of implementing web services.
The decision making body within the public administration will take a decision on the adoption of the SOAP and/or REST specification in the context of implementing web services. The adoption of the specification needs to be communicated and executed within the related public administrations that are implementing web services. Specific guidelines should be provided by the public administration on the execution of this adoption of the SOAP and/or REST specification.
CAMSS Sustainability and Future Objectives
The objective of this section is to provide an initial definition of the sustainability and future objectives of CAMSS. This means the sustainability and future objectives of CAMSS after the development and establishment of a widely adoptable official version in this CAMSS project. As a widely adopted official version, the CAMSS method will offer several benefits for Member States as well as the Commission and other interested parties in the promotion of interoperability including the following:
- - It provides a clear guideline that allows the assessments of formal specifications to be made consistently and in high quality by applying neutral, unbiased and transparent criteria.
- - It helps Member States to make interoperability agreements based on formal specifications, by providing a common assessment method for these needs.
- - It helps Member States to assess formal specifications in public procurement.
- - It allows for re-use of assessments, in whole or in part and consequently eliminates duplication of effort, as already finalised assessments of formal specifications can be shared.
- - The assessed formal specifications will contribute to the interoperability of systems implementing these specifications.
- - It allows for more efficient use of public funds, because of easier and faster assessments and a CAMSS library for formal specifications meant for similar business needs.
- - It improves the expertise of civil servants working with formal specifications.