ESSnet Big Data
ESSnet Big Data is a project within the European statistical system (ESS) jointly undertaken by 28 partners. Its objective is the integration of big data in the regular production of official statistics, through pilots exploring the potential of selected big data sources, and through building and implementing concrete applications.
ESSnet Big Data II has started in November 2018 and is to run for 26 months until December 2020. It is a continuation of ESSnet Big Data I (from February 2016 until May 2018) and consists of 12 workpackages, A to L. Apart from WPA Coordination supporting and coordinating the project overall, these are grouped into an 'Implementation track' covering WPB to WPF, a 'Pilots track' covering WPG to WPK and the stand-alone workpackage WPL on smart statistics.
- 1 Background
- 2 General objectives
- 3 Specific objectives
- 4 Expected impact
- 5 Implementation projects
- 6 New pilot projects
- 7 Preparing for trusted smart statistics
The “ESSnet on Big Data 2018-2020” is the envisaged successor to the “ESSnet Big Data” that existed from January 2016 to May 2018. In this document these ESSnets will be denoted by “ESSnet Big Data II” and “ESSnet Big Data I”, respectively. Both ESSnets have their foundation in the Big Data Action Plan and Roadmap (BDAR), which was adopted by the ESSC on 26 September 2014. The BDAR itself was conceived in the “Scheveningen Memorandum on Big Data and Official statistics”, which was adopted by the ESSC on 27 September 2013. In view of the considered ESSnet Big Data II, the business case “Smart Statistics and Big Data” was developed and adopted by the ESSC on 8 February 2018.
The ESSnet Big Data II will build on the results of the ESSnet Big Data I. That ESSnet was based on a Framework Partnership Agreement (FPA 11104.2015.006-2015.720), in the context of which two Specific Grant Agreements (SGA-1 and SGA-2) were signed and carried out (numbers 11104.2015.007-2016.085 and 11104.2016.010-2016.756, respectively). The results of that ESSnet pertain to seven pilot projects, which were carried out in workpackages (henceforward: WP = workpackage) and one cross-cutting WP. The pilot projects were part of both SGAs, the cross-cutting WP was launched in SGA-2. Thus, the ESSnet Big Data had eight WPs, of which the eighth concerned the cross-cutting subjects :
- WP 1: Webscraping - job vacancies
- WP 2: Webscraping - enterprise characteristics
- WP 3: Smart meters
- WP 4: AIS data
- WP 5: Mobile phone data
- WP 6: Early estimates
- WP 7: Multiple domains
- WP 8: Methodology
In addition, activities were carried out concerning coordination, communication and dissemination. The ESSnet Big Data II will also build on the products of, and experienced gained in, those activities, and also on the big data community that ensued. In particular, a Mediawiki-based wiki was set up in which all project participants could edit and which anyone may freely consult. Among other things, this wiki contains all the results of the ESSnet Big Data I. It also contains a final report which summarizes all results. Of course, the results can also be accessed through CROS-portal.
The overall objective of the ESSnet Big Data II is to further prepare the ESS for integration of big data sources into the production of official statistics. The strategic, practical and urgent need for this is explained in the Scheveningen Memorandum and the BDAR. Given the nature and diversity of big data sources, this objective requires research into the possibilities of and impediments to using these sources. Thus, the previous ESSnet investigated aspects of data access, processing and dissemination including in particular conditions for access and use, the development of methodology, quality and technology, and the production of experimental outputs. These aspects were looked at for the specific data sources and domains of the pilots, and at a more general level to identify communalities and good practices.
The ultimate goal, of course, is to have actual implementation of big data sources into the production of ESS statistics. The research outcomes of the previous ESSnet allow this step to be taken for several of the big data sources and domains, but not yet for all. Moreover, several other potentially relevant big data sources have not been investigated yet for the ESS, therefore there is a need to continue with doing research next to starting with the actual implementation of earlier results. This will, once again, be carried out in the form of pilot projects, as this approach has been shown to be effective in the previous ESSnet.
The approach so far fits well in the traditional way of statistics production, in that big data sources can be seen as adding to the survey and administrative sources already available to NSIs, even though this may give rise to many new challenges. However, the emergence of potentially relevant big data sources associated with the Internet of Things (IoT) may necessitate a new approach that diverges even more from the traditional way of statistics production. This has to do with, among other things, the more distributed character of such sources and the desirability to tap them in a smart way, if they are ever to be utilized on a large scale in official statistics. This new approach, of which the development has only recently started, is denoted by the term “trusted smart statistics”. This will be a major challenge for the ESS in the coming years. The ESSnet is also meant to prepare the ground for future actions in this field.
Thus, the new ESSnet will have three strands or tracks:
- Implementation projects.
- New pilot projects.
- Preparing for trusted smart statistics.
The specific actions (WPs) for each of the tracks are explained in section 1.2.
In addition to the general objectives and related actions, there are also horizontal, cross-cutting and other specific objectives and actions, as was the case for the previous ESSnet, although they are somewhat different from the previous ones. The following may be considered the most important:
Since implementation is based on already developed methodology, the efforts will be mainly aimed at the new pilot projects. The aim is to go one step further than in the previous ESSnet and produce results in a more user-oriented way: not so much an analysis of methodology encountered in the projects but rather providing solutions to potential users (statisticians) struggling with specific issues. A specific subject that will be given attention is combining data sources.
This is closely linked to methodology, which normally addresses quality targets or conditions. Again, the aim is to go one step further than in the previous ESSnet and formulate a quality framework. As the ESS already has a functioning quality framework, work on this will be coordinated with Eurostat and relevant groups.
Process and Architecture
When implementing the results of the previous ESSnet, statistical processes have to be designed within an architectural framework. Although this may be very country specific, elements will presumably be determined by the big data source at hand and the domain of application, and for wider use in the ESS care will be taken that the relationship with international standards such as GSBPM will be elaborated. The same is the case for metadata. This will also be coordinated with Eurostat and relevant groups.
- IT and infrastructure. As was the case in the previous ESSnet, IT is important to the subjects of the pilot projects. However, it may be even more important to implementation, where integration in or linking with existing systems may be an issue. In the new ESSnet, IT will mainly be considered in connection with process and architecture. Concerning infrastructure, the BDTI (Big Data Test Infrastructure) initiative of the European Commission is especially relevant, since it aims at designing a test infrastructure for public administrations at large. As this still has to be implemented, there is a window of opportunity to help shaping this in a way that meets the needs of the ESS best. The ESSnet will assume an assessing and defining role.
Of course this list is not exhaustive. Other relevant issues include privacy and ethics, legal aspects, skills, the development of partnerships – in particular with data holders – , and data governance. However, given the available resources and already high ambitions, there is a need to prioritize. These other relevant issues will be treated in the context of the projects to which they are most pertinent rather than in cross-cutting projects.
A specific – and crucial – objective of another nature is the external communication and dissemination of the results, in particular within the ESS. This is implicit in the general objective of preparing the ESS for integration of big data sources into the production of official statistics. As was the case in the previous ESSnet, a wiki will be used for internal as well as external communication (in addition to CROS-portal), and a major dissemination conference is foreseen towards the end of the ESSnet. Moreover, as the previous ESSnet has shown, those involved in the ESSnet in fact constitute a community. The new ESSnet will reinforce this community and look at sustainability after the end of the grant period (see also section 1.6). The approach to communication and dissemination is further described at several places in the remainder of this chapter.
For assessing the expected impact of the ESSnet, the following factors may be deemed relevant:
- Integration of big data sources into the production of official statistics in the ESS requires involvement of a significant number of ESS members. The ESSnet will have 23 NSIs as partners, plus 5 other ESS members. Of the 23 NSIs, 21 are from the EU (the other two are from CH and NO, respectively) and they include, without exception, the ten largest members of the EU (in terms of GDP as well as population). Also at the level of each of the three tracks (implementation projects, new pilot projects and preparing for trusted smart statistics), a significant number of ESS members are involved.
- The three-track approach maximizes the impact by having earlier research results implemented in parallel with doing new research. Going from research to implementation is generally recognized to be difficult, and the timeframe of the ESSnet is too short to have full implementation of new or adapted statistical processes. Important indicators of success will be the successful development of production prototypes by NSIs and the expectancy (ideally the non-reversibility) of the finalization of the implementation projects at the national level after the end of the ESSnet.
- The impact of the ESS should not only be visible at the level of the individual NSIs, but also at the level of the ESS as a whole. The actions and deliverables of the ESSnet have been designed in such a way that the ESS dimension is fully expressed. Care has been taken to draft a programme of actions and deliverables that is not simply the sum of national endeavors but is aimed at collaboration and usability of results throughout the ESS.
- The impact of the ESS is highest if optimal use is made of available knowledge. Care has been taken to make use of and link to what has been developed already, in the previous ESSnet as well as elsewhere (e.g. in the context of the Global Working Group on Big Data for Official Statistics ). Moreover, not only big data expertise is used, but also other expert knowledge, such as domain knowledge from subject matter experts.
1.2 - Description of the action on the basis of the main activities planned and where it will be implemented Present the overall strategy and description of the different tasks and deliverables to be achieved during the whole duration of the action and, if applicable, the distribution of the partners in each task. Show the soundness of the concept and of the methodological approach and, if necessary, software developments proposed in the action. In case of a multibeneficiary project, explain how the composition of the partnership is well-balanced in relation to the objectives of the action. For each of the tasks proposed, provide details on the objectives, the work to be achieved, indicators to measure progress for reaching the proposed objectives and the added value of the work proposed in regards of the work already carried out by the partners (ESSnet criteria 4 for ESSnet projects).
In total 28 ESS members have expressed interest in participating in the ESSnet, not all of them being NSIs and not all of them being eligible for reimbursement. On a voluntary basis two countries will participate without reimbursement; they will contribute to deliverables. With all 28 partners agreement has been reached about inclusion in this proposal, their expected contribution and the resources involved. These are the partners:
- NSIs with reimbursement (20): o AT o BE o BG o DE o DK o EE o ES o EL o FI o FR o IT o LT o NL o NO o PL o PT o RO o SE o SI o SK
- NSIs without reimbursement (3): o CH o IE o UK
- Non-NSIs (5):
- From DE: Office for Statistics Berlin-Brandenburg (henceforward: “Berlin”) Hesse Statistical Office (henceforward: “Hesse”)
- From FI: Natural Resources Institute Finland (henceforward: “LUKE”)
- From FR: Ministry of Labour, Employment, Vocational Training and Social Dialogue (henceforward: “DARES”) Ministry of Agriculture, Agrifood Industry and Forestry (henceforward: “SSP”)
The expressions of interest in the possible subjects for each track have influenced the choice of WPs for each track. In addition, there will be a WP for Coordination and Communication, described in section 1.9. An overview of participation is given at the end of this section.
In a number of cases the participating ESS members collaborate with other partners, such as specialised institutes or subcontractors. Where this is the case, this is mentioned in the description of the WPs in section 1.9.
For implementation, functional production prototypes will be developed related to statistics on
- online job vacancies;
- enterprise characteristics;
- measuring electricity consumption, identifying energy consumption patterns;
- maritime and inland waterways statistics, environmental statistics.
These will build directly on the results of the previous ESSnet. For each of these four, there will be a workpackage (WP; between brackets the number of partners participating in it): WPB Online job vacancies (12), WPC Enterprise characteristics (9), WPD Smart energy (4) and WPE Tracking ships (3)
In addition, there will be a workpackage for cross-cutting issues, which will cover issues concerning the design of the statistical process and architecture, but also other issues with which the implementation workpackages will be confronted, such as IT and infrastructure: WPF Process and architecture (8).
The aims and approach of the workpackages are briefly described below.
WPB Online job vacancies
The aim of implementing this pilot is to produce statistical estimates in the statistical theme of online job vacancies. Suitable techniques and concrete methodologies were developed during the pilot phase of the project. Implementation will be based on work that was carried out regarding the conditions that web scraping techniques can be used as far as the quality of the scraped data is concerned, as well as the use of mixed sources including job portals and job adverts on enterprise websites, and job vacancy data from third party sources. Relationships with the project of Cedefop will be consequently explored. Within the same statistical theme, the combination of existing data from multiple sources will be promoted and embedded in the methodology. The final aim of the activities is to develop and test the methodology and prototypes as well as capacity building to facilitate their integration into production at the level of individual NSI and at the level of the ESS.
WPC Enterprise characteristics
The aim of WPC is to use web scraping, text mining and inference techniques for collecting and processing enterprise information, in order to improve or update existing information, such as Internet presence, kind of activity, address information, ownership structure, etc., in the national business registers. The implementation involves massive scraping of company-websites, collecting, processing, analysing unstructured data and dissemination of national-level experimental statistics. The enterprise data obtained by this WP is combined with existing data from multiple other sources, such as administrative data and ICT usage in enterprise surveys, in order to maximize the quality and quantity of the statistical output.
WPD Smart energy
The use of smart electricity meter data and applying appropriate analytical methods will enable the ESS to produce new kind of statistics - smart statistics. The European Commission has proposed a deployment plan for smart electricity meters in the EU Member States on the basis of economic assessments of long-term costs and benefits and foresees to achieve almost 72% deployment rate by 2020. The aim for this WP is to implement smart meter data for producing statistics. This can be as a supplement to other statistics e.g. energy statistics of businesses, census, household costs, tourism seasonality or impact on environment. The implementation will include linking electricity data with other administrative sources for eventually producing statistics of businesses, households and identifying vacant living places or seasonal/temporary occupancy of living places. Work will include the identification of energy consumption patterns in households. Each participating country will produce implementation procedures for statistical products.
WPE Tracking ships
Taking into account the developments of the successfully concluded work in the previous ESSnet programme during 2016-2018, the aim of this WP is to develop functional production prototypes including setting up procedures and developing technical solutions, to promote and support the collection, processing and analysis of (big) data from AIS (Automatic Identification System) for statistical production in the participating NSIs related to statistics on maritime, inland waterways and environmental statistics. The output of the work should become available to the ESS and adequately enable other NSIs, in the next implementation phase, to put in place the necessary system(s) and deploy or adapt the proposed solution(s) in order to produce statistical output. The outputs will include key artefacts like target architecture definition and requirements and guidance for the next implementation and deployment phase, hence producing specifications for full-fledged implementation at the ESS level. Along the production of the relevant methodologies, recommendations, specifications and statistical software, the production of experimental statistics demonstrating the capabilities to produce statistics is a prominent objective.
WPF Process and architecture
The aim of WPF is the definition of reference architectures necessary to carry out big data production both at national and European levels. The defined architectures will span from the logical layer to the technological one. The logical architectures will serve the main purpose of sharing common data representations as well as services and processes’ design. The EU-level and national-level dimensions will be identified and cost and quality effective architectural scenarios will be designed. The technological architectures will be strictly connected to the logical architectures and will provide concrete solutions on which NSIs could rely for their own big data production.
New pilot projects
Four new pilot projects will be carried out, resulting in experimental statistics. They concern (1) use of financial transactions data, (2) use of remotely sensed data (e.g., satellite images, in-situ measurements, etc.), (3) use of mobile network operator data and (4) use of innovative sources and methods for tourism statistics. Creating a pilot project for the use of online platforms such as social media and sharing economy platforms was also considered, but for this subject the interest among the partners was considerable less than in the other subjects, and the number of WPs would be rather big already. Moreover, these platforms can also be looked at in the context of the first and the fourth pilot. Some of the pilots, in particular the third and the fourth, will build on the results of the previous ESSnet. For these pilots, where relevant the work done by the Global Working Group mentioned earlier will also be taken into account. For each of the four pilots, there will be a WP (in brackets: the number of partners):
- WPG Financial Transactions Data (6)
- WPH Earth Observation (9)
- WPI Mobile Networks Data (9)
- WPJ Innovative Tourism Statistics (8)
In addition, there will be a WP for cross-cutting issues, which will cover in particular methodological and quality issues, but also other issues with which the pilot WPs will be confronted:
WPK Methodology and Quality (6)
These WPs are described in detail in section 1.9, but their aims and approach are the following:
WPG Financial transactions data
The main aim of the WP is to get an overview of the sources and the data infrastructure (metadata) of financial transaction data in the countries participating in this WP. The objective is to describe to what extent FTD are available as well as whether it is possible for NSIs to access them. Given the infrastructure it is also a main aim of the WP to assess the statistical potential of these data sources. This may be for improving the existing quality or for quality evaluations of some currently produced statistics, or it may be for a completely new portfolio of statistical products. To do this, empirical studies will be carried out. Given that the relevant financial transaction data is acquired in time for this WP, empirical investigations should underpin conclusions about potential and implementing statistics based on such data sources.
WPH Earth observation
Earth observation (EO) creates an unprecedented advantage in Europe and the World for the development of operational applications of remote sensing providing an enormous dataset. Especially the launch of the Sentinels from the Copernicus Programme opened a new chapter in applicability of remote sensing data ensuring free, open access, continuous and systematic acquisition of the satellite images. One of the important economic and commercial applications of EO data is official statistical production and landscape mapping for variable thematic purposes. The use of EO data is particularly promising in the perspective of the upcoming Census 2021 and Agricultural Census as well as other commitments of European Commission or United Nations. The crucial goal of the project is the usage of the EO data from different sources that will contribute to build the geospatial framework to support this. Whitin this project the usefulness and practical usage of EO data in order to fill the gap between statistical and geographical information named as “geospatial breakdown” will be proposed. The objective of the project will be achieved through the elaboration of the several thematic fields like agriculture, build-up area, land cover and settlements, enumeration and forestry.
WPI Mobile networks data
The main objective of the WPI on mobile phone data is to carry out the construction of a production framework already initiated in the previous ESSnet. A modular instead of a linear approach will be followed both to avoid potential blockings in the research due to the obstacles in accessing real data and to optimally develop the different elements in the production framework. The modular approach will enable the project to concentrate on many different aspects of the framework such as the data access itself and the relationship with Mobile Network Operators. Other important aspects are the methodological and IT elements (thus also enabling the identification of necessary skills and capabilities), quality issues as well as proposals for standards and related metadata and dissemination aspects of statistical products (mainly visualization thus orienting the outputs towards stakeholders). As a novelty, the use of instrumental (semi-simulated) data closely resembling real data (when these are not accessible) will be used in this line of work. All in all, the ultimate goal is to push forward an ESS production framework with mobile phone data aiming at a standardised statistical production process.
WPJ Innovative tourism statistics
The main objective of the package is to address the need of a conceptual framework and setting up a smart pilot Tourism Information System that will support statistical production in the field of tourism by integrating various big data sources with administrative registers and statistical databases using innovative statistical methods. Specific objectives are:
- Identification and evaluation of availability and quality of big data obtained with the use of various methods;
- Developing methods of combining data and their spatio-temporal disaggregation;
- Developing a prototype of a solid production system for tourism statistics;
- Recommending which of the identified and improved data sources produce good quality tourism estimates;
- Recommending how to set up a smart and robust prototype of Tourism Information System.
WPK Methodology and quality
The aim of this WP is to consolidate knowledge gained in this ESSnet (and with limitations outside, in academia and the non-ESS official statistics community) in the area of methodology and quality when using big data in the statistical production process and combine it with the insights from the previous ESSnet. Several key deliverables from the previous ESSnet will be used, extended and enhanced, especially:
- the literature overview;
- an overview of quality issues and their possible solutions;
- an overview of methodologies applied and challenges identified.
To achieve a high degree of acceptance, feedback from relevant ESS bodies (e.g. the TF/SG BD, WG Qual, WG Meth, DIME) will be collected for key envisaged deliverables (especially the guidelines and the template).
Preparing for trusted smart statistics
For trusted smart statistics, several cases might be explored, for instance related to the use of citizen science data, smart cities, connected vehicles or smart farming. However, in line with the BDAR several actions beside the new ESSnet are foreseen concerning trusted smart statistics, starting already in 2019. Therefore the efforts will be directed at preparing for future actions rather than doing in-depth studies that take a long period to carry out. Only one WP will be started, and it will have a relative short duration of 12 months, so that its results can still be used for the preparation of actions (outside the ESSnet) that can start in 2019. Nevertheless, when exploring the subject, a number of cases will be looked at in order to generate findings that are rooted in reality. Furthermore, as is the case with all WPs, use will be made of material available outside the ESSnet. For instance, in April 2018 a trusted smart statistics workshop was held in the Netherlands with members of the ESS Task Force on Big Data, the results of which can be used. Thus, this track will have the following WP (in brackets: the number of partners): WPL Preparing smart tatistics (12)
The aims and approach of this workpackage are briefly described below (see here for a more extensive description]].
WPL Preparing smart statistics
The aim of WPL is to explore the extended use of the Internet of Things (IoT) in order to produce trusted smart statistics. As the range of topics regarding the subject IoT is huge, the goal of this WP is to provide an overview of relevant topics for official statistics and to show their variety. As this WP is supposed to set the ground for further studies, another outcome will be to highlight topics that are promising and could be analyzed further with regards to a feasibility study. Therefore, the availability and accessibility of the different data sources will also be checked. The WP will explore how the digital footprints of daily life created by human wearables, city and vehicles sensors and other smart systems could change the way to produce trusted smart statistics taking advantage of societies’ datafication. All the results of this WP will reflect that the overall goal is the development of generic solutions and harmonized as well as standardized approaches and recommendations for the European Statistical System (ESS).
In summary, the following WPs are foreseen with the partners and leading NSI as indicated:
WP WP name Participating partners Lead
WPA Coordination and Communication BE, FI, NL, NO, PL, SE NL WPB Online Job Vacancies BG, CH, DARES, DE, IE, IT, LT, PL, RO, SE, SI, UK SI WPC Enterprise Characteristics AT, BG, DE, FI, IE, IT, NL, PL, UK BG WPD Smart Energy DK, EE, NO, SE EE WPE Tracking Ships EL, NL, PL NL WPF Process and Architecture BG, DE, DK, EE, FR, IT, NL, PT IT WPG Financial Transactions Data BG, DE, IT, NO, PT, SI NO WPH Earth Observation BE, DE, FR, IT, LUKE, NL, PL, PT, SSP PL WPI Mobile Networks Data DE, EE, ES, FR, IE, IT, NL, RO, UK ES WPJ Innovative Tourism Statistics BG, EL, Hesse, IT, NL, PL, PT, SK PL WPK Methodology and Quality AT, ES, IT, NL, PL, PT AT WPL Preparing Smart Statistics AT, Berlin, BG, DE, FI, FR, IT, NL, NO, PL, PT, UK DE
Apart from WPA, the resources have been spread in a more or less even way over the implementation WPs that aim at producing prototypes for statistics production and the pilot projects that aim at producing experimental statistics. The horizontal WPs (i.e., WPF and WPK) and WPL (which has a much shorter lifespan) have less resources allocated.
1.3 - Methodology to be followed/project implementation Explain the methodology, both theoretical and practical, that will be followed in order to meet the general and specific objectives of the action described above. Explain which part of the action will be subcontracted (list of main activities to be carried out) as well as which activities will be undertaken by linked third parties (affiliated entities) (if applicable). Where subcontracting is envisaged please provide its justification and specify in particular, the tasks to be subcontracted, the selection procedure used and, if known,the contractor's name (in particular if an existing framework contract is used). Subcontracting: If the applicant (and affiliate) has to conclude contracts in order to carry out part of the action, the recourse to the award of contracts must be justified having regard to the nature of the action and what is necessary for its implementation. The description of the action must include information on the role, activity and responsibility of each subcontractor. Rules have already been foreseen in the financial regulation and the standard grant agreement where implementation of an action requires the award of procurement contracts. It is assumed that the beneficiary (and his affiliate) is competent for carrying out and managing directly the major part of the action. Situations where a beneficiary simply acts as intermediary must be avoided. In any case the beneficiary is sole responsible for the whole action and for compliance with the provisions of the agreement. Note and take into account that Eurostat should not have direct contact with the subcontractor(s) during the implementation of the action. The justification of subcontracting part of the action is particularly sensitive and important within the framework of grants restricted to the NSIs and other national authorities, due to their acknowledged responsibilities for the development, production and dissemination of European statistics. The justification must be substantiated in this respect.
This ESSnet has to be organized very effectively, since the number of partners is very high, the project involves research as well as implementation, and the scope in terms of subject domains and aspects investigated is very large. The complexity is compounded by the need for flexibility in execution, given the long duration of the project. When doing research and implementation with 28 partners, not everything can be predicted and worked out in advance for the 26 months of the action. Set-backs may occur, new opportunities may arise, results and new insights may induce adjustments. Fortunately, a lot of experience was gained in the previous ESSnet, which was organized in a way that was generally considered to be effective. The methodology of the new ESSnet builds on this.
The implementation methodology has the following elements:
• Overall planning • Organization structure • Physical meetings • Non-physical meetings • Communication and supporting arrangements
In addition, quality and risk management are foreseen. This is explained in section 1.5.
The project has a duration of 26 months, starting November 2018 (M1) and ending December 2020 (M26). Apart from the deliverables from the cross-cutting WPs and the final reports, the main deliverables are produced not later than in M24. For WPL (Preparing Smart Statistics) the deliverables are produced not later than in M12. As the cross-cutting WPs need the final reports of the WPs in the same track as input for the final cross-cutting deliverables, these deliverables are planned in M25. A major dissemination conference will also take place in M25 (see below at physical meetings). M26 is to be used for the report on that conference and for the final ESSnet report. Thus, the WPs are active in the following months:
WP WP name active period
WPA Coordination and Communication M1 – M26 WPB Online Job Vacancies M1 – M24 WPC Enterprise Characteristics M1 – M24 WPD Smart Energy M1 – M24 WPE Tracking Ships M1 – M24 WPF Process and Architecture M1 – M25 WPG Financial Transactions Data M1 – M24 WPH Earth Observation M1 – M24 WPI Mobile Networks Data M1 – M24 WPJ Innovative Tourism Statistics M1 – M24 WPK Methodology and Quality M1 – M25 WPL Preparing Smart Statistics M1 – M12
For the planning of the individual deliverables and milestones, see section 2.2.
The organization structure will be kept as lean as possible. The previous ESSnet had a coordinator and one layer of WP leaders, together forming the so-called Coordinating Group (CG). This worked well. In view of the size and complexity of the new ESSnet, a second layer could be considered, but this would make the organization itself more complex, slower, possibly less responsive and more bureaucratic. Instead, it is accepted that the CG will be bigger, but this will be compensated by giving the cross-cutting WPs (i.e., WPF and WPK) a coordinating role within their respective track and by a smart meetings scheme, which also takes into account the links between the WPs of the same track.
The coordinator has the overall responsibility of the ESSnet, the WP leaders are responsible for the management of their WP and its deliverables and milestones in accordance with the planning. The CG is the management team of the ESSnet and deals with issues related to the work programme, in particular the use of resources, the contents and planning of deliverables and milestones, and other issues that are relevant to all WPs, such as the preparation of meetings. The CG aspires to work by consensus, but if needed the coordinator may have to take a final decision. If this concerns a major issue, this will be done in consultation with Eurostat.
The following persons have been found prepared to assume the role of WP leader:
WP WP name name WP leader country
WPA Coordination and Communication Peter Struijs NL Marc Debusschere (deputy) BE WPB Online Job Vacancies Tomaž Speh SI WPC Enterprise Characteristics Galya Stateva BG WPD Smart Energy Arko Kesküla EE WPE Tracking Ships Anke Consten NL WPF Process and Architecture Monica Scannapieco IT WPG Financial Transactions Data Johan Fosen NO WPH Earth Observation Marek Morze PL WPI Mobile Networks Data David Salgado ES WPJ Innovative Tourism Statistics Marek Cierpiał-Wolan PL WPK Methodology and Quality Alexander Kowarik AT WPL Preparing Smart Statistics Natalie Rosenski DE
The secretary of the ESSnet is Martin van Sebille (NL).
As was the case for the previous ESSnet, the new collaboration requires a certain number of face-to-face meetings other than video-conferences. The following approach is taken:
• The budget required for physical meetings must remain under 10% of the total budget of the ESSnet. • A big kick-off meeting of the whole ESSnet may not be necessary, given the fact that many partners and participants were already involved in the previous ESSnet, and the wiki of that ESSnet provides a starting knowledge base. There is a risk that a big kick-off meeting would be too general to justify the budget involved. • However, at the level of the track for implementation and the pilots track, a kick-off meeting will be needed, if only to bring the WPs concerned on the same footing in terms of approach taken, standards, planning and interaction with the cross-cutting WPs (i.e., WPF and WPK, respectively). In this way, there will be optimal synergy between the WPs concerned. WPL will need a kick-off meeting as well. • Given the duration of the ESSnet of 26 months, a physical meeting after one year makes sense. Again, it stands to reason to organize this at the level of the different tracks. These are called mid-term meetings. • At the end of the ESSnet, a dissemination conference will be held. This will be similar to the very successful final dissemination conference of the previous ESSnet, BDES 2018 (Big Data for European Statistics 2018). The conference will be held at the end of 2020 and will be called BDES 2020. The NSI of PL is willing to host this meeting. • There is also a need for the individual WPs to meet, of course. For each WP, one meeting is foreseen halfway 2019 and one halfway 2020. These meetings may take the shape of sprints, focusing on working with actual data, or may have a more traditional character, depending on need. • For WPI, the need for more but smaller meetings has become clear already. For 2019 three of such meetings with only two or three participants have been budgeted. • For all physical meetings a report will be written. Such a report is considered a milestone that is due the month after the meeting.
For each meeting the location, month, length and participation has to be specified. Apart from the BDES 2020 conference, which will take place in Poland, the location of the meetings still has to be decided. The kick-off meetings are planned to take place in M2, the mid-term meetings in M14 and the BDES meeting in M25. The individual WP meetings are planned to take place in M8 and M20, respectively (with WPI having a different scheme). Meetings are generally budgeted for a duration of two days, but the BDES may take one day longer and include an internal ESSnet meeting. In order to keep the total costs acceptable, meetings will generally be attended by one participant per partner, plus for some meetings one or a very few extra participants in view of specific roles.
This is the total planning of physical meetings, with the organizing WP in the first column:
WP description month participants
F kick-off implementation (2 full days, 3 nights) M2 1 per partner (if involved in WPs B to F) 1 extra for WPF 2 for WPA 1 from Review Board K kick-off pilots (2 full days, 3 nights) M2 1 per partner (if involved in WPs G to K) 1 extra for WPK 2 for WPA 1 from Review Board L kick-off (2 full days, 3 nights) M2 1 per partner (of WPL) 1 extra for WP leader B WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP) 1 extra for WP leader C WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
D WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
E WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
F WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
G WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
H WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
I 1st WP meeting 2019 (2 days, 2 nights) M5 3 partners
I 2nd WP meeting 2019 (2 days, 2 nights) M8 2 partners I 3rd WP meeting 2019 (2 days, 2 nights) M11 3 partners J WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
K WP meeting mid 2019 (2 days, 2 nights) M8 1 per partner (of the WP)
L WP final meeting (2 days, 3 nights) M8 1 per partner (of the WP) 1 extra for WP leader F mid-term meeting implementation (2 full days, 3 nights) M14 1 per partner (if involved in WPs B to F) 1 extra for each WP 2 for WPA 1 from Review Board K mid-term meeting pilots (2 full days, 3 nights) M14 1 per partner (if involved in WPs G to K) 1 extra for each WP 2 for WPA 1 from Review Board B WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP) 1 extra for WP leader C WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
D WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
E WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
F WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
G WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
H WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
I WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
J WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
K WP meeting mid 2020 (2 days, 2 nights) M20 1 per partner (of the WP)
A BDES 2020 (3 days, 3 nights) location: Poland M25 1 per partner (of ESSnet) 2 extra for WPA whole Review Board
The need for meetings may change after the launch of the ESSnet, and flexibility may be needed, but any changes in planned meetings will have to be agreed with Eurostat, and of course have to take into account what is possible within the total budget.
Because of cost-efficiency, the preferred mode for meetings is video-conferencing. For this reason, the project has a WebEx license, managed by WPA (Coordination and Communication). This provides great flexibility, and the experiences with this tool during the previous ESSnet were generally positive.
The CG meetings of the previous ESSnet were attended by all WP leaders, the project secretary, the ESSnet project manager from Eurostat and the members of the Review Board (see section 1.5). This was considered an efficient way of managing the ESSnet, although the number of participants used to be rather high, and time for in-depth discussion of specific WP matters was limited. The new ESSnet will have a larger number of WPs and will be more heterogeneous. Conducting CG meetings in the same way as in the previous ESSnet would leave even less time for specific WP matters and would tend to be process- rather than content-oriented.
To solve this issue, there will be two types of meeting. Full CG meetings will only be held every other month. Such meetings will be process- rather than content-oriented. Progress in relation with planning and resources used will be discussed, with a focus on issues and bottlenecks. Administrative and other matters concerning the whole ESSnet will be addressed as well. The time for content-oriented discussions will be limited. Apart from the WP leaders, the project secretary and the deputy of WPA (who will be responsible for communication and dissemination), the ESSnet project manager form Eurostat and one member of the Review Board (to be chosen by the Review Board) will be invited. For all CG meetings minutes will be written.
Complementary to the full CG meetings, there will be track meetings for the first track (implementation; called CG-I meetings) and for the second track (pilot projects; called CG-P meetings). These meetings will be attended by the WP leaders of the track, the leader and deputy of WPA, and the project secretary. Again, the ESSnet project manager form Eurostat and one member of the Review Board (to be chosen by the Review Board) will be invited. The CG-I and CG-P meetings will be held in the month when there is no full CG meeting. The track meetings have time to discuss process as well as contents. For all CG-I and CG-P meetings minutes will be written.
Furthermore, there will be WebEx meetings for the individual WPs. These don’t necessarily follow a predesigned scheme, this is up to the WP leader concerned. The booking of such meetings, on request, is facilitated by the project secretary. Where useful, the WebEx facilities may also be used for conferences with external partners, such as data providers or experts, including for instance architects or quality experts from Eurostat.
Communication and supporting arrangements
The wiki of the previous ESSnet, which was used as an internal communication and editing tool as well as for external communication, will be made suitable for use by the new ESSnet too. Care will be taken that no confusion will arise between the two ESSnets. CROS-portal will be used as well. Further information on communication arrangements is given in the description of WPA in section 1.9.
For the publication of experimental results of the WPs, the ESSnet will look at what is being developed in the context of the Big Data Test Infrastructure project (BDTI), mentioned earlier where horizontal issues were discussed. In addition, Eurostat has facilities for publishing experimental statistics, which will be used as default. NSIs may, of course, additionally publish experimental results using their own dissemination channels.
Finally, for the project implementation the internal reporting arrangements are relevant. As mentioned, for CG, CG-I and CG-P meetings minutes will be written. Physical meetings have their own reports as milestones. All these reports allow for following the progress of the outputs of the ESSnet. However, for good project management information on resources spent will also be needed. During the previous ESSnet data on resources spent were collected on a monthly basis, but this was considered by some to be more than needed. Since the CG meetings will be process- rather than content-oriented, the data collected on resources spent will be more pertinent to these meetings than to the CG-I and CG-P meetings. Therefore, the new ESSnet will collect data on resources spent on a bimonthly basis, tuned to the CG meetings. The partners of the ESSnet will appoint a contact responsible for providing the requested data.
1.4 - Expected results and their use Specify the benchmarks or deliverables which you intend to employ to achieve the expected results and targets and how they will be used and disseminated.
The expected results and use are described in general in sections 1.1 and 1.2, and in detail in section 1.9, including a specification of the deliverables and milestones. The distinction between deliverables and milestones in this ESSnet is that milestones are process documents and deliverables are the main results for use in the ESS. Dissemination in general is described in section 1.1. It is part of WPA, which has been worked out in section 1.9. A particular aspect of dissemination is the release of experimental results, which is described in the last part of section 1.3.