CROSSMINER: Developer-Centric Knowledge Mining from Large Open-Source Software Repositories

crossminder cloudwatch
Project start: 
Sunday, 1 January, 2017
Project end: 
Tuesday, 31 December, 2019

CROSSMINER enables in-depth analysis, evidence-based selection and monitoring of open source components, and facilitates knowledge extraction from large open source software repositories to support development of systems for a wide range of applications by reusing the best available open source software. Selected open source components are monitored to raise alerts related to quality, and to give suggestions to reduce development effort and increase the quality of new products and services.

While there are several high quality and mature open source software projects, there are also many projects that are dysfunctional due to one or more of the following:
• The development team behind the open source software project invests little time in further development and support
• Development of the open source software has been discontinued due to lack of commitment or motivation
• Documentation is limited or of poor quality making the software difficult to understand and to update
• The community around the open source software project is small, questions receive late/no response, and defects get repaired slowly or ignored

Consequently, developing new software systems by reusing existing open source components raises challenges in:
• Searching for open source candidate components
• Evaluating and selecting the most suitable open source components from amongst identified candidates
• Adapting selected open source components to fit specific requirements of new software products and services

Who is the service/solution designed for?: 

1) Developers of a wide range of software systems applications that wish to use open source software components for new or existing systems
2) Providers of open source software forges
3) Software developers that contribute to the evolution of open source software projects.

How will your solution/service benefit the end-user? 

Recent reports state that the adoption of open-source software (OSS) helps, resulting in savings of about $60 billion per year to consumers. However, the use of OSS also comes at enormous cost: choosing among OSS projects and maintaining dependence on continuously changing software requires a large investment. Deciding if an OSS project meets the required standards for adoption is hard, and keeping up-to-date with an evolving project is even harder. It involves analysing code, documentation, online discussions, and issue trackers. There is too much information to process manually and it is common that uninformed decisions have to be made with detrimental effects.

CROSSMINER remedies this by automatically extracting the required knowledge and injecting it into the IDE of the developers, at the time they need it to make their design decisions. This allows them to reduce their effort in knowledge acquisition and to increase the quality of their code. CROSSMINER uniquely combines advanced software project analyses with online monitoring in the IDE. The developer will be monitored to infer which information is timely, based on readily available knowledge stored earlier by a set of advanced offline deep analyses of related OSS projects.

To achieve this timely and ambitious goal, CROSSMINER combines six end-user partners (in the domains of IoT, multi-sector IT services, API co-evolution, software analytics, software quality assurance, and OSS forges), along with R&D partners that have a long track-record in conducting cutting-edge research on large-scale software analytics, natural language processing, reverse engineering of software components, model-driven engineering, and delivering results in the form of widely-used, sustainable and industrial-strength OSS. The development of the CROSSMINER platform is guided by an advisory board of world-class experts and the dissemination of the project will be led by The Open Group.

How can the solution/service help you become more efficient, more secure, faster or cost-effective?: 

CROSSMINER will deliver an integrated platform for development of complex software systems by (1) enabling monitoring, in-depth analysis and evidence-based selection of OSS components, and (2) facilitating knowledge extraction from large open source software repositories. The following key scientific and technology innovations will be developed:
• Open source software analysis tools to extract and store actionable knowledge from a collection of open source projects
• Natural language analysis tools to extract quality metrics related to community communication channels and bug tracking systems of open source projects
• System configuration analysis tools to provide an integrated development and operational view of Open source software projects
• Workflow-based extractors that simplify creation of customised analysis and knowledge extraction from Open source software projects
• Cross-project analysis tools for understanding a wide range of Open source project relationships (e.g. dependencies and conflicts) based on developer defined similarity measures
• Advanced integrated development environments allowing developers to easily adopt the CROSSMINER knowledge base and analysis tools while receiving open source project alerts, recommendations, and feedback to improve developer productivity

The outcomes of the CROSSMINER innovations will contribute to the definition of a knowledge base addressing complex classifications of OSS projects and will automate identification of complementary and competing open source software projects, project incompatibilities, and prediction of the future of given open source projects based on the evolution of other projects having similar historical characteristics.