Difference: SemistructuredMergeReplication (5 vs. 6)

Revision 62016-04-07 - GuilhermeCavalcanti

Line: 1 to 1
META TOPICPARENT name="GenteAreaProjects"

Assessing Semistructured Merge in Version Control Systems: A Replicated Experiment

Guilherme Cavalcanti, Paola Accioly, and Paulo Borba.

In Proceedings of the 9th International Symposium on Empirical Software Engineering and Measurement (ESEM 2015).




Context: To reduce the integration effort arising from conflicting changes resulting from collaborative software development tasks, unstructured merge tools try to automatically solve part of the conflicts via textual similarity, whereas structured and semistructured merge tools try to go further by exploiting the syntactic structure of the involved artifacts.

Objective: In this study, aiming at increasing the existing body of evidence and assessing results for systems developed under an alternative version control paradigm, we replicate an experiment conducted by Apel et al. to compare the unstructured and semistructured approach with respect to the occurrence of conflicts reported by both approaches.

Method: We used both semistructured and unstructured merge in a sample 2.5 times bigger than the original study regarding the number of projects and 18 times bigger regarding the number of merge scenarios, and we compared the occurrence of conflicts.

Results: Similar to the original study, we observed that semistructured merge reduces the number of conflicts in 55% of the scenarios of the new sample. However, the observed average conflict reduction of 62% in these scenarios is far superior than what has been observed before. We also bring new evidence that the use of semistructured merge can reduce the occurrence of conflicting merge scenarios by half.

Conclusions: Our findings reinforce the benefits of exploiting the syntactic structure of the artifacts involved in code integration. Besides, the reductions observed in the number and size of conflicts suggest that the use of semistructured merge,when compared to the unstructured approach, might decrease integration effort without compromising correctness.

Replication Design

The study design is composed by a mining step, which is different from the original study since we are exploring DVCS repositories instead of CVCS ones; and by a execution step, which is similar to the original study, since we use the tool and scripts provided by the original authors. In particular, in the mining step, we built tools that mine DVCS repositories to collect a number of merge scenarios. Subsequently, in the execution step, we use a prototype of the semistructured approach in order to run the selected merge scenarios using both merge approaches, and R scripts, to collect metrics on the number of textual conflicts, conflicting lines of code, conflicting files and semantic conflicts.


Data, Scripts and Tools

Evaluation Results
R Script to generate graphics and run statistical tests

To generate the results in HTML (see the links below), you might use our R script. To do so, consider the following steps:

  • Install R.

  • Install the following packages and their dependencies:
    • "beanplot"
    • "ggplot2"
    • "R2HTML"

  • Download all .dat files above and rename them by removing "-R1" or "-R2". For your convenience, you may create different folders for each round.

  • Open the script in a text editor and change the values of the variables importPath and exportPath. The importPath should point to a directory that contains the data files (.dat files above). exportPath should point to the directory you want to save the generated HTML results.

  • Load the script in R.

  • Call the main function. Here is its signature: main <- function(round, task, SD=F)
    • round the round number. Valid numbers: 2 (at Federal University of Pernambuco, Round 1 in the paper) and 3 (Federal University of Alagoas, Round 2 in the paper); any different number corresponds to rounds 2 and 3 together.
    • task the task number. Valid numbers: 1 (New-requirement task) and 2 (Unused-variable task).
    • SD if true, the script will generate the results adjusting the time data with the SD-based time penalty. Notice that SD occurs only for task 2 (Unused-variable).
    • Example: main(3, 2, T) generates the results for the round at Federal University of Alagoas, regarding the Unused-variable task by using the SD-based time adjustment.


The tasks we distributed to the participants are available in portuguese and english.


Our tool is available in this link.

Auxiliar plug-in to play/pause/stop time and run test cases

The plug-in is available here.

Software Product Lines

In our experiment, we used two product lines: a commercial one and an academic one:

  • Best Lap (commercial product; we cannot distribute the source code);
  • MobileMedia

Virtual Machine

We have prepared a virtual machine to provide the same environment to all participants of our experiment. Notice the following:

  • You can use VirtualBox to execute our virtual machine;
  • The virtual machine already has the Eclipses, Emergo, and the Play/Pause/Finish plug-in installed.
  • We removed Bestlap from Eclipse 1 and Eclipse 2 due to commercial reasons; MobileMedia, on the other hand, is installed in Eclipse 3 and 4.
  • As pointed out in the paper, when opened for the first time, the cursor points exactly to the variable that should be changed;
  • There is an additional Eclipse to the warmup tasks.

-- GuilhermeCavalcanti - 2015-04-17

META FORM name="MyForm"
FORM FIELD 29 Jan. 29Jan.
FORM FIELD 12 Feb. 12Feb.
FORM FIELD 26 Feb. 26Feb.
META FILEATTACHMENT attachment="ResultsTable.pdf" attr="h" comment="" date="1429712121" name="ResultsTable.pdf" path="ResultsTable.pdf" size="147161" user="GuilhermeCavalcanti" version="1"
META FILEATTACHMENT attachment="figprocedure.png" attr="h" comment="" date="1429712756" name="figprocedure.png" path="figprocedure.png" size="56197" user="GuilhermeCavalcanti" version="1"
META FILEATTACHMENT attachment="gjccEsem15Pre.pdf" attr="h" comment="" date="1439211576" name="gjccEsem15Pre.pdf" path="gjccEsem15Pre.pdf" size="276222" user="GuilhermeCavalcanti" version="1"
META FILEATTACHMENT attachment="gjccEsem15Pre.pdf" attr="h" comment="" date="1460033280" name="gjccEsem15Pre.pdf" path="gjccEsem15Pre.pdf" size="304233" user="GuilhermeCavalcanti" version="2"
META FILEATTACHMENT attachment="sampleDetails.pdf" attr="h" comment="" date="1439212004" name="sampleDetails.pdf" path="sampleDetails.pdf" size="79497" user="GuilhermeCavalcanti" version="1"
META FILEATTACHMENT attachment="gjccEsem2015Presentation.pdf" attr="h" comment="" date="1448467571" name="gjccEsem2015Presentation.pdf" path="gjccEsem2015Presentation.pdf" size="1660289" user="GuilhermeCavalcanti" version="1"
This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback

mersin escort bayan adana escort bayan izmit escort ankara escort bursa escort