What are the practical problems of software effort data sets?

In fact this post is an adaptation of my replies to a reviewer in a journal submission. The reviewer was concerned about the age of some of the software effort data sets. That was really a very valid concern, thinking that some of the most widely used data sets can be as old as 30+ years.

This made me thinking about some other problems related to software effort data sets and I identified 4 main issues. Those issues were usually discussed in the conferences between researchers, but I had never officially complained about them in one of my papers. So hopefully if this paper (fingers crossed) gets accepted, then we will see the below issues expressed regarding the software effort data sets.

1) Why collect data: For most companies, putting together scattered information into a software effort data set requires extra effort. So it is a serious issue for a practitioner to spend his already limited time in data collection activities. Therefore, unless user groups or practitioners in companies are given explicit outcomes for their effort, this difficulty is unlikely to disappear.

2) Data Privacy: Another concern for the companies is exposing their proprietary information. This issue calls for data anonymization techniques in software effort estimation.  A company that is convinced that its proprietary information is securely altered, is more likely to make their data publicly available.

3) Proprietary-right Period: This is quite a delicate issue with lots of pro's and con's. For research labs, sharing data can mean giving up on the exclusive right to publish results from that data set. Thinking the "publish or perish" saying of the academics, the reluctance to share the data is understandable. On the other hand, our ultimate goal is to bring software engineering one step ahead, one paper at a time... The solution to that problem can be to define a proprietary-right period. For example the science of astronomy has good solution examples to that problem.

4) Data Set Aging: The age of the data sets is another concern that lacks consensus. From a practitioner’s point of view, there is limited value in using old data sets as they may no longer represent current standards. On the other hand, for a researcher -like myself- not using old but widely-used data sets would create quite a lot of problems during the review process of a paper.. The easiest solution to that problem seems like deciding on a validity period. However, unless the software effort estimation field is supplemented with new data sets, I believe use of older data sets will be standard practice for most of the researchers.