Tuesday, June 15, 2010

The Role of Secrecy in Science


For the first time that I can remember, a story about data embargoes for a space mission has made the New York Times. A data embargo is an agreement signed between a space agency and a mission science team which grants the science team exclusive rights to the data for a fixed period of time. The specific case in question involves the Kepler Mission, a NASA-led spacecraft in an earth-trailing orbit (at the L5 Lagrangian point) which is searching for earth-sized extrasolar planets. As first reported in Nature, the science team has identified about 700 potential exoplanet candidates which, under the terms of the agreement the team signed with NASA, must be made public within one year of discovery. But because of bad weather and launch delays for other space telescopes, they have been unable to confirm all those putative detections within that time. As such, they are seeking an extension on their data embargo. In a compromise have been granted the right to hold back their best candidates for one more year.

As space missions are funded by taxpayers, you might wonder why data embargoes exist at all. In fact, a case can be made that their presence can impede scientific progress since it limits new data to a small pool of people for a certain amount of time. If the entire scientific community could be brought in, progress would certainly be faster. Furthermore, many space missions excite the imaginations of the public who ultimately provided the funding that makes them possible. Sharing with these people is an obligation that will only help our future prospects. After all, a vociferously supportive public and scientific community increases the likelihood that more missions will take place in the future.

This desire for openness must be balanced against providing an incentive for the Science Team to participate. Mission planning can consume a great deal of a scientist's time for years before launch. This requirement will delay or supplant work that would lead to publications, the keys to career advancement. Furthermore, once the mission starts producing science, the team who put so much work in ahead of the mission is in the worst possible place to analyze their results. They must spend most of their time running the mission, making scientific decisions, and preparing the cleaned up and polished data for public release. This production of RDRs (Reduced Data Records) from the raw data received from the spacecraft (EDRs, or Experimental Data Records) is a long and tedious task, in and of itself.

If the data were freely available as soon as they were collected and anyone could publish immediately, why would anyone make the effort to join a mission team, knowing what it could cost? Without the carrot of a guaranteed first paper, possibly in a premier journal such as Science or Nature, it would be difficult to attract top talent and the quality of those scientists working on the mission and the decisions they make would be lower. Thus the collected data products would also suffer without embargoes.

A balance is required between the needs of the scientific community, the science team and the public at large. For Mars missions, typically EDRs of images are released within a few days and public release of data to NASA's planetary storehouse, the Planetary Data System (or PDS, as we typically refer to it) for dissemination to the scientific community occurs at the 6-month mark. This 6-month embargo is barely adequate to give the scientists a head start, and often in the course of a mission, you can see a gradual change over the first few months in the make-up of the operations team as the experienced scientists step back to write their papers and younger members of the team step into their roles. Often these younger scientists are graduate students, as I was when I worked on Phoenix.

This brings up the potential conflict that can exist amongst the members of a science team. As there are different levels of participation and different roles within the science team, there is also a method for determining how publishing rights will be meted out within the team. After all, a senior scientist who dedicated five years to a project and who has considerable duties during the science phase would not want someone who was added at the last minute and has more free time to write up a paper in Science or Nature. Thus, the the publications that will result are typically codified in advance of the beginning of the science phase in a document called "The Rules of the Road." This document lists the collaboration level of each science team member and what he or she is permitted to publish. For instance, for Phoenix, the main Science Paper announcing the results was restricted to the senior level of researchers, the Co-Investigators (or Co-I's). In the later JGR special issue, any science team member was permitted to contribute (Disclosure: I helmed a paper in the JGR special issue and contributed to one of the subsidiary Science papers for Phoenix).

The peanalties for the science team members breaking an embargo are serious. While not all missions require formal NDAs (Non-Disclosure Agreements) as Kepler did, team members are well aware that infractions could result in the termination of their participation on the mission and their funding, and even disqualify their participation on future missions. Given the prestige of serving on a mission, this is usually incentive enough.

Embargoes can also affect those on the outside in the scientific community. For instance, publication of results in a peer-reviewed journal can take upwards of a year from submission to publication. Thus every day counts. Often, new discoveries will be announced, somewhat informally, at press conferences and at trade conventions. More then a few outside scientists attempted to publish in peer review the material released at these conferences and conventions for Phoenix while the embargo was still in effect. Mostly, the editors refused to accept these papers - after all, the authors did not collect the data they were reporting, and were not experts on its usage. However, unfortunately, at least one paper did get through and became the first record of one of our instruments to occur in the literature. It seems unlikely that the authors of this paper would be asked to participate in a mission in the future.

The public reaction to full disclosure can also be a source of worry. For instance, when I was on the Huygens Mission working for the Descent Imager/Spectral Radiometer in 2005, an accidental release of the image library resulted in internet bloggers providing the first processed versions, many of which made it onto the pages of newspapers. These bloggers did not have the calibration data from the instrument and thus could not eliminate many of the artifacts and properly balance the colour in the way that we could. Thus an incorrect record was widely disseminated. As you can expect, the P.I. of the instrument who had devoted almost 17 years of his life to getting these 3 hours of data from the surface of Titan, was not impressed.

Many, especially those on the outside, would argue that secrecy is anathema to good science. However, a certain amount is important if we are to keep the quality of mission data at its peak. Those who participate are giving up their time to use their talents to bring back the best data possible and are deserving, at the very least, of our respect. They also deserve the time needed to get it right. I do believe that the current state of affairs with respect to the public is very fair. In terms of the Scientific community, I believe we need to continue to work to make space missions as inclusive as possible. The more talent we have working inside of the embargoes, the better our results will be. At all costs, we need to avoid the impression of space missions as an exclusive club where who you know is more important then the quality of your work.

----

As a note, the JPL planetary photojournal provides thousands of images that have been publically released. These images have been processed by the best and should be your first stop when exploring the solar system or looking for something to include in a presentation.

No comments:

Post a Comment