LSST Science Advisory Committee Face-to-face meeting Sunday, March 5, 2017 Glendale Hilton (In coordination with LSST Joint Technical Meeting) SAC members attending: Mansi Kasliwal, Jason Kalirai, David Kirkby, Nelson Padilla, Charles Liu, Michael Wood-Vasey, Michael Strauss, Timo Anguita, Amy Mainzer, Anze Slosar, Niel Brandt Project Members attending: Beth Willman, Steve Kahn, Zeljko Ivezic, Francisco Delgado, Chuck Claver, Sandrine Thomas, Andy Connolly In our August 2016 face-to-face meeting, we made a series of recommendations, and posed a series of questions, to the LSST Project, mostly on two broad topics: -The status of, and future plans for, the LSST observing strategy, with particular emphasis on deep drilling fields, mini-surveys, and implementing rolling cadence. -The commissioning plans and mechanisms/policies for making the commissioning data public. These recommendations may be found on the SAC web page at: https://project.lsst.org/groups/sac/sites/lsst.org.groups.sac/files/2016August15.minutes.txt The March meeting focused largely on the Project's responses to the issues that we raised, and the discussion that ensued. In what follows, we give a high-level summary of our recommendations to the project; those in a hurry may want to read only that. We follow that with a (relatively) brief overview of the meeting itself, then give a series of comments and findings. The SAC website includes links to the presentations that we heard. Please see willman_issues_goals.pptx for a summary of the goals of this meeting. ************************************************************************ ************************High-level recommendations********************** The SAC fully endorses the NCOA concept, whereby LSST operations will be under a common umbrella with Gemini and NOAO. The SAC is excited about the release of Version 4 of the Operations Simulator. In this context: -We recommend that the formal definitions of the boundaries of the Wide-Fast-Deep survey be re-examined with the tools available with the OpSim. -We recommend that the OpSim code be made available to the community to carry out their own experiments in survey design. The SAC endorses the plan and timeline presented to us for engaging the community in further developing the plans for deep drilling fields. We recommend that these plans be expanded to examine the currently planned mini-surveys, as well as ideas for additional such surveys and observing modes. The scientific goals and the parameters for the currently planned mini-surveys need to be re-examined. The call for white papers on additional survey modes should go out only after a reasonable set of Version 4 OpSim runs have been performed, analyzed, and written up for the community. In addition, they should have a clear statement of process, including strict timelines and a description of the review and decision-making process. The project should consider having public events such as Town Halls at the AAS meeting to describe the process and bring it to closure. The SAC urges the project to develop plans for director's discretionary time and/or target of opportunity time, creating a worked example in the context of LIGO object follow-up. The SAC endorses the concept of a Survey Strategy Committee during operations. We recommend that this committee be largely independent of the LSST Operations teams. The SAC is reading the draft version of the observing strategy white paper, and will give feedback, especially as concerns important scientific areas not adequately covered in the white paper to date. The SAC endorses the proposal that commissioning data be made public to the LSST data rights community. We recommend that no restriction be placed on the authorship of scientific papers within that commmunity, including LSST project personnel. We also recommend that the SAC develop a mechanism for people from the community to report problems they find in the commissioning data; this will give a different perspective on the commissioning process. The LSST will include a limited events broker, with additional brokers developed within the community. Given that most users will use these community brokers, they will need to be commissioned as well. In this spirit, we recommend that the LSST commissioning team work to engage those community broker developers that they are aware of, to make sure they can coordinate their commissioning activities with the project. Finally, the SAC recommends that in a future meeting, we return to the question of the longevity of data releases: how long will a given data release be kept available for the community to use? ************************************************************************ ************************Summary of the meeting************************** *****LSST Project Status Steve Kahn gave us a high-level summary of the status of the project. His presentation is available at: kahn_projectstatus.pptx The construction is currently peaking: all the major contracts are let, and metal is literally being cut. There have been glitches and set-backs, but there have been no show-stoppers. The schedule has 13 months of contingency. The critical path right now is through the camera, with the telescope in a close second. There is a major replan of the data management (DM) system, which is nearing completion. A new DM subsystem manager, Will O'Mullane, has now stepped in, and the DM team is heading to a major review to happen in late July. The SAC may be asked in future meetings to view and give advice on prioritizing aspects of the "science platform", of which the Science User Interface is a piece. Steve also described the plans for NCOA, the National Center for Night-Time OIR Astronomy. This would bring the operations of LSST, NOAO, and Gemini under a common framework. Within NCOA, people will be centrally managed, but will be deployed as needed to specific projects within LSST, NOAO and Gemini. AURA will submit a formal plan for NCOA operations in mid-2017, coordinated with the LSST operations proposal submission. The SAC was interested in the way NCOA allows the interface between the LSST operations and the science community to be coordinated within NCOA. We briefly discussed the ways in which LSST might support staff scientists, in analogy to what currently exists at NOAO and STScI. Steve also emphasized the benefits for professional development for the staff. We didn't have time to discuss this, but Beth Willman prepared a set of slides describing recent developments in the interface between the project and the science community: willman_communityscience.pptx *****Observing strategy Our discussion started with a presentation from Mansi Kasliwal, summarizing the process by which the Zwicky Transient Facility (ZTF) has decided on its cadence. The ZTF team wrote a very focused call for white papers from their user's community. They found that the various cadence requests fell into a small number of categories (many of them asking for either nightly, or once-every-three-night revisits). They also found that because of the different length of nights, and different weather patterns in winter and summer, it was beneficial to use different strategies depending on the season. This is something the LSST may well want to consider. She emphasized that the cadence decision process is a tangible problem which can be sensibly solved if a clear and transparent process, with a finite number of options offered, is followed (as opposed to an infinite parameter space problem which never converges). ZTF will revisit their cadence decisions every year, based on science return. Another detail we discussed briefly is the possibility of dynamically changing LSST exposure time as conditions (sky brightness, transparency, seeing) change. The Dark Energy Survey is doing something similar, and has seen an improvement in the observing efficiency. Beth Willman, Zeljko Ivezic and Andy Connolly gave presentations and led discussions on the status of planning the LSST observing strategy. Their slides are: willman_observingstrategy.pptx ivezic_cadence.pdf OpSim Status 2017.pdf Most of the issues and conclusions from this discussion are included in the summary recommendations and questions below. Just a few notes here: The Observing Strategy White Paper is nearing completion; the current draft may be found at: http://ls.st/o5k It is the avenue for community input into the observing cadence. Zeljko and Andy summarized the conclusions of the white paper; please see its final chapter. One important development is the production of a new baseline cadence, "minion_1016" in the obscure language of these things. Note that 6.5% of the total time is devoted to the North Ecliptic Spur, 1.7% to the Galactic Plane observations, and 2.2% for the South Celestial Pole. This baseline was built on V3 of the OpSim code. The new version, V4, has just been released. Improvements include: -An explicit splitting between the Simulated Observatory Control System, and the scheduler. -Far more flexible specification of prioritizing different programs ("proposals" in the OpSim jargon), allowing, for example, rolling cadence and a uniform cadence program to be run simultaneously. -Rolling cadences can be specified with much more precision than previously possible. -One can limit the number of exposures in a given night. -The Western bias, whereby many fields were being chased into the West, is now gone, by better setting priorities as a function of hour angle. -The sky brightness model is much improved, and includes a proper modeling of the twilight sky. -The mini-surveys (North Ecliptic Spur, Galactic Plane, and South Celestial Pole) are now observed fairly uniformly throughout the 10 years; previously they were jammed into the first 5 years or so. The Project now aims to start a series of what will be perhaps several hundred realizations of the survey, varying various parameters that control survey strategy to see their effect. However, V4 is still being vetted, and although initial results indicate that rolling cadence is a big step forward, there is much exploration of parameter space to be done, and quantifying (via metrics) the effects on the Wide-Fast-Deep survey. *****Commissioning Plans LSST commissioning will go through the following phases: Phase 0: Planning Phase 1: Observations with a 1-raft (9 CCD) "Commissioning Camera" Phase 2: Observations with the full (189 CCD) camera Phase 3: Science verification At the end of Phases 1, 2, and 3, there will be sustained observing campaigns. Phase 2 will include the first generation of alerts; it will consist of 1600 deg^2 of imaging, in two epochs separated by some weeks. Phase 3 will include observations to 10-year depths (or even a bit deeper) in 20-30 fields. Community input will be invited, helping the project decide where those should be placed, and what their cadence should be. In 2018, LSST will have a public github site where community suggestions for the specifics of the commissioning observations can be placed. These will be evaluated by the LSST commissioning lead, in coordination with the SAC and the Project Science Team. The Project plans to make these data available to the LSST data rights community several months after they are taken. The Project is preparing a statement to the NSF about their plans here, and is looking for feedback from the SAC. The meeting was called to a close following an executive session and a debrief with the Project leadership. ********************************************************************** ********************************************************************** The following are the findings, questions and comments from the Science Advisory Committee, based on the material we discussed at this meeting. ***NCOA We are happy to hear about the NCOA concept, and fully endorse it. We realize that it is a realistic way by which the NSF can integrate operations support for LSST, NOAO, and Gemini, and the concept explicitly recognizes the synergies between these organizations. The SAC has long been concerned about the challenges of supporting the science community during LSST operations, given that LSST itself is quite constrained in what kind of support it can offer. We hope that NCOA can broaden the base of support for the community as LSST data begin to flow. If it would be useful to the LSST Project Office, the SAC is happy to give feedback on the NCOA proposal as it is being prepared. *****Observing Strategy The SAC was excited by the completion of Version 4 of the Operations Simulator. With this, we understand that it will now be possible to simulate a much larger range of alternative cadences. The first indications are that rolling cadence does significantly improve the spacing of observations for various types of transient and variable phenomena, and we are eager to see a proper exploration of the relevant parameter space there. Qualitatively, it seems that the uniformity needed for much of static sky science is maintained with a rolling cadence (especially in the "sandwich" mode in which the first and last years of the survey aim to get uniform sky coverage over the full Wide-Fast-Deep footprint), but we would like to see this demonstrated quantitatively. Survey design currently draws sharp boundaries between components of the survey, driven in part by the need to keep the observing strategy simple. Examples include the sharp airmass<1.5 boundary for Wide-Fast-Deep, the sharp boundary between Wide-Fast-Deep and the low-latitude observing cadence, and the uniform number of visits over the Wide-Fast-Deep area, without correcting for varying observing conditions or Galactic extinction. Our understanding is that the Scheduler is now sufficiently sophisticated that we can now explore the effects of removing these boundaries, and we urge the Project to carry out this exploration. We recommend that the code for running OpSim (V4) be made available to the community to carry out their own explorations. We understand that it may not be trivial to install, but some members of the community will be interested enough to make the effort to do so. One of our principal recommendations from the August 2016 meeting was for a coherent plan for engaging the community in gathering additional concepts for the deep drilling fields. We also recommended that there be a timeline, to be distributed widely with the community (via the observing strategy white paper; see below) for making decisions about the observing strategy plans overall. The plan and schedule that Beth and Andy outlined is reasonable. In particular, this schedule has an announced detailed observing strategy plan in place at the start of commissioning, with ample opportunity for the community to give input. We stress the importance of deciding on the positioning of the Deep Drilling Fields earlier (as the plan suggests), so as to give the community an opportunity to carry out preparatory or parallel observations of these fields in other wavebands. However, as we learn more about the performance of the LSST system, and as our scientific knowledge grows, it will be important to revisit the assumptions which go into the observing strategy, with the flexibility to adjust the strategy before full LSST operations begin, and once every year or so thereafter. It is our understanding that that will be the role of the Survey Strategy Committee; it would be good to be explicit about this when this plan and timeline are sent out to the community. As we discussed extensively in August, and we will reiterate here, the decision-making process about deep drilling fields needs to be made in coordination with defining the parameters of the mini-surveys. In the current baseline cadence, the mini-surveys take up roughly 10% of the telescope time, the equivalent of tens of millions of dollars in operating costs, yet remain based on a design written down with only qualitative scientific justification over a decade ago. In particular: -The boundaries of the Galactic Plane survey are defined somewhat arbitrarily in terms of stellar density. -The number of visits of the Galactic Plane survey are set in terms of reaching the confusion limit in the coadded data, while much of the science at low latitudes will be for variable phenomena; -The Galactic Plane survey somewhat arbitrarily reaches into the Northern celestial hemisphere; there is no stated scientific justification for this; -The North Ecliptic Spur is largely justified for "Taking an Inventory of the Solar System", one of the four main science goals of LSST. It currently is taking about 6.5% of the observing time, which needs to be explained quantitatively. The SAC reiterates its support for all aspects of solar system science in LSST, including the near-earth asteroid demographics. In summary, we need a specific scientific justification for the fraction of time devoted to these three mini-surveys. While the previous baseline cadence finished observations in the mini-surveys in five years, the latest Version 4 runs of OpSim spread the observations in these fields over the full ten years. Without a more focused science goal, it is not obvious a priori that this is a step in the right direction. We therefore recommend that the Project engage the community (involving, but not limited to, the Science Collaborations) in an exercise to define quantitative science goals for the mini-surveys. This will allow metrics to be written for the observations there, allowing evaluation of the efficacy of different designs of the mini-survey cadences. We recommend the following as the project considers a community call for DDF white papers: -The call should go out only after a reasonably complete set of OpSim V4 realizations and metrics have been run, exploring the cadence parameter space. With those in hand, we will be able to state clearly what fraction of the time can be devoted to the deep drilling fields and mini-surveys. -The call should perhaps be more general than just deep drilling fields, which, as the name implies, are each a single field of view of the telescope. That is, we should consider the possibility of additional more extended mini-surveys. An example would be a shallow extension of the WFD survey into the Northern celestial sphere, which could be useful for the Euclid survey. -On a related note, the call should ask for refinements, and quantitative scientific justifications, of the currently planned mini-surveys, including metrics to quantify how well any given realization of these surveys meets the science goals. -The call should be very clear on process: strict timelines, how the white papers will be reviewed, how decisions will be made, and how the process will be shared with the community. Similar to a Time Allocation Committee, having a firm and final deadline to submit ideas can motivate community engagement. The SAC suggests that the Project consider this and plan the appropriate community summits or Town Halls to bring closure to the white papers. There was discussion of Director's Discretionary Time and Targets of Opportunity during survey operations. The notional number of 1% of the time reserved for such things seems significantly too small (for comparison, HST, which is admittedly a very different type of facility, devotes 10% of its time to Director's Discretionary Time). The obvious scientific case is for LIGO/VIRGO follow-up observations. It is likely that the optimal follow-up strategy for gravitational-wave detections will evolve considerably by the time LSST starts operations, but it is worthwhile developing an observing plan given what we currently know about the problem. We endorse the idea of a Survey Strategy Committee (SSC) which will continue during operations. It was suggested that this be chaired by the LSST Project Scientist and include a mix of external scientists and LSST Project members, but the SAC feels the committee should be largely independent; the chair should not be a member of the Project. The LSST Director and Deputy Director already receive input from the Project Scientist and Project team. Advisory committees for community projects like LSST should be selected from the community, consistent with the approach adopted for other large astronomy projects. This approach maximizes independent feedback and eliminates perceptions of "inside jobs". It is possible that such a committee could be a subcommittee of the LSST SAC. We are looking forward to seeing the public release of the first version of the observing strategy white paper. We understand that this is a document from the community, but are happy to hear that our recommendation that the Project contribute a chapter about the decision-making process has been put in motion. Indeed, the SAC has started the process of scientifically reviewing the white paper. We are concerned that there may be important science cases that are not included among the existing metrics, and we will read the white paper with an eye to identifying such lacunae. ******Commissioning We were pleased to hear about the commissioning data plans. We endorse the proposal that the data be made public to the LSST data rights community within a few months of the completion of each commissioning phase, once an external review committee declares the data ready to be made public. The early operations plans will need to explicitly include the effort needed to make the commissioning data public. The SAC feels that the public will understand, in broad terms, that commissioning data will *not* be perfect, and therefore it is not crucial that the data be fully vetted before being released. There are several concerns that the release of commissioning data bring up. We list these here, and give our responses: -The commissioning data will be accessible only to those in the LSST data rights community, i.e., those in the US and Chile, and named individuals from international partner institutions. To what extent do we wish to control the sharing of these data with those who don't have data rights? That is, will it be a problem if a paper appears using LSST commissioning data with co-authors from those who don't have data rights, and if so, how could this be prevented? The SAC does not have a good answer to this question. -There was a suggestion that to avoid the appearance of preferred access, people paid by the LSST Project not be allowed to lead papers based on commissioning data. Given the fact that the data are public to such a large community, we feel that this concern is misplaced, and would recommend that no such restrictions be placed on LSST personnel. -We reiterate the concern that it will take significant resources to make the data available to the world. These resources will need to be explicitly budgeted in the plans for early operations, and the Project will have to be careful not to promise data on a timescale shorter than is realistic. The commissioning plan will include an evaluation of the limited event broker that the Project is responsible for. However, while it is beyond the Project's purview, the community event brokers, such as the Antares system being developed at NOAO, will also need to be commissioned, and it is important that this be coordinated with the first commissioning of the alert stream during Phase 2 of the commissioning plan. At one point, there was discussion of having members of the scientific community involved directly in the commissioning process. The best way to find subtle problems in the data is to carry out scientific investigations with it. This seems not to be part of the plans as they were presented to us. In any case, people using the commissioning data to do science *will* find problems in the data, and the commissioning plan needs to include a mechanism for people to report these problems, in a way that the project can learn from them. Finally, a topic for a future meeting: We would like to return to a topic we discussed previously, namely whether data from old releases would continue to be made available through the Science Platform after two years (the time after which the data become world-public). The suggestion had been made not to keep it spinning after two data releases, in order to save storage space. We think this is problematic, for several reasons: -It means that results in published scientific papers will be unreproducible after a given data release disappears; -If a given data release disappears while scientists are working on a scientific result, they will in essence have to start their analysis over again in order to finish their paper; -The scientific community has long expected that data, and the tools to efficiently access them, will eventually be made world-public. We recognize that the Project has worked hard to have international partners contribute to operations in return for data rights, but that by itself should not be a reason not to make these data access tools available for more than two years.