Summary of LSST SAC Recommendations From Executive Session discussions, April and May 2016 The following is a series of recommendations to, and questions of, the LSST Project Office from the LSST Science Advisory Committee. These come out of deliberations over the last few months on a variety of topics we have been discussing. In order, these topics include: -Feedback on the Science Operations plans -The nature of Level 1 alerts -Plans for keeping old data releases spinning *****Science Operations Plans******************* Beth has given presentations to the SAC, describing the plans for science operations in some detail. She has asked us to respond to two over-arching questions: 1. What activities and resources must LSST include in its proposed Science Operations directorate, to satisfy our operational requirements? How might we approach estimating the required level of effort within this scope? 2. What additional support, services and capabilities should be provided [for example: by NOAO, an LSSTC Science Institute, DESC], to enable community research/publications with LSST data? She also showed us a draft WBS for science operations, and asked us whether the allocation of personnel for the various activities made sense to us. We make several statements in response: -The high-level Science Operations tasks outlined in Beth's presentation are indeed sensible. The help desk, whereby scientists can get answers to detailed questions about how to use the LSST data and/or software, and to diagnose and identify problems and idiosyncracies in the data, is very important. Detailed documentation, which incorporates important questions that come through the help desk, is key. The documentation needs to be searchable, and needs to be clear about known problems, and what has been fixed from previous data releases. -The scientific community needs a mechanism to comment on operations while they are on-going. One way to do this is to have an external user's committee, perhaps the analog to the SAC, or an outgrowth of the existing science collaborations. We understand that such a user's committee is already being planned. The committee (or perhaps personnel within LSST Operations) would be responsible for actively monitoring community discussions (e.g., on http://community.lsst.org) to identify important issues that need documentation, software, or other operational fixes. -There needs to be close coupling between the help desk activities and the documentation effort. Questions to the help desk will help identify items that need further documentation. Tools such as StackExchange (planned to be used by JWST for community discussion) and community.lsst.org give a mechanism whereby common problems (and their solutions!) can be identified and incorporated into documentation. Keeping documentation up-to-date will be a key challenge for LSST; our experience with other big projects is that often documentation does not get updated. HST and SDSS have the advantage that by this time, relevant expertise has diffused throughout the astronomical community, meaning that often one can find the help one needs from one's colleagues. LSST will not have that luxury, at least in the early years. We will point out that JWST has a mechanism whereby users can ask that a specific question to the helpdesk be kept proprietary (as the data themselves are initially proprietary). We suspect that this will not be an issue with LSST, even if some users wish to keep the details of their on-going science analysis quiet. -The SAC does not have the expertise to determine in detail whether the proposed Science Operations staffing levels given in the WBS are adequate, as our experience with other large surveys has been from the inside looking out. Moreover, the descriptions we were given for the individual tasks in the WBS were too brief to understand the staffing rationale. Having said that, here are some specific comments and questions: -Sections 4.3.4, 4.3.5, and 4.3.7 of the WBS list a total of 2 people for the sum of Level 3 service management, and both the LSST and community event brokers; this seems rather underscoped. We imagine that the community event brokers in particular will change with time, and maintaining those will be a continued challenge. Perhaps the support for this will come from outside the Project, but we are concerned, that as a major way in which many scientists will get their data, problems in the alert brokers could reflect badly on the project. -Item 5.4.1 devotes 3 FTEs for the helpdesk, and 5.4.4 is a single FTE for documentation. Is this balance correct? We argued above that these two items should be tightly coupled. That is, questions to the helpdesk should feed into updates in the documentation, as a knowledgebase is built up. How much of the documentation will be written during construction? If it is a substantial amount, then the documentation need during operations may be smaller (but still significant; see above). And a quick question about the help desk and other aspects of Science Operations: to what extent will it be focused on the US community, and to what extent will the help desk be open to questions from Chileans and international members of the LSST community? We recommend that the help desk be available to all people with LSST data rights. Will the non-data rights community be able to ask questions of the help desk at any point during operations? If so, how will they be handled? As for the second question, about additional resources needed to support the community science activities beyond the formal LSST operations, we can imagine that there will be need for workshops and additional educational tools for those wishing to learn about how to use database resources, as well as support for science collaborations as they do the major analyses that LSST will enable. The current activities of the LSST Corporation Enabling Science initiative go in this direction, although they are limited by the resources now available. ********Level 1 alerts:***** We heard a presentation from Mario Juric describing the current plans for Level 1 alerts. We have several conclusions, and several questions for the Project Office: -Scientists should have full access to the full alert stream. We are concerned that the filters the alert brokers will put on will hamper some people from doing their science. -However, getting the full alert stream with a latency longer than 60 seconds would be fine for many science requirements. -A longer latency (e.g., 24 hours) would be appropriate for classroom settings as well. -Where is the bottleneck which limits access to the alert stream? We believe that the alert processing is done at NCSA, and we (perhaps naively) think that from there, alerts should be accessible to an arbitrary number of people; bandwidth shouldn't be an issue. -We realize that we do not yet have a clear understanding of how the event broker model will work in practice. How will it be decided who will develop the event brokers? It is likely that we'll have substantially more than three teams interested in developing such brokers. What are the requirements placed on an event broker? Who will make the decision? We are concerned that the event brokers will be outside the control of the LSST operations, and that if they are perceived to perform poorly or unfairly in any way, this will reflect badly on the LSST itself. There is likely to be interactions between the Event Brokers (and those who run them), and an LSST Science Center, e.g., run by NOAO; the Science Center may end up playing the role of making sure the Event Brokers are responsive to the needs of the community. ****Data Releases**** The SAC also discussed the yearly data releases. Here are our conclusions: -It is important that all LSST data releases be kept spinning (or at least accessible via tape robot in a user-friendly and database-enabled way) through the life of the project. This is needed to allow published results to be reproduced, and to make sure that science projects started with a given data release can be completed. Statistics from SDSS show that data releases even from many years previous continue to be accessed at a substantial rate. -The LSST Project should prepare a plan by which LSST data will continue to be available after the survey itself ends. This is an unsolved problem in existing large surveys (e.g., SDSS), and the community will need access to LSST data for decades after 2032. -The Project should prepare a detailed description of the alerts and data release plan, to be made available to the LSST community. There is a lot of misinformation and confusion on these topics, even among the SAC members.