Rubin2020_PCW slack archives day2-tue-slot2-rrb-general 2020-07-15---2020-08-17

Wed 2020-07-15 03:04PM
@Ranpal (she/her/hers) has joined the channel
Wed 2020-07-15 03:04PM
@Melissa Graham has joined the channel
Wed 2020-07-15 03:04PM
@Emily (she/her) has joined the channel
Melissa Graham Wed 2020-07-15 03:34PM
We will eventually have N RRB sessions, with topics for each, and will make a slack channel for each of them (e.g., slot2a-rrb-sne, slot2b-rrb-cosmo, slot2c-rrb-lsb)
Melissa Graham Fri 2020-07-31 08:37PM
@Melissa Graham has renamed the channel from "day2-tue-slot2a-rrb-" to "day2-tue-slot2-rrb-general"
Eric Gawiser Sat 2020-08-01 12:54PM
It took me a while to figure out that RRB = Rubin Research Bytes, so perhaps that could get put in the channel description? :slightly_smiling_face:
Melissa Graham Sat 2020-08-01 01:12PM
@Melissa Graham set the channel purpose: Rubin Research Bytes general discussion.
Melissa Graham Sat 2020-08-01 01:13PM
@Melissa Graham set the channel topic: Rubin Research Bytes general discussion.
Individual parallel sessions also have Slack rooms.
E.g., <#C0183QAEN0J|day2-tue-slot2a-rrb-camera>
RRB Webpage: <>
ldacosta Tue 2020-08-04 10:05AM
:raising_hand: Roll call! Who else is here?Luiz da Costa
Merlin Tue 2020-08-04 10:10AM
Not wanting to be a total killjoy, but everyone in this Slack space is in here; everyone was added automatically I think, so please let's not start another channel with >400 people all just saying hello! :upside_down_face:
Melissa Graham Tue 2020-08-04 12:52PM
The topics and speakers for the 10 parallel sessions have been posted to the , which also features links to the 10 Zoom rooms and Slack channels, along with agendas, chair names, titles and abstracts, and communal slide decks.

Please have a look and decide which two parallel sessions to attend.

All sessions will be recorded and posted asap, and all slide decks will be available via the webpage. All are welcome to ask questions and/or discuss any of the flash talks in the Slack channels, whether you were able to attend the live presentation, or viewed the recording or slides.
drphilmarshall Mon 2020-08-10 05:08PM
Do you love living in the Solar System and are you wondering how we are going to learn more about it with Rubin? Are you excited by Rubin's surveying power, intrigued by its survey strategy and want to hear about how we can study gravitational wave sources , investigate Dark Energy , and discover entirely new phenomena with the LSST dataset?

Well then, Session G is for you!

Please join @Lynne Jones and me, with speakers @David Trilling @Andjelka @Humna Awan and @Fabio Ragosta in


for what could be the highlight of your week!
Melissa Graham Mon 2020-08-10 05:11PM
Oh I hope this starts a battle between RRB session chairs to creatively advertise their session!!
Nushkia Chamba Tue 2020-08-11 12:52PM
@Sugata Kaviraj
Sugata Kaviraj Tue 2020-08-11 12:55PM
ah well morphology advertises itself so... :relaxed: @Nushkia Chamba
Robert Blum Mon 2020-08-10 05:12PM
How can anyone beat @drphilmarshall ?!
drphilmarshall Mon 2020-08-10 05:14PM
Some might say, simply by not getting kicked out of the Zoom room you're in together
Robert Blum Mon 2020-08-10 05:15PM
Lynne Jones Mon 2020-08-10 05:19PM
Ah, I was going to say I don't think anyone can .. however, the zoom boot was funny.
Tom J Wilson Tue 2020-08-11 10:40AM
Hi all,

If you would like to know why cross-matching your favourite photometric catalogue to LSST products WILL FAIL* and what you can do about that, please join me in #day2-tue-slot2f-rrb-minimizing-systematics for my talk "The Effect of Unresolved Contaminant Objects on the Cross-Matching of Photometric Catalogues" or talk to me at any time!

*some, but depressingly really not that much, hyperbole used in this session plug
Will Clarkson Tue 2020-08-11 11:07AM
Thanks, Tom (and good to "meet" you): I found the Wilson & Naylor papers to be a very interesting read, and am looking forward to your presentation! Definitely interested to talk to you more about this, since the confusion limit to cross-matching might well be a limiting systematic for a lot of Galactic science in crowded regions.
Raphael Shirley Tue 2020-08-11 11:55AM
Hey everyone, I'm also going to be speaking in #day2-tue-slot2f-rrb-minimizing-systematics about my work with @Manda Banerji processing VISTA imaging with the LSST stack to build a pixel matched NIR imaging dataset along with LSST/VISTA ugrizyJHKs catalogues. It would be great to chat to other people developing 'obs' packages for the stack.
Robert Lupton Tue 2020-08-11 12:02PM
Happy to chat, but I'm not sure how to structure this to be useful.
How bad is the VISTA sampling, or do you start on cleverly (IMCOM??) combined stacks?
Raphael Shirley Tue 2020-08-11 12:06PM
I am trying to start with the VISTA stacks for now and tell the LSST stack not to do any instrument signature removal. I'm mainly trying to understand the various errors the stack is throwing and how to change configurations to deal with NIR specific issues resulting from high backgrounds etc. The main question I have is how best to use both HSC (as a test pre-LSST) and VISTA. Should I add HSC bands into my obs_vista package or try to use my package alongside the obs_suabru code
Raphael Shirley Tue 2020-08-11 12:07PM
But any obvious issues that you can imagine coming up with such a project would be useful
Robert Lupton Tue 2020-08-11 12:08PM
Well, that's easy. But if you have correlated noise, discontinuous PSFs, undersampling, and such like you'll have problems starting at that point.
You should absolutely use obs_subaru for HSC data. Currently it's a pain to mix cameras --- @Jim Bosch (he/him) can tell you more about plans to fix that in gen3
Raphael Shirley Tue 2020-08-11 12:11PM
Oh great in that case perhaps I'll concentrate on VISTA bands only until gen3 is further along. Yes I have just started and am currently debating the benefits/disadvantages of starting with the stacks. We don't want to waste the VISTA efforts that have gone into the raw processing but perhaps we will get the best results by working with raw exposures. So far I haven't managed to get the processCcd task to run on the raw exposures so will clearly have to expend effort there but I'm hoping I can test later stages of the pipline using the stacks as a first pass
Jim Bosch (he/him) Tue 2020-08-11 12:12PM
Before you get to the science-level problems Robert mentioned, you'll probably run into lots of bookkeeping issues related to providing all of the image characterization information our code needs in the form it expects: PSF models, variance planes, photometric and astrometric calibration, aperture corrections. It should not be too hard to put together minimal versions of most of those (or turn off the downstream functionality that needs them), but correct versions may be quite difficult.
Raphael Shirley Tue 2020-08-11 12:13PM
Yes I have managed to get a minimal first pass but very much far from 'correct' at this stage
Raphael Shirley Tue 2020-08-11 12:17PM
We have two levels of stacks in VISTA. Around 6 exposures on a given pointing are stacked. These are then subsequently coadded into a contiguous tile. I'm trying to ingest the pointing stacks such that the LSST stack directly coadds these to the skymap but need to think about incorporating the current variance planes and other image characterisation info you mention
Robert Lupton Tue 2020-08-11 12:19PM
Does your first level stacking preserve the PSF? How do you handle the under sampling?
Raphael Shirley Tue 2020-08-11 12:24PM
As far as I am aware it preserves the PSF but I am currently reading the details of the original processing. I'm not handling undersampling so far. I'm still at an early stage of simply seeing how wrong the outputs are. I was told that obs_decam is designed to do something similar (start from instrument calibrated images) so I am partially reading that code for reference
Robert Lupton Tue 2020-08-11 12:26PM
You should be able to just disable the ISR. DECam uses another set of calibs which is much more complicated.
Raphael Shirley Tue 2020-08-11 12:29PM
Yes I think I have managed to do that. The stack is producing coadded patches and running through the photometry stages. I still need to build calibration catalogues and understand the various errors the stack is throwing. Are you aware of other external groups starting obs packages from scratch. My guess is a lot of telescopes will want to produce pixel matched imaging and use LSST detections to conduct forced photometry
Robert Lupton Tue 2020-08-11 12:33PM
I'm not sure I'd quite do that due to deblender problems. Other people have written obs packages, and I'd recommend asking on
Raphael Shirley Tue 2020-08-11 12:35PM
The community website has been very useful. I'm particularly interested in the effects of the deblender where for instance there are blended objects in Rubin imaging but single detections in VISTA
Raphael Shirley Tue 2020-08-11 12:37PM
Perhaps the deblender is not well suited to dealing with different depths/NIR passbands?
Robert Lupton Tue 2020-08-11 12:37PM
We're building a deblender to handle exactly that issue.
Robert Lupton Tue 2020-08-11 12:38PM
The work that @Peter Melchior is doing is specifically interested in multi-resolution many-band data (including Euclid and Roman)
Raphael Shirley Tue 2020-08-11 12:40PM
We will certainly follow that work with interest. I suppose with VISTA at least the seeing is similar so that is one fewer issue to deal with
Robert Lupton Tue 2020-08-11 12:44PM
But it's significantly different (in general better) than the optical seeing.
Raphael Shirley Tue 2020-08-11 12:46PM
Ok well I look forward to seeing how the current blender performs on HSC/VISTA imaging.
Robert Lupton Tue 2020-08-11 12:49PM
Me too! It'll require good stack reductions of the VISTA imaging. Do you happen to know what your PSF rho statistics look like?
Raphael Shirley Tue 2020-08-11 12:51PM
No, can I get these from the output of processCcd tasks? Or do I need to go back to the original processing?
Robert Lupton Tue 2020-08-11 12:55PM
I hoped your VISTA processing had QA on the PSFs. So you're running PSF estimation on the stacked images. There is code to calculate them that @Arun Kannawadi (he/him) was working on
Raphael Shirley Tue 2020-08-11 12:57PM
The VISTA processing will have that I'm just starting to read my way around all the current VISTA processing and products. Yes I'm running PSF estimation on the stacks (simply because I am starting from the standard pipeline and altering it as I find issues).
Robert Lupton Tue 2020-08-11 12:59PM
We don't do that, we estimate the PSFs on the individual frames because of the effects of stacking.
Raphael Shirley Tue 2020-08-11 01:01PM
Ah yes I see this is because i'm giving stacks to the pipeline as if they are single exposures. Perhaps the first thing I need to do is write my own task to write the stacks to a special VISTA image format (halfway between calexp exposure and coadd)
Robert Lupton Tue 2020-08-11 01:03PM
It all depends on how much clever VISTA processing is involved, removing e.g. IPC and 1/f in the readout electronics, and handling undersampling. What is the pixel scale and average seeing?
Raphael Shirley Tue 2020-08-11 01:07PM
pixel scale is 0.339 arcsec
Raphael Shirley Tue 2020-08-11 01:09PM
I'm starting with VIDEO data which I think is set to FWHM < 0.8 arcsec
Raphael Shirley Tue 2020-08-11 01:10PM
At the moment I'm warping to the default HSC skymap pixel size
Robert Lupton Tue 2020-08-11 01:11PM
Right, so undersampled. So when you take the "Around 6 exposures on a given pointing are stacked" VISTA stacks, what's the plate scale?
Raphael Shirley Tue 2020-08-11 01:14PM
I see. I was giving it the pixel scale (calculated from actual pixel size and sky size) as a first pass.
Melissa Graham Tue 2020-08-11 01:03PM
The Rubin Research Bytes session starts at 10:30am Pacific. There are 10 parallel zoom rooms: each will have four flash talks followed by a panel-style Q&A. The talks+Q&A takes 30 minutes, and then repeats._ So all attendees switch zoom rooms at the 30-minute mark.

Session Webpage is_ _listing all sessions and Zoom links.

All Zoom Rooms
A Camera --_ B Cosmology --_ C Dark Matter and Low Surface Brightness --_ D Deep Imaging --_ E Galaxy Morphology --_ F Minimizing Systematics --_ G Survey Strategy and Solar System Science --_ H Transient Classification --_ I AGN and Microlensing --_ J Supernova Science --_
Keith Bechtol Tue 2020-08-11 01:20PM
We are currently waiting for host in "C Dark Matter and Low Surface Brightness"
mwv Tue 2020-08-11 01:21PM
Locating the host seems like an inherent risk given the session topic
Lynne Jones Tue 2020-08-11 01:21PM
We are also waiting for host in "G Survey Strategy and Solar System Science"
Melissa Graham Tue 2020-08-11 01:22PM
We're aware of issues facing all RRB Zoom rooms and working to respond (follow in #help )
Melissa Graham Tue 2020-08-11 01:33PM
@here pushing back RRB start time to 10:45am to try and fix global access issues.
Keith Bechtol Tue 2020-08-11 01:33PM
we already started, should we stop?
Lee Kelvin Tue 2020-08-11 01:34PM
okay, will do
Melissa Graham Tue 2020-08-11 01:35PM
need to be synced for the switch
mrawls Tue 2020-08-11 01:36PM
room H is forging ahead, 1st speaker just concluded
Margaux Lopez Tue 2020-08-11 01:36PM
same with A
Federica Bianco (she/her/hers) Tue 2020-08-11 01:36PM
soom G works
Tom J Wilson Tue 2020-08-11 01:36PM
F started too
Lee Kelvin Tue 2020-08-11 01:39PM
Room C paused, will re-start at 12:45, we're currently having an informal discussion
Melissa Graham Tue 2020-08-11 01:42PM
If your RRB Zoom is not working follow in #help we're making alternate arrangements.
Yao-Yuan Mao Tue 2020-08-11 01:46PM
cross-post to #general ?
Melissa Graham Tue 2020-08-11 01:45PM
@here alternate Zoom rooms being amalgamated above and on website. Please turn on recording if you can!
Melissa Graham Tue 2020-08-11 01:46PM
@here everyone please start or restart the flash talk and Q&A process and aim to transition at 11:15am Pacific (30 min from now). This will push back the session end time to 11:45am, a total of 15min delay.
Melissa Graham Tue 2020-08-11 01:51PM
(I'm really sorry to eat into your breaks!! But there will still be 15 minutes before our slot 3 sessions start.)
De Marte De QuiƩn Tue 2020-08-11 02:53PM
Thanks for attending us at the Rubin Research Bytes!

If someone wants to spread information in Peru through our platform, you can do it so freely. We invite you to follow and contact us on our Instagram account.

We help transfer your knowledge to a form of communication close to the bulk of the public that use social media. The idea is to get more people involved. So, we invite Rubin to have a funny conversation ;)
Melissa Graham Tue 2020-08-11 03:00PM
@here A big Thank You to all 20 RRB session chairs and all 40 speakers! :thankyou:
Steve Ritz Tue 2020-08-11 03:00PM
Thanks for organizing it! Do we each now need to make one slide from each session for Friday, or is there another way this will be handled?
Melissa Graham Tue 2020-08-11 03:03PM
Oh good question! No, all the RRB parallel chairs do not need to make a breakouts summary slide. I'll make a single slide to capture the RRBs for us.
Tom J Wilson Tue 2020-08-11 03:01PM
Well done people responsible for fixing the technical issues (currently the only thing to even remotely have gone wrong that I know of?!) :slightly_smiling_face:
Tom J Wilson Tue 2020-08-11 03:03PM
Heh, the only thing to even remotely have gone wrong...
Nino Cucchiara (he/him/his) Tue 2020-08-11 03:13PM
Question for Lauren: how EPO is going to, if planned, to increase awarenss among underepresented groups?
Yao-Yuan Mao Tue 2020-08-11 03:15PM
Is this meant for #day2-tue-slot3a-commissioning ?
Nino Cucchiara (he/him/his) Tue 2020-08-11 03:15PM
Yes, it was presented there
Nino Cucchiara (he/him/his) Tue 2020-08-11 03:16PM
I also posted the question on the zoom chat
Melissa Graham Mon 2020-08-17 03:35PM
All the recordings from all sessions have been posted and links are available via the RRB Session Webpage:

All Recordings On YouTube:
A. Camera --_ B. Cosmology --__ C. Dark Matter and Low Surface Brightness --_ D. Deep Imaging and Data Access --__ E. Galaxy Morphology --_ F. Minimizing Systematics --_ _ G. Survey Strategy and Solar System Science --_ H. Transient Classification --_ I. AGN and Microlensing --__ J. Supernova Science --__