Rubin2020_PCW slack archives day4-thu-slot3a-lsb-science 2020-07-15---2020-08-17

Wed 2020-07-15 02:57PM

@Ranpal (she/her/hers) has joined the channel

Thu 2020-07-16 01:20PM

@Melissa Graham has joined the channel

Wed 2020-07-22 11:58AM

@Jeff Carlin has joined the channel

Thu 2020-07-23 05:35PM

@Lee Kelvin has joined the channel

Thu 2020-07-23 05:35PM

@Sugata Kaviraj has joined the channel

Lee Kelvin Thu 2020-07-23 05:47PM

@Lee Kelvin set the channel topic: Session Page: <https://project.lsst.org/meetings/rubin2020/agenda/session/low-surface-brightness-science-and-robust-sky-estimation>

Lee Kelvin Thu 2020-07-23 06:03PM

@Lee Kelvin set the channel topic: Zoom Channel: <https://tinyurl.com/pcw2020-lsb-zoom>
Session Page: <https://project.lsst.org/meetings/rubin2020/agenda/session/low-surface-brightness-science-and-robust-sky-estimation>

Lee Kelvin Thu 2020-07-23 06:05PM

@Lee Kelvin set the channel topic: Low Surface Brightness Science and Robust Sky Estimation
Organizers: @Lee Kelvin, @Sugata Kaviraj
Zoom Channel: <https://tinyurl.com/pcw2020-lsb-zoom>
Session Page: <https://tinyurl.com/pcw2020-lsb-session>

Lee Kelvin Thu 2020-07-23 06:05PM

@Lee Kelvin set the channel topic: Low Surface Brightness Science and Robust Sky Estimation (info below)
Organizers: @Lee Kelvin, @Sugata Kaviraj
Zoom Channel: <https://tinyurl.com/pcw2020-lsb-zoom>
Session Page: <https://tinyurl.com/pcw2020-lsb-session>

Wed 2020-07-29 07:58PM

@Preethi Nair has joined the channel

Fri 2020-07-31 03:06AM

@Ilaria Musella has joined the channel

Fri 2020-07-31 04:55AM

@Ivan Baldry has joined the channel

Fri 2020-07-31 08:34AM

@Robert Szabo has joined the channel

Fri 2020-07-31 08:38PM

@Mireia Montes has joined the channel

Fri 2020-07-31 08:39PM

@Sarah Brough (she/her) has joined the channel

Fri 2020-07-31 08:39PM

@C A Collins has joined the channel

Fri 2020-07-31 08:42PM

@Enrichetta Iodice has joined the channel

Fri 2020-07-31 08:42PM

@Yuanyuan Zhang has joined the channel

Sun 2020-08-02 09:33PM

@Cristina Martinez-Lombilla has joined the channel

Mon 2020-08-03 11:41AM

@K-T Lim has joined the channel

Wed 2020-08-05 02:11PM

@M Temple has joined the channel

Wed 2020-08-05 07:48PM

@Markus Dirnberger has joined the channel

Wed 2020-08-05 10:43PM

@Luis Pineda has joined the channel

Fri 2020-08-07 11:31AM

@Youtsumi has joined the channel

Mon 2020-08-10 06:44AM

@quasar_panda has joined the channel

Sugata Kaviraj Thu 2020-08-13 03:10PM

please put your questions for speakers here or use the chat function in the zoom room

Michael Strauss (he/his) Thu 2020-08-13 03:14PM

Lee: All the recovered sky values in your penultimate plot are positive. An unbiased estimator would give you a sky value at zero on average, which would suggest that sometimes, you have to get slightly negative sky values.

Lee Kelvin Thu 2020-08-13 04:17PM: @Michael Strauss (he/his) , thanks again for this question. To summarize my earlier answer (for posterity), the data we're showing in the session slides are for all sources down to m_r=30 mag. When running a source extraction routine over these data, it typically starts becoming incomplete at m_r~26 mag, resulting in a large number of faint sources entering the extra-galactic background (EBL) light component. For this reason, all of our estimated sky levels fall above zero counts. However, when we account for EBL and remove it, you're indeed correct that our corrected results all scatter about zero. I hope that answers your question, and happy to discuss further/show plots.
Michael Strauss (he/his) Thu 2020-08-13 04:29PM: What exactly does it mean to "account for EBL"? Fitting a constant component to the sky overall?
Lee Kelvin Thu 2020-08-13 04:44PM: Yes, that is one way to do it with real data. However, as we're working with simulated imaging, we were able to construct simulated fields whereby those faint sources were not injected in the first instance (i.e., we create another binary set of simulations containing only the bright and intermediate-brightness sources, excluding the faint and typically non-detected sources prior to source extraction/sky estimation).

Ferguson Thu 2020-08-13 03:19PM

There are some possible additional metrics for evaluating sky subtraction at the end of the notebook at https://spacetelescope.github.io/jdat_notebooks/pages/background_estimation_imaging/background_estimation_imaging.html .

Lee Kelvin Thu 2020-08-13 03:45PM: Thanks for the link!

Sugata Kaviraj Thu 2020-08-13 03:22PM

Lauren -> Lee: do you have a sense of how many false positives the various methods produce?

Lee Kelvin Thu 2020-08-13 03:27PM: Answer copied from Zoom: Lauren, yes, we do - we look at the false positive rate, and how each modification we've made impacts that. Results vary from ~10% false positive rate in the best case scenarios, to very high false-positive rates (>50%) in the worst case scenarios.

Mariano Dominguez Thu 2020-08-13 03:26PM

Lee: I would like to known if it is possible to access the set of simulated HSC images? Alternatively, could you comment how the images were produced. Thank you in advance.

Lee Kelvin Thu 2020-08-13 03:28PM: Yes, absolutely, we've put all of our work on GitHub and I'm more than happy to send them your way (will ping you).
Mariano Dominguez Thu 2020-08-13 03:31PM: That's great Lee, thank you very much! I'll love to read your article soon, m.
Lee Kelvin Thu 2020-08-13 04:20PM: On the image production, we used GalSim to simulate our fields, adding appropriate noise by providing GalSim with an estimate of the initial sky pedestal and gain (for Poisson noise). We inject galaxies according to a fitted power-law model to their number density as a function of magnitude, i.e., we're injecting a large number of faint sources and a relatively small number of large bright sources. All other Sersic parameters (half-light radius, ellipticity, position angle, etc) are similarly assigned in a descriptive manner. If you have any further questions, I'd be happy to discuss it further - please do let me know!
Mariano Dominguez Thu 2020-08-13 04:34PM: Yeap, I would like to known if you explore different observing conditions on your images? and also where I could learn about the correct GalSim parameters configuration for the HSC or LSST cameras. Any help is appreciated.
Lee Kelvin Thu 2020-08-13 04:41PM: We didn't explore different observing conditions here, but happy to discuss further GalSim (please ping me if so!). A good place to start is the https://ui.adsabs.harvard.edu/abs/2015A%26C....10..121R/abstract , and you can also check out the https://github.com/GalSim-developers/GalSim .

Karen Masters Thu 2020-08-13 03:32PM

@Sugata Kaviraj - I liked your plot comparing survey depth in different bands to typical disc and spheroid SEDs. Which SED model did that use? Can you share/point me to the data which was used to make that plot?

Sugata Kaviraj Thu 2020-08-13 03:33PM: I actually pinched it from @Lee Kelvin :slightly_smiling_face: @Lee Kelvin could you help?
Lee Kelvin Thu 2020-08-13 03:37PM: @Karen Masters , I took the SED information from https://ui.adsabs.harvard.edu/abs/2012MNRAS.427.3244D/abstract . Happy to share a parsed data file with results if you'd like?
Karen Masters Thu 2020-08-13 03:42PM: That would be awesome. I think a version of that plot would help a lot to explain the difference between SDSS and UKIDSS Galaxy Zoo (beyond the wavelength - the imaging depth is also different as this clearly reveals!).

Dan Taranu Thu 2020-08-13 03:34PM

Nushkia: are segmentation maps still useful with deeper/highly blended images, and if so how? With shallow images and well-separated sources they're less ambiguous but once images get crowded it seems like they won't have much utility (and it's hard to come up with a "truth" to optimize).

Nushkia Chamba Thu 2020-08-13 03:39PM: hi Dan, thanks for your question! I think it could be useful if you are interested in finding potentially interacting galaxies (like the first example I showed you) or studying intra-group or intra-cluster light
Nushkia Chamba Thu 2020-08-13 03:43PM: but I agree, to optimise, we used FDS simulated images, we have not simulated specific environments etc to do our optimisation, the issue in a crowded field is the computation of the background but I think some people use a nearby isolated fields to measure the background there etc. But I am not an expert in studying objects in highly crowded fields. Maybe some of the others in this session can tell you more :slightly_smiling_face:
Lee Kelvin Thu 2020-08-13 04:23PM: @Nushkia Chamba , @Dan Taranu , crowded fields are the primary motivator behind our developing the modelled masking procedure to apply in advance of some of the background computation procedures you outline here - I agree, crowded field photometry will only become more prevalent going forward!
Dan Taranu Thu 2020-08-13 04:29PM: Is anyone working on methods specifically for galactic cirrus science? If you assume that the cirrus is static it should be easier (not easy :slightly_smiling_face: ) to model with large numbers of exposures and separate from sky backgrounds.
Lee Kelvin Thu 2020-08-13 04:45PM: I think that @Alex Drlica-Wagner (he/him) pinged me a message about this the other day actually?
Alex Drlica-Wagner (he/him) Thu 2020-08-13 05:09PM: The cirrus is pretty cool/nasty depending on your perspective.
Alex Drlica-Wagner (he/him) Thu 2020-08-13 05:09PM: It's not on my todo list, but I'm generally interested
Nushkia Chamba Fri 2020-08-14 05:43PM: Check out my colleagues paper on Galactic cirrus colours and how it is different from other sources (galaxies, stars etc). Its a really awesome paper, and he used the fact that Sextractor CAN NOT detect cirrus (as I showed in my slides) as an advantage as most of the other brighter sources around could be masked (with some manual adjustments). :smile: Here is the paper, Román et al. (2019): https://ui.adsabs.harvard.edu/abs/2019arXiv190700978R/abstract he is an awesome person, so email him for more info :slightly_smiling_face:

Ed Thu 2020-08-13 03:41PM

For a given Data Release, is the overall plan to report "corrected" magnitudes, etc., for each object or else to report the biases as a function of magnitude generally? If the former, would such corrections be environment (as a function of sky, etc.) dependent?

Lee Kelvin Thu 2020-08-13 04:25PM: There will be multiple data products available in the final data releases, and so you should be able to select your data product of interest and work with that. We're working to provide data from various stages of the data processing pipeline, allowing you to step in where you feel it best for your own science.
Ed Fri 2020-08-14 04:06PM: So, the object "catalogue" will really be a complex databases that's queried in very customizable ways? I am just trying to have a better understanding of what this will be like. There are obviously many possible (and reasonable) choices. Thanks.

Yusra AlSayyad Thu 2020-08-13 03:45PM

Wanted to add that Aaron's work was testing the effects of a spine background model with 128-pixel bins. This is not baseline for the LSST pipelines. Looking forward to the analysis on with the full focal plane backgrounds.

Chris Walter Thu 2020-08-13 03:47PM

Quick acronym question: What is the BCG?

Lee Kelvin Thu 2020-08-13 03:48PM: Brightest Cluster Galaxy

A Emery Watkins Thu 2020-08-13 03:47PM

Brightest cluster galaxy

Nushkia Chamba Thu 2020-08-13 03:51PM

@Benne Holwerda and others if interested. MTObjects can be found here: https://github.com/CarolineHaigh/mtobjects still being improved by the team in Groningen :slightly_smiling_face:

Thu 2020-08-13 03:52PM

@Benne Holwerda has joined the channel

Chris Walter Thu 2020-08-13 03:52PM

@Cristina Martinez-Lombilla Thanks for your talk. It was very clear and it was really helpful for me (I'm a non-expert in this science)!

Song Huang Thu 2020-08-13 03:53PM

Question for Cristina: What's the criteria for "over-subtraction" in your case? Negative flux density or just using the GAMA Sersic model as reference?

Cristina Martinez-Lombilla Thu 2020-08-13 03:57PM: It was based on negative flux
Song Huang Thu 2020-08-13 03:57PM: :+1: Thx!

Sarah Brough (she/her) Thu 2020-08-13 03:54PM

Moving conversation about data types over to slack so we can retain the discussion

Robert Lupton Thu 2020-08-13 03:55PM: Sorry; I only just managed to escape my previous meeting
Sarah Brough (she/her) Thu 2020-08-13 03:55PM: There's definitely a lot of data there and it;s not always clear which ones to use - see next comment by @parejkoj
Robert Lupton Thu 2020-08-13 03:56PM: Indeed. It's also a moving target
Jeff Carlin Thu 2020-08-13 03:57PM: One can always ask questions like this on http://community.lsst.org , which has the benefit of being available and searchable for future users with similar questions.
Sarah Brough (she/her) Thu 2020-08-13 03:58PM: @Cristina Martinez-Lombilla and I (and likely @A Emery Watkins ) will re-run on the deepCoadd instead of deepCoadd_calexp. But your question of how to add backgrounds back in - is also something we're interested in testing but some advice on how to use those data would also be helpful.
Chris Walter Thu 2020-08-13 04:03PM: @Robert Lupton and @Yusra AlSayyad is

https://arxiv.org/abs/1705.06766

still a good reference for people (like @Sarah Brough (she/her) to look at for this? I know the deecCoadd_calexp is listed etc but don't know how relevant the specifics are now since it is a bit old.
Yusra AlSayyad Thu 2020-08-13 04:16PM: That paper is a good reference for PDR1. https://arxiv.org/abs/1905.12221 for PDR2. Cristina's images are of unpublished data products processed as part of a continuous integration effort at NCSA and accessed via the science platform. We don't publish release notes for weekly changes to the stack. I agree with @Jeff Carlin that if you want to work with those, community is the best place to ask.
Lee Kelvin Thu 2020-08-13 04:29PM: @Sarah Brough (she/her) on the topic of adding the sky back in, at the coadd stage you can access the deecCoadd_calexp_background data product, which gives you the background subtracted from the deepCoadd_calexp . At the visit level (pre-coadd) there's also the calexpBackground , which is the background subtracted from the calexp .
Sarah Brough (she/her) Thu 2020-08-13 05:38PM: And for the non-Calexp's that we've been talking about?
Lee Kelvin Thu 2020-08-13 06:43PM: deepCoadd = deepCoadd_calexp + deepCoadd_calexp_background
Lee Kelvin Thu 2020-08-13 06:46PM: If you wanted to go even further back in processing, you can access the calexps, re-add the calexpBackgrounds, and perform your own image co-addition after the fact.
Markus Dirnberger Thu 2020-08-13 07:02PM: Looking through the code, I've found the following policy files that detail all the data products produced by the pipeline: https://github.com/lsst/obs_base/blob/master/policy/datasets.yaml , https://github.com/lsst/obs_base/blob/master/policy/exposures.yaml , and specifically for HSC https://github.com/lsst/obs_subaru/blob/master/policy/HscMapper.yaml . It would be nice if this could be included in some written documentation at some point, including suggestions for matching recommended data products with science cases!

parejkoj Thu 2020-08-13 03:55PM

More broadly: where do we document the details of the different types of coadd products?

Mireia Montes Thu 2020-08-13 05:35PM: I think that this is extremely important as it seems many of us are quite confused about the products.

Nushkia Chamba Thu 2020-08-13 03:55PM

the best way i found about learning about different coadd products is joining the Stack Club XD

Dan Taranu Thu 2020-08-13 03:55PM

Aaron: Were you showing single-band CModel magnitudes in your plots? There are also "forced" CModel magnitudes where the structural parameters are fixed to those from the highest signal-to-noise band; these have better colours (similar to fitting every band simultaneously).

Sugata Kaviraj Thu 2020-08-13 03:57PM: @A Emery Watkins
A Emery Watkins Thu 2020-08-13 03:57PM: I wanted to know exactly where the magnitudes I was measuring were coming from, so I grabbed the image cutouts with the Butler and measured the magnitudes myself directly from the images.
A Emery Watkins Thu 2020-08-13 03:58PM: The CModel mags did show similar oversubtraction, though (which I did also check at one point).
parejkoj Thu 2020-08-13 03:58PM: "measured" how?
A Emery Watkins Thu 2020-08-13 03:58PM: Very simple aperture photometry.
A Emery Watkins Thu 2020-08-13 03:59PM: @parejkoj If you subtract the deepCoadd_calexp without the model from the deepCoadd_calexp with the model, it leaves behind ONLY the model+sky subtraction. I made the galaxies well-separated enough that they didn't contaminate each other. Just wanted full control over the process, is why I did it this way.
Dan Taranu Thu 2020-08-13 04:22PM: I see. Well, I'd be curious to see how the forced CModel colours behave (from 'deepCoadd_forced_src' rather than 'deepCoadd_meas'). They're better in DC2, which I think indicates that the oversubtraction is similar across filters and so the colours come out fine even if magnitudes don't.

Song Huang Thu 2020-08-13 03:56PM

To echo @Yusra AlSayyad 's point, using HSC S18A or PDR2, we have found that the background over-subtraction has been improved by a lot. We even found that the overall background in g- and r-band slightly under-subtracted.

Sugata Kaviraj Thu 2020-08-13 03:58PM: thanks Song. this will be interesting for our work. @A Emery Watkins @C A Collins
Song Huang Thu 2020-08-13 03:59PM: A future grad-student, Jia-Xuan Li and I did the work and there is a publication in preparation. I am happy to share the early draft if it helps :blush:
Sarah Brough (she/her) Thu 2020-08-13 04:00PM: @Song Huang do you access through HSC-SSP or through the LSST Science Platform? Partially we've also been working on how to use TSP to undertake LSB science,
Sugata Kaviraj Thu 2020-08-13 04:00PM: that would be great! :slightly_smiling_face: mailto:s.kaviraj@herts.ac.uk
Sugata Kaviraj Thu 2020-08-13 04:00PM: thank you! :slightly_smiling_face:
Sarah Brough (she/her) Thu 2020-08-13 04:01PM: @Song Huang if you could share with @Cristina Martinez-Lombilla and myself too that would be great!
Song Huang Thu 2020-08-13 04:02PM: @Sarah Brough (she/her) We did the S18A analysis when I was at UCSC, so I just used the HSC-SSP website. Actually using the unagi package I wrote: https://github.com/dr-guangtou/unagi :blush:
A Emery Watkins Thu 2020-08-13 04:02PM: @Song Huang That's great to hear. I'd love to see an early draft as well.
Cristina Martinez-Lombilla Thu 2020-08-13 04:02PM: Yes, thank you!
Song Huang Thu 2020-08-13 04:03PM: Ok, great! Let me quickly check with the student and collaborators, but I am sure they will be Ok. I'll send it as soon as possible.
Song Huang Thu 2020-08-13 04:04PM: I think Yusra or Lee have mentioned this, but HSC (and LSST I presume) pipeline has this skyobject photometry that is quite useful for evaluating background subtraction. We found it quite useful.

Robert Lupton Thu 2020-08-13 04:04PM

This "random point" approach is essentially what HSC/Rubin does, using "sky" objects

Robert Lupton Thu 2020-08-13 04:05PM

@Morgan Schmitz is working on subtracting wings of stars

Mariano Dominguez Thu 2020-08-13 04:06PM

Where I could find information about the #lsb_challenge3?

Mireia Montes Thu 2020-08-13 04:06PM: That will be me :slightly_smiling_face:
Sarah Brough (she/her) Thu 2020-08-13 04:06PM: Also, there is an LSSTC slack channel with links to the challenge google doc etc. lsb-challenge3
Mariano Dominguez Thu 2020-08-13 04:11PM: Thank you, Sarah. I wondered if I could join it because our institute is going trough the in-kind contribution process.

Mireia Montes Thu 2020-08-13 04:06PM

Roberto Baena is working on huge HSC PSFs

Morgan Schmitz Thu 2020-08-13 04:07PM: Raul Infante-Sainz came to visit the group at Princeton and helped us get started; he showed us Roberto's extended HSC PSFs
Mireia Montes Thu 2020-08-13 04:08PM: Yep, they are the experts, I just copy them :slightly_smiling_face:

Robert Lupton Thu 2020-08-13 04:06PM

As a function of position in the HSC field? Or is the work we saw in Italy?

Mireia Montes Thu 2020-08-13 04:07PM: I am not sure if there are any new updates, but if it will as a function of position it will be life-saving
Robert Lupton Thu 2020-08-13 04:08PM: You should be interested in @Morgan Schmitz 's work --- I am!
Morgan Schmitz Thu 2020-08-13 04:10PM: Yep - spatial variation is definitely something we will be looking into very soon!
Mireia Montes Thu 2020-08-13 04:11PM: @Morgan Schmitz share it! Don't keep it to yourself
Robert Lupton Thu 2020-08-13 04:12PM: We understand some of the spatial structure for HSC based on the Arcturus data.
Morgan Schmitz Thu 2020-08-13 04:14PM: @Mireia Montes Will do! It is all done within the Rubin science pipelines, so you should be able to use it as soon as it's about ready. I'd be happy to ping you when that happens if you like

Jeff Carlin Thu 2020-08-13 04:08PM

Thanks to the speakers and organizers for a great session! :clap:

Colin Slater Thu 2020-08-13 04:12PM

The rotation-dependent flats that @Mireia Montes showed look exciting, is there any guess at a physical cause for this?

Mireia Montes Thu 2020-08-13 04:15PM: Tbh, Raul Infante-Sainz mentioned he saw something similar in his flats for another telescope so I tried as the i-band image showed weird residuals.
Colin Slater Thu 2020-08-13 04:24PM: it's a cool test!

Robert Lupton Thu 2020-08-13 04:13PM

In HSC?

Mariano Dominguez Thu 2020-08-13 04:13PM

Amazing session and science. Congrats! :clap:

Robert Lupton Thu 2020-08-13 04:13PM

The camera rotates relative to the optics

A Emery Watkins Thu 2020-08-13 04:16PM: Why does it do that? Seems odd.
Colin Slater Thu 2020-08-13 04:17PM: I'd guess the less weight on the rotator, the better
A Emery Watkins Thu 2020-08-13 04:17PM: Hm. So instead of rotating the whole tube, they just rotate the camera. I guess that makes sense for such a large telescope.

Robert Lupton Thu 2020-08-13 04:14PM

A similar problem to the new degrees of freedom needed in going from e.g. DECam to HSC/Rubin

Colin Slater Thu 2020-08-13 04:19PM

So just spatial variation in the transmission of certain optics? It should only matter for the elements where the beam is small

Robert Lupton Thu 2020-08-13 04:27PM

And the spider. And remember that HSC has 26% vignetting at the edge of the field, falling to 0% at c 10% of the field (so most of the field is vignetted)

Colin Slater Thu 2020-08-13 04:28PM

ah and it's the projection of the spider too. Ok, I can see that.

Mireia Montes Thu 2020-08-13 04:29PM

Another reason not to like spiders...

Lee Kelvin Thu 2020-08-13 04:55PM

A few questions that Zoomed by:

Lee Kelvin Thu 2020-08-13 04:56PM

@Benne Holwerda What exactly is "modeled mask"?

Lee Kelvin Thu 2020-08-13 04:58PM: A modelled mask is the procedure we refer to when we're talking about subtracting an initial Sersic model from your image before re-running a sky-estimation/source-extraction pipeline.

You need to run your favourite pipeline of interest first to get positions, magnitudes, etc for the brightest sources to facilitate modelling. Post modelling and subtraction, you apply the initial segmentation map (to ensure that any peculiarities in the core region of your model fit doesn't impact your sky estimation routine either).

It does a great job at removing contaminant flux in the wings of sources, and also appears ideally suited towards handling deep crowded field data - an important concern in the era of Rubin!

Lee Kelvin Thu 2020-08-13 04:59PM

@Benne Holwerda Which would be the optimized package to identify and remove the brightest sources with?

Lee Kelvin Thu 2020-08-13 05:01PM: All packages we tested (SExtractor, Gnuastro, LSST Science Pipelines) performed well at the bright end, only impacted by issues such as deblending. What matters is how these pipelines attribute faint flux to bright sources (e.g., in correctly attributing flux in the wings to their bright centroid).

We found that SExtractor is best for more compact sources (exponential sources, or point sources) whilst Gnuastro is better for more extended sources (e.g., elliptical galaxies).

Perhaps @Nushkia Chamba also has some thoughts on this?
Robert Lupton Thu 2020-08-13 05:05PM: The assumptions that "Gnuastro" uses are different, basically looking for the darkest part of the image. It is not clear that this is what you want for all purposes.
Nushkia Chamba Fri 2020-08-14 05:59PM: hey! Yes, I answered this question live but I didn't have time to write here as well. Something I forgot to mention in my talk is that the Max-Tree algorithm performs statistical tests to ensure that everything it detects is really a source (above a significance level alpha that the user can set, this is one of the two parameters in the settings, but the default is already good enough). But it assumes a constant background to build the tree, so I dont think at this stage of the development whether it can be used for reducing data....I have not tried. But the nice thing is that the background estimator can be changed/improved and the group in Groningen is working on this.

Yes, SExtractor (SE) is better for compact sources because otherwise it generates a lot of false positives, (but something I also forgot to mention in my talk is that a lot of people run SE multiple times and combine the maps...which in all honesty, they should not be doing for any algorithm because you can create artifacts and there is no way to control what happens...) and with NoiseChisel, yes, it may perform for elliptical objects better because they do not have much substructure as compared to spirals, but the main issue is the # of parameters which the author says that they are independent and in fact, the manual says that the tool is meant to the hand tuned, each parameter after the other. I have tried doing this but it doesn't always work as expected and the user can literally tune parameters infinitely without any objective criteria to stop! So when I said a nightmare, I really meant it XD Algorithms > 10 parameters and when you have no idea how each parameters affects the other clearly, makes hand-tuning hard.

So this is why, at this stage, I recommended the Max-Tree approach in general, not perfect, as I showed in my slides it sometimes represents nested sources incorrectly when the sources nearby have similar brightness, but the tool is stable, easy to use and you can visually mask things easier as the segmap follows the shapes of the objects and nested sources well.

This work will be published in a paper, we have re-submitted this summer and hopefully it should be out in the Fall :slightly_smiling_face: Will keep you updated!

I want to talk to @Remyj about SCARLET, because right now the tools we have compared do some kind of de-blending, but this is not in the Max-Tree yet. and SCARLET would be what LSST will use so its probably worth to explore this tool as well using our measures.
Remyj Fri 2020-08-14 07:08PM: Yes Nushkia, I wanted to talk to you too after your great talks! Right now, we are working to have scarlet being able to model faint diffuse sources with wavelets while other sources can be our usual compact objects. It work well for low surface brightness galaxies, but remains hard to automatise.
I understand that one of your issues with scarlet is that it is not performant enough for large images and this is a problem. We are working hard on making scarlet more efficient and Fred recently reported very encouraging results, so we are getting there.
Nushkia Chamba Sat 2020-08-15 05:10AM: thank you very much and I am glad to hear that! I really hope our paper will be out by the end of the year and you could get some ideas from there too. We have learned a lot through the comparison and we hope that it could eventually direct further improvements in the development of all these algorithms. Let's keep in touch. I am off on holiday soon until the end of the month.
Remyj Mon 2020-08-17 11:09AM: Enjoy your holiday! When you come back, if you have data that you can share with me, I could try my methods on them.

Lee Kelvin Thu 2020-08-13 05:03PM

@Yoon6768 What is the surface brightness level that all of these methods give a consistent result?

Lee Kelvin Thu 2020-08-13 05:08PM: All packages we explored here perform well above mu_r~25 mag/arcsec^2. With that said, the issue is in how well each method correctly attributes flux to the wings of bright sources at surface brightnesses fainter than this. Top-down techniques (such as SExtractor or the LSST Pipelines) perform well with sources with relatively compact wings (e.g., spiral galaxies, most stars), whilst bottom-up noise detection routines such as Gnuastro perform better with elliptical galaxy type extended sources.

Lee Kelvin Thu 2020-08-13 05:10PM

Thanks again @Nushkia Chamba , @A Emery Watkins , @Cristina Martinez-Lombilla , @Mireia Montes , @Yuanyuan Zhang for an excellent session, and thanks to all who participated in the active Q&A!

Lee Kelvin Thu 2020-08-13 05:11PM

Please feel free to continue the discussion here, and looking forward to seeing you all again at a future meeting. :clinking_glasses:

Sarah Brough (she/her) Thu 2020-08-13 05:43PM

Hi All, I also just wanted to add that if you're interested in LSB light with LSST you may be interested in joining the Galaxies Science Collaboration and its Low Surface Brightness Working Group where we are currently running 4 data Challenges working towards using LSST for LSB science. There's more information on the Challenges on LSSTC slack at the galaxies-lsb and the lsb-challenge1 etc. channels. You can join the LSB WG by emailing me, and get involved!

Robert Lupton Thu 2020-08-13 05:50PM

With lots of Project folk involved, and eager to work with you.

Nimish Hathi Fri 2020-08-14 02:45PM

Just curious, why 'Live Recording' /youtube link gives error (Video unavailable, This video is private) at https://project.lsst.org/meetings/rubin2020/agenda/session/low-surface-brightness-science-and-robust-sky-estimation ?

Lee Kelvin Fri 2020-08-14 02:56PM: It works for me, but I had that message on the page earlier today - perhaps try refreshing the page?
Nimish Hathi Fri 2020-08-14 02:58PM: Thanks. Yes, it works after I restarted my browser

Ranpal (she/her/hers) Fri 2020-08-14 04:58PM

The recording of the live session is here: https://youtu.be/QZsrdH3NkRo

Fri 2020-08-14 05:59PM

@Remyj has joined the channel