牧夫天文网

 找回密码
 加入牧夫(请注明天文爱好者,否则无法通过审核)

QQ登录

只需一步,快速开始

[天文活动] 老外那边的暗场无用论的几种形态——只转帖, for CMOS only

[复制链接]
ecloud 发表于 2014-5-7 04:08 | 显示全部楼层 |阅读模式 来自: 辽宁省大连市 联通

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有账号?加入牧夫(请注明天文爱好者,否则无法通过审核)

x
One thing to note on this particular forum is that one of the technical hurdles in the development of CMOS was fixed pattern noise. Unlike a CCD, each sensor on the array has it's own amplifier with it's own gain and offset. Precisely balancing millions of detector-amplifier pairs was a huge problem that I suspect has been solved in modern cameras with better manufacturing control and a lot of very low level processing. It's not perfect and under the right conditions, the pattern can become pretty obvious. It is different than what you typically see out of a CCD.
 楼主| ecloud 发表于 2014-5-7 04:09 | 显示全部楼层 来自: 辽宁省大连市 联通
Hey,

This is a great topic and appears in various forms often. Here's my take on astrophotographic noise and its reduction. In keeping it simple, I'm sure I will make a generalization or two that can be argued with... but with a larger goal in mind, here goes.

In low light photography, there are many sources of image noise, (dark noise, bias noise, photon noise, shot noise, thermal noise, ISO noise, stuck pixels, etc). All are things that are unwanted (hence noise), but they all boil down to two fundamental types: true random noise, and pattern noise. There are many ways to label and describe these two, but a convenient pair of labels to me is non-correlated and correlated noise.

True random or non-correlated noise is what it says: it truly is a random, totally unpredictable sequence of values. True random values are non-correlated: there is no signal present, nothing that repeats with a pattern. No matter how long you look at it, nothing repeatable emerges. Taking advantage of that fact, the more times you average the values, the more you reduce the value of random noise. In order to half the noise, you need to take 4 times the averages. (EDIT: I originally stated "squaring the averages, halves the noise", wrong! Noise is reduced by the inverse square root of the number of averages.) ISO noise is pretty much pure random in that it varies across the image from pixel to pixel and varies at each pixel image to image.

Pattern or Correlated noise may appear to the casual observer to be random, but if you look at it long enough a repeating or correlated value emerges. Many correlated noise sources have a true random component superimposed on a correlated or repeating signal. If there is no random component, the repeating or correlated portion of the signal appears immediately. A stuck hot or dead pixel is a very obvious signal with no random component. You see it in each and every image. No matter how many times you average, it just stays the same. It is correlated between frames. you can't average it away.

An interesting example of pattern noise is "poisson noise" such as the number of letters you get in a week. Over a period of a year, if you count up all the mail you get, and got 572 pieces, it would average to 11 pieces of mail per week. (572/52) During the year you might get 9 pieces one week and 13 pieces the next, averaging to 11 per week. Following weeks might be 35 pieces one week, 25 another, and 0 three weeks in a row and 6 the last. Again averaging 11 pieces per week, even though you never actually got 11 pieces in a week. No matter how long you observe, (well, as long as you watch for at least a few weeks) it will always tend to work its way to an 11 piece average. This is an example of pattern noise that has a random component, but a signal hiding below.

Lets look at a few typical noises you experience in an astro image:

A satellite streak: Certainly obvious, predictable and correlated from pixel to pixel in a single frame. But, it doesn't appear in any subsequent frames, so it can be considered "correlated within a frame" but "non-correlated across frames". Averaging a bunch of images will eventually remove or minimize the satellite. However, if it was a REALLY bright Iridium flare, it's gonna take awhile to be removed. As Samir pointed out, there are better techniques for removing satellites and stuck pixels.

Thermal noise: one of the biggies in astrophotography. At first glance, this can appear random across a single image, and even random at a single pixel for a few images. But it is similar to poisson noise: there is an underlying signal caused by a temperature reaction unique and repeatable at each pixel. For a given exposure and temperature, the underlying thermal noise has the same fundamental value, but it varies slightly from image to image. So there is a random component between images, but a fundamental value that is always there. If you average enough images together, like the mailbox, the random component averages towards zero and the fundamental dark value remains. Averaging removes the random component but only reveals the correlated pattern noise beneath.

Okay, now that we know the kinds of noise, we can attack how to remove them. A different technique is required for each.

Pure random noise is removed by averaging a bunch of images together. The more averages, the less noise. However, any signal that is present (desired ones like a star and undesired ones like thermal noise) remain and emerge clearer and clearer.

So averaging gets rid of random noise. How do we get rid of correlated or pattern noise? With dark frames or dithering.

Dark frames: when you shoot a series of dark frames at a single temperature and exposure length and average them, the random component averages towards zero but the thermal and pattern noise emerges as a signal. The same signal that is present in all your sky exposures. If you subtract an averaged dark frame from a light frame, the pattern component is removed from the image. Obviously a single dark frame does not do a lot of good since it still contains the frame-to-frame random component. Hence, that's why we take 10, 25, 50 or whatever dark frames. They have to be at the same temperature and exposure length as the lights since the correlated dark signal changes as the temperature and exposure length changes. And, to top it off, thermal noise is non-linear. A change in temperature affects different pixels differently by different amounts. Shucks. Gotta have a suite of dark frames.

Dithering: This is a cool example. As explained by other posts in this thread, dithering is the act of shifting the sensor position a few pixels between images. Therefore the image appears shifted across the sensor. As long as you dither with random amounts 3 pixels, then 5 pixels, then back 7 pixels, then 4 pixels etc... the pattern noise which was correlated between images before is no longer correlated when shifted some amount each time. Since the dark noise is no longer correlated, averaging a series of images removes that noise.

Summary: Averaging removes random noise. Dark frame subtraction removes pattern noise. Dithering converts pattern noise into random noise so averaging begins to work again.

Back to the original post: Whew. After all that, if you do random dithering between each frame, then average a series of re-aligned frames, both random and pattern noise is reduced. So you don't HAVE to subtract darks too. That being said, if you subtract dark frames from each light AND dither, you will reduce the pattern noise faster, so for any given number of averages, you should end up with still less noise. There is benefit to doing both, although dithering alone really helps. However, that assumes you take the time to make good darks. If the sensor temperature is significantly different in your darks than when you shoot lights... you are actually introducing further pattern noise.

Interesting corollary: I shoot a lot with my CPC1100 on an alt-az mount. I can generally not shoot more than 30 second frames (rule of thumb) because of field rotation. Ahhh! Field rotation is similar to dithering! If you shoot 50 frames with field rotation between each one, align them and average them, you effectively remove the pattern or thermal noise for the same reason dithering works. However, you statistically remove less in the middle of the image than around the periphery where the apparent motion is higher.

So, I often get some nice results when shooting alt-az, even without using dark frame removal.

Double whew! Long winded. But I fell better now.

Best regards,

Jimmy the Geek, in Boulder
人类的灵魂还是没有从重力的束缚中解脱出来
回复 支持 反对

使用道具 举报

 楼主| ecloud 发表于 2014-5-7 04:15 | 显示全部楼层 来自: 辽宁省大连市 联通
Here is a contrary view. Don't do darks. Don't do calibration. If your Canon model is relatively new (i.e., less than 5 years old) the amount of noise that darks and calibration will eliminate can easily be taken care of in Photoshop's Camera Raw. This is the advice of Roger Clark, who does a lot of technical evaluation work on Canons (http://www.clarkvision.com/) and Tony Hallas who has a great video out on using Camera Raw for astronomical image processing. I don't do calibration with my modified Canon 6D images. Admittedly, that camera has very low noise. But I have also processed images from a modfied Canon 20D and the result was the same... no noise problems and great color.

Give it a try. Just load your RAW images into Bridge and then open in Camera Raw. If you select all and synch you can do them all at once. Use the luminance noise reduction slider to eliminate any graininess at 100%. The save as 16 bit TIFF and go from there.

George
回复 支持 反对

使用道具 举报

lygg 发表于 2014-5-7 07:55 | 显示全部楼层 来自: 北京市 电信IDC机房
求翻译~
回复 支持 反对

使用道具 举报

大大大老虎 发表于 2014-5-7 10:38 | 显示全部楼层 来自: 辽宁省大连市 电信
方言看不懂。
回复 支持 反对

使用道具 举报

northwolfwu 发表于 2014-5-8 09:08 | 显示全部楼层 来自: 江苏省南京市 电信
不错的分析!
文章的主要论点是:单反的暗场,必须是同一温度,要叠加很多张(10张以上)才能有效果,现实是没有那么多时间拍,也不能保证温度一致,因此暗场的质量不能达到要求,作用不大.
推荐的方法是:佳能用户用Camera Raw将所有原片的亮度通道降噪(减少杂色)到100%,消除背景的颗粒状噪声,转换成tiff后再进行后续的处理.

Tony Hallas 的文章我看过,他推荐的方法是至少9张高度Dither的原片,然后再进行camera raw处理,不用暗场,不用平场.镜头要缩小光圈,尽量减少暗角的影响.

个人的评价:值得一试,不过必须dither拍摄,这样CMOS噪声才能随机化,而且dither的幅度要大,效果才好.暗场有时间还是要拍,有总比没有好,而且温度的影响并不是想象的那么大.
Camera Raw降噪100%感觉太暴力了,背景的暗星云都被抹掉了,实际操作的时候还是要温柔些.

点评

我个人的意见是,如果一定要靠软件后期降噪,那么应该使用原厂软件的RAW降噪功能,而不是第三方的,那样效果才会最好 不过操作上比较累人,就算批处理也很慢,似乎得不偿失 还有一种方法是深度抖动+数张相机自降噪  详情 回复 发表于 2014-5-8 13:54
回复 支持 反对

使用道具 举报

 楼主| ecloud 发表于 2014-5-8 13:54 | 显示全部楼层 来自: 辽宁省大连市 联通
northwolfwu 发表于 2014-5-8 09:08
不错的分析!
文章的主要论点是:单反的暗场,必须是同一温度,要叠加很多张(10张以上)才能有效果,现实是 ...

我个人的意见是,如果一定要靠软件后期降噪,那么应该使用原厂软件的RAW降噪功能,而不是第三方的,那样效果才会最好
不过操作上比较累人,就算批处理也很慢,似乎得不偿失

还有一种方法是深度抖动+数张相机自降噪的照片叠加,不需要拍暗场,也不用软件后期降噪。相机自降噪其实就是紧接着拍一张同样参数的暗场,温度符合性很好
比如,10张一组,之后跟着2张开机身降噪的。然后冷却3分钟,接着下一组,这样拍暗场的数量不会太多,不会占用太长时间,在一组末尾cmos温度最高的时候使用两张机身自动暗场的降噪照片,也有很大帮助

如果暗角不是很大并且不需要追求边缘成像和拼接,那么具有超声波除尘功能的单反就完全不需要拍平场了
回复 支持 反对

使用道具 举报

rottenweed 发表于 2014-5-8 14:42 | 显示全部楼层 来自: 上海市 联通
CMOS sensor的fix pattern问题比较大,这个不能通过叠加来消除。
必须每张图片减去fix pattern后再叠加。

点评

这个还有个体差异和型号差异,甚至是ISO差异,温度差异 对于比较老的cmos来说,的确fix pattern,可是新一些的,一点也不fix,这里面也有牵扯到佳能半幅cmos,佳能全幅cmos和Sony cmos的不同风格,只能自己试验得出  详情 回复 发表于 2014-5-8 16:18
回复 支持 反对

使用道具 举报

 楼主| ecloud 发表于 2014-5-8 16:18 | 显示全部楼层 来自: 美国 花旗集团
rottenweed 发表于 2014-5-8 14:42
CMOS sensor的fix pattern问题比较大,这个不能通过叠加来消除。
必须每张图片减去fix pattern后再叠加。 ...

这个还有个体差异和型号差异,甚至是ISO差异,温度差异
对于比较老的cmos来说,的确fix pattern,可是新一些的,一点也不fix,这里面也有牵扯到佳能半幅cmos,佳能全幅cmos和Sony cmos的不同风格,只能自己试验得出
比如我自己的那台K5是在ISO2000才开始变得fix起来,而且只限于某一个局部。1600的话,几乎都是random的。听起来似乎不科学,但是其实有道理。因为对于cmos来说,任何一个电路参数的改变,都可能对每一个cmos单元产生不同的影响。因为每个单元所装入的基准AD参数都不一样,外接参数的变化就会对于不同基准参数的cmos单元产生不同的影响
换句话说,cmos的fix pattern对于温度和ISO异常敏感

深度抖动就是把fix变成random的过程
最好的解决fix pattern的方法就是用机身降噪实时拍一张暗场,但是出于效率考虑,不能总这么干
所以我得出的结论是,以抖动为主,辅助以高温情况下的实时暗场降噪。两张机身实时暗场降噪的片子加入叠加序列的话,会使之前的10张的random程度加倍
同样是12张明场+2张暗场的时间。10抖动+2机身降噪,效果应该是好于传统12-2的叠加模式的。当然这样不如dark library省时,但是dark library的暗场温度差异并不能很好的解决fix pattern
回复 支持 反对

使用道具 举报

宋城冬叶 发表于 2015-6-13 06:33 | 显示全部楼层 来自: 河南省郑州市 电信
请教下抖动是不是让目标偏移一点啊

点评

至少5个像素  详情 回复 发表于 2015-6-13 14:52
回复 支持 反对

使用道具 举报

 楼主| ecloud 发表于 2015-6-13 14:52 | 显示全部楼层 来自: 辽宁省大连市 电信
宋城冬叶 发表于 2015-6-13 06:33
请教下抖动是不是让目标偏移一点啊

至少5个像素
回复 支持 反对

使用道具 举报

zbydd 发表于 2015-9-11 18:36 | 显示全部楼层 来自: 辽宁省大连市 联通
提示: 作者被禁止或删除 内容自动屏蔽
回复 支持 反对

使用道具 举报

本版积分规则

快速回复 返回顶部 返回列表