Sharpening with Gaussian and edge-aware blurring kernels; a new experimental approach on High-Radius Low-Amount sharpening; how to separately target edges and texture in the same high frequency range. Gaussian, Bilateral and Mixed pyramid decompositions, efficient platforms on the top of which new sharpening strategies can be developed.
By Davide Barranca
This article is a work in progress collection of personal notes on the subject of sharpening; I’m no digital image processing scientist so, even though I like to play with theoretical problems, I try to ﬁnd my answers within Photoshop (as I suppose they all do in Dan Margulis’ Color Theory Yahoo group, to which attention I ﬁrst would like to turn this one). Chances are that I won’t be rigorous too, my aim is to better understand the subject and share my ﬁndings with people who would like to integrate them, or simply give a feedback on the techniques exposed. If you’re not interested in all the (very trivial indeed) math and graphs, feel free to skip to the how-to sections: nevertheless, I hope everything will be food for thoughts.
What I’m suggesting with this article is, among the rest, the possibility to target with appropriate sharpening different image features, even if they belongs to the same spatial frequency range (plus an experimental approach to high-radius low-amount sharpening). Then I’m showing how to use bilateral and mixed pyramid decompositions to modulate the sharpening within all the image frequencies.
2. Gaussian sharpening
There’s plenty of resources on the web about the subject so I skip much of the basics of Photoshop USM (UnSharpMasking) and its use, why we need it, how to build masks, etc. Let me only stress again that it’s called UnSharp because it assumes the subtraction from the original picture of a blurred (unsharp) version. I’ll use a lot the “subtraction”, so here it goes a quick reminder on it, and why sharpening and blurring are close relatives inside Filter → Sharpen → Unsharp Mask…
A quick note for the reader: Grayscale images will be used throughout the article, to help us keeping the focus on tonal transitions. To mimic the effects with color pictures, use L of Lab or apply a solid white layer Color mode to have what Photoshop would consider a grayscale Luminosity version of your original. First picture (courtesy of my friend the photographer Roberto Bigano) is the starting point; then goes the blurred version and the subtraction:
Adding the subtraction to the original makes the sharpened version:
To do this in Photoshop, I may use offset and scaling, which makes the difference channel look more familiar:
A more abstract way to visualize the effect of Gaussian Sharpening, that I personally ﬁnd useful for comparison purposes, is to plot the signal intensity transition (black) and its blurred version (red), their difference (green), and the original plus the difference (thicker black):
The key concept of Gaussian Sharpening is that the difference between original and blurred version is exactly what will be enhanced. If it’s used a blurring kernel (Gaussian Blur, GB, for instance) which softens edges and texture, then edges and texture will be more prominent when the difference layer will be applied to the original. Depending on the algorithm used and the processing of the blurred picture, we may end with a sharpening that affects separately different image features.
Before going any further, let me review Photoshop’s Calculations and how to use it to sharpen. Open a picture, convert to the grayscale ﬂavor you like the most and:
- duplicate the Gray channel twice;
- call the ﬁrst ORIG and the second BLUR;
- apply a GB to BLUR
- go to the menu Image, Calculation; if you want to perform a simple subtraction: ORIG – BLUR; (Eq. 2.1)
You should setup the Calculations window as follows:
Pay attention that the first term of the subtraction is Source #2, and the second is Source #1 (little confusing, I know). The result you have is not scaled, and for it to be applied with the appropriate blending mode (Linear Light, LL from now on) there are two slightly different ways. Let’s baptize SS1 the Subtraction with Scale = 1 which implies a later LL blend 50% opacity, and SS2 the Subtraction with Scale = 2 which implies a later LL blend 100% opacity. Both ways are useful as we’ll see soon.
Having tested a bit more the subject, I can now affirm that SS1 (LL 50%) and SS2 (LL 100%) are not exactly the same thing, so for the sake of precision, I’m suggesting you to use SS2 only. The issues in SS1 reveals in pyramid decomposition, i.e. blacks not really black (something you can easily test); nevertheless, if you’re not in the middle of a decomposition (an image one, of course 😉 you can pick either SS1 or SS2.
3. Difference of Gaussians
Let’s shift a bit our point of view; what if, instead of subtracting a blurred version from the untouched original, the subtraction is between two differently blurred originals? Have a look to the graph before we investigate the why of such a move with GBs:
The concept behind the Difference of Gaussians (DoG) is quite simple: noise is usually high frequency spatial information, and it’s blown away in both blurred version, so won’t be boosted; on the other hand the two versions keep detail in different frequency ranges. So their subtraction is a way to enhance, precisely, that frequency window:
I’m recalling here the DoG because it represent a ﬁrst step off the traditional sharpening track: I can report that it’s usually applied using a blur ratio ranging from 4:1 to 5:1, while a ratio of 1.6 mimics the Laplacian of Gaussian (LoG: an operator that calculates the second derivative of signal intensity, so it’s good for instance in ﬁnding edges. I plan to add more on LoG here later on).
I still have to test it extensively: nevertheless an important feature that should be noted is its resemblance to HiRaLoAm (as Dan Margulis uses to call USM with High Radius Low Amount). But there’s a remarkable difference: namely, that edges are less or not sharpened at all, because they belong to the high frequency detail window that’s untouched by DoG (less or no difference between edges in GB1 and GB2, so less or no boosting at all).This could be a beneﬁt in workﬂows where different sharpening rounds are planned and an extra step is worth its time.
4. Other blurring kernels
Since it’s clear that subtraction and blurring are the core of this kind of sharpening, I started wondering what if other kernels are used instead of, or at the same time with, GB.
Let’s take Surface Blur (SB), aka Bilateral Filter, a well known edge preserving blurring algorithm. If it keeps edges and wipes the rest out it should lead us to think: SB equal something to enhance everything but the edges. Let’s see:
It works as it’s supposed to do. Besides the fact that SB isn’t the fastest ﬁlter ever in the Photoshop arsenal, there are more annoying things. It’s required a bit of experience to ﬁnd the correct radius/threshold couple; due to the algorithm itself, at large radii a reversion starts to appear and instead of more blurring we get more edges coming back:
A much better edge preserving ﬁlter is the Weighted Least Squares (WLS) operator. Unless you are an experienced programmer or you ﬁnd the way to put your hands on a computer with MatLab and Photoshop both installed, to use it is quite a problem.The results are remarkable indeed, check the links at the bottom of the page.
5. Combined use of different blurring kernels
Here I would like to suggest the joined use of a GB and SF ﬁlters (i.e. two kernels which differs in their edge-awareness) in order to modulate the sharpening. I’m going to write some very simple and very unorthodox math; being O the Original picture, T and E respectively the Texture and Edge features of the picture (both belonging to the high frequency spatial range) we’ve seen that:
GB = O – (T+E); (Eq. 5.1)
SB = O – T; (Eq. 5.2)
A subtraction gives the Edge only component:
(SB – GB) = O – T – O + T + E = E; (Eq. 5.3)
While, rearranging the second equation, we prove that Texture is boosted by SB:
(O – SB) = T; (Eq. 5.4)
This should give us all the instruments needed to modulate the sharpening in texture and edges separately, via subtraction layers/channels to be blended in LL mode. Let’s see. Following images are GB, SB versions and the scaled subtractions we’ve talked about.
What we should see is a Edges only sharpening. So we’ve been able to get a Texture and Edges sharpening (with GB), a Texture only sharpening (with SB) and an Edges only sharpening (using SB -GB). More pronounced effect can be obtained by higher opacity of the LL layers (when possible) or a sigmoid (aka S-shaped) curves adjustment layer clipped to it, whose opacity can be tweaked as well. In the ToDo list there are actions/scripts (and even a Photoshop CS4 panel) to automate the process.
6. Image decomposition
Until now we’ve tried to modulate the sharpening into the same, high frequency range (edges, texture). But an image usually contains several different frequencies: higher ones correspond to ﬁner detail (hair, for instance), lower ones to large, smoother tonal transitions (like the cheeks in a portrait). Now it’s time to try to target those different frequency ranges with appropriate, different sharpening. Pyramid decomposition is just… a way to decompose an image into several frequency ranges, and that makes easier to achieve our end.
Very simple math lies under image decomposition. I’ll use GB ﬁlter (so I’ll build a Gaussian Pyramid of 3 levels, but you can extend it to as many levels as you like); being O the Original picture, GBn(O) the GB ﬁlter applied to O with radius increasing with n:
U0 = O; (Eq. 6.1)
U1 = GB1(O); (Eq. 6.2)
U2 = GB2(O); (Eq. 6.3)
U3 = GB3(O); (Eq. 6.4)
We deﬁne differences Dn as follows:
D1 = U0 – U1; (Eq. 6.5)
D2 = U1 – U2; (Eq. 6.6)
D3 = U2 – U3; (Eq. 6.7)
So the image can be decomposed into:
O = U3 + D1 + D2 + D3; (Eq. 6.8)
In fact, substituting all the elements we get:
O = U3 + U0 – U1 + U1 – U2 + U2 – U3 = U0 = O; (Eq. 6.9)
Let’s switch to Photoshop and try to build this pyramid. We only need to perform GB ﬁltering and subtractions, something we should be familiar with, by now.
The three GB radius will deﬁne the frequency range: I’ve chosen small radii (i.e. 1px, 5px, 15px) because the original picture is quite small; depending on the resolution of yours, they may vary. I can suggest you to select the smaller radius as the one you would use when applying conventional USM ﬁlter, the larger one corresponding to HiRaLoAm radius, and the middle one, guess what, somewhere in between. Here are the three U1, U2, U3 and the D1, D2, D3 as we deﬁned them (I’m using SS2):
Having all the elements, it’s now time to compose the pyramid on the Layers palette.You should make a new Set which contains bottom-up: the most blurred version U3, then D1, D2, D3, all the three LL mode, 100% opacity (remember, I used SS2). Here is how should look like your Channels and Layers palette:
Don’t feel afraid by the mess here, it’s possible to automate everything (ToDo list +1) and end up with a nice tidy Channels palette. Switching on and off the Set should make no difference – and hence the decomposition works: great! So what?
7. A sharpening equalizer
Our recently decomposed picture still smells good, and moreover offers the possibility to build a 3-sliders sharpening equalizer very quickly (to add as many sliders as you want, just increase the numbers of the decomposition levels). If you made the subtractions SS1 then LL blend 50% you can already play with opacity sliders to boost high, middle and low frequencies at the same time. Personally, I ﬁnd more customizable to clip a rough S-shaped curve adjustment layer to each of the Dn layers and drag those opacity sliders:
It reminds me the KPT Equalizer plugin (well, surely it lacks its peculiar looking UI): you can go negative, so not sharpening but smoothing a frequency range, via lowering the opacity under 50% if you used SS1 or lowering the opacity of the Dn layers in SS2. Let’s make a version:
8. Bilateral and WLS Pyramids
We are allowed to use different blurring kernels, the equations of pyramid decomposition still work. Get a look, for instance, to the better SB:
Can you spot the difference between Bilateral (read: which uses SB) and Gaussian pyramid equalizers? Maybe not that much here, but if you’ll use one of your (high res, maybe high bit-depth) pictures, you’ll ﬁnd that being edge-aware, the Bilateral pyramid shows little or no halos, which is quite a remarkable feature in my opinion. Shadow/Highlights with SB (instead of GB) in its “engine” gives the same halo-free look by the way.
As I’ve said before,WLS ﬁlter is a much better edge-aware smoothing operator, therefore it can be used for even better enhancements (have a look to the website of WLS’ creators for examples). Adobe, anyone listening?
9. Mixed Pyramids
Strangely enough if you will, we’re allowed to mix blurring kernels and still the equations work. If you remember: 5. Combined use of different blurring kernels, I’ve found out that:
(SB – GB) = E; (Eq. 9.1)
If we modify the assumptions of the Pyramid decomposition as follows:
U0 = O; (Eq. 9.2)
U1 = SB1(O); (Eq. 9.3)
U2 = GB1(O); (Eq. 9.4)
we end up with a 2 level decomposition with:
D1 = U0 – U1 =O – SB1(O) = T; (Eq. 9.5)
Like we know from eq. 5.4.We then have:
D2 = U1 – U2 =SB1(O) – GB1(O) = E; (Eq. 9.6)
which comes from eq. 5.3. So, D1 is a frequency layer of a mixed decomposition which contains, and hence will be able to enhance,Texture alone; and D2 is a frequency layer of a mixed decomposition which contains, and hence will be able to enhance, Edges alone.
Here are the difference layers:
I’ll show you the full Channels and Layers palettes:
Then I’ll add the same curve adjustments layer clipped to the Dn and play with sliders. Here is an example version:
10. (temporary) Conclusions
I’ve tried to collect here material coming from various sources and my own ﬁndings around the (very personal indeed) subject of sharpening. I was particularly interested in showing how to mix blurring kernels and why pyramid decomposition is a platform on the top of which many sharpening strategies can be developed successfully. Finding new ways to use old tools in order to accomplish even slightly sophisticated tasks pays for the time spent on the project. I still have many open questions, for instance how to simulate the USM’s threshold slider via channel blending (or masks, even though I’d prefer to do the masking with calculations).
Nevertheless I’m pretty happy with the results by now: using those workﬂows means adding a lot of extra steps, even when I’ll ﬁnd the time to put together actions/scripts to automatize the monkey work; I don’t know whether someone but me would ever try to use them in production environments (it depends on the kind of production, by the way). But, again, it’s food for thoughts: I’ve always believed in knowledge sharing and plural researches, so I’m waiting for suggestions, corrections, and smarter workﬂows to sharpen. I plan to keep the article updated, so check for revision number at the top of the page – I have a big fat to-do list (actions, scripts, etc). Drop me an email if you like to be warned when something new appears or just want to give feedback.
Lives and works near Bologna, Italy.
He’s the developer of ALCE, VitaminBW, Double USM, PS Projects and Floating Adjustments.
Davide is also a color-management aware photo-retoucher, focused in color-correction and image enhancement in fine-art photography, art reproduction photography and fine-art digital printing. Interested in academic research around digital imaging, stitching, HDRI, custom filters writing.
Specialties: Pre-press, broad experience in working side by side with photographers trying to convert from artist to technical language.
You can visit his site here >
One of the best writing about the subject I’ve seen.
It’s great you share such thing.