Ed Schlotzhauer

An artists journey

Category: Photography

  • How We Get Color Images

    How We Get Color Images

    Have you ever considered that that great sensor in your camera only sees in black & white? How, then, do we get color images? It turns out that there is some very interesting and complicated Engineering involved behind the scenes. I will try to give an idea of it without getting too technical.

    Sensor

    I have discussed digital camera sensors before. They are marvelous, unbelievably complicated and sophisticated chips. But they are, still, a passive collector of photons (light) that falls on them.

    An individual imaging site is a small area that collects light and turns it into an electrical signal that can be read and stored. The sensor packs an unimaginable number of these sites into a chip. A “full frame” sensor has an imaging area of 24mm x 36mm, approximately 1 inch by 1.5 inch. My sensor divides that area into 47 million image sites, or pixels. It is called “full frame” because that was the historical size of a 35mm film frame.

    But, and this is what most of us miss, the sensor is color blind. It receives and records all frequencies in the visible range. In the film days it would be called panchromatic. That is just a fancy word to say it records in black & white all the tones we typically see across all the colors.

    This would be awesome if we only shot black & white. Most of us would reject that.

    Need to introduce selective color

    So to be able to give us color, the sensor needs to be able to selectively respond to the color ranges we perceive. This is typically Red, Green, and Blue, since these are “primary” colors that can be mixed to create the whole range.

    Several techniques have been proposed and tried. A commercially successful implementation is Sigma’s Foveon design. It basically stacks three sensor chips on top of each other. The layers are designed so that shorter wavelengths (blue) are absorbed by the top layer, medium wavelengths (green) are absorbed by the middle layer, and long wavelengths (red) are absorbed by the bottom layer. A very cleaver idea, but it is expensive to manufacture and has problems with noise.

    Perfect color separation could be achieved using three sensors with a large color filter over each. Unfortunately this requires a very complex and precise arrangement of mirrors or prisms to split the incoming light to the three sensors. In the process, it reduces the amount of light hitting each sensor, causing problems with image capture range and noise. It is also very difficult and expensive to manufacture and requires 3 full size sensors. Since the sensor is usually the most expensive component of a camera, this prices it out of competition.

    Other things have been tried, such as a spinning color wheel over the sensor. If the exposure is captured in sync with the wheel rotation then 3 images could be exposed in rapid sequence giving the 3 colors. Obviously this imposes a lot of limitations on photographers, since the rotation speed has to match the shutter speed. A real problem for very long or very short exposures or moving subjects.

    Bayer filter

    Thankfully, a practical solution was developed by Bryce Bayer of Kodak. It was patented in 1976, but the patent has expired and the design is freely used by almost all camera manufacturers.

    The brilliance of this was to enable color sensing with a single sensor by placing a color filter array (CFA) over the sensor to make each pixel site respond to only one color. You may have seen pictures of it. Here is a representation of the design:

    Bayer Filter Array, from Richard Butler, DPReview Mar 29, 2017

    The gray grid at the bottom represents the sensor. Each cell is a photo site. Directly over the sensor has been placed an array of colored filters. One filter above each photo site. Each filter is either red or green or blue. Note that there are twice as many green filters as either red or blue. This is important.

    But wait, we expect that each pixel in our image contains full RGB color information. With this filter array each pixel only sees one color. How does this work?

    It works through some brilliant Engineering with a bit of magic sprinkled in. Full color information for each pixel is constructed by interpolating based on the colors of surrounding pixels.

    Restore resolution

    Some sophisticated calculations have to be done to calculate the color information for each pixel. This makes each pixel end up with full RGB color values. The process is termed “demosaicking” in tech speak.

    I promised to keep it simple. Here is a very simple illustration. In the figure below, if we wanted to derive a value of green for the cell in the center, labeled 5, we could average the green values of the surrounding cells. So an estimate of the green value for cell red5 is (green2+green6+green8+green4)/4

    From Demosaicking: Color Filter Array Interpolation, IEEE Signal Processing Magazine, January 2005

    This is a very oversimplified description. If you want to get in a little deeper here is an article that talks about some of the considerations without getting too mathematical. Or this one is much deeper but has some good information.

    The real world is much more messy. Many special cases have to be accounted for. For instance, sharp edges have to be dealt with specially to avoid color fringing problems. Many other considerations such as balancing the colors complicate the algorithms. It is very sophisticated. The algorithms have been tweaked for over 40 years since Mr. Bayer invented the technique. They are generally very good now.

    Thank you, Mr. Bayer. It has proven to be a very useful solution to a difficult problem.

    All images interpolated

    I want to emphasize a point that basically ALL images are interpolated to reconstruct what we see as the simple RGB data for each pixel. And this interpolation is only one step in the very complicated data transformation pipeline that gets applied to our images “behind the scenes”. This should take away the argument of some of the extreme purists who say they will do nothing in post processing to “damage” the original pixels or to “create” new ones. There really are no original pixels.

    I understand your point of view. I used to embrace it, to an extent. But get over it. There is no such thing as “pure” data from your sensor, unless maybe you are using a Foveon-based camera. All images are already interpolated to “create” pixel data before you ever get a chance to even view them in your editor. In addition profiles and lens corrections and other transformations are applied,

    Digital imaging is an approximation, an interpretation of the scene the camera was pointed at. The technology has improved to the point that this approximation is quite good. Based on what we have learned, though, we should have a more lenient attitude about post processing the data as much as we feel we need to. It is just data. It is not an image until we say it is, and whatever the data is at that point defines the image.

    The image

    I chose the image at the head of this article to illustrate that the Bayer filter demosaicking and other image processing steps gives us very good results. The image is detailed and with smooth, well defined color variation and good saturation. And this is a 10 year old sensor and technology. Things are even better now. I am happy with our technology and see no reason to not use it to its fullest.

    Feedback?

    I felt a need to balance the more philosophical, artsy topics I have been publishing with something more grounded in technology. Especially as I have advocated that the craft is as important as the creativity. I am very curious to know if this is useful to you and interesting. Is my description too simplified? Please let me know. If it is useful, please refer your friends to it. I would love to feel that I am doing useful things for people. If you have trouble with the comment section you can email me at ed@schlotzcreate.com.

  • Craft Completes Magic

    Craft Completes Magic

    Craft completes magic. I read this in a book on writing poetry by Robert Wallace. This was a new thought to me. It is unusual in my world for a random phrase to seem to crystalize immediately as truth. This did. I have often written about the 2 sides of art as being the creative, the magic, and the technical, the craft. I love the way this brings them together and completes the whole.

    The magic

    Oftentimes we artists focus almost exclusively on the creative aspects of what we do. After all, we think this is what separated us from other artists. And to a large degree, it is true.

    So we look at the work of others we admire. We plan or write or set projects to focus our thoughts. We look for the new and different. The driving challenge is how can we bring a unique perspective to the things we see in the world.

    Sometimes the muse visits us and we feel we have truly made magic. It is a great feeling. Creativity breeds creativity. We try to go on to leverage this new stage into even more.

    But, have you ever had a guilty feeling, looking at your new creative work, that it could have been executed better? Not necessarily more creatively, but with better craftsmanship? Sometimes we don’t know how to make our great idea into a finished work of art. Concentrating too much on just one aspect can throw us off balance.

    The craft

    I believe our craftsmanship is as important as our creativity. Not a replacement, but to balance and complete our work. It’s this completion I want to emphasize.

    There are 2 tendencies I see in a lot of photographers that disturb me. Some seem to feel that a technically perfect image is a good image. Some others take the attitude that “I’m a creative, I don’t know the ‘techie’ stuff”. I believe that either of these, if they drive your behavior too much, lead to bad ends.

    Ansel Adams famously said “There’s nothing worse than a sharp image of a fuzzy concept.” This, to me, is the danger of overemphasizing technical perfection. I see this a lot in online critiques where the objections are things like not enough depth of field or that the color correction may not be completely true to the original scene. The reality in many cases is that no amount of technical improvement is going to give this image life.

    If you don’t have an emotional connection with the scene and a definite point of view to share, then it isn’t going to get great by technical skill.

    On the other hand, it frustrates me to hear even professional photographers dismissively say they don’t do “tech”. Sorry, but photography is a uniquely technical art form. If you don’t understand and appreciate and know how to control the technical aspects you are at a severe disadvantage. You can end up with images that show a great idea but you were unable to produce a gallery-worthy image.

    The whole

    There is a symbiotic relationship between the creative and the craft. Mr. Wallace, who I quoted at the start, related it to the two legs of a runner. The creative leg propels you forward. Then the craft leg helps you bring it into being, which also thrusts you forward to another level. These work together, alternating, each with strengths to add. Neither is complete without the other.

    A comedian doesn’t just walk out on stage and think up funny things. He spends many hours on each skit, refining and rehearsing and tuning it before you ever hear it. Likewise, a magician spends countless hours working on an illusion to make it smooth and believable, to make the magic happen. A musician practices day in and day out for years to get and stay good. Yes, famous musicians still practice scales. It trains their technique.

    Art is hard work. It is hard to do creative things and it requires great skill to make it real. No one can tell you what you can or can’t do, or how you should do your art. But I believe that if we don’t put in as much work on the craft side of our art as on the creative we will never achieve what we could.

    A boring image will never be great because it was technically perfect. On the other hand, you don’t get a free pass to ignore the craft because you are a “creative”. As the initial quote says, craft completes the magic.

  • Beautiful Chaos

    Beautiful Chaos

    I am thinking about some words by William Neill in his book Light on the Landscape, combined with an old country song by Diamond Rio named Beautiful Mess. I’m referring to the visual chaos of the normal world around us. Managing this chaos is one of the great challenges and rewards of outdoor photography.

    Visual chaos

    Alas, the world outside is a chaotic place visually. Things just aren’t naturally arranged to make it convenient for us poor outdoor photographers. Plants are in the way. Trees aren’t in the right place for the best design. Rivers bend the wrong way. Clouds are too much or not enough or arranged wrong. Weather doesn’t cooperate. Sigh.

    I say that facetiously, of course. That chaos and the difficulty of making something pleasing out of a cluttered scene is one of the unique and challenging parts of photography. If it was too easy it would be difficult to create outstanding images.

    Bringing order

    I love this challenge. The inner designer in me rises to it. It is a very satisfying mental exercise to try to mold a chaotic scene into a clean and appealing image. This is one of the defining characteristics of photography. Painters start with a blank canvas and selectively add only the elements they want for their scene. But photographers must start with an existing, disordered scene and simplify it.

    We have many techniques to apply to do this. Lens selection will widen or narrow our field of view. We can change our point of view to include significant parts or exclude distracting elements. Selective focus can emphasize the areas of attention. Exposure can be used to darken or blow out parts of the frame where you don’t want any detail. Long exposure can change moving elements into a different graphical design. These and other techniques give us great control over the arrangements of the parts.

    But above all, it is a design challenge. We have to decide what is key to the scene and how to emphasize that and minimize distractions. Is it the S curve of a river or the graphical arrangement of branches? Is it the forms or the leading lines that draw the eye a certain way? Most scenes can be arranged to bring an interesting view. Some more than others, but most can be improved.

    Refine

    Following on from a previous post, we need to very consciously work to refine our design after we set it up. This is a weakness of mine that I plan to improve. I have long training in composition. When I walk up to a scene I tend to do a tremendous amount of subconscious evaluation to select a composition. My natural tendency is to set up and shoot what I visualized as I came on the scene and stop without taking it further.

    But I know that many designs can be enhanced by exploring variations. I will try to discipline myself to do this more diligently. Move – left, right, up, down – look for improvements in the composition with slight shifts. Look closely at the entire frame to make sure there are no distracting elements that could be eliminated by in-camera techniques. Walk more to see if a more dramatic change of viewpoint could help.

    Most of all, I need to make sure I look and think. What I have is good, but can I make it better?

    Don’t over analyze

    A caution, though. Don’t over analyze the situation. Design and creation should be an act of joy. When you are learning new techniques it is normal to have to concentrate a lot on what you are doing. But try to get to the point where it flows naturally. To where you move with it and follow your instincts. Trust your instincts.

    Shooting in the outdoors should be energizing. We should feel excited about what we are seeing and capturing. Don’t let the joy get sucked out for you. Creativity is exciting and invigorating. Most of us aren’t going to get rich at this. We should at least have fun and feel satisfied.

    This is a journey of discovery. Enjoy the journey and have fun!

    Note on the image

    The image in this article is personally satisfying to me. It is a location that brings me joy and that i return to as often as possible. Despite wading through mud, swatting mosquitoes and trying not to slip in and get swept downstream, I loved the scene. I did follow my advice in 2 significant ways: I worked it until I got to a composition I loved, and I had a great time.

    I hope you will find scenes that bring you such joy.

  • Move

    Move

    No, I am not suggesting you should uproot and relocate. Or join the great resignation and quit your job. These can be beneficial at times, but it’s not what I am talking about. I’m simply saying that art is a physical process. We need to move freely when we are are exploring for images.

    Taking root

    Certain of the images I shoot require a tripod for rock solid sharpness. I actually like this, because it brings a discipline to the process. There is a trap many of us fall into, though.

    When we set the tripod down it’s like it takes root. We’ve gone to the trouble of setting it up, leveling it, composing a shot, and we tend to not move it. It creates inertia. But perhaps that first place we put it was not the optimum location. We need to tell ourselves there is a better placement and we need to find it.

    Use your feet

    “When finding the right angle for a shot…’Move your ass’.” – Jay Maisel

    Photography is a physical activity. At least for the type of outdoor photography I do. I walk. I stop and frame up a scene and take a picture. At this point, though, do I go on or explore options? Either answer is right depending on the situation. But are you confident enough in your compositional prowess that you know you got the best shot of the scene?

    I have learned the hard way that many scenes can be improved by “working” them some. Take some more time. Move. Try another angle. Get higher or lower. Take a few steps to the side to eliminate a distracting background. Wait a minute for the light to improve.

    In other words, once I have the shot, I need to look for ways to improve it. Most often, this involves moving, walking, squatting, thinking. One of the great technical advancements of digital photography is that we can see our image immediately. We can examine it and critique it to see how it could be improved. Do it if you have the time and opportunity.

    I tend to quote the great Jay Maisel a lot. He is very quotable. Here is one that elaborates some on this idea:

    “You find that you have to do many things, more than just lift up the camera and shoot, and so you get involved in it in a very physical way. You may find that the picture you want to do can only be made from a certain place, and you’re not there, so you have to physically go there. And that participation may spur you on to work harder on the thing, . . . because in the physical change of position you start seeing a whole different relationship.” – Jay Maisel

    Try variations

    A great scene often has the opportunity to explore variations. Change the crop. Go in for closer detail. Vary the exposure. Look for an angle that shows better shape or lighting or gets rid of distractions. Moving even a step or 2 can make a large change. Out constant attitude should be, “yes, that’s good; how can I make it better?”.

    Again, an advantage we have with digital imaging is that shooting these variations costs us almost nothing. Yes, we have to edit them, but the reality is, that is an embarrassment of riches. We might end up with 10 great images of the scene to choose from. It can be hard to pick the best.

    Moving is an attitude

    This sounds weird. Moving is an attitude? What I mean is that we should always be ready to explore chances of improvement of a shot. Don’t let our tripod get rooted. Have the flexibility to let ourselves try a different angle. That often involves physical movement.

    I believe I have missed many great opportunities by shooting the first composition I saw. I now try to make myself explore variations and be willing to move. One of the great influences in framing a scene is the position you shoot from. And as Jay said, moving and trying new ideas gives us a new perspective on the scene.

  • Boredom

    Boredom

    Boredom. We’ve all been there. We get in a funk. What we usually see and shoot is distasteful to us. We are discontented. Everything has been done, there is no creativity left. Do you ever feel like this?

    The time of year

    This article will be published in the depths of winter. Many of us don’t think there is anything interesting to see or shoot. After all, there are no flowers or green trees or lush fields.

    I would say, look again. Get out in it. Yes, out in the cold and snow if you have that. Or the rain and clouds. Whatever winter is in your area.

    Forget what you want to be shooting. Look with fresh eyes at what is there. You may discover a whole new world. In the words of an old song, “if you can’t be with the one you love, love the one you’re with.” While I do not recommend this as a way of managing your relationships, it can be very useful artistically.

    I actually love shooting in winter. Today I was out shooting in 70+mph winds and temperatures not much above freezing. Do I love being out in that kind of wind? Not at all. When you have to bundle up in layers of wind-tight clothes and hold your tripod to keep it from blowing over, it looses some of its charm. However, there were great opportunities out there. I have, by chance, been working on a project about wind. Today was a great opportunity to fill in some gaps in the image set.

    It happens to all creatives

    We all feel blocked, bored, empty at times. The “muse” is not around. We fear our best work is behind us and there is nothing to look forward to. Might as well give up.

    You are not alone. We all feel this sometimes. Like temperature and climate and relationships, our mental energy is cyclic. Sometimes the spark seems to be gone. It is a low time for us.

    Recognize that this is a natural part of life. Don’t be (too) discouraged. It will change. The creative energy will flood us again. Just give it time.

    Use it!

    But we don’t have to sit passively waiting for the creativity to return! Use this boredom to propel us to a new level.

    If we are bored perhaps we have plateaued on our current path. Maybe we aren’t reaching far enough. It is a great time to reexamine where we are and how we feel about our art. And actually do something about it.

    Boredom is frustrating to most of us. Use that! That is an energy and motivation. What are we lacking? Should we strike out in a new direction? What would we love to do if we had the opportunity?

    Like with our body, if something hurts, that is a sign that we need to take care of it. Taking care of it doesn’t necessarily mean we should rest it and take a “oh, you poor thing” attitude. Maybe we need to work it, eat right, exercise, build it up.

    The point is, the frustration of boredom can be a motivation to change our self or redirect ours thoughts or energy.

    Opportunity

    I have made some of the best discoveries of my life because I was bored. Really.

    Let me give an example. Way back, we owned a timeshare. For those who don’t know them, it is a vacation ownership scheme that was popular at one time. Don’t buy one. It’s not a good investment. The way it worked was, you “bought” a fraction of a property, say one week during the year. Typically your ownership time would be traded for a week at another property. Because of this, we got to visit beautiful places around the world. But a side effect was that we were “planted” at a single location for the week.

    This led to interesting trips. On one occasion I can think of, we were in a nice place way out in the country in Devon, England. Ten miles or more from the nearest town. It quickly got boring. As a result, we started exploring. Even though this wasn’t an area you would find featured on many tourist itineraries, the things we found were intensely interesting. We still cherish the memories.

    Because of the boredom, we were led to explore with our eyes and minds open. We had to forget the expectations that were not being met and become receptive to the wonderful things that were there. Repeating this experience many times completely changed my travel style. Therefore, now I want to settle in somewhere, get to know it, and have to find out what is there.

    Use boredom to your advantage

    So I encourage you to let boredom be motivating. With the right attitude it can free and empower us. It can lead us to opening our eyes to our surroundings, to learning new subjects or techniques, to re-evaluating our work and making improvements, to getting out and doing something about it. Or, you can sit on the couch and feel sorry for yourself. Your choice.

    The image with this article is one of many surprises we discovered on that “boring” trip to Devon I talked about above. It is a beautiful place with hidden gems all around.