Ed Schlotzhauer

An artists journey

Category: Craft

How We Get Color Images
Have you ever considered that that great sensor in your camera only sees in black & white? How, then, do we get color images? It turns out that there is some very interesting and complicated Engineering involved behind the scenes. I will try to give an idea of it without getting too technical.
Sensor
I have discussed digital camera sensors before. They are marvelous, unbelievably complicated and sophisticated chips. But they are, still, a passive collector of photons (light) that falls on them.
An individual imaging site is a small area that collects light and turns it into an electrical signal that can be read and stored. The sensor packs an unimaginable number of these sites into a chip. A “full frame” sensor has an imaging area of 24mm x 36mm, approximately 1 inch by 1.5 inch. My sensor divides that area into 47 million image sites, or pixels. It is called “full frame” because that was the historical size of a 35mm film frame.
But, and this is what most of us miss, the sensor is color blind. It receives and records all frequencies in the visible range. In the film days it would be called panchromatic. That is just a fancy word to say it records in black & white all the tones we typically see across all the colors.
This would be awesome if we only shot black & white. Most of us would reject that.
Need to introduce selective color
So to be able to give us color, the sensor needs to be able to selectively respond to the color ranges we perceive. This is typically Red, Green, and Blue, since these are “primary” colors that can be mixed to create the whole range.
Several techniques have been proposed and tried. A commercially successful implementation is Sigma’s Foveon design. It basically stacks three sensor chips on top of each other. The layers are designed so that shorter wavelengths (blue) are absorbed by the top layer, medium wavelengths (green) are absorbed by the middle layer, and long wavelengths (red) are absorbed by the bottom layer. A very cleaver idea, but it is expensive to manufacture and has problems with noise.
Perfect color separation could be achieved using three sensors with a large color filter over each. Unfortunately this requires a very complex and precise arrangement of mirrors or prisms to split the incoming light to the three sensors. In the process, it reduces the amount of light hitting each sensor, causing problems with image capture range and noise. It is also very difficult and expensive to manufacture and requires 3 full size sensors. Since the sensor is usually the most expensive component of a camera, this prices it out of competition.
Other things have been tried, such as a spinning color wheel over the sensor. If the exposure is captured in sync with the wheel rotation then 3 images could be exposed in rapid sequence giving the 3 colors. Obviously this imposes a lot of limitations on photographers, since the rotation speed has to match the shutter speed. A real problem for very long or very short exposures or moving subjects.
Bayer filter
Thankfully, a practical solution was developed by Bryce Bayer of Kodak. It was patented in 1976, but the patent has expired and the design is freely used by almost all camera manufacturers.
The brilliance of this was to enable color sensing with a single sensor by placing a color filter array (CFA) over the sensor to make each pixel site respond to only one color. You may have seen pictures of it. Here is a representation of the design:
Bayer Filter Array, from Richard Butler, DPReview Mar 29, 2017
The gray grid at the bottom represents the sensor. Each cell is a photo site. Directly over the sensor has been placed an array of colored filters. One filter above each photo site. Each filter is either red or green or blue. Note that there are twice as many green filters as either red or blue. This is important.
But wait, we expect that each pixel in our image contains full RGB color information. With this filter array each pixel only sees one color. How does this work?
It works through some brilliant Engineering with a bit of magic sprinkled in. Full color information for each pixel is constructed by interpolating based on the colors of surrounding pixels.
Restore resolution
Some sophisticated calculations have to be done to calculate the color information for each pixel. This makes each pixel end up with full RGB color values. The process is termed “demosaicking” in tech speak.
I promised to keep it simple. Here is a very simple illustration. In the figure below, if we wanted to derive a value of green for the cell in the center, labeled 5, we could average the green values of the surrounding cells. So an estimate of the green value for cell red5 is (green2+green6+green8+green4)/4
From Demosaicking: Color Filter Array Interpolation, IEEE Signal Processing Magazine, January 2005
This is a very oversimplified description. If you want to get in a little deeper here is an article that talks about some of the considerations without getting too mathematical. Or this one is much deeper but has some good information.
The real world is much more messy. Many special cases have to be accounted for. For instance, sharp edges have to be dealt with specially to avoid color fringing problems. Many other considerations such as balancing the colors complicate the algorithms. It is very sophisticated. The algorithms have been tweaked for over 40 years since Mr. Bayer invented the technique. They are generally very good now.
Thank you, Mr. Bayer. It has proven to be a very useful solution to a difficult problem.
All images interpolated
I want to emphasize a point that basically ALL images are interpolated to reconstruct what we see as the simple RGB data for each pixel. And this interpolation is only one step in the very complicated data transformation pipeline that gets applied to our images “behind the scenes”. This should take away the argument of some of the extreme purists who say they will do nothing in post processing to “damage” the original pixels or to “create” new ones. There really are no original pixels.
I understand your point of view. I used to embrace it, to an extent. But get over it. There is no such thing as “pure” data from your sensor, unless maybe you are using a Foveon-based camera. All images are already interpolated to “create” pixel data before you ever get a chance to even view them in your editor. In addition profiles and lens corrections and other transformations are applied,
Digital imaging is an approximation, an interpretation of the scene the camera was pointed at. The technology has improved to the point that this approximation is quite good. Based on what we have learned, though, we should have a more lenient attitude about post processing the data as much as we feel we need to. It is just data. It is not an image until we say it is, and whatever the data is at that point defines the image.
The image
I chose the image at the head of this article to illustrate that the Bayer filter demosaicking and other image processing steps gives us very good results. The image is detailed and with smooth, well defined color variation and good saturation. And this is a 10 year old sensor and technology. Things are even better now. I am happy with our technology and see no reason to not use it to its fullest.
Feedback?
I felt a need to balance the more philosophical, artsy topics I have been publishing with something more grounded in technology. Especially as I have advocated that the craft is as important as the creativity. I am very curious to know if this is useful to you and interesting. Is my description too simplified? Please let me know. If it is useful, please refer your friends to it. I would love to feel that I am doing useful things for people. If you have trouble with the comment section you can email me at ed@schlotzcreate.com.
March 2, 2022
Craft Completes Magic
Craft completes magic. I read this in a book on writing poetry by Robert Wallace. This was a new thought to me. It is unusual in my world for a random phrase to seem to crystalize immediately as truth. This did. I have often written about the 2 sides of art as being the creative, the magic, and the technical, the craft. I love the way this brings them together and completes the whole.
The magic
Oftentimes we artists focus almost exclusively on the creative aspects of what we do. After all, we think this is what separated us from other artists. And to a large degree, it is true.
So we look at the work of others we admire. We plan or write or set projects to focus our thoughts. We look for the new and different. The driving challenge is how can we bring a unique perspective to the things we see in the world.
Sometimes the muse visits us and we feel we have truly made magic. It is a great feeling. Creativity breeds creativity. We try to go on to leverage this new stage into even more.
But, have you ever had a guilty feeling, looking at your new creative work, that it could have been executed better? Not necessarily more creatively, but with better craftsmanship? Sometimes we don’t know how to make our great idea into a finished work of art. Concentrating too much on just one aspect can throw us off balance.
The craft
I believe our craftsmanship is as important as our creativity. Not a replacement, but to balance and complete our work. It’s this completion I want to emphasize.
There are 2 tendencies I see in a lot of photographers that disturb me. Some seem to feel that a technically perfect image is a good image. Some others take the attitude that “I’m a creative, I don’t know the ‘techie’ stuff”. I believe that either of these, if they drive your behavior too much, lead to bad ends.
Ansel Adams famously said “There’s nothing worse than a sharp image of a fuzzy concept.” This, to me, is the danger of overemphasizing technical perfection. I see this a lot in online critiques where the objections are things like not enough depth of field or that the color correction may not be completely true to the original scene. The reality in many cases is that no amount of technical improvement is going to give this image life.
If you don’t have an emotional connection with the scene and a definite point of view to share, then it isn’t going to get great by technical skill.
On the other hand, it frustrates me to hear even professional photographers dismissively say they don’t do “tech”. Sorry, but photography is a uniquely technical art form. If you don’t understand and appreciate and know how to control the technical aspects you are at a severe disadvantage. You can end up with images that show a great idea but you were unable to produce a gallery-worthy image.
The whole
There is a symbiotic relationship between the creative and the craft. Mr. Wallace, who I quoted at the start, related it to the two legs of a runner. The creative leg propels you forward. Then the craft leg helps you bring it into being, which also thrusts you forward to another level. These work together, alternating, each with strengths to add. Neither is complete without the other.
A comedian doesn’t just walk out on stage and think up funny things. He spends many hours on each skit, refining and rehearsing and tuning it before you ever hear it. Likewise, a magician spends countless hours working on an illusion to make it smooth and believable, to make the magic happen. A musician practices day in and day out for years to get and stay good. Yes, famous musicians still practice scales. It trains their technique.
Art is hard work. It is hard to do creative things and it requires great skill to make it real. No one can tell you what you can or can’t do, or how you should do your art. But I believe that if we don’t put in as much work on the craft side of our art as on the creative we will never achieve what we could.
A boring image will never be great because it was technically perfect. On the other hand, you don’t get a free pass to ignore the craft because you are a “creative”. As the initial quote says, craft completes the magic.
February 23, 2022
It’s Not Necessarily About the Outcome
Sometimes the muse abandons us or conditions conspire against us or we get interrupted by something urgent. This can make us create pictures that do not live up to our expectations. But unless we are shooting for a client, we probably should not worry so much about the results we get, the outcome. We should remember to enjoy the creative exercise and have fun.
We all want great images
I assume that creating exceptional images is a goal for most of us. I know my expectations are high. We study technique and browse images by great artists we appreciate. We spend a lot of time getting to a location, exploring, setting up, composing. But it doesn’t always work.
Despite our best efforts, we are often disappointed. What we get may not be great. It may not even be very good. This can be very disappointing if we only judge our self by the outcome.
They won’t all be great
It is not uncommon for me to go out for a day of shooting and end up throwing most of them away, with none to add to my portfolio. Does this make me a failure? I try to see it differently.
I hope we can be philosophical about it. Sometimes all we seem to get from our effort is experience. Hopefully we learn from our experiences and improve our craft. That’s a bittersweet benefit. But the reality is we will learn more from a failed shoot than a successful one.
I’m coming to see that I am evaluating it wrong. My attitude was that I failed unless I got a number of great images. I concentrated on the outcome. There are greater goals.
The process may be as important
Sure, it is disappointing to not have captured those scenes that called to us at the time. But it is an opportunity for self-examination. What caused them to be unspectacular? Was there something we could have done different?
The editing process is a mirror where we can see how our mind worked and even see our soul to some degree. The images are captured. For better or worse, the bits are there on the computer. Now we have to deal with them. We can process them, but we cannot change them substantively – well, usually not.
I actually see something cathartic in deleting bad images. I have evaluated them and analyzed the problems and learned what I can. Now I have no more need of them. Remove them from my world. It’s a purging. In most cases I actually have an informal goal of throwing a certain percentage of my images away in the early stages of editing. The thought process is that I should be experimenting and working at the edge of my comfort zone. This causes a lot of failures. Failure is just part of learning.
Enjoy your art
Maybe I’m weird, but I see art as a work of joy. We should love what we do. Loving what we do is not the same as creating great work. They may be related, but they are not the same.
There are times when I go out and don’t end up with anything to keep except the memory of the great scene and the feelings I had. That is enough. Good art should be based on the feelings we are trying to convey. If I had the feelings but couldn’t realize them in the image, that means I am on the right track but I have to learn more. That is a challenge for artistic growth. I have seen too much art that is technically perfect but seems to me devoid of feeling.
There is a lot of talk about “flow” in the artistic process. Have you really experienced it? Not the fake stuff that is hyped by a lot of self-help gurus. There is no “hack” or shortcut to get there.
I developed the ability in my previous professional career, before I ever heard the term defined. There was a “place” I could easily drop into, a creative mode where I did great work and would be completely unaware of time for hours.
I can occasionally find the same place in my art, both when shooting and when processing. This is a reveling in the work regardless of the outcome. Yes, true flow is independent of what we might or might not produce. It is the joy of creation.
Let’s learn to revel in the process, the flow. We will create great things, but that is not the goal in itself. The joy of creation will carry us to become greater. Look at what you are becoming, not just what you are producing.
February 16, 2022
Cooking
No, you’re not in the wrong place. I have not suddenly changed from writing about art and photography to giving cooking lessons. I am exploring an idea that occurred to me recently. I think our cooking style reflects our photographic style, and maybe vise versa.
Weird, but stay with me for a minute.
Cooking styles
Do you cook? I hope so. It is rewarding and satisfying. A kind of art in itself.
What kind of cook are you? Do you follow the “rules” (e.g. follow the recipes) or do you “wing it”? Is your pantry and refrigerator well stocked so you can always come up with something? Or do you take your recipe to the store and buy what you need for it? Is your goal to exactly recreate the dish as specified in the recipe or do you apply creative license? Do you plan our the week or months meal list ahead of time or do you come home and try to decide what is for dinner that night?
We are all in different situations and make different tradeoffs. For instance, if you are cooking for a large family you tend to do things different than if you are cooking for one or two. If you are cooking for someone with food restrictions you may have to plan more carefully.
I’m intrigued by the idea that how we cook gives some insight on us as an artist. I think you will see where I lean in my thinking.
Recipe follower
Some people follow recipes exactly. They will not even try it unless they have all the ingredients and equipment necessary before starting.
If you exactly follow the recipe I think that says something about your style. Could it mean you are likely to follow influential artists and try to create in their style? Do you enjoy going to workshops where a leader will guide you to locations and help you compose shots to get similar results as theirs?
Maybe this means you also browse social media and photo sites looking for images you like to give yourself ideas for your work. Is your reaction “I wish I had shot that; I’ll try to do it”? Then research the location so you can plan to go there and capture something similar.
Recipe is a suggestion
Another approach I observe is the cook who looks at recipes, but mostly for motivation and ideas. They will freely substitute ingredients and end up with something substantially different from the recipe. Good, but not the same.
This cook, I believe, has greater confidence and experience. They know they can cook. That is not the issue. What I want to make tonight and how do I like it seems to be the basis of their decisions. A recipe, to them, is a kind of general guide. Descriptive, not prescriptive, to get sociological.
The recipe calls for an ingredient they don’t care for, so they substitute something else. It calls for something they don’t have, so they use something they have on hand. Not random substitutions, but based on knowledge of the ingredients and their effect on the dish. All the while, they know they will create something good, regardless of how close to the recipe it is.
Artistically, it seems this person is more likely to say yes, thanks, but I see it a little different. I’m going to shoot this other view. They have the confidence to follow their own vision, even if an instructor is trying to lead them in a different way.
What recipe?
Another cooking style I see is someone who seldom if ever consults a recipe. After all, most cuisine styles are fairly simple. There are general principle about how to combine things and what things go together to create certain flavors. Italian food has certain patterns based on certain ingredients, as does Mexican or Chinese or most any other recognizable type. When you learn the patterns almost any dish can be created. Most dishes are variations on the pattern. No recipe needed.
This person is experienced and confident. They can go into their pantry and quickly envision a dish based on what is there. If they served it to you, you would probably say “that is very good. What is it?”. And they would just say it is an Italian inspired dish they made up.
As an artist, they probably would not be in the instructor led workshop. They would just be out on their own, following their own muse, confident in their own decisions and style. Their attitude would be that they may not be as good as that instructor, but they would rather make their own decisions and go their own way.
The best style
Which of these styles is best? I think it is impossible to say. What you are is what is best for you.
But I wonder if there is a progression? When we start don’t we strictly follow recipes? As we get more confidence and experience perhaps we learn to be more free with the recipe. Eventually we learn the principles well enough that we give ourselves wide latitude in creating according to our own tastes.
I will admit that, in cooking, I am somewhere between the “recipe is a suggestion” and “what recipe?”. I have extensive files of recipes and cook books that I used to follow. If I have something in mind to fix I may still consult a recipe, but more for inspiration and to get an idea of what ingredients the recipe designer used. When the cooking starts I am likely to set the recipe aside and “wing it”. What I serve may only slightly resemble the original, but it will be good. 🙂
No one told me how
This comes around to a fundamental truth of being an artist: you are on your own. You are solely responsible for your art. No one can make the artistic decisions for you. It is a lonely but empowering place.
You can either spend your time copying your favorite teacher or develop the skill and confidence to go your own way. Until you find your own way it is not really your art yet.
Even when you are determined to be your own person, it doesn’t come with instructions. It can be very difficult and unsettling: this or that subject, what treatment or color palette, reject what I used to do and go a different direction? No one is there to guide you. It really is a “the buck stops here” situation.
Hence, the idea that the person who can endure, even thrive, in this situation probably also expresses himself in his cooking. I believe the artist is often comfortable also making creative decisions in the kitchen. Recipes become unnecessarily restrictive – just another set of rules.
Unscientific
If you haven’t thought it already, let me be the first to say this is totally unscientific. It is my hypothesis. My own idea. I do not intend to do a scientific study to prove or disprove it. I just put this out here to help us understand ourselves better as artists. Let’s just think about it and kick it around.
I don’t want you to perceive this as a black & white, all or nothing proposition. It is more a metaphor of art. For instance, if I am trying a new recipe for a dish I am unfamiliar with, I usually follow a recipe. Once. 🙂
What do you think? Is there any correlation between artists and their cooking style? Let’s discuss it! I want to hear from you!
Excuse me for now, though. I have to go home and figure out what is for dinner.
February 9, 2022
Beautiful Chaos
I am thinking about some words by William Neill in his book Light on the Landscape, combined with an old country song by Diamond Rio named Beautiful Mess. I’m referring to the visual chaos of the normal world around us. Managing this chaos is one of the great challenges and rewards of outdoor photography.
Visual chaos
Alas, the world outside is a chaotic place visually. Things just aren’t naturally arranged to make it convenient for us poor outdoor photographers. Plants are in the way. Trees aren’t in the right place for the best design. Rivers bend the wrong way. Clouds are too much or not enough or arranged wrong. Weather doesn’t cooperate. Sigh.
I say that facetiously, of course. That chaos and the difficulty of making something pleasing out of a cluttered scene is one of the unique and challenging parts of photography. If it was too easy it would be difficult to create outstanding images.
Bringing order
I love this challenge. The inner designer in me rises to it. It is a very satisfying mental exercise to try to mold a chaotic scene into a clean and appealing image. This is one of the defining characteristics of photography. Painters start with a blank canvas and selectively add only the elements they want for their scene. But photographers must start with an existing, disordered scene and simplify it.
We have many techniques to apply to do this. Lens selection will widen or narrow our field of view. We can change our point of view to include significant parts or exclude distracting elements. Selective focus can emphasize the areas of attention. Exposure can be used to darken or blow out parts of the frame where you don’t want any detail. Long exposure can change moving elements into a different graphical design. These and other techniques give us great control over the arrangements of the parts.
But above all, it is a design challenge. We have to decide what is key to the scene and how to emphasize that and minimize distractions. Is it the S curve of a river or the graphical arrangement of branches? Is it the forms or the leading lines that draw the eye a certain way? Most scenes can be arranged to bring an interesting view. Some more than others, but most can be improved.
Refine
Following on from a previous post, we need to very consciously work to refine our design after we set it up. This is a weakness of mine that I plan to improve. I have long training in composition. When I walk up to a scene I tend to do a tremendous amount of subconscious evaluation to select a composition. My natural tendency is to set up and shoot what I visualized as I came on the scene and stop without taking it further.
But I know that many designs can be enhanced by exploring variations. I will try to discipline myself to do this more diligently. Move – left, right, up, down – look for improvements in the composition with slight shifts. Look closely at the entire frame to make sure there are no distracting elements that could be eliminated by in-camera techniques. Walk more to see if a more dramatic change of viewpoint could help.
Most of all, I need to make sure I look and think. What I have is good, but can I make it better?
Don’t over analyze
A caution, though. Don’t over analyze the situation. Design and creation should be an act of joy. When you are learning new techniques it is normal to have to concentrate a lot on what you are doing. But try to get to the point where it flows naturally. To where you move with it and follow your instincts. Trust your instincts.
Shooting in the outdoors should be energizing. We should feel excited about what we are seeing and capturing. Don’t let the joy get sucked out for you. Creativity is exciting and invigorating. Most of us aren’t going to get rich at this. We should at least have fun and feel satisfied.
This is a journey of discovery. Enjoy the journey and have fun!
Note on the image
The image in this article is personally satisfying to me. It is a location that brings me joy and that i return to as often as possible. Despite wading through mud, swatting mosquitoes and trying not to slip in and get swept downstream, I loved the scene. I did follow my advice in 2 significant ways: I worked it until I got to a composition I loved, and I had a great time.
I hope you will find scenes that bring you such joy.
February 2, 2022