Monday, May 07, 2018

Attempting to Explain RGB

A few years ago, I discovered the dramatic visual improvement from using RGB for my favorite classic game consoles.   The basics of this are often so basic that no one wants to take the time to talk through them because, duh, you should know this!  But I find it useful to rehash what I think is obvious for those corner cases when some nuance of explaining it helps someone to understand it where they couldn't before.

Why RGB is so great
Let's take as read that all analog video pictures are made up of dots (call them pixels or phosphor groups or whatever you like, they're a string of dots).  All color displays, regardless of technology - CRT, LCD, OLED, or whatever, have to separate picture information into discreet red green and blue values before they can assemble it into a screen full of colors.  All picture information starts as RGB and winds up as RGB, but the fewer transitions it has to go through along the way, the more faithful it will be to the original image.  RGB is pretty close to being the fewest transitions needed.

An Analogy:
Think of it like you have a mosaic picture made with thousands of red, green and blue painted marbles, and you want to move your mosaic to the other end of town.  The mosaic is too big to move all at once so you have to pack the marbles up, put them in a truck, drive to the other side of town and then reassemble the mosaic.  Let's say you only have one box to carry the marbles in, so you have to put them all into a big funnel that mixes them together.  In the act of mixing them together, the marbles rub against eachother scratching them and causing their paint to rub off on eachother so that now some of the green ones have red paint streaks and some of the blue ones have red paint streaks etc... When you get to the other side of town you have to sort them back out into red, green and blue using another hopper which also scratches them and gets some of them mixed up.  Now when you put the mosaic back together, the colors are recognizable but the picture is duller because all the marbles are scratched and some of the colors are smeared with streaks from other colors, and others are just in the wrong place.  That's the (grossly oversimplified) gist of how Composite video (the yellow RCA cable) works.  Now, if instead, you put the red, blue and green marbles neatly into their own separate boxes, when you got to the other end, you wouldn't have to scratch them or sort them, you would just simply reassemble them in the same order and the mosaic would barely be scratched all and every marble would be clean and in its proper place.  That's RGB.

Where RGB comes from.
RGB refers to the three primary colors - Red Green and Blue, of which all pixels on a color television are comprised.  By combining and using different values of these colors, a TV can produce a nearly complete spectrum of other colors. The values of these colors are also known as "saturation".  Irrespective of how it gets there, color information is separated into these three channels before being displayed on the screen.  In old school CRT displays this was because there were separate emitters for each of the three colors.  In LCD televisions this is because light is pushed through red green and blue layers of LCD, and the image processor needs to know how opaque to make each pixel in each LCD layer.

Brightness/Luma:
In addition to knowing the relative intensity of red green and blue, the TV also needs to work out how light/dark a pixel should be.  The relative brightness of a pixel is also known as its "Luma" value.  This is essentially the black white version of the picture.  In fact before color television, this signal was by itself.

Sync:
After figuring out what color and how bright a dot is, the TV needs to know where to draw it.
This is accomplished by something known as "sync".  The picture is transmitted as one continuous line.  Think of it like a machine constantly pushing out a colored string that's meant to be chopped up into pieces of a certain width and laid down one line at a time until it forms a rectangle that we'll call a "frame".  If we do our job right, the frame will be a picture of something recognizable.  But the machine doesn't stop moving, so as soon as we're done with one rectangle/frame, we immediately start snipping lines and drawing the next one.  We'd have to have some way of knowing precisely when a new rectangle was supposed to start, and when to snip the string to begin laying down the next line.  Suppose then that to help us along the machine spewing out the string also had two different with distinct sounds to them.  One whistle would blow when we were supposed to start a whole new rectangle, and the other would blow when we were supposed to snip each string and start a new line.  This is basically how sync works - one type of pulse tells the TV to begin drawing lines at the very top of the screen, the other tells it when to end the line and begin drawing on the next line down.  If that wasn't clear, I have one more analogy - think of it like a clock with hours minutes and seconds.  At the top of each hour, you begin drawing the screen at the top left corner.  Each movement of the second hand draws a pixel across the screen from left to right.  When the second hand reaches the top of the next minute, that's the signal to move one row down and begin drawing pixels from left to right again.  When you get to the end of the hour you've drawn a 60x60 grid of pixels.  Each movement of the minute hand is like an H-Sync pulse, and each movement of the Hour hand is like a V-Sync pulse.

A Tale of Two RGBs:
There is a 5-channel type of RGB known as RGBHV (Red, Green, Blue + Horizontal Sync + Vertical Sync) that was commonly used by personal computers and, if I'm not mistaken, some arcade monitors, but not really used by home televisions.  The RGB commonly used by game consoles in the 80's and 90's, and the topic of this article is also called RGBS (Red, Green, Blue + Sync) which only uses 4-channels.  The H and V sync channels are merged into something called "Composite Sync".  While RGBS was a common output format of retro game consoles and a common input format for consumer Televisions in Japan, Europe and Austrailia, it was not featured on consumer TVs in North America. (More on that below).

In a perfect world:
To ensure the best possible picture and the fewest transitions, it would be necessary to transmit all of the needed attributes of a picture on separate wires, or channels.  That would mean Red, Green, Blue, Luma, V-Sync and H-Sync for a total of 6 channels.  And that's all before you start talking about sound - can you imagine having to plug in 8 RCA plugs for a single device to get a picture and stereo sound?!  In a commercial sense, simplifying connections was much more important that image quality.  It didn't make any difference how awesome the picture quality was, if Grandma couldn't figure out how to hook up her VCR.  This led to a huge number of compromises.

RF Back in the Day
When analog TV signals were broadcast over the air back in the day, both the chroma and luma were combined into a single waveform (composite) and then modulated into a radio frequency (RF), and it was up to the TV circuitry to catch the radio signal, demodulate it back into a composite signal then figure out which part of the signal was originally luma and which part was chroma, then which part of the chroma was red, which was green, and which was blue.  The picture information was basically put in a blender and the TV had to sort it all out - problem is, it's impossible to assemble the original picture this way - just like the earlier analogy with the marbles only worse.  Each transition compromised the fidelity.  The colors appear smeared, sometimes light and color information is mixed, the sync signal had numerous errors and the result is a blurry semi-stable approximation of what the original signal looked like.

Composite
As TV technology advanced - particularly with the advent of the VCR, it became possible to skip the RF encode/decode by plugging a "composite" signal (the luma/chroma combined) directly into the TV - the standard marked by the yellow RCA connection.  Cutting out the RF modulate/demodulate step meant more of the original picture was intact and the result was a more stable correct image.  But the TV still had to guess about which information was luma and which was chroma, and then sift the chroma into RGB.

S-Video
While Europe, Austrailia and Japan began incorporating full RGB connectivity in their televisions by means of a standardized SCART cable, the standard was not adopted in the U.S.  Instead we were given an awkward half-way solution called S-Video, sometimes also referred to as "SVHS" for its association with the short-lived Super Video Home System videocassette standard for which it was seemingly created.  S-Video dramatically improved picture quality by separating Luma and Chroma into their own channels.  The TV no longer had to "guess" as to which part of the signal was light and which was color, and the result was a remarkable jump in image clarity.  The distinctiveness of the Sync signal from Luma made them very easy to accurately separate which also meant a super stable image.  The television still had to sift red green and blue values from a combined signal, but the improvement in contrast improved overall image quality so dramatically that many argued full RGB separation was ultimately unnecessary.

RGB(S)
RGB, as the name suggests keeps the color information separated into three distinct channels, with a fourth channel dedicated to C-Sync.  Each of the three color channels also carry Luma (lightness/darkness).  This is about as close as it gets to a perfect picture, but unfortunately it was never adopted as a standard for Televisions in North America.

Component a.k.a. Once Upon A Time in North America
And then things got weird.  When DVD started to become popular, it was clear that the pure digital format was capable of much higher fidelity than even S-Video could provide.  In countries using the SCART standard, this wasn't really an issue because that connector was perfectly capable of transporting the full spectrum of what DVD had to offer.  In the U.S. however we were wrestling with what to do about high definition content because we didn't have an existing connection type that would support it.  S-Video was insufficient, and the all-digital "firewire" being used by digital camcorders and specialty VCRs terrified the copy-averse MPAA.  While RGBS was perfectly equal to the task of both DVD playback and HD content playback, in North America we went a different way.  Instead of the decades old RGBS(Red, Green, Blue + Composite Sync) standard, we adopted something commonly referred to as "component" video, but more accurately described as "YPbPr" or "Color Difference".  (Component technically refers to any video signal where the colors are transmitted separately). Before getting into what that means, the important thing to know is that it is fundamentally incompatible with the already-existing RGBS signal standards in use and consequently supported by many older game consoles.

Component vs RGB
There's a lot of opinion out there about the image quality difference between RGBS and Component (YPbPr), but mathematically they produce images of identical fidelity.  Most opinions to the contrary are anecdotal and based on invalid test procedures.  You'll find a lot of people who think that RGBS is fundamentally superior to YPbPr, but they're wrong.  If anything, there's a slightly stronger argument to the contrary.  Ordinarily, combining color information into fewer channels involves modulating, multiplexing, or otherwise mashing it together to be sorted out later.  The clever thing about YPbPr is that it can effectively transmit three channels worth of data over two channels without compromising the fidelity of either by essentially using triangulation.  By knowing the difference between Blue and Green and the difference between Red and Green, you can mathematically figure out the exact original values of Red Green and Blue without any sifting or guesswork like what happens when Composite or S-Video gets untangled.  Another major difference between the two which is inconsequential in practice is the fact that each of them transmit Luma and Sync differently.  RGB uses a separate channel for C-Sync, whereas Component combines it with Luma.  All things being equal, this would seem to be an advantage for RGB, however sync is such a distinctive portion of the signal that, in practice, separating it from Luma can be done with nearly 100% accuracy (the only reason this isn't 100% is because nothing truly is).  On the flip side, RGBS carries Luma information over all three color channels, this part of the signal is completely redundant - the brightness of a pixel is not changed by its color value (saturation), so the Luma on each channel is exactly the same as that of the others.  While it's considerably easier to separate Luma from a single color than it is to separate Luma from a combined Chroma signal, this is still more error prone than separating C-Sync from Luma - thus the slightly stronger argument that Component/YPbPr can yield a more accurate picture.

Of course, all things are not equal, and the quality of your experience will definitely be more impacted by the quality of the source (game console, DVD player etc...), the quality of the cables (shielded/unshielded, quality conductors/cheap conductors), and the quality of the display (TV, monitor, BVM etc...) than it will by the differences between RGBS and YPbPr transmission methods.



No comments: