Wednesday, May 23, 2018

The road to Dreamcast 480p YPbPr

When I set up my current retro gaming area around 2013, I grouped the Dreamcast with the rest of the consoles of the generation it kicked off - the Playstation 2, the Gamecube and the XBOX.  That particular area has a dedicated CRT TV which natively supports all of the best resolutions those consoles have to offer.  Most games of the era max out at 480p, and all of those consoles have official support for 480p via Component/YPbPr except the Dreamcast.  The Dreamcast is absolutely capable of it, but until this week, all of my attempts to get 480p Component video out of the Dreamcast could be measured in degrees of failure.  If you just want to know how I finally did it, skip to the end. In the middle is the journey.

1999
The Sega Dreamcast was born during the earliest days of the transition from Standard Definition TeleVision (SDTV) to High Definition TeleVision (HDTV).  DVD was an up-and-coming technology which was driving the adoption of cleaner analog video technologies such as RGB and Component (YPbPr).  Though most households did not have the means to take advantage of those technologies in 1999, Sega seemed to be hedging their bets when they designed the Dreamcast's video output, and indeed some of the technology locked inside that little white box was well ahead of its time.

Interlacing
As consumer-level video technology improved in the late 90's in an arc towards HDTV, the first quality leap involved moving  from mixed signals such as Composite, to increasingly discrete signal transmission first with S-Video which separated color from brightness, and Component which separated brightness from color and the colors from each other.  The next step was the leap from "interlaced" to "Progressive Scan".  To thoroughly explain those terms would probably require several paragraphs, but I will spend a little time explaining the basic premise.  In an ideal and simple world, a television picture would be formed by drawing one horizontal line at a time from top to bottom until a whole frame was complete before starting over at the top to draw the next one.  Once upon a time in the early days of Television, engineering compromises were made.  I'm not entirely sure whether these compromises were made to cut down on bandwidth or to deal with slow cathode guns, but the engineers discovered that instead of drawing an entire screen from top to bottom, they could get almost the same effect in half the time and half the bandwidth by only drawing every-other horizontal line from top to bottom, starting over at the top and drawing the opposite lines from top to bottom.  So instead of drawing a whole frame at a time, they could draw a "field" of all of the even lines, followed by another "field" of all of the odd lines. Each field left gaps like the spaces between teeth in a comb, but when laid over top of each other, the "teeth" would mesh to exploit a property of human sight called "persistence of vision".  The screen was flickering so quickly as the teeth of the combs were interlocked, your eyes would sort of glue the two fields together and to your brain you would see a whole picture.  This is known as interlacing, and it's the reason that a SDTV image looks like it's bobbing up and down rapidly if you look at it up close.

Progressive Scan
Progressive Scan was sort of bringing us back towards that ideal state where an entire image is drawn top-to-bottom before the next one is displayed.  This has the effect of making the image appear more stable because it doesn't have the bobbing effect caused by alternating fields.  Actually, most 80's and early 90's game consoles tricked the CRT televisions into displaying a progressive image by always drawing odd or or even lines instead of alternating between them.  While this was never a ratified standard, it is commonly referred to as 240p, and it's what allowed those consoles to display a stable image on a TV that wasn't meant to do so - it's also what causes those characteristic "scan lines".  But when we talk about the turn of the century, Progressive Scan almost universally refers to 480p - which was a real ratified standard that not only created a rock solid image without flicker, but (arguably) doubled the horizontal resolution of the image.  The switch to 480p was a dramatic picture quality improvement, however it did not qualify as "high definition" because that term had been reserved for 720p and above. For this reason a new designation was given for this in-between format: Extended Definition TeleVision (EDTV) - a term that never really saw popular use.

In 1999 when the Dreamcast was released, progressive scan Televisions were on the horizon, but hadn't quite hit the market yet.  However almost every computer monitor available at the time did support progressive resolutions, so Sega's solution was to design a video output that could be configured to work with a PC monitor via a special adapter.  In 2000, I purchased one of these "VGA adapters" and connected my Dreamcast to a 17" PC monitor and I have to say the picture was unbelievably good - colors were more vibrant than ever, the image was razor sharp and lightning quick.  But a 17" PC monitor didn't really fit well in my gaming setup so I found myself falling back on the S-Video output to play on my television.

Consumer televisions capable of supporting 480p wouldn't be widely available until after the unfortunate demise of Sega's hardware business and therefore the Dreamcast.  So while all of the capabilities were right there in the Dreamcast, no one ever made official hardware to connect the Dreamcast to an EDTV to enjoy 480p on a larger screen.  While some early 480p televisions provided VGA inputs, it was hardly a common feature.

The Problem:
While the Dreamcast technically outputs everything needed to display 480p over component, some translation has to happen to make it work.  Firstly the Dreamcast has to be switched into "VGA mode" by grounding one of the pins on the AV out.  When it does this, the Dreamcast outputs video in a format called RGBHV - the "HV" part refers to the fact that horizontal and vertical sync pulses are delivered on separate channels.  However Component video requires a C-Sync which is the two pulses combined into a single channel.  The Dreamcast uses C-sync for everything else BUT VGA mode.  Activating VGA mode deactivates C-Sync.  So the next stage in the process needs to combine the H and V sync into C-Sync.  Lastly the color transmission needs to be converted from RGB to YPbPr.

My First Solution:
This was my first attempt at a Dreamcast 480p "box".
I hesitate to call my first solution a "success", though it did result in outputting 480p YPbPr from the Dreamcast to my TV, the picture was awful.

The first stage was the Dreamcast to VGA portion.  I initially used the Naki Dreamcast VGA adapter I bought back in the day but ended up buying a new Dreamcast VGA cable because the picture was too dark and I thought the box might be at fault (it wasn't). - what these adapters basically do is ground the "VGA Select" pin of the av-out and put the Dreamcast into VGA mode (RHBHV).

When I initially connected the VGA adapter to the VGA-to-Component adapter, the picture was very dark and hard to see.  Aside from trying another VGA adapter for the Dreamcast, I also ended up inserting a device called a "Gamma-X" as the second stage.  The purpose of this device is to boost brightness - and it did help, though not as much as I needed it to.

The third stage was a Monoprice VGA to Component YPbPr adapter.  There were several things wrong with this setup.  I lay most of it at the feet of the VGA to Component adapter.  The device was designed for maximum compatibility, not for the simple task I needed.  In order to always output a working 480p/720p/1080i/1080p signal, every single signal input was run through a scaler and the sync was regenerated.  This effectively crushed the contrast and introduced completely unnecessary scaling artifacts.

No matter what I did, I could never get the picture to look right.  Aside from not being progressive scan, the picture over S-Video looked waaaay better.

Forlorn Hope #1
Round about the time I was wrestling with this, the guys at HDRetrovision teased an image of a Dreamcast outputting 480p component video along with a blurb that they were in the early stages of working on a product that did exactly what I was looking for.  Unfortunately that was 2015 and as of the middle of 2018 they still haven't even really begun work on it.
The HD Retrovision cable that no one can actually buy.


Forlorn Hope #2
A couple of years ago I learned about The Behar Bros. (www.beharbros.com) - a couple of guys from Spain that produce a variety of output adapters for the Dreamcast.  They have several different types of boxes, including some that output directly to HDMI for a great signal on modern TVs.  They also make a replacement VGA adapter that will add artificial scanlines, and a simple sync-combiner etc...  Since most of their designs were clearly purpose-built for several different uses, I didn't think it would be unreasonable to reach out to them and ask if they could produce a box that just provided a straight-up 480p YPbPr output.  The first time I asked they were in the process of designing several new items and they told me to check back because they would soon have just the thing.  The second time I reached out about a year later, this is the response I received: "No, we don't have a component video solution. RGB is always better."  Not really, guys.

My Second Attempt
Emboldened by several recent successes in modding and a dawning understanding of circuit analysis, I decided that it would no longer be overly ambitious to think I could build my own adapter.  So I studied the problem for a couple of weeks, studied solutions to various pieces of it, made a plan, made a parts list, placed an order and got to work when it arrived.    My plan was to build my own sync-combiner and switch to throw the Dreamcast into VGA mode, but use an HDRetrovision Genesis Component cable to transcode the signal from RGBs to YPbPr.  It actually came together quite beautifully until I made a mistake and ended up burning out the HDRetrovision cable.  Not only are those things pricey, but it seems like it takes ages for them to be restocked.  That was a huge letdown, and i'm still not entirely sure what I did wrong - which tells me I still have quite a lot to learn.


My Final Solution
Simple Dreamcast VGA Cable

(Retrotek VGACTV1) VGA to YPbPr transcoder.


Amidst crying over my burned out HD Retrovision cable, I started re-exploring my options.  A lot has changed over the last few years since I started trying to do this.  I started by thinking I could use a sync-combiner that the Behar Bros. produced along with an RGB to YPbPr transcoder.

I'm currently using such a transcoder for my older 240p/480i systems, but I wasn't sure what 480p would do to it, and the seller had no idea either.  That led me to Retrotek on Ebay.

They sell a handful of specialty devices, including a RGB to YPbPr transcoder - which is what I went looking for in the first place, but somewhere along the line they also added a VGA to YPbPr transcoder they call the VGACTV1.

Unlike my general-purpose Monoprice adapter, this one is just a sync-combiner and transcoder - it doesn't do anything to scale or otherwise alter the resolution.  It does exactly what I had been trying to achieve all along.   The picture comes through sharp and clean with no obvious degradation at all - the Dreamcast is now a proper 480p console on my TV alongside its peers.

The Retrotek box is a little on the expensive side, but compared to how much I've spent over the last three or four years trying to make this happen and the fact that it not only works but works perfectly, it was a bargain.





Monday, May 07, 2018

Attempting to Explain RGB

A few years ago, I discovered the dramatic visual improvement from using RGB for my favorite classic game consoles.   The basics of this are often so basic that no one wants to take the time to talk through them because, duh, you should know this!  But I find it useful to rehash what I think is obvious for those corner cases when some nuance of explaining it helps someone to understand it where they couldn't before.

Why RGB is so great
Let's take as read that all analog video pictures are made up of dots (call them pixels or phosphor groups or whatever you like, they're a string of dots).  All color displays, regardless of technology - CRT, LCD, OLED, or whatever, have to separate picture information into discreet red green and blue values before they can assemble it into a screen full of colors.  All picture information starts as RGB and winds up as RGB, but the fewer transitions it has to go through along the way, the more faithful it will be to the original image.  RGB is pretty close to being the fewest transitions needed.

An Analogy:
Think of it like you have a mosaic picture made with thousands of red, green and blue painted marbles, and you want to move your mosaic to the other end of town.  The mosaic is too big to move all at once so you have to pack the marbles up, put them in a truck, drive to the other side of town and then reassemble the mosaic.  Let's say you only have one box to carry the marbles in, so you have to put them all into a big funnel that mixes them together.  In the act of mixing them together, the marbles rub against eachother scratching them and causing their paint to rub off on eachother so that now some of the green ones have red paint streaks and some of the blue ones have red paint streaks etc... When you get to the other side of town you have to sort them back out into red, green and blue using another hopper which also scratches them and gets some of them mixed up.  Now when you put the mosaic back together, the colors are recognizable but the picture is duller because all the marbles are scratched and some of the colors are smeared with streaks from other colors, and others are just in the wrong place.  That's the (grossly oversimplified) gist of how Composite video (the yellow RCA cable) works.  Now, if instead, you put the red, blue and green marbles neatly into their own separate boxes, when you got to the other end, you wouldn't have to scratch them or sort them, you would just simply reassemble them in the same order and the mosaic would barely be scratched all and every marble would be clean and in its proper place.  That's RGB.

Where RGB comes from.
RGB refers to the three primary colors - Red Green and Blue, of which all pixels on a color television are comprised.  By combining and using different values of these colors, a TV can produce a nearly complete spectrum of other colors. The values of these colors are also known as "saturation".  Irrespective of how it gets there, color information is separated into these three channels before being displayed on the screen.  In old school CRT displays this was because there were separate emitters for each of the three colors.  In LCD televisions this is because light is pushed through red green and blue layers of LCD, and the image processor needs to know how opaque to make each pixel in each LCD layer.

Brightness/Luma:
In addition to knowing the relative intensity of red green and blue, the TV also needs to work out how light/dark a pixel should be.  The relative brightness of a pixel is also known as its "Luma" value.  This is essentially the black white version of the picture.  In fact before color television, this signal was by itself.

Sync:
After figuring out what color and how bright a dot is, the TV needs to know where to draw it.
This is accomplished by something known as "sync".  The picture is transmitted as one continuous line.  Think of it like a machine constantly pushing out a colored string that's meant to be chopped up into pieces of a certain width and laid down one line at a time until it forms a rectangle that we'll call a "frame".  If we do our job right, the frame will be a picture of something recognizable.  But the machine doesn't stop moving, so as soon as we're done with one rectangle/frame, we immediately start snipping lines and drawing the next one.  We'd have to have some way of knowing precisely when a new rectangle was supposed to start, and when to snip the string to begin laying down the next line.  Suppose then that to help us along the machine spewing out the string also had two different with distinct sounds to them.  One whistle would blow when we were supposed to start a whole new rectangle, and the other would blow when we were supposed to snip each string and start a new line.  This is basically how sync works - one type of pulse tells the TV to begin drawing lines at the very top of the screen, the other tells it when to end the line and begin drawing on the next line down.  If that wasn't clear, I have one more analogy - think of it like a clock with hours minutes and seconds.  At the top of each hour, you begin drawing the screen at the top left corner.  Each movement of the second hand draws a pixel across the screen from left to right.  When the second hand reaches the top of the next minute, that's the signal to move one row down and begin drawing pixels from left to right again.  When you get to the end of the hour you've drawn a 60x60 grid of pixels.  Each movement of the minute hand is like an H-Sync pulse, and each movement of the Hour hand is like a V-Sync pulse.

A Tale of Two RGBs:
There is a 5-channel type of RGB known as RGBHV (Red, Green, Blue + Horizontal Sync + Vertical Sync) that was commonly used by personal computers and, if I'm not mistaken, some arcade monitors, but not really used by home televisions.  The RGB commonly used by game consoles in the 80's and 90's, and the topic of this article is also called RGBS (Red, Green, Blue + Sync) which only uses 4-channels.  The H and V sync channels are merged into something called "Composite Sync".  While RGBS was a common output format of retro game consoles and a common input format for consumer Televisions in Japan, Europe and Austrailia, it was not featured on consumer TVs in North America. (More on that below).

In a perfect world:
To ensure the best possible picture and the fewest transitions, it would be necessary to transmit all of the needed attributes of a picture on separate wires, or channels.  That would mean Red, Green, Blue, Luma, V-Sync and H-Sync for a total of 6 channels.  And that's all before you start talking about sound - can you imagine having to plug in 8 RCA plugs for a single device to get a picture and stereo sound?!  In a commercial sense, simplifying connections was much more important that image quality.  It didn't make any difference how awesome the picture quality was, if Grandma couldn't figure out how to hook up her VCR.  This led to a huge number of compromises.

RF Back in the Day
When analog TV signals were broadcast over the air back in the day, both the chroma and luma were combined into a single waveform (composite) and then modulated into a radio frequency (RF), and it was up to the TV circuitry to catch the radio signal, demodulate it back into a composite signal then figure out which part of the signal was originally luma and which part was chroma, then which part of the chroma was red, which was green, and which was blue.  The picture information was basically put in a blender and the TV had to sort it all out - problem is, it's impossible to assemble the original picture this way - just like the earlier analogy with the marbles only worse.  Each transition compromised the fidelity.  The colors appear smeared, sometimes light and color information is mixed, the sync signal had numerous errors and the result is a blurry semi-stable approximation of what the original signal looked like.

Composite
As TV technology advanced - particularly with the advent of the VCR, it became possible to skip the RF encode/decode by plugging a "composite" signal (the luma/chroma combined) directly into the TV - the standard marked by the yellow RCA connection.  Cutting out the RF modulate/demodulate step meant more of the original picture was intact and the result was a more stable correct image.  But the TV still had to guess about which information was luma and which was chroma, and then sift the chroma into RGB.

S-Video
While Europe, Austrailia and Japan began incorporating full RGB connectivity in their televisions by means of a standardized SCART cable, the standard was not adopted in the U.S.  Instead we were given an awkward half-way solution called S-Video, sometimes also referred to as "SVHS" for its association with the short-lived Super Video Home System videocassette standard for which it was seemingly created.  S-Video dramatically improved picture quality by separating Luma and Chroma into their own channels.  The TV no longer had to "guess" as to which part of the signal was light and which was color, and the result was a remarkable jump in image clarity.  The distinctiveness of the Sync signal from Luma made them very easy to accurately separate which also meant a super stable image.  The television still had to sift red green and blue values from a combined signal, but the improvement in contrast improved overall image quality so dramatically that many argued full RGB separation was ultimately unnecessary.

RGB(S)
RGB, as the name suggests keeps the color information separated into three distinct channels, with a fourth channel dedicated to C-Sync.  Each of the three color channels also carry Luma (lightness/darkness).  This is about as close as it gets to a perfect picture, but unfortunately it was never adopted as a standard for Televisions in North America.

Component a.k.a. Once Upon A Time in North America
And then things got weird.  When DVD started to become popular, it was clear that the pure digital format was capable of much higher fidelity than even S-Video could provide.  In countries using the SCART standard, this wasn't really an issue because that connector was perfectly capable of transporting the full spectrum of what DVD had to offer.  In the U.S. however we were wrestling with what to do about high definition content because we didn't have an existing connection type that would support it.  S-Video was insufficient, and the all-digital "firewire" being used by digital camcorders and specialty VCRs terrified the copy-averse MPAA.  While RGBS was perfectly equal to the task of both DVD playback and HD content playback, in North America we went a different way.  Instead of the decades old RGBS(Red, Green, Blue + Composite Sync) standard, we adopted something commonly referred to as "component" video, but more accurately described as "YPbPr" or "Color Difference".  (Component technically refers to any video signal where the colors are transmitted separately). Before getting into what that means, the important thing to know is that it is fundamentally incompatible with the already-existing RGBS signal standards in use and consequently supported by many older game consoles.

Component vs RGB
There's a lot of opinion out there about the image quality difference between RGBS and Component (YPbPr), but mathematically they produce images of identical fidelity.  Most opinions to the contrary are anecdotal and based on invalid test procedures.  You'll find a lot of people who think that RGBS is fundamentally superior to YPbPr, but they're wrong.  If anything, there's a slightly stronger argument to the contrary.  Ordinarily, combining color information into fewer channels involves modulating, multiplexing, or otherwise mashing it together to be sorted out later.  The clever thing about YPbPr is that it can effectively transmit three channels worth of data over two channels without compromising the fidelity of either by essentially using triangulation.  By knowing the difference between Blue and Green and the difference between Red and Green, you can mathematically figure out the exact original values of Red Green and Blue without any sifting or guesswork like what happens when Composite or S-Video gets untangled.  Another major difference between the two which is inconsequential in practice is the fact that each of them transmit Luma and Sync differently.  RGB uses a separate channel for C-Sync, whereas Component combines it with Luma.  All things being equal, this would seem to be an advantage for RGB, however sync is such a distinctive portion of the signal that, in practice, separating it from Luma can be done with nearly 100% accuracy (the only reason this isn't 100% is because nothing truly is).  On the flip side, RGBS carries Luma information over all three color channels, this part of the signal is completely redundant - the brightness of a pixel is not changed by its color value (saturation), so the Luma on each channel is exactly the same as that of the others.  While it's considerably easier to separate Luma from a single color than it is to separate Luma from a combined Chroma signal, this is still more error prone than separating C-Sync from Luma - thus the slightly stronger argument that Component/YPbPr can yield a more accurate picture.

Of course, all things are not equal, and the quality of your experience will definitely be more impacted by the quality of the source (game console, DVD player etc...), the quality of the cables (shielded/unshielded, quality conductors/cheap conductors), and the quality of the display (TV, monitor, BVM etc...) than it will by the differences between RGBS and YPbPr transmission methods.