HD & HDTV content delivery

... hope everyone is having a great summer, this has been a really excellent one - no hurricaines yet, Great private A-List parties http://dsmj.net/starroom/aug5.htm etc

So this time i will rant yet again, before getting to buiness and finally launching www.hd1080i.com this fall. ( oh and BTW i got another HD monitor - the westinghouse digital 1080p 37" LVM-37w3, and its also what i'm typing this on, demonstrating to myself that total digital experience convergence in my house is in full swing, since it's my TV, internet, video editing monitor and Game screen)

MACRO BLOCKING
bah. mpeg-ts.

The method used to get your digital video into your screen space is an MPEG transport stream.
why do you need to know this? ... because it is where the worst quality problems are happening.\ e.g. Why get a High Definition TV and digital Cable service/Dishwhatever and then end up with larger than life junky video frames. I see it everywhere, all services of all types, no one is immune.

Generally speaking a high quality well authored DVD or High Def DVD will be your best viewing experience, but quite frankly most of us wont be seeing the full real deal there in this calendar year, so i will speak to the broadcast quality digital HD and SD.

Ok so what is Mpeg and what is a transport stream?
Its your picture. Its Digital, and it can occasionally suck.
MPEG is a motion picture encoding scheme for compressing digital content, and transport stream is how it gets compressed at the source and decompressed on your end, that is the short of it. BOTH ends matter, but you basically have very little control over it, selecting the right provider company and having good equipment is about all you can do.

Basically the transport stream seeks to compress in a frame sequence optimization by only sending the content that is different in the next frame. So if a frame is mostly an unmoving background and the only things changing are the people talking ( headshots typical of news and talk shows ) then compression works really well and detail is preserved nicely because the bandwidth or size of the transport stream is composed of only the peoples heads in motion.
So in an HD video frame (1/30 sec) maybe only 10-20% of the 2 million pixels are different ,only 10-20% of the frame image data is needed to properly create the next frames. This is why Conan O'Brien in HD looks pixel perfect and really scary Conan's facial detail is always there, that guy can make his face move in such extreme ways and you do not miss a thing.

It all starts to go to hell when the entire frame is in motion and almost every pixel of the 2million in an HD frame are dramatically different, properly done, an Mpeg transport stream will then issue what is called a Key Frame whis is a much less compressed but complete image frame. That means 90-100% of the 2million pixels is needed to construct a new image for the next video frame. Uses More Bandwidth, Bigger data load.

"I See Big Squares"

Unfortunately, bandwidth numbers for the range of low data load to high data load is usually less than what is needed to send full frame transport stream keyframes every time, and the result is that you see the math in all that fail and the screen image is Blocky, There are small ones and Big ones, my biggest beef is the Big Blocks. Now your know what to call that problem. IT is a MACRO BLOCK. they are always there and the variation in its quality is what you notice, transport stream block failure, they come in various sizes 8x8 and 16x16, but bad Macro blocks are big and really botch an image past the point of tolerable pain.

In a worst case you may actually see completely missing blocks on your screen (black squares), or little squares of total garbage. These blocks are usually next to each other in a row when that happens, and it is a transport stream error correction problem , for cable that is a really lousy broadcast quality or system bandwidth overload at the provider/distribution end.

Macro Blocks are the BIG ONES = 16 x 16 pixels ...that is a data block of 256 computed pixels that is constructed at your end by your cable box / Dishthing and and then assembled to the screen memory of your TV by what is usually some very advanced firmware code. Welcome to the DIGITAL world, even if your equipment is totally high end your provider can and will send an MPEG Transport Stream that will make you feel like you are in a trailer park with a coathanger antenna...its the static crap of the digital age.

Ok lets get active. IF YOU HAVE GOOD EQUPMENT & SEE LOTS OF MACRO BLOCK ERRORS in your stuff, then i want you to IMMEDIATELY COMPLAIN in full detail about channel and time, by phone, to your provider.

... and conversely if you DO NOT COMPLAIN, then they will think thier stuff is delivering ok and continue to deliver junk possibly considering that you dont care or that such and issue is tolerable.

Most of these people behind the scenes of it all are trying hard but they cant always see everything at every endpoint in thier system and your feedback is in fact a component of thier quality control. There are a bazillion possible points of failure, including connection quality and signal corruption in the line between you and the Central Office, and if your dish or antenna is having issues, then system bandwidth overload is not the only problem, weather and sun-spots and whatall noise can corrupt the signal.

Ok i know i am way over-simplifying things, but if your video investment is important to you, you really need to know this at a minimum level. Geeklink = http://www.zenith.com/sub_hdtv/mpeg_tutorial/pixtypes.HTM

In Another Note:
Blah. dumb junk i get to read where people comment about the difference between 1080i and 1080p. IF YOU HAVE AN LCD type HD display, then you are seeing images presented full frame at a time. Progressive OUTPUT however is not a guarentee that the input is any good...Interlace errors you may see are almost always going to be the product of bad video editing at the source that was then made into a bad TS . Almost all HD source transport streams are HD1080i & therefore arrive to you interlaced, and your monitor assembles that into your image in a full frame (progressive), and scales the result to your screen.
This means that a nice 1366x768 is going to have 2 times the data incoming that it needs to create a Progressive HD image and basically it will look great. People with 1920 x 1080 displays need to be a bit pickier since all the data is used and not scaled, so all the transport stream data is seen. Fortunately most all 1080 screen manufacturers are very quality conscious and most if not all current 1080 screens are well equipped with internal computing engines and firmware.

1080i 1080p Myth Busting

We a lot of seemingly authoritative bad and old writing out there, it is time for some basic...

MYTH BUSTING HD #1


1080i is worse than 1080p.

Not really. Broadcasts are in HD1080i and properly managed at both ends, is very good. Since Flat panel displays do not scan but instead assemble each frame in video memory, it is always a 1080p screen result. Nearly all the 1080p displays available in 2007 do a great job of de-interlacing, although i have a preference for DCDi ( Directional Correlation De-interlacing ) as done by Faroudja.

1080p in HD-DVD and Blu-ray is better mostly from the reduced compression and not the interlace (i) or progressive (p) delivery. In Fact some 1080p 24 at 24 frames per second from film productions is more "filmic" ( a word i have heard used often ) at a lesser frame rate and more motion blur than the usual 30 or 60 frames persecond that a good 1080p LCD HD display can show you. Odd - even arrival of interlaced data is alternate rows of pixels requiring processing to assemble an image. Over and above that is the 16 x 16 pixel block used in compression codecs.

-- So really where is the problem for 1080 video? It is not Interlacing.

MPEG and Compression

The usual problem for digital video in any form is not in the progressive or interlaced delivery of the image frames, but in the over-compression of the digital processing. Current broadcast standards are delivery of TS or Transport Streams in MPEG2. This is required because the MPEG compression de-compression process must happen at the both send and recieve ends. So the Broadcaster uses MPE2 to compress and your Cablebox firmware uses a built-in standard MPEG2 chipset to de-compress. Its old technology, from back in the DVD days. Direct-TV and others are migrating to MPEG4 variants which are somewhat better, but blocking errors are still the outcome when this technology is mis-used.

Worst Case Scenario: BIG BAD MACROBLOCK

The problem is in bad data or over-compression used to save bandwidth, and visible evidence of this is square chunks of the image looking out of place and time. That artifact is known as a MacroBlock... if there is any detriment to digital works, it is abuse of the mpeg2 compression codecs that reveals blocking in the frame. All your digital video is made up of these blocks and when its all good, you probably do not see them or know that they are there. This is different from interlacing which is a very fine comb-like edge artifact.

Basically a cheap broadcaster running out of Bandwidth will smash and squeeze the video stream right up to the point of customer complaint. Then you see macro blocking.

Now that you know what all this is, complain when you see it. Click to see below -- a very extreme case of compression recovery failure in macroblocks. Click Play and then move the slider to see and select frames for view. ( there is no audio ) A variety of error cases is visible but they happen fast so the slider helps see this in detail.

You should never see anything this bad, but it serves to expose what your digital video is really doing, in that you are always seeing blocks of data. Extreme cases like this make it much more obvious. This example was sent to me on a DVD from a friend who thought his system was broken. It was recorded direct from his cablebox. It is important to note, that this processing problem is not your new flatpanel display, but happens before that at the Cablebox/Converter, and can be from line noise in the cable, a problem at the source, or bandwidth overload, among other things.

1080 and the SXRD

Sony again, this time on BRAVIA and its 50" SXRD.

Jeff Goldstein, Sony A/V product marketing VP, said, “It's time to ask ourselves, 'What kind of HD experience is our industry really offering?' As always, when it comes to high definition, Sony believes that we need to set the right expectations for consumers by developing technology that provides the very best HD performance possible.”

This... is true. But dont take my word for it. Go into any Best Buy store ( Magnolia ) and ask to see the new 50 " SXRD from Sony. or the 55 or the 60" . its all good.

Trust me 50 inches is a sweet size for most larger living rooms, and at 1920 x 1080 , sitting close to it is a real pleasure... at 5-6 feet away it covers your field of vision and visual immersion is assured. however it is so good at detail that you will not enjoy the macro blocks from over-compression and Standard tv signals and some dish network channels will express all the sins of a poor source quality in stunning detail.

This unit is so big bright and clean that you will not believe that 2800 bucks can get you all that.
now granted, it is rear projection and it has a few possible caveats, but in the grand scope of flat wide and big, this is a great buy. You may find that its at the back of the store, and away from the plasmas and "screen door" displays, mostly because it will blow away anything near it in a side by side viewing, and it makes competing units look like a poor choice for the equivalent dollar.