Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

Dr. Dobb's Data Compression Newsletter Issue #46 - September 2003

DDJ Data Compression Newsletter Issue #46

Welcome to this issue of Dr. Dobb's Data Compression Newsletter! This newsletter keeps you up to date with news items, product information, and the occasional editorial aside about the world of Data Compression. Your input is always welcome in the form of comments, submissions, or original material.

PDF Compressor
September 14, 2003

This nice program showed up in the submission queue earlier in the week from Tom Phelps. Rather than the usual nonsense I cook up to describe something, I'll give you Tom's text verbatim:

Free PDF Compressor that removes duplicate PDF objects, optionally takes advantage of new compression features of latest PDF specification (1.5), and optionally takes advantage of a new proposed format called "Compact PDF" that for many classes of documents compresses 30 - 60% better than what is possible in PDF 1.5.
To actually download the package, you'll need to browse up a couple of levels - there is no link on this page. Worth the search.

PDF Compress documentation
Tom's download page

Image to EPS/EPS to Image
September 14, 2003

Open Source package imgtops takes JPEG, PNG, GIF, Targa, BMP, and other image files and converts them to Encapsulated PostScript. Without going into too much detail, it's also important to point out that it does this in such a way as to minimize the size of the resulting EPS file. Examples and proof given here!

And to make this even more of a beautiful thing, I'm here to report that this version 1.0 is now available for download. That first official release always brings a tear to my eye.

imgtops project page

Sand Video Demos H.264 Chip
September 13, 2003

This story in EE Times is another announcement about a cool new video chip. Sand Video has created a chip that will do transcoding of MPEG-2 to H.264. This might be just the thing if you are a cable provider or some other distributor of content. Sand suggests it might be nice for a PVR.

This article also helps to introduce a new acronym. You've heard this video compression technology referred to as H.26L, H.264, MPEG-4 Part 10, now get ready for a new one: AVC, or Advanced Video Coding. Don't be the last one on your block to drop that into a cocktail party conversation.

EE Times: Sand to Demonstrate AVC/H.264 High Definition PVR/DVR Technology

Linux File System Compression
September 13, 2003

This open source package, e2compr, provides transparent compression and decompression of Linux ext2 file systems. I'm not a Linux configuration expert, but I think this is the first dynamic version of a compressed file system on ext2 file systems.

However, before you rush off and use this, I should say that the whole project looks like it might be just a tad bit shaky. Even though it has a version number of 0.4.43, there's no history of earlier versions, and no docs to speak of.

And just thinking out loud here, but how often do you install software that is distributed as a diff file?

e2compr project page

Huffman encoder/decoder library
September 12, 2003

Let's say you created a Huffman coding library and released it as open source. What would be an apt name for this product? How about huffman? Good one!

Looks as though this library is by a fellow named Douglas Ryan Richardson, who appears to also have an interest in steganography.

This is the first release of the LGPL-licensed library, so it's a bit early to pass any sort of judgment on it. But if you're interesting in influencing the direction this product takes, now is definitely the time.

huffman project page

A Few Benchmarks From Berto
September 12, 2003

A fellow named Berto sent me an email the other day telling me about this compression corpus, and passed the link below. He said he compiled the data into a spreadsheet for easy access, and had been accumulating results since 1997.

Well, I went to the page, and instead of being from some guy named Berto, it was owned by Matt Mahoney, an instructor at compression hotbed Florida Institute of Technology. What was up with this?

Confused, I did a minute or two of sleuthing, and finally figured out that Berto has a link to his spreadsheet on Matt's page. And it is indeed a nice spreadsheet. Links to both pages below.

Matt Mahoney's Benchmark Page
Berto's spreadsheet

Maximum Compression Update
September 12, 2003

Speaking of benchmarks, this might be a good time to point out that Werner Bergmans updated his most excellent Maximum Compression Web site on September 1.

Werner's goal is to see who does best at various file types. In addition the standard Calgary corpus, Werners compresses PDF files, DOC files, and more. It's exhaustive. I get tired just looking at it.

Maximum Compression site

Emblaze Video Supports H.264
September 11, 2003

MPEG-4 is good, but H.264 is even better. This new video compression standard is just getting a toe-hold in the commercial market, and this announcement from Emblaze helps to make the whole thing a little more real.

Emblaze is making an accelerator chip that is targeted for the mobile market, with the goal being to pump video down to the phone at a G3-speed bandwidth while retaining a reasonable picture quality.

Of course, for the high-end phones, this chip can also capture and encode video at CIF resolution at 15 frames per second. I wouldn't be surprised to hear of this guy showing up in webcams as well, although the $10 price might be a problem for the consumer market.

EE Times: Emblaze uses accelerators to handle video for handhelds

Linksys Debuts MPEG-4 WebCam
September 11, 2003

Disclosure time: I work for Cisco Systems, which recently acquired SOHO networking company Linksys. But I can assure you I'm totally objective about this when it comes to issues of data compression.

This press release announces a nice little wireless webcam from Linksys. What makes it pretty cool is that it is is an independent network client that broadcasts video in MPEG-4 format at 320x240 resolution. In theory this means you could even watch a live feed over the Internet if you have a decent uplink speed on your broadband connection.

As soon as this shows up on the internal Employee Purchase Web site, I'll be snagging one and reporting on its performance.

Yahoo!: New Linksys Wireless-B Internet Video Camera Sends Live Video Over the Web

JPEG-2000 Real Yet?
September 11, 2003

An anonymous poster to comp.compression recently asked a pretty reasonable question. He points out that the JPEG 2000 standard has been in force for a while, and where are the commercial apps that use it?

The posters did manage to come up with a few examples, but not enough to satisfy me that this standard is real yet. I say wait another year and see if it's picked up critical mass.

comp.compression thread courtesy of Google

Data Compression in the Retina
September 10, 2003

The more I learn about the human retina the more complicated it seems. If you're a normal geeky inhabitant of the early 21st century, without any other info you would probably suspect that the retina is a lot like the imaging sensor in a digital camera. After all, that's how you would design it, right?

Maybe that's how you would do it, but the motive force behind biological innovation has chosen a different path. The long path that starts that the surface of your eye and finishes in your mind's eye involves some very circuitous work, and some pretty intelligent peripherals. Apparently things like motion detection and edge sensing are taking place right there in your eye, not your brain.

This article from the MIT Technology Review talks about the fact that the information passed between the retina and the brain actually undergoes a pass of data compression. A frustrating lack of detail doesn't help, but it sounds as though most of the information is being thrown out in favor of just transmitting edge information. Cool.

Technology Review: Vision Chip Shines

September 10, 2003

Vector Quantization is not getting the press that it once received. So kids these days studying data compression may not learn enough about this noble technology.

One way I got a better appreciation for VQ was by writing my own color quantizer. And I'm betting that others can benefit from a little info about color quantization as well.

This article on the Code Project Web site by Davide Pizzolato uses the Octree algorithm to reduce the number of colors in an image in an nice fashion. One thing that's fun about working with VQ in this fashion is you get to see interesting pictures while you're doing the compression.

The CodeProject: CQuantizer

Another Suffix Tree
September 10, 2003

Just yesterday I ran a pointer to some Suffix Tree code, not a particularly common occurrence here. It just happens that yesterday's article gives much credit to a Suffix Tree implementation by Shlomo Yona, which is pointed to here.

Shlomo's code is designed as a library, and includes some sample code to drive it, as well as some Perl code to make it easier to exercise.

Get a copy while you can; these student Web sites on University servers come and go quickly.

ANSI C implementation of a Suffix Tree

Hacking the Hasbro PVR
September 9, 2003

For only $50, you can buy a video player from Hasbro that plays movies on a grayscale 80x80 LCD. The content comes from Hasbro on 3 inch CDs.

This might be a fun thing to play around with, if only there was some way to create content of your own. It would also be nice to be able to play the special CDs on your own computer.

Well sure enough, somebody named Zocher is having a go at this format, and seems to be having some pretty good success so far. Might be time to go out and buy one of these players - at least if I can think up a reasonable use for it.

VideoNow PVD Reverse Engineering
Hasbro: VideoNow Personal Video Player

September 9, 2003

This project describes itself as "An Open Source Implementation of a Suffix Tree." It's actually a bit more than that, it's an interactive application that can be used to explore the ins and outs of a suffix tree. That's kind of cool.

The current version number is 0.13, so this is obviously an immature product. It's hard to tell if this is an active project or just a one-shot deal.

Suffix Tree

Sun, Real Hook Up
September 9, 2003

Sun is apparently hard at work promoting a desktop O/S called Project Mad Hatter, which is based on UNIX. Mad Hatter is supposed to compete with Windows, so it's going to need some basic media player capabilities, and it looks as though Real is going to help provide that.

That's right, Real is going to provide a version of their client to run under this O/S. Is this important? I don't know, this article represents the first I've ever heard of this project from Sun.

Wired: Sun, RealNetworks in Desktop Deal

New Paper from Jürgen Abel
September 8, 2003

Dr. Jürgen Abel dropped a line a couple of days ago letting me know that a preprint of his new paper, Record Preprocessing for Data Compression, was available on his fine data-compression.info Web site. This follows close on the heels of another preprocessing paper Jürgen published a little earlier this summer.

This paper describes a nice way of preprocessing files that have a record-oriented structure. By transposing the files by the record length, standard lossless compression algorithms see excellent gains in the resulting compression ratios. Makes sense, doesn't it?

Jürgen explains how this transposition allows compression programs to exploit symbol repetitions that occur at the same position inside a sequence of records. He is able to demonstrate the effect on files drawn from familiar corpi used for data compression benchmarks. A good read!

Record Preprocessing for Data Compression

CNet Helps With Christmas List
September 7, 2003

Everyone wants an iPod, and every manufacturer is dreaming of making the iPod killer. This review article takes a look at the next five candidates to go after this lucrative market.

The short story here is that these five products are all pretty nice, get good ratings, and all stack up well against the iPod. Chances of any of them suddenly picking up the mystical aura of the iPod? Near zero.

ZDNet Down Under: Groove armada: 5 MP3 players previewed

ZVUE Redux
September 6, 2003

This $99 video player story has been kicking around for a while now, but so far the story has been long on rumor and short on fact. Now we actually get a hands-on report from someone who has actually touched a ZVUE.

The player definitely looks cool, but it seems to me that we're going to have a little bit of a problem getting the content we want into this guy. I think we're seeing a case where the intention is to lose money on the player and make it up on the rental fees. My guess is the DRM on this guy will be super-tight. No way to put your own video on the player, and no way to share your purchased content with another player.

But I could be all wet.

LiveJournal entry

Celoxica Demos System Solution
September 6, 2003

So you've got some sort of idea for a hardware design kicking around in your head. You'd like to be able to do some image decoding in hardware, and then use your embedded software to do the display work. But how do you put all the pieces together?

Good news! The folks at Celoxica want to show you just how their software can help you design the entire package. These guys want to demo some serious system design. And if you're willing to head on over to Programmable World 2003, you can see the whole thing.

Celoxica press release

Dvorak on DivX
September 5, 2003

John Dvorak of PC Magazine gives a little history of the DivX codec. Well-schooled readers know that DivX is essentially a wrapper for MPEG-4 video, and has long been available as a free add-on for various packages on Windows, Linux, and more.

John goes into some interesting detail behind the company that's now driving the DivX codec, DivXNetworks. These guys are now dropping this codec into DVD players, which has the possibility to make things really interesting, something Dvorak explores a bit. (With the usual sip of Dvorak hyperbole.)

Ziff Davis: DivX Reloaded

MPlayer 1.0 Prelease!
September 5, 2003

I'm a confirmed user of Windows, which usually works out okay. But at times there are some really great apps that just aren't available on my platform of choice.

One example of this has been MPlayer, which is an excellent media player that until now was strictly a *NIX operation. The good news is that the 1.0 release has been advertised as having a port to Win32, and it looks like this is going to come true!

The folks at Slashdot ran an article that let me know the 1.0 prerelease was on the home site, and sure enough, there it is with a Win32 port! As the Slashdot cognoscenti pointed out, perhaps now I won't need to install players from Microsoft, Real, and Apple to meet my media needs. One can hope.

Slashdot: MPlayer 1.0Pre1 Is Here
MPlayer project page

Prodigy Has Head for Music, Compression
September 5, 2003

Reading about Erin Earl's schedule will make your head swim. But try to read the whole article. The reward is that you'll learn that this remarkable girl has spent some of her considerable talents on the study of data compression. Go Erin!

AScribe: New IU Bloomington Student Finds Working More Works to Her Advantage

RIAA Hashing MP3 Files
September 4, 2003

Okay, so the RIAA decides to take a peek at your PC to see if those MP3 files are really yours. What if you tell them you ripped them directly from your own CDs? Wouldn't the burden of proof be on them to to show that assertion to be false?

Apparenty, the RIAA is worried enough about this problem to actually do some serious research. Turns out that in their latest court filings, they are contending that they can use hashes of large segments of the sampled files as indicators of copying.

I'm not sure this is going to fly either. Let's say you and I each choose to rip the latest Justin Timberlake CD, and let's say we both decide to encode our MP3 files at 128Kbps. What are the chances that our files will turn out to be identical?

I'd say pretty good. If we both happen to use the same software, the chances are excellent. Even if we use different encoders, I suspect there is a fairly good chance that segments of the sampled code could be identical.

Try explaining that to a jury though...

New Scientist: Music industry claims MP3s are traceable

Data Compression Feel Good Story
September 4, 2003

Yes, we all love data compression, some of us to the point of otaku-hood. But how often can we really hold our heads up high and point to our passion as something that really makes a difference in the world?

Well, fellow travelers, I recommend you zip down to the newsstand and pick up a copy of the September New Yorker. There's a nice article describing the soon-to-be end of the Galileo probe. A gripping part of the article describes how the mission was effectively saved from disaster by an incredible rewrite of the system software. After losing its high-speed antenna, Galileo was limited to sending back data at 10 bits per second - meaning one picture per month.

The team responsible for Galileo was able to effectively do a complete upgrade on Galileo's system software, enabling data compression of the transmitted images, allowing transmission of over 200 images per month. Mission accomplished, thanks to data compression!


7-Zip numbering system
September 4, 2003

Back on August 23, I took note of the fact that 7-Zip posted a new update with a completely new version numbering system. No explanation of why the version changed.

Anyway, Claus Dissing took the trouble to point out the obvious about the new system:

I just wanted to inform you of Igor Pavlovs new Version-number-system:
It's the release-date! year: 2003 - Month 08 - version xx (still beta of course!)
Thanks, Claus, I guess that was fairly obvious! Now if we can just get Igor to release a version of 7-Zip that is no longer listed as beta, we'll really be in business!

Livingston on HTTP Compression
September 3, 2003

Brian Livingston, author of a zillion different books on tips and secrets for various versions of Windows, has a very useful little article here on using HTTP compression for your web pages.

Brian starts the article with kind of a goofy hook, saying that Intel is wasting more network bandwidth than any other company, simply by failing to use HTTP compression. Okay, Brian, thanks for the tip!

But the article goes on to provide pointers to products that you can use to provide compression with various Web servers, including the big dogs such as Apache and IIS.

Datamation: Intel Blows Bandwidth

Helix Server Vulnerability
September 3, 2003

You've probably seen an article or two in this space about the Open Source Helix project. RealNetworks has nobly released a media server that can be used to deliver a nice variety of streams.

This is all fine of course, unless the server is a wide open magnet for hackers and vandals. Turns out that Real had to eat a bit of crow and announce that the Helix server has one of those nasty bugs that enables exploits that can "execute arbitrary code."

We don't like anything that allows vandals to execute arbitray code, so it's good to know that Real has given pointers to a workaround, and promises to get on a permanent fix ASAP.

internetnews.com: 'Critical' Security Hole in Real's Helix Server

Revolutionary Audio Compression
September 3, 2003

What if you had an audio encoder that was capable of reproducing the quality of MP3 files in 25% of the bandwidth? And what if you knew nothing about how to go about evaluating such a codec? What if you didn't know anything about marketing? Maybe you don't even know if this new idea was worth your time. What would you do?

Well, I know one thing I wouldn't do, which is turn to the denizens of comp.compression for advice. That group generally turns quite a jaundiced eye towards any incredible claims. But John Reidar Mathiassen doesn't share my same sense of caution, and he was quite happy to post away.

And oddly enough, he seems to have made it through the gauntlet with all body parts intact. Way to go John! I'd love to know more about that codec, but so far John is remaining mum.

comp.compression thread courtesy of Google

Compressing a Dictionary
September 2, 2003

This thread on comp.compression brings up a question I've seen posted quite a few times: the poster wants to know the best way to compress a dictionary. (What I think he actually wants to compress is a sorted list of words.) It's an interesting topic, because it can be optimized to take advantage of the nature of the data. Most general-purpose compression programs can't.

In this case I think the poster didn't really get the response he was hoping for - but I think a painstaking search through groups.google.com might turn up some more good tips. Unfortunately, the keyword "dictionary" generates an extraordinarily high number of hits that aren't directly related to the topic at hand.

comp.compression post courtesy of Google

Reverse Engineering an MPEG Driver
September 2, 2003

The folks at VIA have motherboards with integrated MPEG decoders, which is pretty cool. Not only that, but they've released Linux drivers for said chipsets, which is even more cool. But sadly, they decided to release these drivers in binary form only, which is definitely not cool.

Binary-only means lots of bad things. For one thing, if you aren't running on a compatible x86 platform, you are no doubt out of luck. And of course, if you feel the urge to write your own code to talk to the chips, you are not going to get anywhere.

The folks at Slashdot picked up a pointer to an article about an intrepid soul who decided to reverse engineer these drivers. A noble pursuit indeed, and what's better, this guy, Ivor Hewitt, decided to document the process on his Web page. Well done, Ivor!

Slashdot: Reverse Engineering an MPEG Driver
Ivor Hewitt's CLE266 MPEG Library

An Image Viewer with Lossless Rotation, EXIF and Other Goodies
September 2, 2003

I love it when the title of a Web page says it all. That means I don't have to do too much work summing it up here. Go to the CodeProject Web page and I think you know what you can expect.

The CodeProject article

More Research From Drew Scott Daniels
September 1, 2003

A few weeks ago I took note of the fact that IBM was countersuing archenemy SCO over a variety of things, including at least one data compression patent. The press release I had didn't detail what specific patent was involved, so I had to bluff my way through the details.

Fortunately, faithful correspondent Drew Scott Daniels, who keeps track of LZW patent issues, happened to have the details on this, and passed along the following info:

In IBM vs. SCO, IBM made a counterclaim [1] that SCO was infringing on patent 4,814,746 [2] which is thought [3] to cover LZW. It's not clear whether this patent is valid as Unisys got their LZW patent first, but IBM did file 3 days earlier [3].

I'm not too worried about patent 4,814,746 as it should have expired already[4]. Patent infringement can be sued over, up to 6 years after the infringement[5], which is what I think is happening here. If you're not a patent lawyer, then be careful about looking up patents [6].

Since IBM filed their patent first, perhaps LZW should really be called Lempel-Ziv-Miller-Wegman or LZMW?

[1] http://lwn.net/Articles/43363/
[2] USPTO Link to 4,814,746
[3] http://cloanto.com/users/mcb/19950127giflzw.html
[4] Unisys's patent from within a few days expired months ago. Not being a lawyer and not having read the patent I can't be sure.
[5] http://lists.debian.org/debian-legal/2003/debian-legal-200303/msg00416.html
[6] http://www.advogato.org/article/7.html

The Patent Nuclear Weapon
September 1, 2003

John Carroll of ZDNet has a bit more to say about the same lawsuit. He seems to be echoing the point I made, which is that IBM's patent portfolio is deep enough to grind just about anyone into the dirt.

Write a program of even minor signficance, then ask IBM's lawyers to find some code that infringes on their library of patents. Think they'll have any trouble? Don't bet your product on it - you'll lose!

ZDNet: The patent nuclear weapon

August 31, 2003

Imagero is a Java library that knows how to read a wide variety of compressed image files. This includes JPEG, PNG, and TIFF.

Better yet, Imagero can also read metadata and thumbnails from TIFF, PNG, JPEG, and ESIX files.

I wish they'd add GIF to the list of supported formats. Seems like it wouldn't be too hard, seeing as how they already support LZW compression in TIFF files.

Imagero home page

August 31, 2003

Sure, there are lots of media players for Windows. I don't go looking for them, but this one was submitted by the author or somebody close to him, so I happy to pass along a pointer.

MediaOne+ is a Win32 media player that handles all the standard audio and video types you could ever shake a stick at. Yes, that includes DVD, MP3, DivX, and more.

You're going to have to pay for this media player, but maybe you'll find it's worth it. Maybe just for the fast load times, maybe for the super-optimization. Give the downloadable demo a spin and see what you think!

Media1Moon Home Page

PowerArchiver Update
August 30, 2003

In a world full of Win32 archiving program, PowerArchiver manages to hang in there with a decent product that is in the same ballpark as WinZip and other leaders. Version 8.6 shipped earlier this month, and includes some optimizations and bug fixes.

Rather than bore you with the incredible list of features that this product supports, I'll just point you to the feature list on the PowerArchiver Web page. It's pretty awesome.

PowerArchiver feature list

Common Lisp JPEG Library
August 30, 2003

If you take a look at your keyboard, do you happen to see that the ink is completely worn off the parentheses keys? Do you use the words CDR and CONS in normal speech? If so, perhaps you are a Lisp programmer.

And if you are, you just might want to work with JPEG images in some of your Lisp programs. Turns out there's a group who has created a library to do just that!

Common Lisp JPEG Library

FlyZip 2.5
August 29, 2003

Over in Latvia there's a company called Elegant Algorithms. They happen to make a really spiffy Zip tool that is targeted squarely at users of devices running the Palm OS. Small footprint and high speed seem to be the things they're bragging about most.

However, there's something about this utility that confuses me a little bit. They clearly talk about working with Zip files in their docs, which makes sense given the name. But they also talk about having a patented lossless data compression method which they claim is the fastest in the world. Is this simply an implementation of the deflate method, or do they actually have their own proprietary algorithm?

Regardless of this confusion on my part, you have to love a product that carries a customer testimonial advising you that "It compresses and decompresses at the speed of vertigo." Amen.

FlyZip home page

Evil Empire to PNG: Drop Dead
August 29, 2003

No surprise to hear that Internet Explorer is a bit deficient in its support of PNG files. After all, PNG has been the red-headed stepchild of the Internet for some time, despite its aura of patent-free virtue. (No offense to those of you with auburn locks.)

This latest Microsoft Knowledge base article has a headline of "Cannot View Some PNG Images." Well, we suspected that all along. What's funny about it is the listed cause "This behavior occurs because the PNG graphic is either 4097 or 4098 bytes."

The programmers among you will immediately note the proximity of that value to 2^12, and make an assumption that is a fencepost error of some type.

Conspiracy theory or not, the good news is that this is fixed by simply bringing IE6 up to the latest service pack. Of course, if you're like me, you feel like you've been applying patches to IE6 on an hourly basis. But it's free, so I guess we can't complain.

By the way, viewing this Knowledge Base article highlights one of the things that I really hate about Microsoft's knowledge base. Check out the list of keywords attached to this article:

  • kbQFE
  • kbIE600preSP2fix
  • kbfix
  • kbbug
  • KB822071
  • kbAudEndUser
  • kbAudITPRO
Boy, those are handy, aren't they?

Microsoft Knowledge Base Article - 822071

ExtremeTech Trashes Neuros
August 29, 2003

You've seen me drool over the new Neuros MP3 player in these pages. I've been pretty worked up about various features it has, including the FM transmitter, OGG support, hard drive, Linux support, and more.

Well, the folks at ExtremeTech gave this puppy a spin, and I'm sorry to say that they couldn't give it a big thumbs up. Mostly, they found that the really cool new features just didn't work. Bummer.

ExtremeTech.com: Neuros Audio Computer Rethinks MP3

Aladdin Systems Announcement
August 28, 2003

Aladdin Systems recently posted an announcent on comp.compression stating that they licensed something called the PWC image compression algorithm for use in their products.

I waited around for this announcement to show up in an official press release, but that doesn't seem to be in the cards, so I'm just pointing you to the original post.

If the performance of this lossless image compressor is to be trusted, I think this is a pretty good strategic move on Aladdin's part. This will give them the ability to squeeze a lot of extra space out of GIF and PNG files stored in their products, which typically aren't compressed at all by other archivers. Naturally, the PWC algorithm is lossless, so this is a completely reversible transformation. I'm wondering if it could be applied to JPEG images as well?

comp.compression post courtesy of Google.

August 28, 2003

Way back in the spring of 2002, I added a library called Rubyzip to my database. Rubyzip is a Ruby library that lets you read and write zip files from Ruby. At the time, it was in beta at version 0.4.

Well, things in Rubyzip land are progressing, albeit not very quickly. They're still in beta, but now up to version 0.5.1. A fair number of changes during that time, although I'm sure much of the work was just keeping up with Ruby releases.

Rubyzip project page

GNOME Structured File Library
August 28, 2003

Boy, the first time I looked into the possibility of reading Microsoft's OLE2 structured files, I didn't really like what I saw. I could see the wisdom for having structured files, but for some reason Microsoft never makes an API particularly easy to use. At the time using COM from C++ seemed to me to really suck, although I guess things are a little better now.

In any case, the folks at the Linux GNOME project apparently have this structured file stuff on the brain as well, so they've created a library that will read OLE format files. Not that exciting so far, but the good news is that it can treat a Zip file as if it were a structured file. A little more interested now? I thought so.

Freshmeat pointer to the project

The Open Compression Toolkit for C++
August 27, 2003

Here's a pretty cool open source project. The Open Compression Toolkit for C++, otherwise known as Octane, provides a nice set of classes that help you write, test, compare, and validate algorithms.

As you might expect, this includes classes to do modeling, bit-oriented I/O, and more. The first release came out just this month, so you probably haven't bumped into this product yet. I encourage you to take a look.

Project page

August 27, 2003

Although it's not exactly a data compression topic, I like to keep any eye on programs that are used to compare files and analyze differences. After all, a program that can generate a diff very efficiently is already halfway to being a pretty good compressor.

This new program, Fastdiff, is written by somebody named Divon Lan, who claims to be able to chew through as much as 300 MB/sec when comparing files. That's pretty fast.

Divon is giving the program away, but claims to have constraints on the source which prevent him from posting it on the net. He says you can get it under a special contract, which he hopes will not be required in the future.

Fastdiff home page

Any File Down to 8K
August 27, 2003

Yes, it's been too long since we had a good set of claims for infinite compression. Fortunately, Timo Makinen has come along and opened up our favorite can of worms by claiming to compress anything down to 8K bytes.

Naturally, if this were true it would mean that significant portions of what we know about mathematics would have to be rewritten. Bad news, but it would be worth it, because I could get rid of the 120G hard drive MP3 repository and put all my music on a floppy.

I can compress anything down to 8K!
They said the world was flat and man couldn't fly!
I'm willing to demonstrate, but won't release my source!
Again, this works!

Drew Scott Daniels on LZW Patents
August 26, 2003

Drew Scott Daniels worries about the state of LZW patents. Of course, at this time we all suppose that the US patents are dead, and the Japanese and other patents are close behind. Drew has been doing some research on this, and I'll pass it along for your benefit.

I'm still worried about quotes from UNISYS like "Variants on the basic LZW patent run for about 20 years and further U.S. applications are pending" [2] and "Be advised that Unisys has many patents issued and pending on variants from basic LZW." [3]

Although Unisys does say [4]

After expiration of the U.S. LZW patent on June 20, 2003, liability for patent infringement will occur only if an infringing act with respect to a product or service (e.g., developing, selling, offering to sell, making, using, distributing, downloading, exporting and/or importing) occurs in a country where the LZW patent has not expired.
right after talking about how the LZW algorithm is used in the GIF file format.

Unisys also says [4] "...products using LZW (GIF, TIFF-LZW, PostScript, Portable Document Format (PDF), V.42bis, etc.)...".

[1] http://cloanto.com/users/mcb/19950127giflzw.html says IBM patent 4,814,746 is on LZW and British Telecom holds a similar patent.
[2] http://lpf.ai.mit.edu/Patents/Gif/Gif.html
[3] http://lists.debian.org/debian-legal/2003/debian-legal-200303/msg00526.html
[4] http://www.unisys.com/about__unisys/lzw/

August 26, 2003

ModZipRead is an Apache module that reads Zip files. It displays the files to the user in a typical folder view, which allows them to view or download contents at will.

Looks pretty good, although the author cautions that the current version is 0.1, so you shouldn't be using it for any mission-critical functions. Fair enough.

ModZipRead project page

New Toshiba MP3 Player
August 26, 2003

Looks like Toshiba is creating a pretty sweet little portable audio player. Smaller than an iPod, a 20G hard drive, the ability to play WMA and MP3 - what more could you ask for?

Well, you could ask for it to be sold in the US. Right now it looks like this is only for sale in Japan. For right now all you can do is drool. Enjoy reading the Japanese press release as translated by BabelFish. (And don't complain, the fish understands Japanese a lot better than you do.)

Toshiba Press Release - translated courtesy of BabelFish
The Register: Toshiba debuts smallest HDD MP3 player

HD Radio Hits Snag
August 25, 2003

We've been hearing for a while about a revolutionary new broadcasting system that was going to revitalize AM radio here in the US. The HD Radio system apparently transmits digital information on sidebands of a station's standard frequencies, using that space to send both high-quality audio and text information.

Now it turns out that there's some question about whether the audio codec used in this system is going to cut the mustard. I have to admit that I'm not surprised that there's concern about this - the codec is advertised as giving CD-quality music, but this article in EE Times seems to say that it runs at just 36-kbit/second. Getting CD-quality audio out of that would be quite a feat!

EE Times: U.S. digital radio scheme hits static

DivX Now Adware?
August 25, 2003

For many years you've been able to get a few different versions of the DivX codec for Win32 platforms. There was a free version, an adware-supported version, and a pro version. Now the folks on Slashdot seem to have decided that adware is now the only option for this codec.

Browsing through all the messages makes it seem as though this might all be a bit of bogus hysteria, but the good news is that there's plenty of info on other video codecs, so the thread is worth a read anyway.

Slashdot: Divx Now Adware Supported Only

PKWare Data Compression Library
August 25, 2003

Over the years I've seen a regular roster of victims of the PKWare Data Compression Library. This proprietary library from PKWare doesn't ship with source, and thus can put you into various bad positions when you need to decompress data that it has created.

Fortunately there have been a couple of kindly people who have posted source code that can be used to decompress DCL output, but here on comp.compression is a guy named SW who isn't happy with that. Turns out that SW wants to continue creating data in this poorly-supported format.

Mark Adler gives him some gentle discouragement, which seems

like the right thing to do from my vantage point. And since it seems germane, Mark brings up the whole question of whether reverse engineering is even legal these days, given the DMCA. Mark provides some pointers to info on that as well.

comp.compression thread courtesy of Google

PostScript 3 Supports JPEG 2000
August 24, 2003

Adobe announced that they're releasing PostScript 3 version 3016 to OEMs this month. This is of course big news if you make high-end printers, but why do you or I care?

Simple, we care because Adobe has now added support for JPEG-2000 to the format. And that's cool.

Adobe press release

More Recursive Compression
August 24, 2003

Scanning SourceForge the other day I see a new project pop up with the name Recursive Compression. Well of course that kind of nuttiness always causes a blip on my radar, so I certainly wanted to check it out.

I have to say so far it's a bit of a disappointment. The name implies that we're going to be able to compress any file, regardless of content or whether it's already been compressed. But the first download is just a text file describing something called the "Reversable Scrambling Method." Looking at the text file I get some complex description of a process that looks a lot more like encyrption than compression.

I'll keep on eye on it. Maybe the incredible claims will pop up later. We can always hope!

Recursive Compression project page

Decompressing Pictures?
August 23, 2003

Good citizen Bob got on comp.compression a few days back and asked a simple question:

What are good codecs that offer fast decompress, for still pictures? I only need 256 colors, more is fine, but I don't really need it. Images will usually be that of a text document, but may contain pictures.

For my application, JPEG offers good compression, lossy is fine, but JPEG decompresses too slow. So, compared to JPEG, is there a codec that has faster decompress, and comparable compression and quality?

A pretty reasonable question to ask, and the denizens of the newsgroup were happy to toss out a few reasonable suggestions. However, I have to feel that Bob walked away without an answer that satisfied his need for info. Perhaps you can post a follow up?

comp.compression thread courtesy of Google

7-Zip Update
August 23, 2003

I like to keep an eye on 7-Zip. It's a free archiver with a pretty good proprietary compression algorithm. But sometimes I get a bit confused.

For example, 7-Zip version numbers are goofy. For a period of a couple of years, all the releases have been labeled 2.30 Beta xx. What's up with that? They are clearly meant as production releases, but the beta designation apparently means that you aren't going to be able to count on the format remaining around forever.

So all of a sudden, after years of this consistently odd version number, we get a 3.08 beta release. Now I'm really lost.

But looking at the release notes tells me that this guy has added support for multi-threading under P4 Hyper-Threading, so that's pretty cool.

Plus there's a new localization for Esperanto, which means we've opened up some new markets. I say you go, Igor Pavlovl!

7-Zip home page

You Think You Have Problems
August 22, 2003

A Poster identified only as Tom posted this plaintive missive on comp.compression recently:

I need to extract an 11 gig zip file made with an older version of PKZIP. We have been able to get the first 4 gigs but nothing beyond that. Are there any tools or utilities that will scan the zip file and actually extract the data? I have tried a couple that claim to recover corrupted zips, but they only get the first 2 or 4 gb.

The data is a full offline backup and I need it ASAP!

What makes life really bad for Tom is the fact that older versions of PKZip were limited to a maximum of 4GB of data - a 32 bit word problem. There's no telling what was written to the end of the archive file, but there's a good chance it is completely invalid. For sure the entries in the central directory that point to that space in the file are no good.

Follow Tom's thread on comp.compression and see if anyone could offer this poor soul a modicum of relief. And of course, this brings up the cardinal rule of backups: validate your data! Every time is the only frequency that is "often enough!"

comp.compression thread courtesy of Google

Speaking of that 4GB Limit
August 22, 2003

If you're still using an archiver that uses the PKZip 2.04 format, you might find yourself in the same trouble that Tom ran afoul of. And the 4GB limit is not the worst of it. Newer versions of PKZip support the deflate64 algorithm, which is just a hair different from your father's deflate. Get ahold of a file with that format and you might find yourself in trouble.

Given these kind of problems, it's good to see the makers of BitZipper keeping up with the times. With the 3.4.1 release, this archiver added support for WinZip 9 Beta archives that use the "Enhanced Deflate" compression method.

Just think, if enough people add support for these new methods, we might start talking about having a "standard" again!

BitZipper Home Page

Roxio Releases Toast 6 Titanium
August 22, 2003

Just burning CDs isn't enough any more, and Roxio knows this. So when you buy their new Toast 6, you're getting a Mac application that is a complete authoring system for CDs and DVDs.

So what's new in Toast 6? How about sharing your burner across a network? How about adding menus, buttons, and chapters to your DVDs and VCDs? How about iPhoto integration? Yep, all that and more.

I'm particularly interesting in creating "stunning movies with Hollywood-style pan and zoom effects" using nothing more than my still photos.

Love Mac OS X? Love working with video? Get a copy of Toast!

Roxio Press Release

TurboPower Abbrevia Goes Live
August 21, 2003

Venerable Delphi library retailers TurboPower bowed out of the market early this year, apparently tired of how tough it is to make a buck in that line of work. But fortunately for all of us, TurboPower announced that it was going to move all of those fine libraries into Open Source status as time allowed.

Well good news! The Abbrevia compression library now seems to be officially out in the wild. Version 3.05 is now up and running, ready for beta testing.

So if you're a Delphi or Kylix developer, and you want to work with Zip, CAB, or Tar files, you need to get in on this.

Abbrevia Home Page

August 21, 2003

So, you've got your giant mailbox compressed on your Linux system. That thing was getting pretty big, and bzip2 compresses the heck out of it. Why not?

But what happens when you need to find that e-mail offering you just the right source for mail-order Viagra? Well of course, it's a bit hard to find in that compressed mailbox.

At least I should say, it was hard to find, until you got a copy of Grepmail. Now you can grep that compressed folder as easily as if it were plaintext. And grepmail not only knows how to look through the compressed file, it also knows how to ignore headers, inspect content types, and all sorts of other nifty mail-oriented things.

And it's free!

Grepmail home page

August 21, 2003

Sure, I have to respect a product named PigZip. But it looks like this one has a long way to go. Listed as Alpha status on SourceForge.net, and really very little information to go on.

For example, digging through the docs couldn't tell me what sort of compression algorithm PigZip is using. The closest thing to real information I found was this:

The comprssion ratio is quite low now. Noo... it's not because it is a bad algoritm.. it's because it is not done yet
It's easy to pick on someone for spelling and grammar, but when you see that the owner of the project is one Lars Holmqvist, you have to stop and think, hey, maybe this guy's language of choice isn't English? Let's give him props for just being intelligible at all and not worry about having a cherry on top, okay?

SourceForge.net Project Page for PigZip

Prismiq Media Player Review
August 20, 2003

You've seen a lot of news here about various networked media players. ZDNet seems to have now found one that they're ready to pronounce as the king of all media. The Prismiq player handles a really nice variety of audio, video, and image formats, even going so far as to play back MPEG-4 video.

On top of it all, it currently supports 802.11b, but will be able to upgrade to 802.11g with a new firwmware load. (Although you'll need an 802.11g capable PCMCIA card to drive it.) Based on what I'm reading here, this guy could be pressed into service in Nelson labs quite soon.

ZDNet: Prismiq Media Player

Real's Helix Client
August 20, 2003

I've passed a long a few mentions of RealNetworks' Helix Client project. The goal is to create a highly capable media player for UNIX and Linux systems. Solaris too.

At least that's the description I was originally working on. The official page is on the net now (finally), and it has this description:

The Helix DNA Client is a universal playback engine supporting the decode and playback of any data type on any device.
So we probably still need some clarification on all that. In any case, it's here, it's real, and you can use the code or program in your totally non-commercial product for a zero-dollar royalty.

Helix Client home page

In-Memory Image Compression
August 20, 2003

This article by Amer Gerzic uses GDI+ to load an image, then compresses it into memory. In the sample code he loads a BMP file and converts it to an in-memory JPEG file.

As Amer himself points out, this article is not so much about compression as how to work with the GDI+ code. That's okay, it's still useful.

CodeProject article

Iomega Entering Backup Business
August 19, 2003

How long since you've felt the need to purchase a Zip drive, once Iomega's flagship product? Probably a long time. With CD-R media now dirt cheap, it doesn't seem logical to pay $10 for a cartridge that holds a lot less data.

Iomega sees the logic in this, and just recently axed a big percentage of their staff in a major concession to reality. But they're not out yet!

No, in an apparent attempt to prove that there are indeed third acts in American business life, Iomega is betting the farm on a new backup technology. They're apparently creating some sort of disk-like technology that can be used to cheaply backup major amounts of data in no time. The goal is to be way better than tape cartridges, but not too expensive.

Naturally, since this is backup media, we can expect that the controller for the device will be compressing data on the fly. Unfortunately, no specs on what compression format is being used, although Iomega is tossing out numbers like "2:1 compression ratio." Makes me think they may be using one of the standard LZS-like algorithms that tape-drive manufacturers are happy with.

Hardware Zone article with unreasonably long title

The Fortran interface to read gzipped files
August 19, 2003

Okay, I have to admit it's been a while since I last wrote a line of FORTRAN. I'm guessing 1977, on an IBM card punch, target a CDC mainframe.

But just because I gave it up doesn't mean there aren't scads of physics researchers and mechanical engineers who still depend on the dawn-era language to do their number crunching. And now those folks have access to a library that lets them read and write Gzip files.

This is all thanks to John Donners, who may or may not be a resident of the Netherlands. We are particularly thankful to John for making this code available free of charge to the last of the FORTRAN faithful.

libfgz home page

Keep an Eye on MPlayer
August 19, 2003

>From time to time I check in on the status of MPlayer, one of the more popular movie players for the Linux platform. A few days ago I noticed that the 0.91 release was shipping, so I went to the home site to see what was up.

Nothing too exciting with the 0.91 release, but some pretty interesting developments seem to be coming up quite soon. It looks like when Mplayer 1.0 ships, the team is going to attempt to be rid of the binary codecs currently used for some formats. They've already got H.264, Sorenson, Indeo3, and others converted.

Better yet, a Win32 player built around MinGW appears to be in the works. This will of course open up the player to the masses, for better or worse. Look for big things from this team soon!

Mplayer home page

Compressing the Entire Universe
August 18, 2003

I happened across this interesting article about some astronomers who have decided that the era of star formation in our universe is coming to an end. That's not particularly germane to the world of data compression, but what is is the fact that these guys collected enormous amounts of data from the Sloan Digital Sky Survey.

Enormous amounts of data? Sure enough, they had to develop a special type of data compression just to keep up with it all. The article describes it as something called MOPED.

Sadly, details about MOPED aren't easy to find on the net. Looks like the work is owned by the University of Edinburgh, and they'd love to license the technology to you, but they aren't going to give you too much in the way of details. I'll let you read the page to find out just how spiffy it is.

Space.com: Dim Future for the Universe as Stellar Lights Go Out
University of Edinburgh MOPED Licensing page

Nero 6 Review
August 18, 2003

You might not be inclined to think of Nero as a data compression program. Think again! The latest version of what used to be a simple CD burning program now has quite a few data compression options, including the ability to create MPEG-2 and MPEG-4 videos.

PC Magazine takes a good look at this new (albeit late) release of the latest version of this title. Overall they seem to be pretty impressed.

PCMag.com: Nero 6 Moves Ahead of the Disc-Creation Pack

CoffeeCup Free Zip Wizard
August 18, 2003

This is yet another free zip program, which includes the basic archiving functions as well as email, ftp, search, and a few other niceties. So how do the CoffeCup Software people pay for the development of this program? I'm not sure - it could be ad-ware. Download a copy and tell me, okay?

CoffeCup Software product page

BusinessWeek On Fracturing Zip Standard
August 17, 2003

You're well aware that PKWare and WinZip have released new versions of their archivers that create non-standard Zip archives.

It appears that the news has now leaked out of the geek-o-sphere and into the real world. BusinessWeek has a pretty complete article on the whole mess, so the word is out. (Yes, this is a feed from CNET.com, but still, here it is in a business publication.)

BusinessWeek: Is Zip coming undone?

August 17, 2003

This is a Win32 program that manages SoundFont files. These files are the ones used by Creative Labs to program their Midi cards. I think.

The program acts much like you would expect a conventional archiver to behave - giving you an interface similar to a retro version of WinZip. Big difference is that the contents are those crazy SF files.

SFZip home page

On2 Gets Personal
August 16, 2003

The good folks at On2 Technologies have decided to do you a favor. Now that their ultra-powerful VP6 coder is on the market, they've decided they don't need to spend much time working on sales and marketing for their older VP4 codec.

But instead of parking it in in some dark corner of their warehouse, like most companies would do, On2 is offering this codec free of charge for personal use. I like that.

EETimes: On2 Technologies Releases VP4 Personal Codec

Update on Rio Karma
August 16, 2003

We mentioned the Rio Karma portable music player a few days ago. News about this player leaked out on the net in a few places, but now we finally have an official announcement from Rio about the product. (No, you can't buy one just yet.)

It looks to me as though this is going to be a pretty sweet little player. It has a nice-sized 20G hard driver, and can play MP3, OGG, and WMA format music. For what it's worth, I suspect that WMA format will give you about the same performance as Apple's new AAC format at the same bit rates. (No, I can't prove that.)

This player has a few other nice features, including a nifty docking station that has both an ethernet port and RCA jacks to connect to your stereo. I think the Karma could be a winner in this product category.

Rio Karma product page

August 15, 2003

Ziproxy is a very cool proxy server that takes ordinary web pages and squeezes them down to minimal size before serving them up over what we assume are low-speed connections.

How does Ziproxy work this magic? Two ways. First, it compresses all the text using gzip encoding, which is supported by virtually every browser worth its salt these days. Second, it takes images on that Web page and converts them to low-quality JPEGs. This won't give you great images, but it will save a ton of space.

So this is the perfect proxy to use when serving up pages to users with low-speed dialup connections. I only wish there was a demo page so I could see it in action!

Ziproxy project page

JPEG 2000 Dropper
August 15, 2003

Are you ready to start converting all the images on your system to JPEG 2000? You might save a lot of space in your photo album or your Web page. If you've got a couple of gigabytes of J-Lo photos, this is a way to avoid upgrading to a new hard drive.

The folks at Monkeybread Software appreciate your problem, and they want to help you make the conversion quickly and easily. Accordingly, they've created an OS X program that lets you drag and drop your image files, converting them to the new format in a jiffy.

Best of all, this is freeware written with a German UI. Can't beat the combination of these two features!

JPEG 2000 Dropper Home Page

August 15, 2003

Truxton King Fulton is a programmer who follows The One True Path: write good programs, then give them away. You can find much of Truxton's good works here.

One thing in particular we're interested in is Truxton's motion program. It takes a couple of JPEG images and compares them to detect motion between the frames. Why would you care about this? For the simple reason that detecting motion is the core of video compression algorithms such as our beloved MPEG-2.

motion.tar.gz on Truxton's home page

SCO/IBM Tangle Over Data Compression

August 14, 2003

Everyone in our business knows by now about the giant lawsuit SCO filed against IBM just recently. SCO claims that IBM inserted copyrighted UNIX code into Linux, violating SCO's intellectual property rights.

Naturally, IBM fired back with a lawsuit of their own, charging that SCO is infringing on some of IBM's patents. Listed among the patents in question is one titled "Data Compression Method."

Unfortunately, I haven't been able to track down the details on this patent just yet, as the press releases only show the name. But SCO ought to be a little bit worried about this. IBM has a patent portfolio that can be used to crush smaller competitors, and they know how to use it. It will be a lot easier to back out any infringing code from the Linux kernel than it will be to scrub the entire SCO product line free of IBM patented software.

PCWorld.com: IBM, SCO Go to Court Over Linux

PentaZip 6 Review
August 14, 2003

PentaZip 6.0 from PentaWare is a full-featured archiving program that has the goods to be found in the same league as WinZip and PKZip, the two segment leaders.

The 6.0 release comes with a big batch of goodies, including crazy things such as the ability to save to a CD-burner and to display a slideshow from your image files.

Is it all worth a buy? Well, the folks at ZDNet say that you'll probably get better service and support from PKZip. What's interesting is that they start the review with that statement, then say absolutely nothing about service and support anywhere in the review! If I were working for PentaWare I think I'd be on the phone to Mr. Ziff or Ms. Davis ASAP to discuss this little oversight.

ZDNet: PentaWare PentaZip 6: Comprehensive compression

Vibrant Communicator ActiveX Library
August 14, 2003

Want to stream low-latency compressed audio through your application? Well, you are welcome to write directly to Microsoft's APIs, but that can often be somewhat of a chore.

Fortunately, the folks at Lake of Soft have created an ActiveX library that manages the chore for you. They don't do any compression themselves; instead they pass the encoding and decoding tasks to Microsoft's Audio Compression Manager. No DirectX technology is used.

This is a commercial library, but a demo version is yours for the asking.

Lake Of Soft Product Page

Compress My Music, Please
August 13, 2003

A poster to comp.compression named Csaba T posted an inquiry recently, asking for help in getting better compression of his audio files:

I've had this on my mind for a while... What I am looking for is maximum compression possible of audio files, without the ability of playing back the file. In other words, I want to get as small a compressed version of an audio file as possible. If uncompression takes some time and resources, that's ok. The compressed file would not have to be playable, just heavily compressed.
What do you suppose the denizens of comp.compression had to say about this? A very minor quibble over philosophies, but in general a recommendation to follow current standards.

comp.compression thread courtesy of Google

Windows Media Monopoly Endangered
August 13, 2003

We all know that Microsoft would really love to own the keys to the kingdom when it comes to audio and video formats. They've created some pretty spiffy codecs that compete well with the best open standards, and are working on locking their desktop users into those formats.

One way to achieve monopoly in this area would be to force users to use Windows Media Player for all their audio and video needs. And while Microsoft can't really do that, they can do the next best thing, which is to make Media Player an integral part of the system that ships with every version of Windows.

This strategy worked pretty well for Internet Explorer, and apparently European antitrust regulators feel that it might work in the media world as well. As a result, they propose to slap down this type of behavior before it has a chance to take hold. Read the story for the sordid details.

Cnet.com: Microsoft media strategy at risk?

Compression Worker Gone Bad?
August 13, 2003

It's nice to look around our quiet little community of data compression programmers and suppose that we're all a bunch of nice, friendly types. For the most part, everyone is polite unless we start arguing about the value of bijective techniques in compression, or perhaps get sucked into an recursive compression thread.

If you dig below the surface, however, you might find that things aren't always so pleasant. In a recent conclusion to a well-known case, Maher "Mike" Mofeid Hawash pled guilty to conspiring to aid the Taliban in its quest to destroy much of western society.

It turns out that not only was Mike a former employee of Intel, but he was a lead engineer on the Intel's Multimedia Extensions (MMX) software team. Part of his job at Intel apparently involved working on MPEG decoders, and he co-authored an Addison-Wesley book that covered some of this ground. Now it looks like his skills will be eroding as he serves a sentence of up to ten years in federal prison. Bummer.

ZDNet.com: Ex-Intel worker guilty of aiding terror

Reed-Solomon Coding
August 12, 2003

Reed-Solomon coding is not specifically a data compression topic, but data compression programmers often want to add forward error correction codes to their programs. After all, if you are taking the trouble to archive stuff to CD-ROM for backup purposes, there's nothing wrong with adding a little protection in case of media errors.

This was apparenty the goal of a fellow named Matthew who posted just such a question to comp.compression last week. He got a couple of good answers, both short, to the point, and containing links to stuff that proved very useful.

comp.compression thread courtesy of Google

More MMX Tips
August 12, 2003

I ran a pointer a few weeks ago to a CodeProject article that gave some tips on using Intel's MMX instructions from inside your C++ code.

Well, the good folks at my favorite programming site have just posted another article on both MMX acceleration and other optimization techniques. All of this stuff is of course near and dear to the hearts of programmers working on data compression projects, so I encourage you to check it out!

CodeProject: High performance computing from C++ to MMX

AVI/MPG ScreenSaver
August 12, 2003

Well, here's a pretty nifty Win32 shareware program. It lets you turn any video into a screensaver - which means you can have bootleg sequences from The Matrix instead of that lame Flying Windows stuff your cube-mates are dishing out.

The only thing that would improve the program would be if it were free. As is, the $17.95 price isn't too bad, but I bet it's hard to get people to cough up that much for a novelty.

AVI/MPG Screensaver

Real Ups Ante on Helix Project
August 11, 2003

The great folks at RealNetworks set up the Helix project last year, dedicated to moving major chunks of their code into Open Source status.

Up until now, most of the effort in this project seems to have been concentrating on server side components. Well, that's about to change. Real has announced the start of the Helix Player initiative.

The goal of this project is to create a nice media client for *IX platforms, including Linux, Unix, and Solaris. That sounds great, although there is one annoying catch: it will require the use of proprietary binaries from Real.

Real Press Release

Avi-Ogm Info
August 11, 2003

When cataloging and indexing all your Win32 AVI video files, it's nice to be able to quickly look at one and get the key stats. What codecs it uses for audio and video, what the sampling rates are, what the frame rate is, etc.

The good news is that you can now get all this info from the aptly named Avi-Ogm Info program. It returns info on both AVI and OGG files, and does it using either a neatly formatted GUI or old-school console output.

Did I mention that the program's UI seems to be entirely written in French? No problem, I think it's still completely usable for speakers of any Western European language, including the most western language of that family, American.

AVI-OGM Info home page

August 11, 2003

If you like Avi-Ogm Info, I think you're going to like MediaInfo even better. This free Win32 program gives you complete info on the attributes of scads of media files, including MP3, MPEG2, OGG, WAV, AVI, and more.

Like the previously mentioned French cousin, it has both GUI and console modes of operation, and it appears to have both French and English localizations as well.

MediaInfo home page

Rio Karma Unveiled
August 10, 2003

It looks like Rio is going to be producing a pretty sweet looking 40G MP3 player. The original rumor on this piece had it shipping in October.

Gizmodo: Photos of the Rio Karma
Gizmodo: The Rio Karma vs. the iPod

Apple Rumor of the Day
August 10, 2003

As long as the news today is concentrating on phantom products, I might as well pass along a pointer to the latest rumors on future iPod products. The folks at MP3newswire.net are trying to convince me that Apple is going to be creating a video iPod sometime soon.

It's a nicely fleshed-out rumor, and perhaps the best thing about is the notion that Apple will be selling content via the iTunes channel in order to get people watching stuff on the player. After all, without content, this thing would not be much good.

Still, Apple fans seem to love nothing more than spreading rumors about upcoming products. Frequently the rumors are based on nothing more than some fan's notion of what seems logical. So take all this with a giant semi-load of salt.

MP3newswire.net: Rumor: The iPod Movie Player

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.