April 29, 2003
Dr. Dobb's Data Compression NewsletterMark Nelson
The most recent compression techniques, algorithms, products, tools, and utilities. Edited by renown data compression expert Mark Nelson (author of "The Data Compression Book").
Welcome to this issue of Dr. Dobb's Data Compression Newsletter! This newsletter keeps you up to date with news items, product information, and the occasional editorial aside about the world of Data Compression. Your input is always welcome in the form of comments, submissions, or original material.
Lossless Audio Compression Tutorial
I don't know too much about Monkey's Audio, a free, lossless audio compressor currently supported on Win32. I do know that my DataCompression.info site gets a lot of hits from its link on the Monkey's Audio site, so I guess that means it's actively visited by folks with an interest in compression.
While scanning comp.compression a few days ago, I saw a query from a user regarding the storage of delta-encoded audio data. Rainer Nausedat posted a link to a nice tutorial on the Monkey's Audio site that I hadn't seen before.
This document describes a lossless audio compressor that does some straightforward prediction in combination with a Rice coder. The tutorial is credited to Matt Ashland, author of the Monkey's Audio codec.
Since Monkey's Audio is free and has source available, maybe starting with this paper and finishing with the source code would be a good way to get yourself up to speed on an adaptive delta compression scheme for stereo audio.
BitMagic Compression Articles
BitMagic is a C++ library that deals with dynamic bit vectors. There are a few pages on their Web site devoted to the various types of compression they use to manage these vectors in memory.
Media Wizard 8.0: Yet Another Media Convertor
Description from the Web site: Media Wizard is a powerful, all-in-one multimedia application supporting playback of all popular formats, audio and video conversions (CD to WAV; CD to MP3; CD to WMA; CD to OGG, WAV to MP3; MP3 to WAV; WAV/MP3 to WMA; WMA to WAV/MP3; WAV/MP3 to OGG; OGG to WAV/MP3; WAV/MP3 to CD; AVI to MPG; MPG to AVI; etc.)
It must be getting to be pretty easy to write a program like this - it seems as though I'm getting several new links every week for Windows programs that can convert from Column A to Column B for a list of common media formats.
I don't know if Media Wizard can do anything to stand out from the field, but it does have more than the average amount of documentation and support on their Web site, so you can at least make an informed decision before laying out the $50 they are asking for it.
Quick Review for Miliki Super Compressor from QuikCAT
The folks at QuikCat Technologies recently sent me a review copy of their Miliki archiving program. I had seen their claims that Miliki's content-specific compression allows it to destroy the competition in head-to-head comparisons, so naturally I wanted to put this to the test. Lines such as 5 to 10 times more than popular Zip compression software are provocative.
I put Miliki up against WinZip, which provides nominal compression, and WinRAR, which uses a few tricks to provide something closer to state-of-the-art compression.
I first tested the three compressors with the Canterbury Corpus, (a total of 2.67 MB), and again with a few files from the Large Canterbury Corpus (a total of 10.6MB.) The archive size for the three compressors was:
Not particulary impressive, but not surprising. These corpi don't have any files that Miliki targets for special processing. I then decided to compress 11 MB of uncompressed image files from the Waterloo BragZone. I converted the TIFF files to BMP, an image format the Miliki claims to understand. The first cut on this test yielded good results, with Miliki showing much better compression than the competition. I then ran the Settings program for Miliki, and told it to reduce image quality down to 1%, the lowest possible setting. The 100% and 1% quality results are shown below.
That's a lot more impressive! However, it must be pointed out that at this point, Miliki has parted ways with the other two archivers by using a lossy algorithm for the images. Even when compressed at the 100% settings, my tests show that the format is lossy, although not detectable with just a glance.
JPEG files got good results from Miliki as well, in spite of the fact that they are already compressed.
Finally, I wanted to see how Miliki did for lossless compression with file types that it knew something about. It advertises special compressors for Microsoft Word, Excel, and PDF documents. Here are some results on sample files:
Given these results, here is the nutshell conclusion:
ViXS Gets In Bed With Toshiba
Toronto-based ViXS was able to get their video bit-rate controller into a Toshiba reference design, which hopefully will lead to inclusion in real products. This chip is able to receive up to eight MPEG bit streams (MPEG-1, -2, or -4) and selectively reduce their bit rates to adjust to available downstream bandwidth.
According to the article in EE Times, the bit-rate reduction is done in a way such that the streams remain MPEG-compliant, so no special decoding hardware is needed by remote clients.
EE Times adds some nice detail to the story, discussing the processes ViXS actually went through when designing and implementing the chip.
Audio File Format FAQ
This FAQ provides good information about various audio file formats, which naturally includes quite a few compressed formats. It also has a few links to other relevant pages.
The file hasn't been updated since 1998, so the info can't be completely up to date. If you don't find it here, you can always try the Wotsit's Format site, which is very complete, but generally only has pointers to other resources.
Windows Media 9 Authoring Utilities
Microsoft knows that if Windows Media 9 is to rule the world, it must have authoring tools that are cheap or free, and of high quality. The tools on this web page are all free downloads, but I'll let you be the judge as to whether they are high quality.
The list of authoring tools includes the following:
New Submission: Fractal Image Compression for Spaceborne Transputers
Keith Howell dropped me a line with a link to his M.Sc. dissertation from way back in 1993. This paper evaluates the suitability of Fractal Compression for spacecraft images. The project includes complete source code for Borland C, which is ought to be easy to port to other C compilers.
Fractal compression doesn't seem to have the cachet that it did back in 1993, which I think is mostly a result of the superior performance of wavelet-based lossy methods. Keith's paper reminds me of just how much promise this field once held.
Update: ImageMagick 5.5.6
ImageMagick is a nice package of tools for manipulating and converting image files. The code is essentially free, and you can access the tools from the command line, or by calling library functions.
Version 5.5.6 comes very close on the heels of 5.5.5. It doesn't look like there are any exciting new features that have been added, but I guess that's par for a point-point release. The ChangeLog lists a couple of bug fixes and minor changes.
Meet Michael J. Gormish, JPEG 2000 Maven
Working for a big company in a research capacity has always sounded to me like the ideal way to make a living. Michael J. Gormish seems to have achieved this state of nirvana with Ricoh. Ricoh obviously has a big stake in the world of imaging, and Gormish appears to make sure their interests are represented in the JPEG 2000 world. Michael clearly spends a lot of time doing real work at Ricoh, as he has managed to rack up a couple of dozen US patents, along with publishing a nice list of papers. But he also seems to have a role as a cheerleader/advocate for the JPEG 2000 standard, which probably means making presentations, serving as a reviewer, and no doubt plowing through a lot of email.
If this kind of stuff sounds exciting to you, take a few weeks to plow through all of Michael's online papers, and perhaps his dissertation as well. Having done that, you might want to drop him an email and ask him to update his home page - it looks like he is about a year behind!
Update: ReaConverter Pro 3.0
ReaConverter Pro is a batch image converter, which might not be that exciting. However, with this new release they advertise their product as supporting over 340 different image formats, and that has to get you just a little bit worked up.
Naturally, in addition to simply converting from one format to another, ReaConverter does the usual: sharpen, smooth, blur, add watermarks and borders, etc. And you can download a demo version of the product to give it a try before purchasing.
On the plus side, ReaSoft gives you a lifetime license to this product. Once you buy it, you will get all updates for free. On the downside, ReaSoft is one of those companies that doesn't make it easy to see what new features have been added in the 3.0 update. Hey guys, it's not really too tough to post RELEASE_NOTES.TXT on the product's web page. Think about it!
Addition to Datacompression.info: LuraDocument
LuraDocument is a professional-quality (read: expensive) set of libraries and programs for management of document storage. LuraTech has the following description on their Web site: LuraDocument represents an innovative compression software custom-tailored to the compression of scanned color documents composed of both images and text. The documents are partitioned into pure picture and pure text segments through the use of its state-of-the-art segmentation algorithm. Each of these segments is then compressed using an algorithm optimized for this type of data. As a result, scanned color documents can be stored at extremely high quality despite small file sizes. LuraDocument is capable of reaching compression rates of 1:150 to 1:500 while still preserving full readability of the text and excellent quality of the pictures. This makes LuraDocument far superior to even the widespread PDF format.
DRM For Dummies
Digital Rights Management (DRM) doesn't really have a direct connection with Data Compression, but there's no doubt that the future of digital media rests on the success of DRM.
In a nutshell, DRM is generally used to protect content owners from unauthorized copying of their material. Interscope Records is willing to license Eminem's latest CD to Listen.com only because they trust the DRM built into the Rhapsody client. Lack of DRM in MP3 files leads to Napster-like situations, where content can be copied by anyone at any time.
Ths PC Magazine article touches on the basics of DRM without going into too much depth. If your goal is to sound lucid at the next cocktail party when talking about copy protection in streaming media, just commit this article to memory and you'll be fine.
Microsoft Beta: Windows Media Video 9 Video Compression Manager
Microsoft would really like you to package Media Series 9 content in the new ASF file container, but realizes that you may have older apps tied to the AVI file format. The Video Compression Manager (VCM) gives you a bridge that allows you to use the new codecs in the old container.
Note that this package is needed for encoding and editing apps, not playback applications. This tool has been released in a beta format, and is listed as Not Supported. Kind of hard to figure out what that means in this context.
How File Compression Works
The How Stuff Works Web site tackles the question of just how Data Compression works, pitched to the general audience. All in all not a bad stab at it, with links to many other How Stuff Works pages for the curious. Just remember that this is written for the curious civilian, not you.
DivX-Certified Networked DVD Player from Kiss
Kiss Technology has demonstrated the first DivX-certified DVD player at the giant CeBit trade show. This is pretty exciting, because it means you can store MPEG-4 movies on CD, then watch them from this player. And better still, it has an Ethernet port, so it can play those same MPEG-4 files right off your hard drive!
Added to datacompression.info: MoviX
Directly from the MoviX Web site: The MoviX project is a series of three different tiny Linux CD distributions containing all the software to boot from a CD and play multimedia files through MPlayer, the best multimedia player in the Unix world:eMoviX is at release 0.8.0rc1, MoviX is at 0.8.0pre2, and MoviX2 is at pre0.3.0pre2 as of April, 2003.
Perl and JPEG 2000 - A Match Made In Heaven
SouthDowns is a set of Perl scripts for the creation of JPEG 2000 files from JPEG 2000 codestreams. If you're interested in the format used by the standard, this might be an easy way to experiment with it. The sample program appears to insert some metadata into the image file.
AOL Jilts Real
According to this CNET news article, AOL is going to replace RealNetworks with Dolby AAC as their default streaming format, for both narrowband and broadband users. No word on the software packages that will be used, but the article seems to imply that the software will be coming from Dolby, who I haven't previously seen as a supplier of codecs or server software.
This has got to be a bitter setback for Real, who has seen the merciless Microsoft machine slowly chip away at their market share. Real is still a force to contend with, but there was a time when they enjoyed kingpin status, and I'm afraid now they are reduced to just-another-player.
DjVuLibre - Document Distribution Champ
The DjVu standard for document encoding was once an ATT research project, but now has been commercialized by LizardTech. Although LizardTech owns much of the DjVu technology, they seem to understand that popularization of DjVu requires free, multi-platform support.
The folks at the DjVuLibre project aim to provide that free support, and have created a pretty comprensive set of tools for using this format. When it comes to document storage, DjVu claims advantages over all competitors. From the DjVuLibre site: DjVu documents download and display extremely quickly, and look exactly the same on all platforms with no compatibility problems due to fonts, colors, etc. DjVu can be seen as a superior alternative to PDF and PostScript for digital documents, to TIFF (and PDF) for scanned bitonal documents, to JPEG and JPEG 2000 for photographs and pictures, and to GIF for large palettized images. DjVu is the only Web format that is practical for distributing high-resolution scanned documents in color. No other format comes close.I'm not about to argue with that!
Added to datacompression.info: AVPhone ActiveX Controls
These ActiveX controls provide real-time compression and rendering of audio and video streams, supporting standard formats such as H.263, MPEG-4, G.723 and more. It appears that they are designed for IP Telephony, but they also advertise the ability to do straight-up compression to/from disk or memory. You can use a demo version for free, with limitations, but upgrading to the pro release is going to set you back a bit. Redistribution is included in the price, though.
Meet Rieni Otten, Goodwill Ambassador From the Netherlands
Last week I was checking my referral logs and I noticed that I was getting some links to DataCompression.info from a Yahoo! discussion group titled JPEG 2000. I thought that would make a good addition to the database, but when I checked up on it at Yahoo! Groups, I saw that membership was listed as being restricted. I didn't want to list the discussion group on my site if membership was off-limits to the general public, so I sent a message to the list owner inquiring as to entry requirements.
The response I got was somewhat remarkable, and I thought it was worth sharing with you: I really don't know how to deal with your stupidity. If there isn't any restriction, people keep complaining about spam, now there is some restriction to avoid spam, I get people asking why there is restriction. Did you EVER consider just trying to join the forum and see what happens? Or are you afraid that the FBI will show up at your door to search your house?Now, I could understand this reaction if Rieni was getting bombarded with thousands of requests each day, which might be the case if he was operating a discussion group dedicated to photos of Anna Kournikova. But really, this is JPEG 2000 we're talking about. How many emails like this does Reini have to answer? (Incidentally, the IEEE remark is in reference to my email address, which is forwarded through the IEEE servers. Other than membership, I have no connection with the organization, although I may apply for that coffee boy position if it opens up.)
Well, we in the US are already on the outs with Germany and France, and apparently I've managed to botch things with the Netherlands as well! Time to put a Union Jack bumper sticker on my car and do my best to avoid alienating our stalwart mates in the UK next. And for Reini, I have to recommend the same course of action that Hall 9000 once gave Dave Bowman: I honestly think you ought to calm down; take a stress pill and think things over.
FollowUp - Premium ISP Services
A couple of days ago I ran a story about how ISPs such as Earthlink were offering a premium service that used data compression to offer faster access to the net over dialup lines. Drew Scott Daniels provided me with a link to a nice explanation of how this works.
This link to the Propel Web site explains how they manage to accelerate web connections using a combination of various techniques. I don't know if Propel has inked any agreements with ISPs, but you can sign up for their service without the assistance of your local provider.
Update: Vlorb 1.3
Vlorb is a high-quality Audio CD to Ogg Vorbis encoder. It does this by operating as a console front end to cdparanoia and CDDB on Linux systems. The first release of Vlorb wasn't too long ago, but this group is cranking out updates at quite a clip.
FullAudio launches MusicNow music service
MusicNow is yet another online music service that charges $9.95/month and has deals with all five major labels. They appear to be using Windows Media Player DRM to protect their music, which has the benefit of allowing users to download music directly to their PCs. Navigate to the site and take advantage of the 7-day free preview. Tell me if there is anything to differentiate this service from the other half-dozen that are pitching their wares right now.
Update: PicoZip 2.8
PicoZip is a nice commercial archiver, seems to get good user reviews, has a great feature set, and is being actively supported and improved. PicoZip 2.8 was released at the end of March, 2003 with bug fixes, plus support for Zip files greater than 4GB, and support for extraction from multi-volume Zip files. Worth a look for the 30 day trial!
My favorite PicoZip feature is the free upgrade to all future versions of PicoZip. This type of dedication to the customer base is pretty rare, and it is worth taking advantage of.
Two MPEG-2 Tutorials
I've added pointers to a couple of new tutorial papers on MPEG-2 to DataCompression.info. Tutorial papers aren't something we have a shortage of, but these two articles go into a nice level of detail, include some good diagrams, and in general are a nice job.
MPEG-2 Video Compression by P.N. Tudor
MPEG : Standards, Technology and Applications by Shanawaz Basith
Windows Media 9 Going For Movie Dollars
You've heard me say it over and over: Microsoft is determined to have Windows Media 9 rule the world. This press release from the Redmond behemoth announces that they've managed to finagle a deal with a major chain to equip their theaters with Media 9 cinema playback systems.
Interesting news for the movie industry, because digital distribution can save a ton of money. I've read estimates in the past that indicate that major releases could save literally millions of dollars by ditching filmstock. I guess making all those prints of LOTR:T2T gets pretty expensive. Not to mention delivery, etc.
Theater chains are the first step, but we still have to wait for the second shoe to drop. That will be the announcement that Microsoft has signed agreements with one or more major studios to distribute their films digitally. I expect announcements of this type within 30 days, followed by a spate of hype when we hear about a major movie that is produced without ever committing a pixel to film.
You might want to take a peek and see what the folks at Slashdot think of all this. Even more interesting will be what the film purists have to say of all-digital movies. I predict they'll miss the grain!
Initial Release: Jarchiver 1.0
This is an open source tool that is described as a Java based archiving tool that is ideal for housekeeping tasks such as backups, pruning and more. Part of the archiving task is the ability to create zip archives, which explains its presence here.
This is a brand new product; let me know if it fills a need for you.
Update: Real GIF Optimizer 3.05
A GIF optimizer seeks to reduce the size of your GIF files, usually without compromising image quality. Not enough detail on this site to pass judgment, but good GIF optimizers will also let you reduce the number of colors in the palette dynamically, so you can see the effect.
Real GIF Optimizer 3.05 shipped in February of 2003. I wish I could tell you what new features are in this release, but RealMultiMedia Development is one of those companies that doesn't bother to post release notes on their Web site!
RealMultiMedia Development Web site
Enhanced Dial-up: Sweet Spot or Dead End?
Earthlink and other ISPs are attempting to use data compression to create a new class of service: "enhanced dial-up." By paying a few extra bucks per month, customers get faster access to their web pages by the use of server-side compression. Earthlink is offering their product for an extra $7 per month, and touts it as up to five times faster than standard dial-up.
In theory this is supposed to be for the customer who is not willing or able to take the big jump to broadband, but still wants improved perfomance. From this vantage point, it looks like a hard sell - I'm not sure customers are going to be convinced that they are getting a big enhancement beyond their normal dial-up service. We'll see if Earthlink is still offering this product a year from now.
New Release: OpenShiiva 0.8.2
OpenShiiva does just one thing: converts VOB files to MPEG-4 format under MacOS X. VOB files are wrapped MPEG-2 format streams, such as you might find on DVDs. You can expect a nice decrease in size when going from VOB to MPEG-4, so this is ideal for archiving DVD files to your hard drive.
IE6: Data is Truncated When You Download a gzip-encoded Excel
Most Web browsers are happy to accept gzip-encoded data. Using gzip to encode data on your web server is a good thing, as it saves both hard disk space on the server side and download time for the client.
Unfortunately, IE6 seems to have a bit of trouble with gzip-encoded Excel spreadsheets. This Knowledge Base article describes the problem, and happily presents an easy workaround.
Update: SoX - Sound Exchange 12.17.4
SoX aims to be the Swiss Army Knife of sound processing programs. It shows up on the data compression radar by virtue of the fact that SoX is able to convert between a wide variety of compressed formats, ranging from MP3 and Ogg to ADPCM and linear PCM. SoX has ports to a really nice variety of platforms, including Win32, Linux, NetBSD, and more.
While not really a compression feature, SoX also sports a great roster of effects you can apply to audio files, including various filters, panning, reverb, looping, etc. If someone slapped a GUI on this project, it might rule the world. As it is, the command line interface means you have to pass an entrance exam to use it, but the bar is not too high.
Update: AVI File Support Library 0.7.34
This open source project gives you the ability to read and write AVI files under Linux. The really interesting part about all that is that this is done using the Win32 DLLs from Microsoft to do the compression and decompression. Clever!
Knowledge Base: Hardware-Created ASF Files Stop Playing in Media Player 9 Series
Apparently, certain media files created by certain hardware will cause Windows Media Player 9 to generate the following message: 0xC00D1199 -- Windows Media Player cannot play the file. The file is either corrupt or the Player does not support the format you are trying to play.Microsoft knows how to fix this problem, but they don't seem to be willing to place the fix on the web. You have to contact the support team and make it past the screener before you get the update.
HoloDynamic Compression: Mapping Miracles into the Machine
A great example of an incredible compression claim. There's just a teeny-tiny bit good thinking in here, and a huge amount of tangential speculation attempting to walk upright on two legs. Read it and just be glad the author doesn't seem to be raising funds.
James Redfield's site: The Celestine Prophecy Home Page
New Watermarking Technique
SunnComm Technologies has announced a new technique for embedding data in digital media files. Like all good watermarking techniques, this data can in theory be embedded in the media source, and will survive through compression cycles, yet remain undetectable by human eyes and ears.
Although this technique could be used for metadata purposes, e.g. liner notes, lyric sheets, etc., the company is presently concentrating on copy protection as a profitable application.
New Library: Compress::PPMd
Admit it, you never thought you would use the terms Perl and PPM compression in the same sentence. Wrong you were, for Salvador Fandiño García has created a Perl interface to Dmitry Shkarin's PPMd compression library. PPMD is still available, but I don't know what has become of Dmitry - his home page has disappeared.
Update: Xine Beta 9 Xine is a versatile media player that seems to be mostly oriented towards Linux. It comes with a bewildering array of add-ons and packages, and guarantees frustration for novice users by not supplying binaries. Nonetheless, it looks pretty capable, and will probably do a nice job playing your CDs, DVDs, MP3 files, and most other multimedia formats. With Beta 9 out, I'm not sure what's left for Xine. The biggest step would undoubtedly be Win32 compatibility, but I'm not sure if if that's where their heads are right now.
Xerox Invents New Print Compression Technology
Xerox has invented something called Xerox Multimode Compression (XM2). They've inserted XM2 into their digital color presses, and are ready to add it to their monochrome presses. Somehow this system compresses pages as they are laid out so they can be delivered to the printer more quickly. I wish I could tell you more about it, but this article doesn't really give too many hints. As a matter of due diligence, I checked the Xerox site and found their original press release, but it didn't offer too much more info. So we'll have to take their claim of using 1/3 of the normal bandwidth at face value, and wait for more details later.
English Text Frequencies This page has some data on frequencies of words and letters in English text. Besides the useful data, the site has some rather odd warnings, structure, etc. I can't really tell what the page is all about; perhaps it is devoted to PIC Microcontroller programming. In any case, the author of the site seems to be quite worried that people are going to pull data off the site for nefarious purposes. I'm still not sure whether we are going to offend him by posting this link. Guess I'll wait to see if I merit a nasty email, or perhaps a restraining order.
zlib and libpng for Windows CE Kazuo Adachi ported both of these important packages to Windows CE and Windows CE .NET. This includes binaries for what I believe are all the currently supported CPUs. Both of these libraries are up-to-date ports of the current versions of the libraries, zlib 1.1.4 and libpng 1.2.3. We definitely need to give Kazuoa a big domo arigato from the bottoms of our hearts. At least from those of us who are doing CE development.
Beta release: XViD Quantizer Parser (XvidQP)
The XVidQP project has released Beta3 of their product. XvidQP is a tool that is designed to tell you something about the quality of an MPEG-4 stream. It does this by giving you a batch of reports, including:
You can get additional information at the Source Forge project page.
Guide to Capturing, Cleaning, and Compressing
Video Matt Woodward has written an article for Ars Technica that gives a good rundown on capturing and compressing video. There's a great deal of detail here, which you should be able to use to walk through the process, even if you are a rank beginner.
File Compression: New Tool for Life Detection?
In a startling act of hubris, we in the compression world can now claim to be arbiters of what constitutes life itself. This article in Astrobiology Magazine discusses the quest of some researchers to use complexity as a sign of life. A good way to get a rough estimate of complexity is to use gzip on your data set. Actually, the researchers here are viewing complexity as an inverse sign of life, figuring that noise is a result of random processes, whereas living organisms impose structure on their works. They plan on looking at rock structures called stromatolites, gzipping the 3D shapes, and then hoping that the highest compression ratios point towards biological processes. If all goes well, this analysis might be used to direct a future Mars roving robot to concentrate on samples that offer some hope of excitement. Jean-loup must be so proud!
JPEG 2000 Discussion Group on Yahoo!
Yahoo! has a discussion group that is devoted to the technical aspects of JPEG 2000. Newcomers are kept on a short leash for a time in an attempt to avoid some of the downside of standard USENET newsgroups. In addition to this group, there are some related groups with similar charters:
JPEG 2000 Part 4 Conformance Test Files
As long as we're on the topic of JPEG 2000, it seems a good time to look into Part 4 of the standard, which relates to conformance. The files on this site are used in conformance testing. This Web site implies that this part of the standard will soon be available for free, but as of March, 2003, this is not the case.
Meet David Taubman David Taubman is on the faculty of the University of New South Wales, in Sydney, Australia. Taubman is the author of the book JPEG 2000: Image Compression Fundamentals, Standards and Practice and has been very influential in the current development of the standard. He is also the author of Kakadu, a commercial library that is advertised as a complete implementation of Part 1 of the standard.
New product - OGG-S OGG-S is an open source project that aims to create a Digital Rights interface for all media files, but particularly for Ogg Vorbis. A good implementation might be real helpful for Ogg. Commercial music services such as Rhapsody or PressPlay aren't going to let you download music to your portable player unless they have a guarantee of copy protection. These guys aim to offer just that. This is a new project which has just moved into the beta phase. They are probably still looking for help, so check out this page and the SourceForge page to see if you can help. Migrator 2000 March 30, 2003 This group says that they will develop, demonstrate and disseminate tools for the JPEG 2000 digital imaging standard. At this time that seems to include a couple of demonstration programs, some text and web pages, test images, and a link to a Perl package.
Chaotic Compression? Kevin Short at the University of New Hampshire has created a company called Chaoticom that is raising money to develop his new idea called Chaotic Compression Technology. This sounds suspiciously like a Magic Function Theory. Yes, random function generators and other chaotic sources can indeed produce complex outputs, but it's a real trick to get them to produce the complex output you need! Maybe Kevin Short is on to something, but a quick peek at the company Web site doesn't seem to indicate that any breakthroughs have dropped over the transom yet. So for the short term at least, this gets categorized under the Incredible Claims topic. It would be pretty exciting to see it manage to move into the real world, but don't hold your breath.
BrowseBlast Web Accelerator I received an email from PC Magazine last week with a pointer to an announcement from Ikano and SlipStream launching something called BrowseBlast: Ikano and SlipStream launched the BrowseBlast Internet service for U.S. consumers. SlipStream's Web Accelerator is supposed to increase connection speed significantly for dial-up and wireless connections. The software compresses Web content once it leaves the server, increasing browsing speeds to five times the traditional dial-up or wireless speed Seems like it might work. Their technical explanation says that they use state-of-the-art lossless compression on the server side to speed up your delivery. In addition, they apparently let you select a level of quality for lossy compression of graphics. No doubt this could work well in theory, but the devil is in the details. Any users out there who would care to give a report?
HTTP Compression There is a happy little industry out there consisting of products that offer to compress your web pages in one way or another. These products have to contend with the fact that most browsers will already accept Web sites that have been compressed with gzip, which is free and easy, so a commercial product really has to show some big advantages.
In any case, I added pointers to a batch of pages with information on this topic, including some commercial products. You might want to check them out:
CWinamp - more than just a Winamp2 API wrapper
I love the Code Project. It has a huge array of information, user contributions ranging from the sublime to the mundane, from eloquent to illiterate. CWinamp caught my eye as soon as it appeared, because it was a great approach to a problem I'd recently worked on myself. (See Louis, Louis.) CWinamp creates a nicely wrapped package that allows you to control Winamp, one of the more popular Win32 MP3 players. If this had been available last year, my pet project, Louis, Louis would have never been created! Gregor S. is to be congratulated on his work.
More Links to the NIST DADS The National Institute of Standards and Technology has a fairly large Web site. Tucked into one corner of said site is their odd little dictionary of Algorithms and Data Structures. It has what looks like a couple hundred entries to a lot of topics in the basic catalog of computer science topics, which naturally includes quite a few related to data compression and information theory. The entries are fairly short and terse, and usually give just one or two links for further study. But I guess that's what you get with a dictionary - if it were an encyclopedia we would be able to expect more. In any case, I've added a bucketful of these items to my site. If I've kept track properly, they are:
NIST Dictionary of Algorithms and Data Structures
Controlling Winamp2/3/Sonique Programmatically
As long as I'm giving props to the Code Project, I might as well drop in a link to another new aticle that popped up last week. This is a description of how to control WinAmp from an external program. As a bonus, the technique works for Sonique 1.x as well. Daniel Bright seems to have come up with this totally via reverse engineering, so bully for him!
Meet Davide Pizzolato DataCompression.info has links to scads of projects that people have written for the good of their fellow programmers. A good example of that would be CxImage, a nice image library for Win32. CxImage makes it easy to to load, save, and dsiplay BMP, JPEG, GIF, PNG, MNG, and J2K images. It's a free library, and seems to get frequent updates and bug fixes. Well, if you like the library, maybe you ought to take some time to meet the guy behind the library. Follow the link here to check up on Davide Pizzolato, from Vicenza, Italy. Davide doesn't have too much personal information here, but you can get links to his other projects, to useful programming information, and sign up for email from Davide when he has an interesting announcement to make. Davide Pizzolato's personal Web site
WAN Accelerators Followup I recently ran a pointer to an article in ComputerWorld on WAN Accelerators. These black boxes perform compression on packets as they wend their way through your corporate network. The makers of these boxes have to be clever to squeeze the maximum compression out of these packets - it's a lot harder than compressing files. If you're interested in learning more about these products, you can try some of these links for starters: Of course, if you're the typical DataCompression.info reader, you'll probably toss out all this high-priced gear and just plug in your own compressor in your corporate routers, right?
MJPEG
Tools There are a growing number of video capture cards and PC video cameras that use Motion JPEG as their preferred format. Motion JPEG is a lot easier to encode than MPEG, because you don't have to do all that complicated comparison between successive frames in order to do motion estimation. Much easier! This is nice for keeping the cost of your acquisition hardware down, but once you've captured some video, you might want to convert it to true MPEG format, which will not only save some space but make your video a bit more universally accepted. The folks at MJPEG Tools make a set of tools that help you with all that. If you check out this site you'll find a wide variety of packages available for free download. Although it isn't super-active, there do seem to be fairly regular bug fixes and updates to the products supported here.
EXEmp3 This program has a novel approach to the distribution of MP3 files. Instead of just shipping MP3 files around, the folks at EXEmp3 propose packaging the MP3 file with a mini-player executable, as well as a copy of the song's lyrics. The player only takes up 48K under Windows, and you can unpack the original MP3 file if you like.
RIP Vinyl by Wieser Software Ltd Ripping CDs to MP3 format is pretty easy - the CD format lets the ripping program know when one track ends and another beings. Not true for old-school vinyl records, and that's where RIP Vinyl wants to make a difference. It attempts to partition your input into tracks by looking for the gap between songs. Naturally, it has many other options, and will record directly to MP3 format. The program gives you control over the level sampling begins at, and the minimum amount of time that must elapse between tracks, and can record from any audio source on your PC.
DropMP3 The author of this program, Philippe Laval, says You are looking for an application to encode your sound files in MP3, DropMP3 is done for you. The program runs on MacOS, uses the LAME encoder, and is released under the GPL. Note that the web page is more or less duplicated in English and French. The English version of the DropMP3 site
PowerArchiver 2000 8.1 PowerArchiver 8.0 was a really good release for this product, providing a raft of features that can match up with segment leader WinZip one-for-one. For example, PowerArchiver 8.0 added full support for the new PKZip archive format, allowing for huge archives and > 65535 files, as well as support for the deflate64 compressor. The 8.1 release added a full German language version of the program, along with a list of minor enhancements. You can see the whole list here.
Roger Walker's DOS Port of mpg123
This mp3 player uses the DJGPP C compiler to port the MPG123 player to MS-DOS. Binaries and source available. Version 3.0 appears to have been worked on in 2002, but no history file seems to be available.
David Weekly's MP3 Stream Code Snippet
David posted this C code with the following comment: I have no idea if this is useful to folks, but since I had to beat my head against the silly Microsoft APIs for quite some time to get a useable result, I thought it might be helpful to post this little snippet showing you how to find an ACM decoder for MP3s, intiialize it, and use it to decode streaming MP3 buffers.
SXSW Streams Content Using Ogg March 24, 2003 I don't know how much the rest of the country cares about the South by Southwest Music Festival, but it's pretty big news here in Texas. If you enjoy live music, particularly in the pop, rock, alternative and americana modes, it is a bit of heaven on earth in our fair capital city of Austin. This year the festival saw fit to stream selected live shows out over the net, using lots of open source software, including the Ogg Vorbis package. This site implies that you might be able to come back some day and listen to archived shows. For the time being there are some preview tracks from the majority of artists on the bill. South By Southwest Press Release
12Ghosts Zip 6.20 A company called SuperGee has this package of utilities called 12Ghosts, which includes two different Zip archivers. Nothing wrong with that, but it is quite annoying when trying to track changes to the package. SuperGee does a point upgrade and re-releases the whole shebang every time any one of the things changes. So after each release, which seems to happen twice a week, I have to pore over the release notes to see if anything significant has occurred. Anyway, since my last update of the package at 6.0.3, there have been a few minor changes to the zip packages. You can track the changes here.
Music Industry Wets Pants Again Sony has announced the release of a new music recording system that will allow users to save as many as 30 hours of music on a single CD. Philips has announced a system that will allow the recording of up to 1000 hours of music on a DVD. The music industry sees nothing good coming out of this, but anticipates more piracy. In this forum, we're more interested in finding out exactly what that new compression format is that they talk about in the New Scientist article. A thread on Slashdot discusses just that. I'm not sure if we have resolved whether the new Sony system will use ATRAC or not, or whether they'll use this ATRAC enhancement called MDLP.
Michael Walden's Compression Pointers
A comprehensive set of compression pointers. Unfortunately, Michael is using some sort of software that makes bookmarking into his index impossible. So instead, you must link to the main page, shown here, and locate the links to "Data Compression." Under that he has links to General Resources, Software, and Theory.
Bob Crispen Ships Win-GZ 1.2 Bob Crispen has created a GUI wrapper for Gzip so that Win32 users can take advantage of this fine piece of free software. Naturally, Win-GZ is free as well. Version 1.2 of Win-GZ is built using zlib 1.1.4. If you're interested, Bob has also created a port of zlib to Jacob Navia's lcc-Win32 development environment. You can get a copy of it here.
JPEG to MJPEG-AVI converter Ships 1.2
This Open Source Linux project does one thing and one thing only: it converts a series of JPEG files to an AVI MJPEG file. The 1.2 release notes show a couple of bug fixes, nothing more.
Digital Photography from Agfa A complete short course on digital photography from Agfa. It shows up here because Lesson 6 is on The Right Compression. You get a discussion of various formats, including GIF, JPEG, JPEG2K, and PNG. Best of all, it looks like the entire page is available in English, German, Spanish, and Portuguese.
Introduction to Wavelets for Image Compression
Aleks Jakulin has created a tutorial that will walk you through the necessary steps to compress images using a wavelet transform. The steps in the process are illustrated using Mathematica code. This page goes beyond a basic tutorial in that it shows a proposal to improve image rendering by adding noise.
Arithmetic Coding The NIST page on arithmetic coding from their Dictionary of Algorithms and Data Structures. A terse definition and a couple of links. Like most of the definitions in the NIST DADS, this is a starting point and not much more. The NIST Dictionary of Algorithms and Data Structures
Context Tree Weighting Context Tree Weighting (CTW) has been a technique with great promise, but it hasn't ever been able to reach the critical mass needed to become more than a curiousity. Jürgen Abel is doing his best to overcome that problem. He's created a nice reference page for CTW on his data-compression.info Web site. He has references to a few papers, a few people, and one piece of source code. I've been through a few of these papers in the past without achieving a good understanding of CTW, but perhaps with Jürgen's new links I'll be able to finally get somewhere.
MINC Lossless Digital Data Compression System
Back in 1998 I heard from a poor soul who was being encouraged to put some money into Premier America, makers of the MINC Compression System. Seeing that MINC promised an iterative compression system that guaranteed any ratio you cared to dial in, I encouraged him to run, not walk, away from this opportunity. This led to an email conversation with someone from the company, assuring me that they would have a real product on the market quite soon now. I haven't heard a peep out of Premier America since then, so hopefully they have stopped raising funds for their unlikely system. Fortunately, a good soul named Chez DuLac has preserved MINC's manifesto on his Web site, along with his reasoned rebuttal. Chez DuLac's Library of Laughs
Creative To Ship Media2Go Video Player
Microsoft recently debuted their Media2Go reference platform, which is more or less a video iPod. Naturally, their short term goal is to sign up a raft of companies who are willing to take a stab at selling an implementation of this platform. In this case it looks as though the first company to bite is Creative, who will be shipping an unnamed video player based on Micrsoft's Windows CE .NET O/S. Creative didn't release device specs, but they did say that the gizmo would hold up to 175 hours of high-quality video, 8,000 music files, or 30,000 photographs. Look to see the device in time for Christmas, 2003. The Register - Story Dated March 13, 2003
MMPlayer for the Palm David Janssens, author of the open-source J2000 JPEG 2000 codec, let me know that he has a couple of other interesting projects keeping him busy. The first is called MMPlayer, which is a Mobile Media Player for the Palm OS. It supports a wide variety of audio formats, including Windows Media, MP3, Ogg and WAV. Video isn't supported in the current release, but is anticipated as part of MMPlayer 2.0. You can download an alpha copy of MMPlayer and see how you like using your PDA as a music source. No word on the Web site regarding a release schedule or pricing - I think David is probably deep in the development phase, so we'll hope for more details as work progresses.
Archos Media Player Archos decided that they could release a portable video player without waiting for Microsoft's Media2Go specification to be complete. The result is the AV340, which sells for a retail price of around $560. It's got a 40G hard drive, which you can pack full of MP3 files or MPEG-4 footage. If your video files are formatted to fit on the Archos screen, you can get as much as 80 hours. The AV430 can also record video directly your VCR or TV, and can playback on your TV using standard video cables.
Jürgen Abel's BWT Page Jürgen Abel has done an enormous amount of research on the Burrows-Wheeler Transform, and has published the results on his Web site. On this page you will find:
BWT page on data-compression.info
DJWrap Approaches Release From the SourceForge development page for DJWrap: The DJWrap format is an effort to create an open format for combining several MPEG audio streams into one, without losing information about the original files and without disturbing the stream with erroneous or misplaced data. I'm a bit unaware of who needs DJWrap, but it seems like a interesting project nonetheless. It's a version 0.9.2 right now, so the first official release ought to be coming up really soon!
Introduction to MrSID MrSID is a wavelet-based image format that seems to be pretty popular for people creating mapping databases (which tend to be huge). This page has a nice little explanation, provided for you courtesy of the state of Massachusetts. MrSID is a proprietary format owned by LizardTech.
Compression Relieves Congestion Are you managing an internal network that has links between geographically separated locations? And are you paying a ton of money to Sprint or AT&T or MCI every month? Then you might have thought about using a compression appliance to cut down on your bandwidth requirements. The folks at Computerworld want to help you in your decision making process, so they provided a nice article discussing some of this. In addition to the main story, you can follow this link to get a little Q and A from a real-live user. Worth a look.
Windows Media Metadata Usage Guidelines
Digital media needs metadata to be supremely useful. For example, wouldn't it be nice if your CD player showed you artist, album, and track information while it was playing a song? MP3 tracks have ID3 tags to supply this information, but naturally, Microsoft has created an extensive set of APIs for embedding metadata in all types of media. This article will bring you up to speed on metadata support for media encoding, playback, and more.
Lempel-Ziv Welch - NIST Reference Page
Our National Insitute of Standards and Techology uses our tax dollars to maintain a Dictionary of Algorithms and Data Structures, which is probably worth an entry in your bookmark file. The entry for LZW compression has pointers to a definition, implementation, and further information. NIST Dictionary of Algorithms and Data Structures
Optimizing Microsoft Windows Media Services 9
Series An article from Microsoft giving you the lowdown on the best way to use all the tools shipping with Windows Media 9. This includes basic guidelines on setting up a streaming media system, comparison to Media Services 4.1, locations of bottlenecks, performance evaluation, and advanced tuning tips. There's quite a bit of content in this article. If you are planning on using Microsoft servers to deliver audio content, you owe it to yourself to plow through this.
Contributions à la compression de données
Steven Pigeon's Ph. D Thesis from the University of Montreal. Proposes a new set of universal codes, which he dubs taboo codes, as well as new optimization algorithms for (Start, Step, Stop) codes. Plus lossy variations on LZW. The paper is in French, no English translation available at this time.
i-Sound WMA/MP3 Recorder 5.60 Professional
This app from AbyssMedia is advertised as being able to turn your computer into complete audio recording studio. That means you have the ability to record from external inputs directly to MP3, OGG, WMA, or WAV format. Includes niceties such as MP3 and OGG tag editors, Voice Activated recording, skin support, and more. New Features in 5.6 appear to include support for multiple sampling rates, Hotkeys that work in any application, Skin support, and Mono/Stereo operation.
EasyAudio 3.0 Ships The EasyAudio ActiveX Control adds speech handling capabilities to your Win32 program. Listed features include nice things such as support for popular codecs including G.729, G.711, and ADPCM, unicast and multicast support, AGC, jitter buffer management, and noise reduction. The web page gives a price of $1000 for the control, and $2000 for the source code. I hope that big price tag includes free distribution rights, but the Web site is woefully short on license information. Actually, I should say that the web page is woefully short on any sort of information about the company, the products, the licensing, etc. Technically this product has great looking specs, but if I'm going to hit the company credit card with a $3K purchase, my boss often wants little details such as a company name and phone number!
Easy Video 3.0 Ships The EasyVideo ActiveX control lets you integrate real-time video into any Win32 app that can host an ActiveX control. Video features supported include video capture, transmission, compression, and decompression, using standard Windows codecs as well as MPEG-4. Additional niceties include multicast support and bandwidth adjustment. And since this is a sister product of EasyAudio, you should feel free to assume that all the same gripes I listed in its entry apply here as well.
Jürgen Abel's BWT Paper This is a preprint of a paper by Jürgen Abel describing the functionality of a basic Burrows-Wheeler compressor. Dr. Abel reports that this will be published in PIK, a German-language journal on Communications and Information Theory. This paper is in German, and unfortunately, translation sites such as AltaVista and Google don't seem to translate PDF files. Any good citizens willing to translate the paper pro bono?
Blaze MediaConvert 2.0 Ships Blaze MediaConvert is a pretty nice looking convert-all program that takes you from one format to another. They advertise an interface that makes it a piece of cake to convert your MP3 files to WMA or OGG, your GIF files to PNG, and even your video files to animated GIFs. A truly impressive list of supported formats, especially for image files. Blaze shipped version 2.0 of this product in February. I wish I could give you a nice list of features, but unfortunately, Mystik Media is one of those companies that neglects to include release notes on their Web site. So, would the first person to download and install version 2.0 of Media Convert let me know what's new? Thanks!
Jocsoft - MP3 For Developers Jocsoft makes some software products used for searching the web, and one of those is designed to search for MP3 files. As a result, they decided to include a nice page of MP3 links for developers on their web page. It's a little dated, with a high ratio of dead links, but still worth a look if you're an MP3 afficionado.
Adobe Blesses JPEG2K Algo Vision's latest newsletter let me know that Adobe has added support for JPEG 2000 to Photoshop with their addition of a plugin for version 7.01. They're quoting a price of EUR 119, which seems like quite a bit of money for a plugin. The folks at Algo Vision are quick to point out that you can buy their plugin for quite a bit less, and it will work with earlier versions of Photoshop as well. Algo Vision LuraTech Newsletter
That's all until next month. Write me with your feedback at markn@ieee.org.
|
 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
