Springy 1.5.5 released
Dragan posted Aug 5th 2009 at 8:55PM
Another version of Springy was released about two hours ago. This one slightly deviates from a newly established practice that each, even very minor, release should bring some new features, but I think it still brings some very important improvements and fixes, especially to those using popular Path Finder as their primary file manager in Mac OS X.
As Path Finder users probably already know, since the version 1.5 Springy hasn’t coped very well with Path Finder, actually not at all. The problem was introduced by me and my workaround to bring SpringyCM back into the root of the contextual menu, not to be placed in “More” submenu like Apple wants since introduction of Leopard. I did this since many users have asked for it. In general, SpringyCM was appearing in the Path Finder contextual menu, but not respecting Path Finder preferences and even worse, besides appearing in the menu nothing else worked correctly. When a user clicked any some Springy command in the Path Finder menu, some other Path Finder command would be called instead.
This problem is now resolved. SpringyCM is now (almost) fully functional when working with Path Finder! The only thing which won’t work is extracting particular file from an archive while browsing through hierarchical contextual menu. Namely, browsing is still possible, but clicking on a menu item representing a file in the archive will NOT work. This functionality has to be implemented using Carbon command event and that event cannot be caught in a Cocoa application, such Path Finder is. By the way, this particular functionality has never worked, I hadn’t even known until I’ve tried it recently.
Another improvement in this release is an option to show Springy menu in either root of the Finder contextual menu, or in the “More” submenu (like it was prior version 1.5) in Leopard. In Tiger, it is in its standard place where it always has been, root of the Finder contextual menu. This will apply only when working with Finder. When working with Path Finder, setting from Path Finder preferences (“in Submenu”, “at End”) will be used. Some users still prefer having 3rd party CM plug-ins shown in the “More” submenu, now they have that option with Springy too. The option is available in the General preferences panel.
This version also brings some minor bug fixes. The full list of changes is available on the version history page.
Enjoy using the new version.
Dragan
Springy and RAR archive creation/modification
Dragan posted Jul 8th 2009 at 10:00PM
In the previous blog post I wrote in great detail about the status of the support for 7-zip archives in Springy. Like promised then, this post will treat the support for creating and modifying RAR archives.
If you use Springy more than just occasionally, you probably know that it already supports working with RAR archives. However, that support is limited to opening/browsing/extracting only. No creation, nor modification. The reason for it is quite simple: licensing. RAR format is completely proprietary, there is no publicly available specification document of it. The consequence is lack of publicly available library which can be linked against, since nobody is able to make it using the specification. Off course, some sort of reverse engineering is always possible, but would certainly involve legal dispute with RAR format and tool author, Eugene Roshal and his brother Alexander, who is more prominent in public since he mainly runs all the business for the RARLAB company.
I’ve tried everything I could to include full RAR support in a reasonable way. I’ve contacted Eugene long time ago, when Springy was still just a bunch of ideas and non-functional code. My idea was to get some kind of RAR library or full RAR source code in order to link my code against it and provide full RAR support. I was ready for any king of licensing. Regrettably, I received a short response from Eugene that they didn’t license RAR compression anymore. This effectively meant no RAR compression support in Springy for the time being, since I didn’t want to make a UI wrapper around CLI tool (I explained my reasons in the previous blog post).
However, Eugene still contributes significantly to the community by offering very portable source code for his unrar command line utility. That is the stripped-down version of his full-featured rar command line tool, which can do only opening and extracting files. That code is used in Springy to handle RAR archives. It is not included directly, nor is it compiled as a separate tool which runs in the background. I modified it in a way which could produce a proper library and then linked against it. The final result is very satisfactory; having native support for RAR archives, while preserving full control over the processing and progress notifications. This support was included in Springy in version 1.2 (somewhere in January 2007) and since then, as my knowledge of RAR code grows, every new release brought significant improvements in the way RAR archives are handled, becoming faster and more disk I/O efficient.
Unfortunately, creation and modification of RAR archives is still an issue. In the meanwhile several Mac archiving utilities appeared on the marked, which use rar CLI tool as an engine to do RAR processing, including creation and modification. Since licensing issues doesn’t allow for that, those utilities don’t include rar tool directly. On the contrary, users are expected to download and PURCHASE it separately and then instruct the utility to use it for RAR processing. Many such utilities don’t even go at least as far as mentioning that requirement in documentation, leaving users scratching their heads when receiving strange error messages. However, I stayed determined not to go with separate CLI tool and UI wrapper around it.
Somewhat this time last year I decided to make another try. I contacted Eugene again, making very unusual proposal that they develop RAR compressing dynamic library and offer it to the end users, like they offer their utilities. That way, developers could license and link against it during development, while they would’ve been forbidden of distributing it with their product. Users would have to license the library as well, so it would be used by the utility on run-time to provide full RAR functionality. Their business wouldn’t have been affected, every user would’ve had to come to them and purchase the library. I even went that far to offer my own services to make all necessary changes to the existing rar source code and to compile it as a library, and they should’ve only approve the changes and distribute it as their product. Regrettably again, I was informed that their stance on RAR compression licensing wasn’t changed, no RAR compression licensed. Consequently, I had to accept that.
As of lately, more and more people are asking to provide any mean of creating and modifying RAR archives with Springy, putting me on the edge to bite the bullet and to do the similar to what others do; require users to download and purchase rar utility for Mac and then instruct Springy to use it as an engine to create and modify RAR archives. I still want to avoid a lot of unnecessary I/O disk activity and processing this approach brings, involved especially when dealing with files which contain HFS+ extended attributes and resource forks (since Leopard, most of the files on your disks DO HAVE these attributes). For that , I needed rar tool to be able to compress data not located on disks only, but also data from some “non-physical” source, like for example memory buffers, pipes or similar. Luckily, I’ve discovered that rar utility can compress data coming from standard input. That’s completely enough to implement RAR archiving, not exactly the way I would’ve liked, but still good enough until Roshal brothers (hopefully) decide to license RAR compression again.
In clear words, RAR compression will be supported in Springy, starting from version 1.7. I expect that to happen somewhere around New Year 2010, possibly one or two weeks after New Years Day. As you probably realise by now, in order to use this new feature, you’ll have to download and PURCHASE rar CLI utility separately. I hope this will finally fulfil wishes and demands of many users. It will also extend Springy usability, although I have to make a strong statement one more time that kind of implementation is not exactly what I’d like to do. But at the end, user is The King.
Dragan
Springy and 7-zip archives
Dragan posted Jul 2nd 2009 at 7:52PM
A considerable number of existing and potential users asked about the status of support for 7-zip and RAR archives in Springy. I’ve already promised to write a few words about it, but due to some other higher priority issues I haven’t be able to keep that promise so far. So, I’m just about to do it now.
Since a lot of things can be written about support for these two archive types, this story will be divided in two blog posts; the first one (the one you read) will look more closely into the 7-zip support, since things are more certain there, while the second one, which I’ll post in a couple of days, will be dedicated to the RAR support.
When including support for a new archive type in an archiving utility, one can choose one of the following paths:
1. Take a library which deals with specific archive type files, developed by someone else, which has clearly defined API and is preferably already proven and in use on the market. This approach has all the benefits; it’s quick and enables handling archived files in a proper way. Furthermore, if the library is open sourced, it can be easily modified (of course, if licensing permits that) to fulfil specific needs. This is how ZIP, TAR, PAX, CPIO, SIT, GZIP, BZIP2 and UNIX Compress are supported in Springy. More information on underlying libraries, their names and authors (big kudos to them) can be found in Springy About Panel.
2. Take the archive format specification documents (if available), sit behind your desk, take a deep breath and start developing a library yourself. The benefit is clear; you can implement anything you like in the way you like. The drawback is clear as well; it requires a lot of time and effort. And if you’re like me, developing your product not as a primary job (something else still puts bread on my table), but in your spare time just for the passion of developing it (and earning some extra cash, it never hurts), you usually don’t have that much time. Especially if the list of other features users wants you to put into your product is rather long.
3. If neither a library nor specification of the archive format are available, one can try to find a source code of the utility doing the job already. Including the source code unmodified, as for completely separate application, is usually not very effective thing to do, but if it is portable enough and if licensing allows for it, it can be modified and turned into a proper library. From then on, the path is the same as in #1. This is how RAR is supported in Springy. The support is not complete (creation and modification of RAR archives are not possible, but that’s the topic for the next blog post).
4. Finally, one can take already existing command line (CLI) utility handling desired archive type and use it as a processing engine. The application itself would be just a UI wrapper around the CLI utility, receiving users actions and handing them to the engine and present the results of processing done by the engine to the user. I’ll stay a bit more at this particular option for the obvious reason that many archiving utilities for Mac use exactly that approach and since it gives me the nice opportunity to answer question/comment/rant of some users “How come that many others support that archive type, and Springy still doesn’t?”
I have to say that I really dislike concept of an UI front-end application with CLI tool running in the background doing all heavy works. That is the reason Springy doesn’t use any such tool running in the background for archive processing (apart from system built-in hdiutil, which does disk image mounting and ejection). There are many reasons why I don’t like going this path, and here I’ll quote just what authors of Version Control with Subversion book, Ben Collins-Sussman, Brian W. Fitzpatrick, C. Michael Pilato say about that, which sums it very nicely:
Binding Directly: A Word About Correctness
Why should your GUI program bind directly with a libsvn_client instead of acting as a wrapper around a command-line program? Besides simply being more efficient, this can address potential correctness issues as well. A command-line program (like the one supplied with Subversion) that binds to the client library needs to effectively translate feedback and requested data bits from C types to some form of human-readable output. This type of translation can be lossy. That is, the program may not display all of the information harvested from the API, or may combine bits of information for compact representation.
If you wrap such a command-line program with yet another program, the second program has access only to already-interpreted (and as we mentioned, likely incomplete) information, which it must again translate into its representation format. With each layer of wrapping, the integrity of the original data is potentially tainted more and more, much like the result of making a copy of a copy (of a copy …) of a favorite audio or video cassette.
As they explained nicely about correctness issues, Springy talks for itself when it comes to efficiency; Springy, while being much richer with its UI options and views, proves to be faster in archive processing than many of numerous applications available for Mac OS X which serve just as a UI wrapper around CLI tools running in the background. There are so many things that can be done when using proper library or implementing things your own way than running a separate tool in the background. They usually lead to faster operation, lower memory consumption and most often (and most important I would add) in case of archiving utilities, much less disk read/write activity (which contributes to speed as well).
Now with this clear conclusion, let’s go back to the original topic of this post: support for 7-zip in Springy. There isn’t publicly available library that can process 7-zip archives. Implementing it from level zero myself would definitely take much, much longer than anyone would’ve wanted and been ready to wait for. My first attempt was to use LMZA SDK. Unfortunately, the source for this SDK is full of Windows-isms, and completely undocumented and non-commented. It’s not portable at all and it’s very hard to use it as a foundation of a custom made library which would handle 7-zip archives. Then, I tried my luck with 7-zip for Windows source code. The situation with that was even worse. Nearly impossible to change it and compile for some other platform, at least not in a short period of time. Then, I though that the source code of p7zip would be the solution. It’s available for many flavours of UNIX, even for Mac, so it’s portable, right? Unfortunately again, it’s not much better. It’s just the same 7-zip code with a Win32 compatibility layer compiled in alongside. The only really portable 7z code is 7z-C code, which is part of LMZA SDK distribution. But it’s very limited; it supports only extraction/decompression, no support for file attributes, split and solid archives, encrypted files and archives etc. So, I didn’t want to think of it at all.
Of course, I could’ve gone the undesired path as well; take the existing p7zip CLI utility compiled for Mac OS X (its distribution supplies make files to compile it on Mac), embed it into Springy application bundle and use it as an engine for 7z archiving/extraction with Springy as a UI wrapper. But as already concluded above, that isn’t going to happen.
So, the question remains: what is going to happen with support for 7-zip archives in Springy? The good news is: it will come! And it will come very soon! I decided to go path #2 from the above list. I took the source code of the p7zip CLI utility and started modifying it all over the place in order to make a proper library out of it. I started this long time (more than a year) ago and I still haven’t finished due to time (un)availability, other features in a queue, but most significantly, due to the p7zip code itself. It really needed a lot of rework and modifying in order to make it the way I wanted. Please, don’t get me wrong, I’m not saying the code is bad or anything similar. I just want to point out that it’s nowhere near to being easily adapted to make a library out of it. And that’s exactly what I want.
What this all mean for the users? The reading/browsing/extracting part of functionality is already finished and working beautifully. Currently I’m struggling with archiving/compressing part, which should work properly and include numerous options and archiving parameters 7-zip archive supports. I still don’t know when it will be ready, I’m working hard to finish it before the version 1.6 is released, and that should be somewhere in October. If I don’t succeed, the reading/browsing/extracting part will be included in version 1.6 anyway, so at least users will be able to open, browse and extract files from 7-zip archives. Archiving part will follow soon after. Off course, if I manage to finish it before October, the full 7-zip support (including archiving and modifying of archives) will be available in version 1.6.
I hope this post finally clarifies the status of support for 7-zip archives in Springy. I also hope it explains why it still isn’t supported and why it took me so long to make it. In the next blog post I’ll provide similar information (not that detailed though) on support for creating and modifying RAR archives using Springy.
Dragan
