Home > Uncategorized > Building directly from a .tgz file

Building directly from a .tgz file

May 15th, 2010

Working on lots of different code bases means I am forever having to extract the contents of tar/zip files before compiling/analysing the files in the extracted directory tree, then deleting the directory tree when I am done. It is about time development tools such as make and compilers had the ability to build directly from an archive.

Vi (well actually Vim and other editors) supports the editing of files contained within an archive and thanks to libarchive the latest version of Numbers also has this functionality.

This is not about saving disc space, it is a way of working that creates a barrier between files created elsewhere and files created by me; it would make it harder for me to accidentally leave my work files in the directory I happened to be sitting, in the directory tree, when working on the source.

There are some design issues that need to be sorted out if build configuration via .configure files is to work correctly. I leave these design issues to the people who know about configuration management.

We will need to extend the existing directory-path/file syntax to support the specification of a file contained within an archive. How about using ::tgz:: as a file prefix to indicate that the subsequent directory/file specification is to be interpreted as referring to the contents of an archive file, e.g.,

cc /home/stuff/::tgz::app.tgz/src/foo/bar.c

I don’t think a separate prefix is needed for each kind of archive, any character sequence that is sufficiently unused at the moment will do.

Go readers, spread the word!

  1. Juha
    May 24th, 2010 at 09:19 | #1

    I think you should read The Hideous Name by world-renowned Unix gurus Rob Pike and P.J. Weinberger as it looks like you have just reinvented the same old design flaw they warned against decades ago.


  2. May 25th, 2010 at 00:30 | #2

    Juha, you are right to say that the naming convention /home/stuff/::tgz::app.tgz/src/foo/bar.c has nothing to recommend it on aesthetic grounds, it is ugly. The Pike and Weinberge paper does a good job of pointing out the disadvantages of some naming conventions but says little about the advantages they might provide.

    The ideal naming convention would be no change to existing practice, i.e., tools would deduce that app.tgz was a compressed tar file and treat the rest of the path as specifying a file within the archive.

    How easy would it be to leave existing practice unchanged? I don’t know.

    Accessing files contained within an archived/compressed file is likely to be the rare case, at least in the short term, and we don’t want to force implementors to do lots of work updating existing code to handle it. Adding the ::tgz:: carbuncle may reduce the effort needed to add this functionality to existing code (I am guessing here), but at the expense of future generations having to live with it. When designing C++ Stroustrup decided on C compatibility for shared features as a means of gaining acceptance and now C++ is widely used this compatibility is regarded as a ball-and-chain.

  3. Anonymous Cowherd
    October 4th, 2010 at 23:56 | #3

    Seems like all you’d have to do is add the functionality to your filesystem of choice. Once your filesystem understands that “app.tgz” is a directory, and understands how to access files in that directory (via compressing and decompressing), any program using the open() system call will be able to access the compressed files too. Then not only will you be able to do

    cc /home/stuff/app.tgz/src/foo/bar.c

    but you’ll also be able to do “ls -l /home/stuff/app.tgz” where you used to have to do “tar -ztf /home/stuff/app.tgz”. Possible unintended consequences with “cp -R /home/stuff/app.tgz /home/otherstuff”, though; I’m not sure whether that would work out of the box in a sensible manner or if you’d need to mess with some of the common command-line tools to make sure they do sensible things.

  1. No trackbacks yet.
Comments are closed.