not always searching files in zip stored with the "Store" method

[swg0101]

On the latest version 7.13, it seems searching a zip file with files that are using method 0 (Store), the search will fail with the following message:
ugrep: cannot decompress crowdstrike-1.49.0.zip: unsupported zip compression method 0

The file in reference can be found here, but any zip file using the Store method should have the same error.

https://epr.elastic.co/epr/crowdstrike/crowdstrike-1.49.0.zip

Any thoughts for adding support for this method? Most ZIP files are either DEFLATE (supported) or Store (not supported).

[genivia-inc]

The STORE method is supported in ugrep (src/zstream.cpp:530), where we issue an error when the zip compression method is unknown and it is not STORE, or if the method is STORE, then the size must be known:

else if (method != Compression::STORE || (flag & 8) != 0)
{
        std::string message("unsupported zip compression method ");

However, if the "general purpose bit flag" bit 3 is set in the zip local file header data descriptor then the CRC-32 and file sizes are not known when the header is written with the STORE method. This prevents it from being searchable right now, because we don't know the size.

EDIT: perhaps when bit 3 is set and a very large file is stored in Zip64 format then the size is known, because the size is in the Zip64 extended header? I need more information on this!

EDIT-2: before someone comments about the "zip central directory": yes, I am aware of the central directory located at the end of the zip file. However, we don't necessarily have to use the central directory to get the (de)compression sizes, because the compression formats are all self-terminating. Only for the STORE method we may not have its size when it is not specified as per general purpose flag bit 3 and when instead it is specified in the central directory. For this, we can't just search from the end of the zip for the central directory when we have streaming zip file data being piped through. But we can use it when the zip file is seekable i.e. a physical file, not a stream. <CAUTION:RANT> It sucks to be a zip decompressor when a zip compressor tool puts all this extra burden on the decompressor. If the STORE size is eventually known by the zip compressor, it should back-patch it in the zip file and not leave it to the decompressor to find it at the end of the zip in the central directory. </CAUTION:RANT> In order to read the STORE data that has its size specified in the central directory, I will add some code to read the central directory when the zip file is seekable. If the zip file is not seekable, such as a pipe or a nested zip file (--zmax=2), then we can't do this.

[genivia-inc]

It is possible to use the zip central directory to locate STORE files of indeterminate size, meaning when no size is specified in the local header. I will add this feature to ugrep in the next couple of days, though I have to add that the priority to do so is medium, because this problem should rarely if ever happen in practice.

Note the ugrep does in fact search STORE files in zip archives. Only if the archiver for some reason used the STORE method instead of a compression method and when it did not specify a size with the the STORE method in the local header of the file in the zip, then we can't search the zip-archived file at the moment.