Join Log in
Score:10
avatar Ubuntu

Can't unzip 6 GB file: bad zipfile offset

es flag
Rich Hard Fine Man

I need to unzip a file of about 6 GB. However, I can't do it neither right-clicking (it gives an error saying "empty archive") or in the terminal, showing the following for the latter:

$ ls
 fhs-3.0.pdf   IDrive   Tese.zip  'Transport Studies of Dual-Gated ABC and ABA Trilayer Graphene: Band Gap Opening and Band Structure Tuning in Very Large Perpendicular Electric Fields - Zou.pdf'

$ unzip Tese.zip
Archive:  Tese.zip
warning [Tese.zip]:  4294967296 extra bytes at beginning or within zipfile
  (attempting to process anyway)
file #1:  bad zipfile offset (local header sig):  4294967296
  (attempting to re-compensate)
 extracting: Tese/.DS_Store          
error: not enough memory for bomb detection

$ jar xvf Tese.zip
java.util.zip.ZipException: only DEFLATED entries can have EXT descriptor
    at java.base/java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:313)
    at java.base/java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:125)
    at jdk.jartool/sun.tools.jar.Main.extract(Main.java:1361)
    at jdk.jartool/sun.tools.jar.Main.run(Main.java:409)
    at jdk.jartool/sun.tools.jar.Main.main(Main.java:1681)

I don't think the file is corrupt because I unzipped the same file in a Mac recently.

My PC has 15 GB of RAM and 2 GB of swap memory.

1845
0 + 0
zip
us flag
Didier L
Note that 4294967296 is exactly 4 GiB so that’s another hint toward’s Ferrybig’s answer.
0
Reply
co flag
Gábor
It doesn't warrant a new answer because it's basically already described in the existing ones, but by googling the error message you got, it seems that this ZIP file was created in a so-called streaming mode that not all ZIP implementations are known to support. The Java one is especially mentioned as one without support. So, while the file itself might be OK, you simply won't be able to handle it with those tools. Use different tools or unpack it on another platform that has those tools.
0
Reply
nobody avatar
gh flag
nobody
`unzip -l /path/to/your/Tese.zip` the zip file could be corrupt.
0
Reply
es flag
Rich Hard Fine Man
@nobody I don't think that's the case because I recently unzipped the same file on a mac. Nevertheless, I did what you wrote and it listed the files inside the zip
0
Reply
nobody avatar
gh flag
nobody
`unzip -v | head -n1` please [edit] your question please.
0
Reply
es flag
Rich Hard Fine Man
@nobody edited the question. I can read the files, but I need to download them.
0
Reply
Score:23
avatar Ubuntu
in flag
Ferrybig

Java only supports file in the ZIP format, not ZIP64. The same goes for the simple zip command

Zip files have a size limit of 4GB minus 1 byte, Zip64 have a limit of 16 EB minus 1 byte. MacOS makes zip64 files when the size of bigger than supported. The tool your desktop environment gives may have the same limitation

You need the proper tooling to read zip64 files. On the command line ditto can read those files, while on Java, you can use Apache Commons Compress

0 + 0
es flag
Rich Hard Fine Man
thanks for your explanation! However, running ditto, it says that the command is not found. Looking at the Apache Commons Compress link you provided, https://commons.apache.org/proper/commons-compress/zip.html, I don't understand how to use it.
0
Reply
in flag
user1937198
Java added support for zip64 in 1.7 so any recent JRE _should_ be able to read zip64 files. I have used this in the past for jars that had more than 65k classes. However there is a regression in certain JREs: https://bugs.openjdk.java.net/browse/JDK-8226530. So check your JRE version.
0
Reply
in flag
Austin Hemmelgarn
I’m 99% certain that there is no `ditto` port for Linux, and therefore no way to use it on Ubuntu (barring Darling, but that’s _way_ too complex to talk about here). The best option in my experience for this type of thing on Linux is `unar` (the CLI version of [The Unarchiver](https://theunarchiver.com/)), which _is_ available for Linux (`sudo apt-get install unar` on Ubuntu and Debian systems).
0
Reply
Ismael Miguel avatar
om flag
Ismael Miguel
And if all else fails, just use 7Zip with WineHQ. I've successfully used it to extract weird RAR files on a Raspberry Pi with WineHQ with Box86.
0
Reply
Score:14
HuHa avatar Ubuntu
es flag
HuHa

Both tools, unzip and jar, explicitly told you that the file is corrupt:

miguel@...$ unzip Tese.zip
...
warning [Tese.zip]:  4294967296 extra bytes at beginning or within zipfile
  (attempting to process anyway)
file #1:  bad zipfile offset (local header sig):  4294967296
  (attempting to re-compensate)

miguel@...$ jar xvf Tese.zip
java.util.zip.ZipException: only DEFLATED entries can have EXT 
...

If you could unzip it on a Mac, it might be possible that some Mac software created that ZIP file, but maybe in a nonstandard format (as far as ZIP file standardization goes) that only some Mac software can understand.

You could try to go to that Mac again, unzip it there and then create a compressed tar file from it, then take that tar file to your PC and unpack it.

0 + 0
Joshua avatar
cz flag
Joshua
I've been filing bugs for stuff that explodes on encountering large zip files. ZIP64 is more than a decade old and an extension to the format put out by the format makers as a direct legal descendant of the original PKZIP. Implement it already or at least raise the error of "Zip64" not supported.
4
Reply
es flag
Rich Hard Fine Man
@HuHa where does it say that the file is corrupt? Nevertheless, did what you suggested, good idea! Thanks!
0
Reply
HuHa avatar
es flag
HuHa
I edited my answer to highlight the parts that indicate that.
0
Reply
barbecue avatar
us flag
barbecue
Might want to try using 7-zip on it, which has supported ZIP64 forever.
0
Reply
Matthias Winkelmann avatar
mx flag
Matthias Winkelmann
The way this answer mentions Macs reminds me of 12-year old boys talking about girls.
0
Reply
ilkkachu avatar
co flag
ilkkachu
@MatthiasWinkelmann, which part? The thing about "only Mac software understanding", or the thing about "going to a Mac, unzipping and creating a compressed tar file"? If the latter, I'm not sure I want to know more about what 12-year-olds of today think about girls...
0
Reply
Score:8
pmqs avatar Ubuntu
ua flag
pmqs

You can test if your copy unzip supports the zip64 extension (and archives >4Gig) by running unzip -v and checking for the presence of ZIP64_SUPPORT. Below is what I get on my Ubuntu setup. Note it does have ZIP64_SUPPORT

If your unzip doesn't display ZIP64_SUPPORT, you need a newer version of unzip.

Alternatively, if your unzip does have ZIP64_SUPPORT the problem is something else. Can you share any details about how the zip file was created? Is the zip file publicly available anywhere?

$ unzip -v
$ unzip -v
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 9.2.0 for Unix (Linux ELF).

UnZip special compilation options:
        ACORN_FTYPE_NFS
        COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
        SET_DIR_ATTRIB
        SYMLINKS (symbolic links supported, if RTL and file system permit)
        TIMESTAMP
        UNIXBACKUP
        USE_EF_UT_TIME
        USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
        USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
        UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)
        LARGE_FILE_SUPPORT (large files over 2 GiB supported)
        ZIP64_SUPPORT (archives using Zip64 for large files supported)
        USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.8, 13-Jul-2019)
        VMS_TEXT_CONV
        WILD_STOP_AT_DIR
        [decryption, version 2.11 of 05 Jan 2007]

UnZip and ZipInfo environment options:
           UNZIP:  [none]
        UNZIPOPT:  [none]
         ZIPINFO:  [none]
      ZIPINFOOPT:  [none]
0 + 0
es flag
Rich Hard Fine Man
my unzip seems to have ZIP64_SUPPORT. The zip file was created automatically, by downloading a folder that I had previously uploaded to an online backup, so it's not publicly available.
0
Reply
pmqs avatar
ua flag
pmqs
From the other answers there is a suggestion that the zip file was created in streaming mode. The standard `unzip` utility can extract most streamed zip files without any issues. To know for sure, can you run [zipdetails] (https://github.com/pmqs/zipdetails/blob/main/bin/zipdetails) against the file and post the results. This will dump all the metadata from the zip file. Output is a bit verbose, If you don't want to disclose the filenames in the zip file, use the `--redact` option. [Full disclosure: I'm the author of zipdetails]
2
Reply
es flag
Rich Hard Fine Man
thanks for your help! Running what you told me yields the following 000000000 LOCAL HEADER #1 04034B50 Can't use an undefined value as an ARRAY reference at /usr/bin/zipdetails line 757. I include the terminal image, https://imgur.com/a/kPjQLra. I guess I'm not doing what you told me in the right way. Sorry, this is very new to me.
0
Reply
pmqs avatar
ua flag
pmqs
Even though you broke my code, it's a really interesting result. It means the backup service you are using creates non-standard zip files. Can you tell us the backup service you are using? That would allow me to see if I can reproduce your issue. One last thing to try. Run `zipdetails - - scan Teste.zip` This may take an age to run, so you need to be patient.
0
Reply
pmqs avatar
ua flag
pmqs
@RichHardFineMan the line number referenced in the error message you supplied, 757, comes from an old version of the `zipdetails` script that ships with Ubuntu. The version on github has lots of fixes to allow it to work with more unusual zip files. In the same directory where your zip file is stored, run `wget https://raw.githubusercontent.com/pmqs/zipdetails/main/bin/zipdetails` to get the most recent copy of `zipdetails`. Then run `perl ./zipdetails Teste.zip`. If that still spits out an error message, please post the message here and try running `perl ./zipdetails --scan Teste.zip`.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.