[TRUEZIP-374] Iterator for TPath throws exception Created: 26/Mar/17  Updated: 26/Mar/17

Status: Open
Project: TrueZIP
Component/s: TrueZIP Archetype Path
Affects Version/s: TrueZIP 7.7.10
Fix Version/s: None

Type: Bug Priority: Major
Reporter: kriegaex Assignee: Christian Schlichtherle
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 10, Java 8



 Description   
    TPath tPath = new TPath("C:/Users/Alexander/Downloads/PowerShellArsenal-master.zip");
    for (Path path : tPath.toRealPath()) {}

Please note `toRealPath()`, basically turning the pure file system path into an URI-like path like file:/C:/Users/Alexander/...

Exception in thread "main" java.lang.IllegalArgumentException: de.schlichtherle.truezip.util.QuotedUriSyntaxException: Relative path with non-empty prefix.: "C:"
	at de.schlichtherle.truezip.nio.file.TPath.name(TPath.java:395)
	at de.schlichtherle.truezip.nio.file.TPath.<init>(TPath.java:157)
	at de.schlichtherle.truezip.nio.file.TPath$SegmentIterator.next(TPath.java:883)
	at de.schlichtherle.truezip.nio.file.TPath$SegmentIterator.next(TPath.java:869)
	at de.scrum_master.app.Application.main(Application.java:19)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

What is also a bit ugly is the fact that if I want to use the TPath instances returned by the iterator, I have to cast them from Path to TPath. It would be nice to also have a method returning a TPath iterator.






[TRUEZIP-373] Zip files corrupting with Norwegian characters Created: 30/Jan/17  Updated: 02/Feb/17  Resolved: 30/Jan/17

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: mital_highq Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As per solution given in
https://java.net/jira/browse/TRUEZIP-371
I have upgraded truezip 7.7.10 but still character " Ø " gets changed by " ¥ ".

Please download the zip file from below link and try to upload and check the behavior.
https://highq.in/wi474u2jg

Please let me know what result are you getting after uploading this zip.



 Comments   
Comment by Christian Schlichtherle [ 30/Jan/17 ]

Please, do not use the ticket system for general support. There is a mailing list for that: https://truezip.java.net/mail-lists.html .

The default charset for writing ZIP files is IBM-437, but you can change it to UTF-8 using the two parameter constructor of the ZipDriver class. As per https://truezip.java.net/faq.html#installDriverFile, here's an example for how to install it:

TConfig.get().setArchiveDetector(
        new TArchiveDetector(
            TArchiveDetector.NULL,
            "zip", new ZipDriver(IOPoolLocator.SINGLETON, StandardCharsets.UTF_8)));

Note that you shouldn't use anything else than IBM-437 or UTF-8 or it will not work with third party tools.

You can play around with the encodings on the command line, too. Here's what happens if you put non IBM-437 characters into a ZIP file:

$ java -jar ~/.m2/repository/de/schlichtherle/truezip/truezip-samples/7.7.10/truezip-samples-7.7.10-jar-with-dependencies.jar touch test.zip/hellØ
de.schlichtherle.truezip.fs.FsArchiveFileSystemException: hellØ (java.io.CharConversionException: hellØ (entry name not encodable with IBM437))
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem.newCheckedEntry(FsArchiveFileSystem.java:317)
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem.access$200(FsArchiveFileSystem.java:45)
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem$PathLink.newSegmentLinks(FsArchiveFileSystem.java:441)
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem$PathLink.<init>(FsArchiveFileSystem.java:416)
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem.mknod(FsArchiveFileSystem.java:390)
	at de.schlichtherle.truezip.fs.FsBasicArchiveController.mknod(FsBasicArchiveController.java:326)
	at de.schlichtherle.truezip.fs.FsContextController.mknod(FsContextController.java:178)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsCacheController.mknod(FsCacheController.java:157)
	at de.schlichtherle.truezip.fs.FsSyncController.mknod(FsSyncController.java:201)
	at de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:207)
	at de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:204)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
	at de.schlichtherle.truezip.fs.FsLockController.mknod(FsLockController.java:211)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:431)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:426)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.mknod(FsFalsePositiveArchiveController.java:436)
	at de.schlichtherle.truezip.file.TFile.createNewFile(TFile.java:2381)
	at de.schlichtherle.truezip.sample.file.app.Nzip.touch(Nzip.java:422)
	at de.schlichtherle.truezip.sample.file.app.Nzip.runChecked(Nzip.java:166)
	at de.schlichtherle.truezip.sample.file.app.Application.work(Application.java:93)
	at de.schlichtherle.truezip.file.TApplication.run(TApplication.java:59)
	at de.schlichtherle.truezip.sample.file.app.Nzip.main(Nzip.java:122)
Caused by: java.io.CharConversionException: hellØ (entry name not encodable with IBM437)
	at de.schlichtherle.truezip.fs.FsCharsetArchiveDriver.assertEncodable(FsCharsetArchiveDriver.java:94)
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem.newCheckedEntry(FsArchiveFileSystem.java:314)
	... 27 more

You should get that from your code, too. But you can "trick the system" by using test.jar instead and then rename it to test.zip:

$ java -jar ~/.m2/repository/de/schlichtherle/truezip/truezip-samples/7.7.10/truezip-samples-7.7.10-jar-with-dependencies.jar touch test.jar/hellØ
$ mv test.jar test.zip 
$ java -jar ~/.m2/repository/de/schlichtherle/truezip/truezip-samples/7.7.10/truezip-samples-7.7.10-jar-with-dependencies.jar ls test.zip 
hellØ

This works because if you use UTF-8 when writing the ZIP entry (which is the default for JAR files), then a flag gets set in the file which marks the entry as using UTF-8, so the setting "survives" renaming the file. This should work with modern third party tools, too, but may fail with older software. Here is an example:

$ unzip -l test.zip 
Archive:  test.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  01-30-2017 15:51   hell+?
---------                     -------
        0                     1 file
$ unzip
UnZip 6.00 of 20 April 2009, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Usage: unzip [-Z] [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]
  Default action is to extract files in list, except those in xlist, to exdir;
  file[.zip] may be a wildcard.  -Z => ZipInfo mode ("unzip -Z" for usage).

  -p  extract files to pipe, no messages     -l  list files (short format)
  -f  freshen existing files, create none    -t  test compressed archive data
  -u  update files, create if necessary      -z  display archive comment only
  -v  list verbosely/show version info       -T  timestamp archive to latest
  -x  exclude files that follow (in xlist)   -d  extract files into exdir
modifiers:
  -n  never overwrite existing files         -q  quiet mode (-qq => quieter)
  -o  overwrite files WITHOUT prompting      -a  auto-convert any text files
  -j  junk paths (do not make directories)   -aa treat ALL files as text
  -C  match filenames case-insensitively     -L  make (some) names lowercase
  -X  restore UID/GID info                   -V  retain VMS version numbers
  -K  keep setuid/setgid/tacky permissions   -M  pipe through "more" pager
See "unzip -hh" or unzip.txt for more help.  Examples:
  unzip data1 -x joe   => extract all files except joe from zipfile data1.zip
  unzip -p foo | more  => send contents of foo.zip via pipe into program more
  unzip -fo foo ReadMe => quietly replace existing ReadMe if archive file newer

As you can see, Info-ZIP doesn't support this. To avoid problems like this, it's generally recommended to use JAR files if UTF-8 encoding is required.

Comment by mital_highq [ 01/Feb/17 ]

Hi Christian,

I was not aware about mailing system that is why I was using ticket system. Sorry for that.

I have a question on this same ticket issue so commenting here ,

Above you have mentioned to use ,

"TConfig.get().setArchiveDetector(
new TArchiveDetector(
TArchiveDetector.NULL,
"zip", new ZipDriver(IOPoolLocator.SINGLETON, StandardCharsets.UTF_8)));"

  • but I cannot find two parameterized constructor of ZipDriver in latest version 7.7.10 also. So I am not able to add "StandardCharsets.UTF_8" charset .

Kindly help.

Thank you.

Comment by Christian Schlichtherle [ 01/Feb/17 ]

I am sorry, it's been a while since I've last worked on this code base. The two-parameter constructor is protected und subclassed by the JarDriver. So you can either subclass ZipDriver to get access to it or use the JarDriver directly.

Comment by mital_highq [ 02/Feb/17 ]

Thanks a lot Christian ,

I tried this way too using UTF-8 charset , but problem is not resolved.

My file name is "TESTÅØÆ.zip" .

  • Here if I am using UTF-8 charset than Å , Ø , Æ three of this character replaced with junk character , I checked three of these characters are listed under UTF-8 character list. Still its not showing them as it is.
  • And if I am using IBM437 or CP437 , than Å , Æ works correctly but " Ø " gets replaced by " ¥ ".

Thank you

Comment by Christian Schlichtherle [ 02/Feb/17 ]

If the name of the ZIP file itself gets mangled up, then this has nothing to do with TrueZIP. The character encodings used by the different drivers (ZipDriver, JarDriver et al) is only used for the file names of the entries within a ZIP file. This is shown in the third code example above: "test.jar/hellØ"

Comment by mital_highq [ 02/Feb/17 ]

Sorry for the confusion ,

My file name is "TESTÅØÆ.zip/TESTÅØÆ.txt" . Actually my file name gets mangled up as I mentioned above.

Comment by Christian Schlichtherle [ 02/Feb/17 ]
$ echo Hello world! > file
$ java -jar ~/.m2/repository/de/schlichtherle/truezip/truezip-samples/7.7.10/truezip-samples-7.7.10-jar-with-dependencies.jar cp -utf8out file TESTÅØÆ.zip/TESTÅØÆ.txt
$ java -jar ~/.m2/repository/de/schlichtherle/truezip/truezip-samples/7.7.10/truezip-samples-7.7.10-jar-with-dependencies.jar cat TESTÅØÆ.zip/TESTÅØÆ.txt
Hello world!

As you can see, it works as designed. Please have a look at the Nzip class if you're interested how this works.

Comment by mital_highq [ 02/Feb/17 ]

Hi Christian,

I have not hands on experience on UNIX command line but I understood your above implementation , where you have applied -utf8out charset for encoding. I am implementing UnZip functionality in JAVA. Can you please try with my below JAVA file and suggest me what is wrong with code.

================================================ START =============================================================================

import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import de.schlichtherle.truezip.file.TArchiveDetector;
import de.schlichtherle.truezip.file.TConfig;
import de.schlichtherle.truezip.file.TFile;
import de.schlichtherle.truezip.fs.archive.zip.ZipDriver;
import de.schlichtherle.truezip.socket.IOPoolProvider;
import de.schlichtherle.truezip.socket.sl.IOPoolLocator;

public class ZipUploaderNew extends ZipDriver
{
protected ZipUploaderNew(IOPoolProvider provider, Charset charset)

{ super(provider, charset); // TODO Auto-generated constructor stub }

public static void main(String[] args) throws IOException

{ TConfig.get().setArchiveDetector(new TArchiveDetector(TArchiveDetector.NULL, "zip", new ZipUploaderNew(IOPoolLocator.SINGLETON, StandardCharsets.UTF_8))); Extract(new String("D:/zip-data/new/TESTÅØÆ-mital.zip"), new String("D:/Source/Extract Result/zip"), ""); }

private static void Extract(String src, String dst, String incPath)
{
TFile srcFile = new TFile(src);
TFile dstFile = new TFile(dst);
try

{ TFile.cp_rp(srcFile, dstFile, TArchiveDetector.NULL); }

catch (IOException e)
{
}
}
}

========================================== END ====================================================================

Thank you.

Comment by Christian Schlichtherle [ 02/Feb/17 ]

a) The ticket system is for tracking issues, not for support enquiries.
b) Please start with a project generated from a Maven archetype and RTFM: https://truezip.java.net/kick-start/index.html .
c) Never suppress exceptions. You may get one if your class path setup is wrong for example.





Uploading Zip files corrupting with Norwegian characters (TRUEZIP-371)

[TRUEZIP-372] Not able to Attach File So writing the Norwegian characters in descrription Created: 24/Jan/17  Updated: 25/Jan/17  Resolved: 25/Jan/17

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Sub-task Priority: Critical
Reporter: mital_highq Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Example - TESTÅØÆ.txt

Create ZIP of this file and try to upload.






[TRUEZIP-371] Uploading Zip files corrupting with Norwegian characters Created: 24/Jan/17  Updated: 25/Jan/17  Resolved: 24/Jan/17

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: mital_highq Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Sub-Tasks:
Key
Summary
Type
Status
Assignee
TRUEZIP-372 Not able to Attach File So writing th... Sub-task Closed Christian Schlichtherle  

 Description   
  • I am using truezip 7.6 version for uploading ZIP file.
  • I am trying to upload Zip folder which has one text file and file name contains "Norwegian characters" , its get uploaded but with some junk character.
  • Here I have attached file to test.


 Comments   
Comment by Christian Schlichtherle [ 24/Jan/17 ]

File names in ZIP files are effectively restricted to ISO8859-1. If you want UTF-8, use a JAR file.

Note that there is a way to override this configuration in TrueZIP, but it's discouraged because it inhibits interoperability with other software.

Comment by mital_highq [ 24/Jan/17 ]

Thanks for the quick response.

So as per your comment , If I just have to use ZIP file than I am just restricted with ISO8859-1 , I can not use UTF-8 supported characters . My application where I have used this functionality is restricted to ZIP upload only. So as per you there is no way to use UTF-8 with ZIP files.

Comment by Christian Schlichtherle [ 25/Jan/17 ]

I am sorry, but I was remembering this wrong: In fact, newer versions of the ZIP File Format Specification do support UTF-8. TrueZIP supports this, too, but you should upgrade to the latest version 7.7.10. If it still doesn't work, then it's because the third party software does not support this. Sorry for the confusion again.





[TRUEZIP-370] Using Nzip command line utility, how to unzip entire zip file (need syntax)? Created: 23/Oct/16  Updated: 23/Oct/16

Status: Open
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Critical
Reporter: cp10000 Assignee: Christian Schlichtherle
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The Nzip command line utility is great for getting high-levels Truezip commands. However, it's use is not always intuitive, despite the best effort.

How can I uncompress (unzip) an entire zip file to a local directory? Need concrete syntax with example.






[TRUEZIP-369] Files.deleteIfExists(TPath) throws an exception if the file does not exist. Created: 27/Sep/16  Updated: 27/Sep/16

Status: Open
Project: TrueZIP
Component/s: TrueZIP Path
Affects Version/s: TrueZIP 7.7.10
Fix Version/s: None

Type: Bug Priority: Major
Reporter: cyberiantiger Assignee: Christian Schlichtherle
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As per the subject.

Exception follows:

java.io.UncheckedIOException: de.schlichtherle.truezip.fs.FsArchiveFileSystemException: test.added (archive entry does not exist)
	at de.schlichtherle.truezip.fs.FsArchiveFileSystem.unlink(FsArchiveFileSystem.java:543)
	at de.schlichtherle.truezip.fs.FsBasicArchiveController.unlink(FsBasicArchiveController.java:338)
	at de.schlichtherle.truezip.fs.FsContextController.unlink(FsContextController.java:192)
	at de.schlichtherle.truezip.fs.FsDecoratingController.unlink(FsDecoratingController.java:126)
	at de.schlichtherle.truezip.fs.FsCacheController.unlink(FsCacheController.java:168)
	at de.schlichtherle.truezip.fs.FsSyncController.unlink(FsSyncController.java:217)
	at de.schlichtherle.truezip.fs.FsLockController$1Unlink.call(FsLockController.java:222)
	at de.schlichtherle.truezip.fs.FsLockController$1Unlink.call(FsLockController.java:219)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
	at de.schlichtherle.truezip.fs.FsLockController.unlink(FsLockController.java:226)
	at de.schlichtherle.truezip.fs.archive.zip.KeyController.unlink(KeyController.java:100)
	at de.schlichtherle.truezip.fs.FsDecoratingController.unlink(FsDecoratingController.java:126)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Unlink.call(FsFalsePositiveArchiveController.java:449)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Unlink.call(FsFalsePositiveArchiveController.java:444)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.unlink(FsFalsePositiveArchiveController.java:471)
	at de.schlichtherle.truezip.nio.file.TFileSystem.delete(TFileSystem.java:369)
	at de.schlichtherle.truezip.nio.file.TPath.delete(TPath.java:971)
	at de.schlichtherle.truezip.nio.file.TFileSystemProvider.delete(TFileSystemProvider.java:320)
	at java.nio.file.spi.FileSystemProvider.deleteIfExists(FileSystemProvider.java:739)
	at java.nio.file.Files.deleteIfExists(Files.java:1165)





[TRUEZIP-368] Files.walk(TPath) causes an UnsupportedOperationException Created: 27/Sep/16  Updated: 27/Mar/17

Status: Open
Project: TrueZIP
Component/s: TrueZIP Path
Affects Version/s: TrueZIP 7.7.10
Fix Version/s: None

Type: Bug Priority: Major
Reporter: cyberiantiger Assignee: Christian Schlichtherle
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Files.walk() requires that TFileSystem.FsEntryAttributes.fileKey() is implemented (at least on Oracle JDK 1.8).

Please fix.

Exception:

java.lang.UnsupportedOperationException: Not supported yet.
	at de.schlichtherle.truezip.nio.file.TFileSystem$FsEntryAttributes.fileKey(TFileSystem.java:527)
	at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:310)
	at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:72)
	at java.nio.file.Files.walk(Files.java:3574)
	at java.nio.file.Files.walk(Files.java:3625)


 Comments   
Comment by kriegaex [ 27/Mar/17 ]

This would come in handy when implementing filtering or matching algorithms, e.g. finding a file within an archive with a case-insensitive path, see e.g. http://stackoverflow.com/a/43010542/1082681.

Comment by kriegaex [ 27/Mar/17 ]

Another remark: When I copy the TPath source code into my own project and change the inner class method

private final class FsEntryAttributes implements BasicFileAttributes {
  // (...)
  @Override
  public Object fileKey() {
    throw new UnsupportedOperationException("Not supported yet.");
  }
}

into

private final class FsEntryAttributes implements BasicFileAttributes {
  // (...)
  @Override
  public Object fileKey() {
    return null;
  }
}

it works. The interface JavaDoc even says that a return value of null is fine. So even if you do not want to implement fileKey so as to return some unique file key, at least you can use the simple default implementation.





[TRUEZIP-367] Add "aar" suffix to TrueZIP Driver ZIP Created: 12/Apr/16  Updated: 13/Apr/16  Resolved: 13/Apr/16

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.7.9
Fix Version/s: None

Type: Bug Priority: Major
Reporter: loutente Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hello,

We would like to add the suffix "aar" on JarDriver.
aar is an axis archive suffix.

https://axis.apache.org/axis2/java/core/docs/adv-userguide.html



 Comments   
Comment by loutente [ 13/Apr/16 ]

Reading documentation helps.

TConfig.get().setArchiveDetector(new ArchiveDetector("aar|jar|war|ear", new JarDriver(IOPoolLocator.SINGLETON)));
Comment by Christian Schlichtherle [ 13/Apr/16 ]

Definitely!

Well, I guess adding "aar" as a default mapping would not be a good choice because this use of the JarDriver would be too niche, so I am going to close this issue. If you object, I would kindly ask you to discuss this on the users mailing list before reopening this ticket.

Comment by loutente [ 13/Apr/16 ]

Thanks for replying,
I finally solved my problem using this config

TConfig.get().setArchiveDetector(
				new TArchiveDetector(
						TArchiveDetector.ALL,
						"aar|ear|jar|war", new JarDriver(IOPoolLocator.SINGLETON)));

You can close the issue.





[TRUEZIP-365] RawZipOutputStream.setLevel fails for level = 0 Created: 06/Mar/16  Updated: 20/Mar/16  Resolved: 07/Mar/16

Status: Resolved
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.7.9
Fix Version/s: TrueZIP 7.7.10

Type: Bug Priority: Major
Reporter: jwleigh Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

All


Issue Links:
Cloners
is cloned by TRUEVFS-160 AbstractZipOutputStream.setLevel fail... Resolved
Duplicate
duplicates TRUEZIP-363 truezip7.7.x.jar is not supporting NO... Closed

 Description   

ZipOutputStream.setLevel() will only accept values between 1 and 9. Level 0 should be valid.

This affects TrueVFS as well.






[TRUEZIP-364] InvalidPathException: Illegal char <:> at index 58 Created: 23/Feb/16  Updated: 24/Feb/16  Resolved: 24/Feb/16

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.7
Fix Version/s: None

Type: Bug Priority: Major
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows XP



 Description   

The following stacktrace came in from a user. Not sure if this is a problem with TrueZIP. It looks like the user is accessing a Linux file system mounted on Windows XP.

program.name=DocFetcher
program.version=1.1.17
program.build=20160211-2357
program.portable=true
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.7.0_17-b02
java.version=1.7.0_17
sun.arch.data.model=32
os.arch=x86
os.name=Windows XP
os.version=5.1
user.language=en
java.nio.file.InvalidPathException: Illegal char <:> at index 58: Z:/home/gerardo/.local/share/wineprefixes/vlc/dosdevices/d:/.Trash-999/files/ADD TO LENOVO (from AM PC)/ADD TO LENOVO/RamCleaner v7.1 With Crack (SEE 500GB for COPY)/
at sun.nio.fs.WindowsPathParser.normalize(Unknown Source)
at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at sun.nio.fs.WindowsPath.parse(Unknown Source)
at sun.nio.fs.WindowsUriSupport.fromUri(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.getPath(Unknown Source)
at java.nio.file.Paths.get(Unknown Source)
at de.schlichtherle.truezip.fs.nio.file.FileController.<init>(FileController.java:73)
at de.schlichtherle.truezip.fs.nio.file.FileDriver.newController(FileDriver.java:28)
at de.schlichtherle.truezip.fs.FsDriver.newController(FsDriver.java:73)
at de.schlichtherle.truezip.fs.FsAbstractCompositeDriver.newController(FsAbstractCompositeDriver.java:35)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController0(FsDefaultManager.java:95)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController0(FsDefaultManager.java:93)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController(FsDefaultManager.java:78)
at de.schlichtherle.truezip.file.TFile.getController(TFile.java:1497)
at de.schlichtherle.truezip.file.TFile.getController(TFile.java:1492)
at de.schlichtherle.truezip.file.TVFS.mountPoint(TVFS.java:123)
at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:236)
at de.schlichtherle.truezip.file.TVFS.umount(TVFS.java:83)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.runFinally(FileIndex.java:431)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:59)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:393)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:145)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:57)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:282)
at net.sourceforge.docfetcher.model.index.file.FileIndex.doUpdate(FileIndex.java:159)
at net.sourceforge.docfetcher.model.TreeIndex.update(TreeIndex.java:148)
at net.sourceforge.docfetcher.model.index.Task.update(Task.java:98)
at net.sourceforge.docfetcher.model.index.IndexingQueue.threadLoop(IndexingQueue.java:193)
at net.sourceforge.docfetcher.model.index.IndexingQueue.access$100(IndexingQueue.java:46)
at net.sourceforge.docfetcher.model.index.IndexingQueue$2.run(IndexingQueue.java:118)



 Comments   
Comment by Christian Schlichtherle [ 24/Feb/16 ]

The colon is a path separator on Unix and therefore it's truly invalid. The exception is raised by the JRE, not TrueZIP, BTW.





[TRUEZIP-363] truezip7.7.x.jar is not supporting NO_COMPRESSION Created: 19/Nov/15  Updated: 20/Mar/16  Resolved: 20/Mar/16

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP, TrueZIP File*
Affects Version/s: TrueZIP 7.7.6, TrueZIP 7.7.9
Fix Version/s: TrueZIP 7.7.10

Type: Bug Priority: Major
Reporter: venubwal Assignee: Christian Schlichtherle
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by TRUEZIP-365 RawZipOutputStream.setLevel fails for... Resolved

 Description   

truezip7.7.6 / 9.jar are not supporting NO_COMPRESSION
where as 6.8.4 version is supporting.
I see, there is a bug in 6.7 which was addressed in 6.8.
https://java.net/projects/truezip/lists/users/archive/2007-11/message/31
I verified in truezip-6.8.4 that support it.

We used have 6.8, We upgraded to support larger then 4gb archival size.
After upgrading to 7.7.6, setting NO_COMPRESSION level is stop working..
It throws 'Time_Invalid compression level'

Using 6.8xv for 338MB of images,
Default Compression (-1): taking 115 seconds
Best Speed (1): 105 seconds
No Compression (0):24 seconds

Hence, using 7.7.6, it cause huge perforamnce issues if I need to zip huge no of images which anyway wont zip/compressed.

1.Please suggest me what I should use
2.Is there any other(inbetween) version that support No_compression as-well as Large Zipping size
3.If I use v7.7.x, what is alternative better option for SPEED equals to NO_compression.
4.Is supporting it intentially removed.
5.If it is a bug, any other version has solution.
6.Please suggest me to have both solution






[TRUEZIP-362] Update dependencies Created: 03/Jun/15  Updated: 05/Jun/15  Resolved: 03/Jun/15

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: TrueZIP 7.7.9

Type: Task Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-361] `PaceController` does not call `PaceManagerController` if a file system operation fails with an `IOException` Created: 03/Jun/15  Updated: 05/Jun/15  Resolved: 03/Jun/15

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Extension PaceManager
Affects Version/s: TrueZIP 7.7, TrueZIP 7.7.1, TrueZIP 7.7.2, TrueZIP 7.7.3, TrueZIP 7.7.4, TrueZIP 7.7.5, TrueZIP 7.7.6, TrueZIP 7.7.7, TrueZIP 7.7.8
Fix Version/s: TrueZIP 7.7.9

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
clones TRUEVFS-147 `PaceController` does not call `PaceM... Closed

 Description   

For example, when calling TFile.exists() on a non-existing file, the file system controller operation fails with some `IOException`. In consequence, the `PaceManager` is not called and hence cannot enforce the `maximumFileSystemsMounted` property limit.






[TRUEZIP-360] "Enter key for writing" popup shouldn't be shown Created: 13/Mar/15  Updated: 20/Mar/16  Resolved: 20/Mar/16

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.7.8
Fix Version/s: None

Type: Bug Priority: Major
Reporter: lbinkiewicz Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I've noticed that "Enter key for writing" popup appears while it shoudn't.
Attached file should explain the issue.



 Comments   
Comment by lbinkiewicz [ 13/Mar/15 ]

package issue;

import java.io.File;
import java.io.IOException;
import java.io.Writer;
import java.nio.charset.Charset;

import de.schlichtherle.truezip.crypto.param.AesKeyStrength;
import de.schlichtherle.truezip.file.TArchiveDetector;
import de.schlichtherle.truezip.file.TConfig;
import de.schlichtherle.truezip.file.TFile;
import de.schlichtherle.truezip.file.TFileWriter;
import de.schlichtherle.truezip.fs.FsController;
import de.schlichtherle.truezip.fs.FsDriverProvider;
import de.schlichtherle.truezip.fs.FsModel;
import de.schlichtherle.truezip.fs.FsOutputOption;
import de.schlichtherle.truezip.fs.archive.zip.ZipDriver;
import de.schlichtherle.truezip.fs.archive.zip.ZipDriverEntry;
import de.schlichtherle.truezip.socket.sl.IOPoolLocator;
import de.schlichtherle.truezip.util.BitField;
import de.schlichtherle.truezip.zip.WinZipAesParameters;
import de.schlichtherle.truezip.zip.ZipCryptoParameters;
import de.schlichtherle.truezip.zip.ZipKeyException;

public class TrueZipTest {

final class CustomWinZipAesParameters implements WinZipAesParameters {
final byte[] password;

CustomWinZipAesParameters(final byte[] password)

{ this.password = password.clone(); }

@Override
public byte[] getWritePassword(String name) throws ZipKeyException

{ return password.clone(); }

@Override
public byte[] getReadPassword(String name, boolean invalid)
throws ZipKeyException

{ if (invalid) throw new ZipKeyException(name + " (invalid password)"); return password.clone(); }

@Override
public AesKeyStrength getKeyStrength(String arg0)
throws ZipKeyException

{ return AesKeyStrength.BITS_256; }

@Override
public void setKeyStrength(String name, AesKeyStrength keyStrength)
throws ZipKeyException

{ // We have been using only 128 bits to create archive entries. assert AesKeyStrength.BITS_256 == keyStrength; }

}

final class CustomZipDriver extends ZipDriver {
final ZipCryptoParameters param;

public CustomZipDriver(byte[] password)

{ super(IOPoolLocator.SINGLETON); param = new CustomWinZipAesParameters(password); }

@Override
protected ZipCryptoParameters zipCryptoParameters(FsModel model,
Charset charset)

{ return param; }

@Override
public <M extends FsModel> FsController<M> decorate(
FsController<M> controller)

{ return controller; }

@Override
protected boolean process(ZipDriverEntry input, ZipDriverEntry output)

{ return false; }

}

private void setConfig1()

{ final TConfig config = TConfig.get(); FsDriverProvider provider = new TArchiveDetector( TArchiveDetector.ALL.toString()); config.setOutputPreferences(config.getOutputPreferences().clear( FsOutputOption.ENCRYPT)); config.setArchiveDetector(new TArchiveDetector(provider, ".zip")); }

private void setConfig2(String password)

{ final TConfig config = TConfig.get(); FsDriverProvider provider = new TArchiveDetector( TArchiveDetector.ALL.toString()); config.setOutputPreferences(config.getOutputPreferences().or( BitField.of(FsOutputOption.ENCRYPT))); config.setArchiveDetector(new TArchiveDetector(provider, ".zip", new CustomZipDriver(password.getBytes()))); }

private File createSampleFile(String dir, String zipFile, String innerFile)
throws IOException {
File f = new TFile(dir, zipFile);
File file = new TFile(f.getPath() + File.separator + innerFile);
Writer writer = new TFileWriter(file, true, Charset.forName("UTF-8"));
try

{ writer.write("TEST"); }

finally

{ writer.close(); }

return f;

}

public static void main(String[] args) throws IOException,
InterruptedException {
String workingDir = "/home/lukasz/ar1";
String zipName = "a.zip";
String innerFile = "a.txt";

TrueZipTest tzt = new TrueZipTest();

File finit = new File(workingDir, zipName);
if (finit.exists())

{ finit.delete(); System.out.println(finit.exists()); }

/*UNCOMMENT
tzt.setConfig2("test2");
File f0 = tzt.createSampleFile(workingDir, zipName, innerFile);
TFile.rm_r(f0);
System.out.println(f0.exists());
*/

// create archive without pass
tzt.setConfig1();
File f1 = tzt.createSampleFile(workingDir, zipName, innerFile);
TFile.rm_r(f1);
System.out.println(f1.exists());
//

// create archive with pass
tzt.setConfig2("test");
File f2 = tzt.createSampleFile(workingDir, zipName, innerFile);
TFile.rm_r(f2);
System.out.println(f2.exists());

// create archive with pass
tzt.setConfig2("test2");
File f3 = tzt.createSampleFile(workingDir, zipName, innerFile);
TFile.rm_r(f3);
System.out.println(f3.exists());

tzt.setConfig1();
File f4 = tzt.createSampleFile(workingDir, zipName, innerFile);
TFile.rm_r(f4);
System.out.println(f4.exists());

}

}

Comment by lbinkiewicz [ 13/Mar/15 ]

This issue occurs only while first file is created as not encrypted, If You uncomment this section:
/*UNCOMMENT
tzt.setConfig2("test2");
File f0 = tzt.createSampleFile(workingDir, zipName, innerFile);
TFile.rm_r(f0);
System.out.println(f0.exists());
*/
There is no popup displayed.

Comment by Christian Schlichtherle [ 20/Mar/16 ]

This is not a bug, the sample program violates the Third Party Access constraints (https://truezip.java.net/truezip-file/usage.html#Third_Party_Access): Your custom ZipDriver gets associated with different TArchiveDetector instances for the same archive file. In general, this change would only get effective if...

1. You would unmount the archive file using TVFS.umount( * ). Actually, your call to TFile.rm_r( * ) has the same effect.
2. You would make all TFile* objects referencing this archive file eligible for garbage collection.
3. A garbage collection is actually performed.

Because this is too much if-and-but to be of practical value, the rule of thumb is: Don't change the configuration for the same set of archive files.

In your case, you want to change the password for the archive file. The best way of doing this is to write a custom KeyManager or a custom view for the built-in PromptingKeyManager. An example of the latter is provided here: https://truezip.java.net/truezip-driver/truezip-driver-zip/key-management.html#Setting_Passwords_For_All_WinZip_AES_Entries_By_Implementing_A_Custom_View . This is the second example on the page from which you have copied your custom ZipDriver.





[TRUEZIP-359] Refactor away from deprecated code Created: 22/Jan/15  Updated: 07/Feb/15  Resolved: 22/Jan/15

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.7
Fix Version/s: TrueZIP 7.7.8

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-358] Update dependencies and plugins Created: 22/Jan/15  Updated: 07/Feb/15  Resolved: 22/Jan/15

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.7
Fix Version/s: TrueZIP 7.7.8

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
clones TRUEVFS-127 Update dependencies and plugins Closed
Dependency
depends on TRUEZIP-357 Refactor code to depend on Bouncy Cas... Closed




[TRUEZIP-357] Refactor code to depend on Bouncy Castle Provider 1.51 Created: 22/Jan/15  Updated: 07/Feb/15  Resolved: 22/Jan/15

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP, TrueZIP Driver ZIP.RAES (TZP)
Affects Version/s: TrueZIP 7.7.7
Fix Version/s: TrueZIP 7.7.8

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
clones TRUEVFS-134 Refactor code to depend on Bouncy Cas... Closed
Dependency
blocks TRUEZIP-358 Update dependencies and plugins Closed




[TRUEZIP-356] Wrong parameter parsing for cp and mv commands in the NZip class Created: 22/Jan/15  Updated: 07/Feb/15  Resolved: 22/Jan/15

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Samples
Affects Version/s: TrueZIP 7.7.7
Fix Version/s: TrueZIP 7.7.8

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
clones TRUEVFS-140 Wrong parameter parsing for TrueVFS.C... Closed

 Description   

Destination path names need to be expanded if and only if the last parameter is a directory, not when it's a directory or names a (possibly non-existent) archive file.






[TRUEZIP-354] Some integration tests hang in the TrueZIP File* and TrueZIP Path modules Created: 09/Jan/15  Updated: 07/Feb/15  Resolved: 22/Jan/15

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*, TrueZIP Path
Affects Version/s: TrueZIP 7.7.7
Fix Version/s: TrueZIP 7.7.8

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
clones TRUEVFS-131 Some integration tests hang in the Tr... Closed




[TRUEZIP-352] Web site links to Road Map and Change Log are broken Created: 21/Nov/14  Updated: 07/Feb/15  Resolved: 21/Nov/14

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.6
Fix Version/s: TrueZIP 7.7.7

Type: Bug Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
is cloned by TRUEVFS-126 CLONE - Web site links to Road Map an... Closed




[TRUEZIP-349] Tree.scala contains a name conflict with new methods in the Collections API in Java SE 8 Created: 17/Nov/14  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Archetype Path
Affects Version/s: TrueZIP 7.7.6
Fix Version/s: TrueZIP 7.7.7

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
is cloned by TRUEVFS-124 CLONE - Tree.scala contains a name co... Closed




[TRUEZIP-348] TrueZIP Archetype File does not correctly resolve Scala version Created: 17/Nov/14  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Archetype File*
Affects Version/s: TrueZIP 7.7.6
Fix Version/s: TrueZIP 7.7.7

Type: Bug Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-347] CLONE - TAR Big Numeric Values Created: 17/Nov/14  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR
Affects Version/s: TrueZIP 7.7.6
Fix Version/s: TrueZIP 7.7.7

Type: New Feature Priority: Major
Reporter: edmann Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Java 7 or 8


Issue Links:
Cloners
clones TRUEVFS-122 TAR Big Numeric Values Closed

 Description   

This new Feature will be for the TarDriver and allow when creating the tar file the ability to set the bigNumberMode option of TarArchiveOutputStream.
http://commons.apache.org/proper/commons-compress/tar.html is the link for details on bigNumberMode.

This feature will allow TrueVFS to support larger than 8Gb archive files.



 Comments   
Comment by Christian Schlichtherle [ 17/Nov/14 ]

Please don't comment on this ticket. See the comments on the original ticket.





[TRUEZIP-346] Can't create a ZIP Archive if it's direct parent is a symlink Created: 21/Aug/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE
Affects Version/s: TrueZIP 7.7.5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: zet Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux



 Description   

When i try to create a ZIP file for a folder of stuff it fails if the parent of the ZIP file i want to create is a symbolic link:
TFile srcFolder = ...;
TFile dstZipFile = ...;
srcFolder.cp_rp(dstZipFile);

will result into:
java.nio.file.FileAlreadyExistsException: <path to parent, last path element is a symlink to a directory>

this is the stacktrace:

sun.nio.fs.UnixException.translateToIOException(UnixException.java:88)
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:383)
java.nio.file.Files.createDirectory(Files.java:628)
java.nio.file.Files.createAndCheckIsDirectory(Files.java:732)
java.nio.file.Files.createDirectories(Files.java:681)
de.schlichtherle.truezip.fs.nio.file.FileOutputSocket.begin(FileOutputSocket.java:109)
de.schlichtherle.truezip.fs.nio.file.FileOutputSocket.newOutputStream(FileOutputSocket.java:222)
de.schlichtherle.truezip.fs.archive.zip.OptionOutputSocket.newOutputStream(OptionOutputSocket.java:48)
de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop(ZipDriver.java:586)
de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop0(ZipDriver.java:576)
de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop(ZipDriver.java:561)
de.schlichtherle.truezip.fs.FsTargetArchiveController.makeOutputArchive(FsTargetArchiveController.java:247)
de.schlichtherle.truezip.fs.FsTargetArchiveController.mount0(FsTargetArchiveController.java:182)
de.schlichtherle.truezip.fs.FsTargetArchiveController.mount(FsTargetArchiveController.java:155)
de.schlichtherle.truezip.fs.FsFileSystemArchiveController$ResetFileSystem.autoMount(FsFileSystemArchiveController.java:85)
de.schlichtherle.truezip.fs.FsFileSystemArchiveController.autoMount(FsFileSystemArchiveController.java:37)
de.schlichtherle.truezip.fs.FsBasicArchiveController.mknod(FsBasicArchiveController.java:319)
de.schlichtherle.truezip.fs.FsContextController.mknod(FsContextController.java:178)
de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
de.schlichtherle.truezip.fs.FsCacheController.mknod(FsCacheController.java:157)
de.schlichtherle.truezip.fs.FsSyncController.mknod(FsSyncController.java:201)
de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:207)
de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:204)
de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
de.schlichtherle.truezip.fs.FsLockController.mknod(FsLockController.java:211)
de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:431)
de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:426)
de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.mknod(FsFalsePositiveArchiveController.java:436)
de.schlichtherle.truezip.file.TFile.mkdir(TFile.java:2425)
de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:157)
de.schlichtherle.truezip.file.TBIO.cp_r(TBIO.java:138)
de.schlichtherle.truezip.file.TFile.cp_rp(TFile.java:3188)

The problem described in TRUEZIP-339 sounds quite simular, but an upgrade to 7.7.6 did not solve my problem.



 Comments   
Comment by zet [ 21/Aug/14 ]

A possible workaround is to disable the "create parents" feature:
TConfig config= TConfig.push();
config.setOutputPreferences(config.getOutputPreferences().clear(FsOutputOption.CREATE_PARENTS));

Comment by Christian Schlichtherle [ 21/Aug/14 ]

I suspect you are using the API incorrectly. dstZipFile needs to be a non-existing path, but apparently it already exists, which is what the exception indicates. Mind you that the TrueZIP API does not auto-complete paths like many shell programs (e.g. cp, mv, ln) do. This is to avoid ambiguities. So the usage pattern should look like this:

TFile src = new TFile("directory"); // this needs to exist
TFile dst = new TFile("archive.zip"); // this must not exist
src.cp_rp(dst);

Comment by zet [ 21/Aug/14 ]

I am using it exaktly as you describe it, but it does not work when the parent of dst is a symlink:

TFile src = new TFile("directory"); // this does exist
TFile dst = new TFile("symlink/archive.zip"); // symlink exists and points to an existing directory, archive.zip does not exist in that directory
src.cp_rp(dst)

This does work:
TFile src = new TFile("directory"); // this does exist
TFile dst = new TFile("symlink/somesubdir/archive.zip"); // symlink exists and points to an existing directory, somesubdir exists and archive.zip does not exist in that directory
src.cp_rp(dst)

Comment by Christian Schlichtherle [ 21/Aug/14 ]

I suspect you need to update your JRE. I tried to reproduce your problem on Mac OS X 10.9.4 with all of the following JDKs and it worked flawlessly:

<pre><code>
$ /usr/libexec/java_home -V
Matching Java Virtual Machines (4):
1.8.0_11, x86_64: "Java SE 8" /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home
1.7.0_65, x86_64: "Java SE 7" /Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home
1.6.0_65-b14-462, x86_64: "Java SE 6" /Library/Java/JavaVirtualMachines/1.6.0_65-b14-462.jdk/Contents/Home
1.6.0_65-b14-462, i386: "Java SE 6" /Library/Java/JavaVirtualMachines/1.6.0_65-b14-462.jdk/Contents/Home
</code></pre>





[TRUEZIP-345] Paths containing symbolic links (outside of zip archive) are not resolved when calling exists() on TFile object Created: 07/May/14  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*
Affects Version/s: TrueZIP 7.7.6
Fix Version/s: None

Type: Bug Priority: Major
Reporter: borgille Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 3.2.0-48-generic #74-Ubuntu SMP Thu Jun 6 19:43:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux


Tags: symlink

 Description   

It seems that TFile doesn't resolve symbolic links within a path when calling exists() or other file attribute methods. This appears to be inconsistent with java.io.File which will confirm the existence of a file having a symbolic link in its path.

I confirmed this using the Pickr example in the TrueZIP archetype. For example, if I perform the following in the example's root directory:

mkdir -p some/directory
cp test.zip some/directory/
ln -s some/directory linktodir

and then I start the Pickr application:

mvn exec:java -Dexec.mainClass=com.gxs.Pickr

I am able to navigate in the diaglog to the test.zip file (i.e. linktodir/test.zip), but when I try to navigate into the test.zip folder, I get the following exception:

Exception occurred during event dispatching:
java.lang.IllegalArgumentException: URI is not hierarchical
at java.io.File.<init>(File.java:363)
at sun.awt.shell.ShellFolder.getNormalizedFile(ShellFolder.java:257)
at javax.swing.plaf.basic.BasicFileChooserUI$Handler.mouseClicked(BasicFileChooserUI.java:421)
at sun.swing.FilePane$Handler.mouseClicked(FilePane.java:1713)
at java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:253)
at java.awt.Component.processMouseEvent(Component.java:6300)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3275)
at java.awt.Component.processEvent(Component.java:6062)
at java.awt.Container.processEvent(Container.java:2039)
at java.awt.Component.dispatchEventImpl(Component.java:4660)
at java.awt.Container.dispatchEventImpl(Container.java:2097)
at java.awt.Component.dispatchEvent(Component.java:4488)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4575)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4245)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4166)
at java.awt.Container.dispatchEventImpl(Container.java:2083)
at java.awt.Window.dispatchEventImpl(Window.java:2489)
at java.awt.Component.dispatchEvent(Component.java:4488)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:674)
at java.awt.EventQueue.access$400(EventQueue.java:81)
at java.awt.EventQueue$2.run(EventQueue.java:633)
at java.awt.EventQueue$2.run(EventQueue.java:631)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:98)
at java.awt.EventQueue$3.run(EventQueue.java:647)
at java.awt.EventQueue$3.run(EventQueue.java:645)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:644)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:178)
at java.awt.Dialog$1.run(Dialog.java:1052)
at java.awt.Dialog$3.run(Dialog.java:1104)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.Dialog.show(Dialog.java:1102)
at javax.swing.JFileChooser.showDialog(JFileChooser.java:723)
at com.gxs.Pickr$1.run(Pickr.java:30)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:199)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:672)
at java.awt.EventQueue.access$400(EventQueue.java:81)
at java.awt.EventQueue$2.run(EventQueue.java:633)
at java.awt.EventQueue$2.run(EventQueue.java:631)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:642)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

I did notice that I can call (new TFile("linktodir/test.zip")).getCanonicalFile().exists() as a workaround for now, but as I mentioned earlier, using getCanonicalFile to resolve the symbolic links isn't necessary with java.io.File.



 Comments   
Comment by borgille [ 07/May/14 ]

I played around with this a bit more, and it seems that TFile is working even with the symbolic link. I have tried to reproduce the original error, but haven't been able to do so, and I'm unable to explain why it was failing for me originally.

I still get the Exception for the Pickr example, however.

Comment by Christian Schlichtherle [ 17/Nov/14 ]

I cannot reproduce this issue. I tried the Pickr app on Mac OS X using Java SE 6, 7 and 8 and it worked on all platforms. Maybe you should update your JRE? I know that older versions had quite some bugs in the java.nio package and the JFileChooser class.





[TRUEZIP-344] TFile.listFiles does not work correctly for archives containing entries with absolute paths Created: 29/Apr/14  Updated: 07/Feb/15  Resolved: 21/Nov/14

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.5, TrueZIP 7.7.6
Fix Version/s: TrueZIP 7.7.7

Type: Bug Priority: Minor
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
is cloned by TRUEVFS-125 CLONE - TFile.listFiles does not work... Closed

 Description   

Hi,

On zip archives containing entries with absolute paths, the method TFile.listFiles returns the archive itself rather than the entries in the archive. Here's some sample code:

package test;

import de.schlichtherle.truezip.file.TFile;

public class Test {

	public static void main(String[] args) {
		String path = "C:\\test.zip";
		TFile file = new TFile(path);
		for (TFile child : file.listFiles()) {
			System.out.println(child.getAbsolutePath());
		}
	}

}

The output of the above program is "C:\test.zip" rather than a path pointing to the file inside the archive.

listFiles returning the archive itself has the unfortunate effect of crashing the program with a StackOverflowError, if the program traverses the file system recursively and comes across an archive with absolute entries.

Best regards
Quang



 Comments   
Comment by qforce [ 29/Apr/14 ]

Note: I have the "test.zip" file that is used in the code above, but I can't find a way to attach it to this bug report. If you need the test file, just tell me where to send it to.

Comment by qforce [ 29/Apr/14 ]

Another remark: The above problem was found on Windows, but not on Linux.

Comment by Christian Schlichtherle [ 29/Apr/14 ]

Entries with absolute path names should be ignored and there should be no way you can address them because they are considered to be a security threat. So if "C:
test.zip" contains nothing but entries with absolute path names, then .listFiles() should return an empty array, just as if the app would encounter an empty directory. Can you verify this behavior with TrueZIP 7.7.6 please or change the initial path to be relative? I suspect this is a Windows specific issue.

Please add the ZIP file in question. You can attach it in the Operations menu on the left.

Comment by qforce [ 29/Apr/14 ]

Can you verify this behavior with TrueZIP 7.7.6 please or change the initial path to be relative?

If I put the test file in the current directory and set the initial path to "test.zip", the program's output is still the test file itself:

C:\Documents and Settings\Nam-Quang Tran\workspace\Test\test.zip

TrueZIP 7.7.6 gives exactly the same output as TrueZIP 7.7.5.

Please add the ZIP file in question. You can attach it in the Operations menu on the left.

I can't see any attachment-related actions in the Operations menu. Maybe I don't have the permission to add files. Anyway, here are the contents of the file encoded in base64:

UEsDBAoAAAAAACmGnUQAAAAAAAAAAAAAAAALAAAAQzovdGVzdC50eHRQSwECFAAKAAAAAAAphp1E
AAAAAAAAAAAAAAAACwAAAAAAAAAAACAAAAAAAAAAQzovdGVzdC50eHRQSwUGAAAAAAEAAQA5AAAA
KQAAAAAA
Comment by qforce [ 06/May/14 ]

Hi, is this bug report still going somewhere? Should I upload the test file elsewhere?

Comment by Christian Schlichtherle [ 07/May/14 ]

Sorry, I am just very busy these days.

Comment by qforce [ 07/May/14 ]

No problem. I just wanted to confirm that you weren't waiting for me to upload the test file, or something like that.





[TRUEZIP-343] Using TPath at least twice as slow as TFile when creating files before writing to them Created: 11/Mar/14  Updated: 07/Feb/15  Resolved: 11/Mar/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Archetype Path
Affects Version/s: TrueZIP 7.7.6
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: qui.tiqe Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Using TPath is at least twice as slow as TFile when each file in an archive is created before the contents is written. I've created a benchmark application for this based on a benchmark I found when searching through the mailing list (created by Gergely Csucs). The tests I've run using a profiler suggest that the difference is caused by calls to TempFilePool.createTempFile() and TempFilePool$Buffer.pool(). Is this the result of something that can be optimized or is it expected?

The results on my machine for the benchmark:

Start (TPath)
Completed writing (TPath): 28285
Finalized (TPath): 64513
Start (TFile)
Completed writing (TFile): 27540
Finalized (TFile): 27873

The code:

ZipBenchMark.java
package com.quintiq.vfs.test.manual;

import java.io.OutputStream;
import java.nio.file.Files;
import java.util.Random;

import de.schlichtherle.truezip.file.TFile;
import de.schlichtherle.truezip.file.TFileOutputStream;
import de.schlichtherle.truezip.file.TVFS;
import de.schlichtherle.truezip.nio.file.TPath;

public class ZipBenchMark
{
  private static final String FILE_TPATH = "testA.zip", FILE_TFILE = "testB.zip";
  
  public static void main(String args[]) throws Exception
  {
    long start;
    Random r = new Random(0);
    byte bytes[] = new byte[65536];
    
    // Test TPath
    start = System.currentTimeMillis();
    TPath base = new TPath(FILE_TPATH);
    System.out.println("Start (TPath)");
    
    for (int i=0; i<10000; i++)
    {
      OutputStream os = Files.newOutputStream( Files.createFile( base.resolve("dump" + i + ".byt") ) );
      r.nextBytes(bytes);
      os.write(bytes);
      os.close();
    }
    
    System.out.println("Completed writing (TPath): " + (System.currentTimeMillis() - start));
    TVFS.umount();    
    System.out.println("Finalized (TPath): " + (System.currentTimeMillis() - start));
    
    // Test TFile
    start = System.currentTimeMillis();
    TFile file = new TFile(FILE_TFILE);
    System.out.println("Start (TFile)");
    
    for (int i=0; i<10000; i++)
    {
      TFile fileInArchive = new TFile(file, "dump" + i + ".byt");
      fileInArchive.createNewFile();
      
      OutputStream os = new TFileOutputStream(fileInArchive);
      r.nextBytes(bytes);
      os.write(bytes);
      os.close();
    }
    
    System.out.println("Completed writing (TFile): " + (System.currentTimeMillis() - start));
    TVFS.umount();
    System.out.println("Finalized (TFile): " + (System.currentTimeMillis() - start));
  }
}


 Comments   
Comment by Christian Schlichtherle [ 11/Mar/14 ]

There are several things to note first:

1) Please specify the OS version. The behavior of file systems and the JRE is quite different among Unix and Windows. I am running the following code on Mac OS X 10.9.2, Oracle JDK 1.7.0_51-b13.
2) The JVM optimizes hot spots at run time, so for a real benchmark please wrap the test in another loop to compensate for JVM warm up time.
3) In production, please make sure to ALWAYS close your streams, even in case of a throwable, e.g. within a finally block or with a try-with-resources statement.
4) For comparable results, you must delete the archive files before the test runs because TrueZIP behaves differently depending on whether the archive file already exists or not.

Now here is the simple solution: You have introduced superfluous calls to TFile.createNewFile() and Files.createFile(TPath). These calls have side effects, and in the case of Files.createFile(TPath), when looping over an archive file this results in quadratic runtime, which is what you were witnessing.

The following code is like yours except that I have removed these superfluous calls:

package com.company.product.mavenproject1.java;

import de.schlichtherle.truezip.file.TFile;
import de.schlichtherle.truezip.file.TFileOutputStream;
import de.schlichtherle.truezip.file.TVFS;
import de.schlichtherle.truezip.nio.file.TPath;
import java.io.OutputStream;
import java.nio.file.Files;
import java.util.Random;

public class ZipBenchMark
{
  private static final String FILE_TPATH = "testA.zip", FILE_TFILE = "testB.zip";

  public static void main(String args[]) throws Exception
  {
    long start;
    Random r = new Random(0);
    byte bytes[] = new byte[65536];

    // Test TPath
    start = System.currentTimeMillis();
    TPath base = new TPath(FILE_TPATH);
    System.out.println("Start (TPath)");

    for (int i=0; i<10000; i++)
    {
      OutputStream os = Files.newOutputStream(base.resolve("dump" + i + ".byt"));
      r.nextBytes(bytes);
      os.write(bytes);
      os.close();
    }

    System.out.println("Completed writing (TPath): " + (System.currentTimeMillis() - start));
    TVFS.umount();
    System.out.println("Finalized (TPath): " + (System.currentTimeMillis() - start));

    // Test TFile
    start = System.currentTimeMillis();
    TFile file = new TFile(FILE_TFILE);
    System.out.println("Start (TFile)");

    for (int i=0; i<10000; i++)
    {
      TFile fileInArchive = new TFile(file, "dump" + i + ".byt");
      OutputStream os = new TFileOutputStream(fileInArchive);
      r.nextBytes(bytes);
      os.write(bytes);
      os.close();
    }

    System.out.println("Completed writing (TFile): " + (System.currentTimeMillis() - start));
    TVFS.umount();
    System.out.println("Finalized (TFile): " + (System.currentTimeMillis() - start));
  }
}

On my machine, I get the following sample results:

Start (TPath)
Completed writing (TPath): 26980
Finalized (TPath): 27537
Start (TFile)
Completed writing (TFile): 24568
Finalized (TFile): 24966

To demonstrate the effect of the JVM warmup, when I swap the usage of the APIs, I get the following sample results:

Start (TFile)
Completed writing (TFile): 26320
Finalized (TFile): 26848
Start (TPath)
Completed writing (TPath): 24751
Finalized (TPath): 25121

To sum it up, there really isn't a noticeable performance difference between these APIs, but you need to be careful about the side effects of the API calls.

Comment by qui.tiqe [ 12/Mar/14 ]

Thank you for taking the time to reply to my original "issue".

"1) Please specify the OS version. The behavior of file systems and the JRE is quite different among Unix and Windows. I am running the following code on Mac OS X 10.9.2, Oracle JDK 1.7.0_51-b13."

I'm running JDK 7u51 on Windows 7.

"2) The JVM optimizes hot spots at run time, so for a real benchmark please wrap the test in another loop to compensate for JVM warm up time."

True for actual benchmarking, though in this case what happens after "TVFS.umount();" is the interesting part.

"3) In production, please make sure to ALWAYS close your streams, even in case of a throwable, e.g. within a finally block or with a try-with-resources statement."
"4) For comparable results, you must delete the archive files before the test runs because TrueZIP behaves differently depending on whether the archive file already exists or not."

The code was intended ONLY as an example; it reproduces the "slowness" which I'm experiencing in the actual code but is short enough to serve as an example (as error handling, etc. is left out). For me it was obvious that the code I posted was not intended as anything else than a quick and dirty example, but I guess I should have mentioned that (and especially the part where the created files should be deleted after running the code).

"Now here is the simple solution: You have introduced superfluous calls to TFile.createNewFile() and Files.createFile(TPath). These calls have side effects, and in the case of Files.createFile(TPath), when looping over an archive file this results in quadratic runtime, which is what you were witnessing."

Indeed, the calls are superfluous. But without them, the issue is not reproducible. That's why I stated in my original post: "Using TPath is at least twice as slow as TFile when each file in an archive is created before the contents is written". The question was therefore why this makes such a huge difference (as using TFile.createNewFile() has a negligible impact on performance)?

Comment by Christian Schlichtherle [ 12/Mar/14 ]

The reason is that TFile.createNewFile() and Files.createFile(TPath, ...) are implemented in fundamentally different ways:

TFile.createNewFile() simply creates a node in the virtual file system for the archive file. No I/O is done at all. This is the equivalent of an mknod() POSIX call.

Files.createFile(Path, ...) allocates a new SeekableByteChannel and then immediately closes it. This is very expensive because the TrueZIP Kernel needs to allocate a temporary I/O buffer (usually a file) so that you can seek in it, connect a new SeekableByteChannel to it and then immediate close it. Then again, you start overwriting the same temporary I/O buffer. Finally, when calling TVFS.umount(), all 10.000 temporary I/O buffers get assembled into the archive file, which is why you are witnessing double runtime. Note that this implements a write-back caching strategy. There is also a write-through caching strategy implemented in the kernel, but not actually used. The write-through strategy would even result in quadratic runtime because the kernel would have to do an implicit umount upon the second write to the same archive entry within the loop.

Unfortunately I have no influence on the implementation of Files.createFile(Path, ...) because it is a static method. I once contacted Alan Bateman, the specification lead for the NIO.2 API on the matter, but it doesn't seem Oracle would add the equivalent of an mknod() call to the NIO.2 API.

Comment by qui.tiqe [ 12/Mar/14 ]

Thank you for the detailed explanation, it is much appreciated!





[TRUEZIP-342] Update dependencies and plugins Created: 27/Feb/14  Updated: 07/Feb/15  Resolved: 27/Feb/14

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: TrueZIP 7.7.6

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-341] CLONE -Too restrictive POSIX permissions when creating an archive file on JSE 7 Created: 27/Feb/14  Updated: 07/Feb/15  Resolved: 27/Feb/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE
Affects Version/s: TrueZIP 7.3.4, TrueZIP 7.4.3, TrueZIP 7.5.5, TrueZIP 7.7.3, TrueZIP 7.7.4, TrueZIP 7.7.5
Fix Version/s: TrueZIP 7.6.6, TrueZIP 7.7.6

Type: Bug Priority: Minor
Reporter: T-Gergely Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

Linux 64bit


Issue Links:
Duplicate
is duplicated by TRUEVFS-61 CLONE - Too restrictive POSIX permiss... Closed

 Description   

When I create a zip with TFile.mkdir(), its permissions become "rw-----" instead of "-rw-rr-" when my umask is 0022.



 Comments   
Comment by T-Gergely [ 27/Feb/14 ]

This bug came back with TrueZIP 7.7.3-7.7.5.

java version "1.7.0_07"
Java(TM) SE Runtime Environment (build 1.7.0_07-b10)
Java HotSpot(TM) 64-Bit Server VM (build 23.3-b01, mixed mode)

Linux 2.6.18-371.4.1.el5xen





[TRUEZIP-339] Exception when creating an archive in the root folder (Windows) Created: 16/Jan/14  Updated: 03/Feb/14  Resolved: 23/Jan/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE
Affects Version/s: TrueZIP 7.7.5
Fix Version/s: TrueZIP 7.7.6

Type: Bug Priority: Minor
Reporter: qui.tiqe Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Oracle JDK 1.7.0_51 on Windows 7 and Mac OS X 10.9.1.



 Description   

When creating an archive in the root folder of a Windows drive (e.g. C:), TrueZIP throws an exception when using Java 7. The following sample code reproduces the issue:

QTrueZipExportToRoot.java
import java.io.BufferedWriter;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import de.schlichtherle.truezip.file.TVFS;
import de.schlichtherle.truezip.nio.file.TPath;

public class QTrueZipExportToRoot
{
  private static final Charset ENCODING = Charset.forName("UTF-8");
  private static final String ARCHIVE_PATH = "C:\\TestTrueZipRootIssue.zip";
  
  public static void main(String[] args)
  {
    try
    {
      Path archivePath = Paths.get(ARCHIVE_PATH);
      
      // Ensure that the file does not already exist
      if ( Files.exists(archivePath) )
      {
        Files.delete(archivePath);
      }
            
      // Create the archive, add a file to it, and write to the file within the archive
      TPath archiveFile = new TPath(archivePath);
      Files.createDirectory(archiveFile);
      TPath newFilePath = archiveFile.resolve("File1");
      Files.createFile(newFilePath);
      
      try ( BufferedWriter writer = Files.newBufferedWriter(newFilePath, ENCODING) )
      {
        writer.write("abcd");
      }
      
      // Commit all pending changes
      TVFS.umount();
      
      // Delete the archive if it exists
      if ( Files.exists( archivePath ) )
      {
        Files.delete( archivePath );
      }
      
      System.out.println("Test completed successfully.");
    }
    catch (Exception e)
    {
      e.printStackTrace();
    }
  }
}

This exception is thrown:

java.nio.file.FileAlreadyExistsException: D:\TestTrueZipRootIssue.zip
	at de.schlichtherle.truezip.nio.file.TFileSystem.createDirectory(TFileSystem.java:362)
	at de.schlichtherle.truezip.nio.file.TPath.createDirectory(TPath.java:970)
	at de.schlichtherle.truezip.nio.file.TFileSystemProvider.createDirectory(TFileSystemProvider.java:315)
	at java.nio.file.Files.createDirectory(Unknown Source)
	at QTrueZipExportToRoot.main(QTrueZipExportToRoot.java:31)
Caused by: java.io.IOException: Root directory does not exist
	at java.nio.file.Files.createDirectories(Unknown Source)
	at de.schlichtherle.truezip.fs.nio.file.FileOutputSocket.begin(FileOutputSocket.java:109)
	at de.schlichtherle.truezip.fs.nio.file.FileOutputSocket.newOutputStream(FileOutputSocket.java:222)
	at de.schlichtherle.truezip.fs.archive.zip.OptionOutputSocket.newOutputStream(OptionOutputSocket.java:48)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop(ZipDriver.java:586)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop0(ZipDriver.java:576)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop(ZipDriver.java:561)
	at de.schlichtherle.truezip.fs.FsTargetArchiveController.makeOutputArchive(FsTargetArchiveController.java:247)
	at de.schlichtherle.truezip.fs.FsTargetArchiveController.mount0(FsTargetArchiveController.java:182)
	at de.schlichtherle.truezip.fs.FsTargetArchiveController.mount(FsTargetArchiveController.java:155)
	at de.schlichtherle.truezip.fs.FsFileSystemArchiveController$ResetFileSystem.autoMount(FsFileSystemArchiveController.java:85)
	at de.schlichtherle.truezip.fs.FsFileSystemArchiveController.autoMount(FsFileSystemArchiveController.java:37)
	at de.schlichtherle.truezip.fs.FsBasicArchiveController.mknod(FsBasicArchiveController.java:319)
	at de.schlichtherle.truezip.fs.FsContextController.mknod(FsContextController.java:178)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsCacheController.mknod(FsCacheController.java:157)
	at de.schlichtherle.truezip.fs.FsSyncController.mknod(FsSyncController.java:201)
	at de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:207)
	at de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:204)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
	at de.schlichtherle.truezip.fs.FsLockController.mknod(FsLockController.java:211)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:431)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:426)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.mknod(FsFalsePositiveArchiveController.java:436)
	at de.schlichtherle.truezip.nio.file.TFileSystem.createDirectory(TFileSystem.java:355)
	... 4 more


 Comments   
Comment by Christian Schlichtherle [ 18/Jan/14 ]

When changing the path to "/TestTrueZipRootIssue.zip" and running the program on Mac OS X, I do get pretty much the same stack trace:

java.nio.file.FileAlreadyExistsException: /TestTrueZipRootIssue.zip
	at de.schlichtherle.truezip.nio.file.TFileSystem.createDirectory(TFileSystem.java:362)
	at de.schlichtherle.truezip.nio.file.TPath.createDirectory(TPath.java:970)
	at de.schlichtherle.truezip.nio.file.TFileSystemProvider.createDirectory(TFileSystemProvider.java:315)
	at java.nio.file.Files.createDirectory(Files.java:628)
	at QTrueZipExportToRoot.main(QTrueZipExportToRoot.java:28)
Caused by: java.io.IOException: Root directory does not exist
	at java.nio.file.Files.createDirectories(Files.java:711)
	at de.schlichtherle.truezip.fs.nio.file.FileOutputSocket.begin(FileOutputSocket.java:109)
	at de.schlichtherle.truezip.fs.nio.file.FileOutputSocket.newOutputStream(FileOutputSocket.java:222)
	at de.schlichtherle.truezip.fs.archive.zip.OptionOutputSocket.newOutputStream(OptionOutputSocket.java:48)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop(ZipDriver.java:586)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop0(ZipDriver.java:576)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newOutputShop(ZipDriver.java:561)
	at de.schlichtherle.truezip.fs.FsTargetArchiveController.makeOutputArchive(FsTargetArchiveController.java:247)
	at de.schlichtherle.truezip.fs.FsTargetArchiveController.mount0(FsTargetArchiveController.java:182)
	at de.schlichtherle.truezip.fs.FsTargetArchiveController.mount(FsTargetArchiveController.java:155)
	at de.schlichtherle.truezip.fs.FsFileSystemArchiveController$ResetFileSystem.autoMount(FsFileSystemArchiveController.java:85)
	at de.schlichtherle.truezip.fs.FsFileSystemArchiveController.autoMount(FsFileSystemArchiveController.java:37)
	at de.schlichtherle.truezip.fs.FsBasicArchiveController.mknod(FsBasicArchiveController.java:319)
	at de.schlichtherle.truezip.fs.FsContextController.mknod(FsContextController.java:178)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsCacheController.mknod(FsCacheController.java:157)
	at de.schlichtherle.truezip.fs.FsSyncController.mknod(FsSyncController.java:201)
	at de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:207)
	at de.schlichtherle.truezip.fs.FsLockController$1Mknod.call(FsLockController.java:204)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
	at de.schlichtherle.truezip.fs.FsLockController.mknod(FsLockController.java:211)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsDecoratingController.mknod(FsDecoratingController.java:120)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:431)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Mknod.call(FsFalsePositiveArchiveController.java:426)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.mknod(FsFalsePositiveArchiveController.java:436)
	at de.schlichtherle.truezip.nio.file.TFileSystem.createDirectory(TFileSystem.java:355)
	... 4 more

It turns out that this is actually a bug in the NIO.2 implementation of the JRE (at least with Oracle JDK 1.7.0_51). The following program reproduces the issue:

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class ShouldNotThrowException {
    public static void main(String[] args) throws IOException {
        Files.createDirectories(Paths.get("/"));
    }
}

When running, this simple program produces:

Exception in thread "main" java.io.IOException: Root directory does not exist
	at java.nio.file.Files.createDirectories(Files.java:711)
	at ShouldNotThrowException.main(ShouldNotThrowException.java:7)

The exception message is bogus. Of course, my root directory exists. But more importantly, the Javadoc for Files.createDirectories(Path, FileAttribute<?>...) reads:

Unlike the {@link #createDirectory createDirectory} method, an exception is not thrown if the directory could not be created because it already exists.

Of course, I could work around this issue, but creating an archive file in the root directory is not a good idea anyway (your permissions may not even allow it) and I would not like to issue a new release of TrueZIP and TrueVFS just because of a workaround for a broken NIO.2 implementation.

Comment by qui.tiqe [ 21/Jan/14 ]

Thanks for investigating the issue. I'd have to agree that the impact on non-Windows systems is very limited. However, on Windows systems exporting to the root folder of a drive is fairly common as every drive represent either a folder (incl. a folder on a network) or a partition. Therefore the impact of this issue is in my view more than "minor". That being said, as it's not a TrueZIP issue I fully understand you decision to not issue a new release. However, I would like to know if you have any plans to add a workaround to the development version?

Comment by Christian Schlichtherle [ 23/Jan/14 ]

I've added a workaround to the code base. Please pull the source code repository and build yourself using

$ mvn clean install

Comment by qui.tiqe [ 03/Feb/14 ]

I created a build that includes the workaround and it indeed solves the issue. Many thanks!





[TRUEZIP-338] TFile cannot find all expected files Created: 24/Dec/13  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR
Affects Version/s: TrueZIP 7.7.5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: diamss Assignee: Christian Schlichtherle
Resolution: Invalid Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64 bit with JDK 1.6.38 / 1.7.11



 Description   

For some TAR.GZ archives generated by Maven Assembly Plugin, TruzZip canno find all files.

  • Do not work on TAR.GZ
  • Do not work on TAR if GZ was decompressed with TrueZip.
  • Work on TAR if GZ was decompressed with 7-zip.

I cannot upload the example TAR.GZ at the moment because of restrictions in my company, but I could do it soon.

Test.java
public static void main(String[] args) throws Exception {

	final TArchiveDetector detectorGz = new TArchiveDetector("gz", new TarGZipDriver(IOPoolLocator.SINGLETON));
	final TArchiveDetector detectorTar = new TArchiveDetector("tar", new TarDriver(IOPoolLocator.SINGLETON));
	final TFile gz = new TFile(args[0], detectorGz);
	final TFile tar = new TFile(gz.getParentFile(), "extracted.tar", detectorTar);

	TFile.cp_r(gz, tar, detectorGz, detectorTar);
	recursive(tar);
}

public static void recursive(final TFile file) {

	if (!file.isDirectory()) {

		System.out.println(file);
		return;
	}

	final TFile[] children = file.listFiles();

	System.out.println(file + " (" + children.length + ')');

	for (final TFile child : children) {

		recursive(child);
	}
}


 Comments   
Comment by Christian Schlichtherle [ 24/Dec/13 ]

The driver setup is wrong: The TarGZipDriver reads and writes TAR.GZ files , not just GZ files. The latter is not an archive file format, just a compressor format.





[TRUEZIP-337] CLONE -Please include EPL license text Created: 18/Nov/13  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.5
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: grdryn Assignee: Christian Schlichtherle
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Fedora



 Description   

Hi Christian,

I'm packaging TrueCommons and TrueVFS for Fedora (we currently have TrueZip, but it seems like TrueVFS is the successor to that). As part of the review process for a new package in Fedora, we check to see if the license text is included with the source code, and if not we query the maintainers (you!) to see if it would be possible to get it included. It's not a blocker, and we include it anyway in our RPM packages, but it's nicer to see it included in source repository!

So, would it be possible to get the license text included in the 3 repositories (truecommons-parent, truecommons, truevfs)?

Thanks for creating this great software!

Regards,
Gerard.






[TRUEZIP-336] Update dependencies. Created: 07/Nov/13  Updated: 27/Feb/14  Resolved: 07/Nov/13

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: TrueZIP 7.7.5

Type: Improvement Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-334] NullPointerException in CanonicalStringSet.contains Created: 06/Nov/13  Updated: 27/Feb/14  Resolved: 07/Nov/13

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.3, TrueZIP 7.7.4
Fix Version/s: TrueZIP 7.7.5

Type: Bug Priority: Major
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hi, received the following bug report from a user:

program.name=DocFetcher
program.version=1.1.9
program.build=20130903-1749
program.portable=true
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.7.0_15-b03
java.version=1.7.0_15
sun.arch.data.model=64
os.arch=amd64
os.name=Windows 7
os.version=6.1
user.language=fr
java.lang.NullPointerException
at java.util.TreeMap.getEntry(Unknown Source)
at java.util.TreeMap.containsKey(Unknown Source)
at java.util.TreeSet.contains(Unknown Source)
at de.schlichtherle.truezip.util.CanonicalStringSet.contains(CanonicalStringSet.java:148)
at de.schlichtherle.truezip.file.TArchiveDetector.getScheme(TArchiveDetector.java:276)
at de.schlichtherle.truezip.file.TFile.scan(TFile.java:862)
at de.schlichtherle.truezip.file.TFile.scan(TFile.java:782)
at de.schlichtherle.truezip.file.TFile.(TFile.java:577)
at de.schlichtherle.truezip.file.TFile.filter(TFile.java:2296)
at de.schlichtherle.truezip.file.TFile.listFiles(TFile.java:2265)
at de.schlichtherle.truezip.file.TFile.listFiles(TFile.java:2205)
at de.schlichtherle.truezip.file.TFile.listFiles(TFile.java:354)
at net.sourceforge.docfetcher.util.Util.listFiles(Util.java:794)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:93)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:386)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:386)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
...


 Comments   
Comment by Christian Schlichtherle [ 07/Nov/13 ]

Apparently the directory contains a file name which ends with a '.' character.





[TRUEZIP-333] Adding file to zip doesn't work if in the zip park there is symbolic link. Created: 18/Oct/13  Updated: 07/Feb/15  Resolved: 17/Nov/14

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.6.6, TrueZIP 7.7.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: pskierczynski Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, Debian Wheezy 64bit, ext4 fs, Oracle JDK 1.7.0_45, Oracle JDK 1.6.0_45



 Description   

I'll can workaround this, but I really think that it should be resolved in the library.

To reproduce:
1) create zip file in symlinked directory
2) create directory in zip
3) try to create file in directory (in zip)

Test that fails:
@Test
public void testAddingZipFilesInSymlinkedDirectory() throws Exception
{
final String OS = System.getProperty("os.name").toLowerCase();
final boolean isWindows = (OS.contains("win"));

if (!isWindows)
{
final File originalDirectory = Files.createTempDir();
final File placeForSymlink = Files.createTempDir();

final String symlinkPath = placeForSymlink.getAbsolutePath();

try

{ //create symlink instead of directory placeForSymlink.delete(); Runtime.getRuntime().exec(String.format("/bin/ln -s %s %s", originalDirectory.getAbsolutePath(), symlinkPath)); //zip will be in symlink final String zipArchivePath = placeForSymlink + File.separator + "test.zip"; final TFile zipArchive = new TFile(zipArchivePath); zipArchive.mkdirs(); //some directory in zip final TFile innerDirectory = new TFile(zipArchive, "someDir"); innerDirectory.mkdirs(); //and finally try to create file in directory final TFile innerFile = new TFile(innerDirectory, "someFile"); TFileOutputStream fos = new TFileOutputStream(innerFile); }

finally

{ //trying to clean up after Runtime.getRuntime().exec(String.format("rm %s", symlinkPath)); Runtime.getRuntime().exec(String.format("rm -rf %s", originalDirectory.getAbsolutePath())); }

}
}



 Comments   
Comment by Christian Schlichtherle [ 17/Nov/14 ]

Your test runs fine on OS X 10.10 with Oracle JDK 1.7.0_72 and 1.8.0_25. I suppose to update your JDK version. I know that older versions had issues in their I/O implementation.





[TRUEZIP-332] Update dependencies Created: 28/Sep/13  Updated: 27/Feb/14  Resolved: 28/Sep/13

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.3
Fix Version/s: TrueZIP 7.7.4

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

E.g. XZ from 1.3 to 1.4.






[TRUEZIP-330] Cannot open any "mobi" document Created: 06/Sep/13  Updated: 07/Feb/15  Resolved: 06/Sep/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 6.8.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: radu_coravu Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: 2 hours
Time Spent: Not Specified
Original Estimate: 2 hours


 Description   

Whenever I try to open a mobi document (any online free ebook from example) I get an internal error in the library like:

java.util.zip.ZipException: expected End Of Central Directory signature
at de.schlichtherle.util.zip.BasicZipFile.findCentralDirectory(Unknown Source)
at de.schlichtherle.util.zip.BasicZipFile.mountCentralDirectory(Unknown Source)
at de.schlichtherle.util.zip.BasicZipFile.init(Unknown Source)
at de.schlichtherle.util.zip.BasicZipFile.<init>(Unknown Source)
at de.schlichtherle.io.archive.zip.Zip32InputArchive.<init>(Unknown Source)

Is this a know issue? I'm currently using TrueZip 6.6



 Comments   
Comment by Christian Schlichtherle [ 06/Sep/13 ]

The exception is quite expressive. It means that there is no End Of Central Directory record, so for whatever reason, the ZIP file appears to be corrupted.

Comment by radu_coravu [ 09/Sep/13 ]

I used 7-ZIP to test the mobi archive and it reported on problems with it.
Christian, could you make a quick check to open any mobi free-book document using the latest TrueZip library sources?
The situation could be one of:

Either the latest TrueZip code no longer has this problem.
Or if the problem persists and no MOBI archive can be opened:
1) Either all MOBI archives are created broken.
2) Or TrueZIP has a problem.

Comment by Christian Schlichtherle [ 28/Sep/13 ]

I don't know why you think a MOBI file is a ZIP file. I just downloaded a sample "Alice in Wonderland" MOBI file and it isn't a ZIP at all. Also, please upgrade to a newer version of TrueZIP. TrueZIP 6.X is dead. If you run into issues, there's little I could do on it.

Comment by radu_coravu [ 30/Sep/13 ]

Hi Christian,

Yes, sorry about that. We'll also try to upgrade the library we are currently using.
We produce an XML Editor called Oxygen XML Editor and we have an Archive Browser which is based on TrueZip.
If you work with XML we can offer you a free license key for it (support@oxygenxml.com).
Thanks for making a great library that we and our users appreciate.

Regards,
Radu





[TRUEZIP-329] IllegalArgumentException in IntervalReadOnlyFile Created: 19/Aug/13  Updated: 27/Feb/14  Resolved: 28/Sep/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.5, TrueZIP 7.7.3
Fix Version/s: TrueZIP 7.7.4

Type: Bug Priority: Major
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hi,

Got this exception from a user, it looks like the program tried to open an invalid zip file:

program.name=DocFetcher
program.version=1.1.7
program.build=20130408-2002
program.portable=true
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.7.0_25-b17
java.version=1.7.0_25
sun.arch.data.model=64
os.arch=amd64
os.name=Windows 7
os.version=6.1
user.language=cs
java.lang.IllegalArgumentException
at de.schlichtherle.truezip.rof.IntervalReadOnlyFile.(IntervalReadOnlyFile.java:109)
at de.schlichtherle.truezip.rof.IntervalReadOnlyFile.(IntervalReadOnlyFile.java:81)
at de.schlichtherle.truezip.zip.RawZipFile$EntryReadOnlyFile.(RawZipFile.java:1122)
at de.schlichtherle.truezip.zip.RawZipFile.getInputStream(RawZipFile.java:993)
at de.schlichtherle.truezip.fs.archive.zip.ZipInputShop.access$100(ZipInputShop.java:30)
at de.schlichtherle.truezip.fs.archive.zip.ZipInputShop$1Input.newInputStream(ZipInputShop.java:135)
at de.schlichtherle.truezip.socket.DisconnectingInputShop$Input.newInputStream(DisconnectingInputShop.java:168)
at de.schlichtherle.truezip.socket.LockInputShop$1Input.newInputStream(LockInputShop.java:147)
at de.schlichtherle.truezip.socket.ClutchInputSocket.newInputStream(ClutchInputSocket.java:75)
at de.schlichtherle.truezip.fs.archive.FsTargetArchiveController$1Input.newInputStream(FsTargetArchiveController.java:321)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
at de.schlichtherle.truezip.fs.archive.FsContextController$Input.newInputStream(FsContextController.java:304)
at de.schlichtherle.truezip.fs.FsResourceController$Input.newInputStream(FsResourceController.java:257)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
at de.schlichtherle.truezip.fs.FsSyncController$Input.newInputStream(FsSyncController.java:412)
at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:494)
at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:491)
at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:364)
at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:303)
at de.schlichtherle.truezip.fs.FsLockController$Input.newInputStream(FsLockController.java:499)
at de.schlichtherle.truezip.fs.FsFinalizeController$Input.newInputStream(FsFinalizeController.java:176)
at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input$NewInputStream.call(FsFalsePositiveController.java:363)
at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input$NewInputStream.call(FsFalsePositiveController.java:356)
at de.schlichtherle.truezip.fs.FsFalsePositiveController$TryChild.call(FsFalsePositiveController.java:536)
at de.schlichtherle.truezip.fs.FsFalsePositiveController.call(FsFalsePositiveController.java:104)
at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input.newInputStream(FsFalsePositiveController.java:353)
at de.schlichtherle.truezip.file.TFileInputStream.newInputStream(TFileInputStream.java:104)
at de.schlichtherle.truezip.file.TFileInputStream.(TFileInputStream.java:95)
at net.sourceforge.docfetcher.model.parse.ParseService.doParse(ParseService.java:260)
at net.sourceforge.docfetcher.model.parse.ParseService.parse(ParseService.java:232)
at net.sourceforge.docfetcher.model.index.file.FileContext.index(FileContext.java:146)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleFile(FileIndex.java:288)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:123)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:386)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.switchSolidToArchive(FileIndex.java:806)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitSolidArchive(FileIndex.java:571)
at net.sourceforge.docfetcher.model.index.file.FileIndex.switchDirZipToSolid(FileIndex.java:491)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$000(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleFile(FileIndex.java:280)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:123)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:386)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:386)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.doUpdate(FileIndex.java:159)
at net.sourceforge.docfetcher.model.TreeIndex.update(TreeIndex.java:142)
at net.sourceforge.docfetcher.model.index.Task.update(Task.java:98)
at net.sourceforge.docfetcher.model.index.IndexingQueue.threadLoop(IndexingQueue.java:161)
at net.sourceforge.docfetcher.model.index.IndexingQueue.access$100(IndexingQueue.java:44)
at net.sourceforge.docfetcher.model.index.IndexingQueue$2.run(IndexingQueue.java:116)

Best regards
<= Quang



 Comments   
Comment by Christian Schlichtherle [ 19/Aug/13 ]

Please upgrade to the latest version. There have been changes meanwhile which should resolve this issue. If not, please reopen this ticket.

Comment by qforce [ 19/Aug/13 ]

Understood, thanks!

Comment by qforce [ 03/Sep/13 ]

Unfortunately, the bug is still present with TrueZIP 7.7.3:

program.name=DocFetcher
program.version=1.1.8
program.build=20130821-2316
program.portable=false
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.7.0_11-b21
java.version=1.7.0_11
sun.arch.data.model=32
os.arch=x86
os.name=Windows 7
os.version=6.1
user.language=de
java.lang.IllegalArgumentException
at de.schlichtherle.truezip.rof.IntervalReadOnlyFile.(IntervalReadOnlyFile.java:109)
at de.schlichtherle.truezip.rof.IntervalReadOnlyFile.(IntervalReadOnlyFile.java:81)
at de.schlichtherle.truezip.zip.RawZipFile$EntryReadOnlyFile.(RawZipFile.java:1124)
at de.schlichtherle.truezip.zip.RawZipFile.getInputStream(RawZipFile.java:995)
at de.schlichtherle.truezip.fs.archive.zip.ZipInputShop.access$100(ZipInputShop.java:29)
at de.schlichtherle.truezip.fs.archive.zip.ZipInputShop$1Input.newInputStream(ZipInputShop.java:134)
at de.schlichtherle.truezip.socket.DisconnectingInputShop$Input.newInputStream(DisconnectingInputShop.java:165)
at de.schlichtherle.truezip.socket.LockInputShop$1Input.newInputStream(LockInputShop.java:147)
at de.schlichtherle.truezip.fs.FsTargetArchiveController$1Input.newInputStream(FsTargetArchiveController.java:299)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
at de.schlichtherle.truezip.fs.FsContextController$Input.newInputStream(FsContextController.java:273)
at de.schlichtherle.truezip.fs.FsResourceController$Input.newInputStream(FsResourceController.java:249)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
at de.schlichtherle.truezip.fs.FsSyncController$Input.newInputStream(FsSyncController.java:397)
at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:455)
at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:452)
at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
at de.schlichtherle.truezip.fs.FsLockController$Input.newInputStream(FsLockController.java:459)
at de.schlichtherle.truezip.fs.FsFinalizeController$Input.newInputStream(FsFinalizeController.java:177)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input$NewInputStream.call(FsFalsePositiveArchiveController.java:333)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input$NewInputStream.call(FsFalsePositiveArchiveController.java:326)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input.newInputStream(FsFalsePositiveArchiveController.java:323)
at de.schlichtherle.truezip.file.TFileInputStream.newInputStream(TFileInputStream.java:104)
at de.schlichtherle.truezip.file.TFileInputStream.(TFileInputStream.java:95)
at net.sourceforge.docfetcher.model.parse.ParseService.doParse(ParseService.java:260)
at net.sourceforge.docfetcher.model.parse.ParseService.parse(ParseService.java:232)
at net.sourceforge.docfetcher.model.index.file.FileContext.index(FileContext.java:146)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleHtmlPair(FileIndex.java:324)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:169)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:275)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:51)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:386)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
...

Comment by qforce [ 16/Sep/13 ]

Hi Christian, still no time to look into this issue?

Comment by Christian Schlichtherle [ 16/Sep/13 ]

I'm sorry, but I'm still terribly busy with other things. I'll look into it ASAP.

Comment by qforce [ 16/Sep/13 ]

No need to hurry. It just seemed as if you had forgotten about it, so I bumped it up.





[TRUEZIP-326] Proper exception Message not shown upon exception (this time with Zip archive file) Created: 08/Jul/13  Updated: 27/Feb/14  Resolved: 19/Jul/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.7.2
Fix Version/s: TrueZIP 7.7.3

Type: Bug Priority: Critical
Reporter: vayu_rw Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

with Java 7 (1.7.0_15) on MacOSx (10.7.5) with IntelliJ IDE version 12



 Description   

The data : a Zip archive consisting of 40K+ zip archives. Each of the 40K+ zip archives are comprised of two text files (an XML file and a plain text file).

Problem : WHile processing a Zip file of zip files, upon processing 10147 files, the process throws an exception

Exception in thread "main" java.lang.NullPointerException
    at com.trgr.rd.elasticsearch.IndexElastic.main(IndexElastic.java:76)
java.lang.IllegalStateException: Can't overwrite predecessor!
Disconnected from the target VM, address: '127.0.0.1:51225', transport: 'socket'
    at de.schlichtherle.truezip.io.SequentialIOException.setPredecessor(SequentialIOException.java:194)
    at de.schlichtherle.truezip.io.SequentialIOException.initPredecessor(SequentialIOException.java:184)
    at de.schlichtherle.truezip.io.SequentialIOExceptionBuilder.update(SequentialIOExceptionBuilder.java:86)
    at de.schlichtherle.truezip.io.SequentialIOExceptionBuilder.update(SequentialIOExceptionBuilder.java:22)
    at de.schlichtherle.truezip.util.AbstractExceptionBuilder.warn(AbstractExceptionBuilder.java:77)
    at de.schlichtherle.truezip.fs.FsManager.sync(FsManager.java:107)
    at de.schlichtherle.truezip.fs.FsDefaultManager.sync(FsDefaultManager.java:190)
    at de.schlichtherle.truezip.fs.FsSyncShutdownHook$Hook.run(FsSyncShutdownHook.java:93)
Caused by: de.schlichtherle.truezip.fs.FsSyncWarningException: zip:zip:file:/Users/U0102180/data/lemur/harvest/2012-02-10.zip!/2012-02-10/3eebbcfc-1190-3bd5-b95f-f6501dbea1be__2012-02-10-03-40-32-696.zip!/
    at de.schlichtherle.truezip.fs.FsResourceController.syncResources(FsResourceController.java:98)
    at de.schlichtherle.truezip.fs.FsResourceController.sync(FsResourceController.java:72)
    at de.schlichtherle.truezip.fs.FsCacheController.sync(FsCacheController.java:178)
    at de.schlichtherle.truezip.fs.FsSyncController.sync(FsSyncController.java:234)
    at de.schlichtherle.truezip.fs.FsLockController$1Sync.call(FsLockController.java:235)
    at de.schlichtherle.truezip.fs.FsLockController$1Sync.call(FsLockController.java:232)

    at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
    at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
    at de.schlichtherle.truezip.fs.FsLockController.sync(FsLockController.java:240)
    at de.schlichtherle.truezip.fs.archive.zip.KeyController.sync(KeyController.java:128)
    at de.schlichtherle.truezip.fs.FsDecoratingController.sync(FsDecoratingController.java:131)
    at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.sync(FsFalsePositiveArchiveController.java:480)
    at de.schlichtherle.truezip.fs.FsManager.sync(FsManager.java:105)
    ... 2 more
Caused by: de.schlichtherle.truezip.fs.FsResourceOpenException: Thread-local / total number of open I/O resources (streams, channels etc): 0 / 2
    at de.schlichtherle.truezip.fs.FsResourceController.syncResources(FsResourceController.java:94)
    ... 14 more

Instead of throwing the reason for the actual exception, the "IllegalStateException: Can't overwrite predecessor" exception is thrown.



 Comments   
Comment by Christian Schlichtherle [ 08/Jul/13 ]

If your application is single-threaded then could you please attach some sample data and test case? This would be really great!





[TRUEZIP-325] Proper exception Message not shown upon exception Created: 08/Jul/13  Updated: 27/Feb/14  Resolved: 19/Jul/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR
Affects Version/s: TrueZIP 7.7.2
Fix Version/s: TrueZIP 7.7.3

Type: Bug Priority: Major
Reporter: vayu_rw Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Running Java 7 (1.7.0_15) on MacOSX (10.7.5) within IntelliJ IDE



 Description   

strong The data being processed:
TAR GZip Compress archive file (tgz/tar.gz) consisting of 40k+ Zip archive files. Each of the Zip archive file consists of two text files (XML file and a plain text file).

strong Problem :
While processing every single one of the entries in the TAR GZip archive file, after processing 10115 files, (on the 10116th file), I get an exception :

Exception in thread "main" java.lang.IllegalStateException: /var/folders/0w/0wgwst553_l060bk95gtdmx40000gp/T/tzp7043037030083864275.tmp (already released)
        at de.schlichtherle.truezip.fs.nio.file.TempFilePool$Buffer.release(TempFilePool.java:99)
        at de.schlichtherle.truezip.socket.IOCache$Buffer.release(IOCache.java:429)
        at de.schlichtherle.truezip.socket.IOCache$InputBufferPool.allocate(IOCache.java:315)
        at de.schlichtherle.truezip.socket.IOCache$Input.getDelegate(IOCache.java:269)
        at de.schlichtherle.truezip.socket.DelegatingInputSocket.getBoundSocket(DelegatingInputSocket.java:43)
        at de.schlichtherle.truezip.socket.DelegatingInputSocket.newReadOnlyFile(DelegatingInputSocket.java:53)
        at de.schlichtherle.truezip.socket.DelegatingInputSocket.newReadOnlyFile(DelegatingInputSocket.java:53)
        at de.schlichtherle.truezip.fs.FsSyncController$Input.newReadOnlyFile(FsSyncController.java:397)
        at de.schlichtherle.truezip.fs.FsLockController$Input$1NewReadOnlyFile.call(FsLockController.java:443)
        at de.schlichtherle.truezip.fs.FsLockController$Input$1NewReadOnlyFile.call(FsLockController.java:440)
        at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:316)
        at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
        at de.schlichtherle.truezip.fs.FsLockController$Input.newReadOnlyFile(FsLockController.java:447)
        at de.schlichtherle.truezip.fs.FsFinalizeController$Input.newReadOnlyFile(FsFinalizeController.java:171)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input$NewReadOnlyFile.call(FsFalsePositiveArchiveController.java:298)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input$NewReadOnlyFile.call(FsFalsePositiveArchiveController.java:291)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input.newReadOnlyFile(FsFalsePositiveArchiveController.java:288)
        at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newInputShop(ZipDriver.java:479)
        at de.schlichtherle.truezip.fs.FsTargetArchiveController.mount0(FsTargetArchiveController.java:197)
        at de.schlichtherle.truezip.fs.FsTargetArchiveController.mount(FsTargetArchiveController.java:155)
        at de.schlichtherle.truezip.fs.FsFileSystemArchiveController$ResetFileSystem.autoMount(FsFileSystemArchiveController.java:85)
        at de.schlichtherle.truezip.fs.FsFileSystemArchiveController.autoMount(FsFileSystemArchiveController.java:37)
        at de.schlichtherle.truezip.fs.FsBasicArchiveController.autoMount(FsBasicArchiveController.java:113)
        at de.schlichtherle.truezip.fs.FsBasicArchiveController.getEntry(FsBasicArchiveController.java:138)
        at de.schlichtherle.truezip.fs.FsContextController.getEntry(FsContextController.java:66)
        at de.schlichtherle.truezip.fs.FsDecoratingController.getEntry(FsDecoratingController.java:56)
        at de.schlichtherle.truezip.fs.FsDecoratingController.getEntry(FsDecoratingController.java:56)
        at de.schlichtherle.truezip.fs.FsSyncController.getEntry(FsSyncController.java:92)
        at de.schlichtherle.truezip.fs.FsLockController$1GetEntry.call(FsLockController.java:101)
        at de.schlichtherle.truezip.fs.FsLockController$1GetEntry.call(FsLockController.java:98)
        at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
        at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
        at de.schlichtherle.truezip.fs.FsLockController.readOrWriteLocked(FsLockController.java:257)
        at de.schlichtherle.truezip.fs.FsLockController.getEntry(FsLockController.java:104)
        at de.schlichtherle.truezip.fs.archive.zip.KeyController.getEntry(KeyController.java:72)
        at de.schlichtherle.truezip.fs.FsDecoratingController.getEntry(FsDecoratingController.java:56)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$GetEntry.call(FsFalsePositiveArchiveController.java:153)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$GetEntry.call(FsFalsePositiveArchiveController.java:148)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.getEntry(FsFalsePositiveArchiveController.java:145)
        at de.schlichtherle.truezip.file.TFile.listFiles(TFile.java:2268)
        at de.schlichtherle.truezip.file.TFile.listFiles(TFile.java:2212)

The response apparently should be the original exception which caused the problem instead of the IllegalState Exception that is thrown.



 Comments   
Comment by Christian Schlichtherle [ 09/Jul/13 ]

I've patched the source code repository in order to fix this issue. I have no test code to cover this yet. For verification, could you please:

hg clone -b "TrueZIP 7" https://hg.java.net/hg/truezip~v7
cd truezip~v7
mvn clean install

and then declare a dependency on TrueZIP version 7.7.3-SNAPSHOT in your POM?

After that, you will hopefully get the root exception.





[TRUEZIP-323] Maven distribution does not work (error finding file while maven downloads....) Created: 16/Jun/13  Updated: 07/Feb/15  Resolved: 16/Jun/13

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: colbertphilippe Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The latest version distribution does not work! Maven can't find the artifact even if the entry is registered.

http://mvnrepository.com/artifact/de.schlichtherle.truezip/truezip-file/7.7.2



 Comments   
Comment by Christian Schlichtherle [ 16/Jun/13 ]

If you use a profile, make sure to add <type>pom</type> to the dependency.

Comment by Christian Schlichtherle [ 16/Jun/13 ]

You should use the official search site for Maven artifacts. You can find all TrueZIP 7.7.2 artifacts here: http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22de.schlichtherle.truezip%22%20AND%20v%3A%227.7.2%22

Comment by colbertphilippe [ 17/Jun/13 ]

The link you have given is confusing to me. It does not give a complete <dependency> snippet and <repository> snippet in order to access to the latest TrueZip library in Maven.

Let me suggest to you to put these two elements on your website so that there is no confusion.

I still can't get access to your TrueZip from Maven.

Comment by Christian Schlichtherle [ 17/Jun/13 ]

The best way to get started is by using a Maven archetype to generate a sample project.

Please follow the instructions here: https://truezip.java.net/kick-start/index.html .

If you run into issues, please follow the instructions here: https://truezip.java.net/help.html





[TRUEZIP-322] Why not allow appending of file to existing zip file by appending without creating new zip ? Created: 13/Jun/13  Updated: 07/Feb/15  Resolved: 13/Jun/13

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: colbertphilippe Assignee: Christian Schlichtherle
Resolution: Complete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: append_new_file

 Description   

TrueZip does have functions that will append new files to an existing zip archive file but it does this by creating a new zip archive file and copying over all file entries from the prior zip file and the new files to be appended. That can be very inefficient if the prior zip file is very big.

It seems to me that the proper way of doing this is to reverse-engineer the prior zip file. While respecting the official format of zip file, the prior zip file should be opened and the content lookup table (located at the end of the zip file) should be read in memory or even in a temporary file. The new files should be written (thus overriding the location of the content table) and at the end the content lookup tables (with entries of new files) should be written at the end of the augmented file.

This is the most efficient way of doing it, but nobody that I know has done it.

Why can't TrueZip do that?



 Comments   
Comment by Christian Schlichtherle [ 13/Jun/13 ]

This feature has been added with FsOutputOption.GROW in TrueZIP 7.3.

Here's an introductory blog post: http://truezip.schlichtherle.de/2011/07/26/appending-to-zip-files/

And here's another blog post explaining how to compact a grown archive file again: http://truezip.schlichtherle.de/2011/07/27/compacting-archive-files/





[TRUEZIP-321] TrueZip does not detect sub archives when Java's temp directory does not exist Created: 06/Jun/13  Updated: 27/Feb/14  Resolved: 19/Jul/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE, TrueZIP Kernel
Affects Version/s: TrueZIP 7.7, TrueZIP 7.7.2
Fix Version/s: TrueZIP 7.7.3

Type: Bug Priority: Minor
Reporter: palich Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7, 32 or 64 Bit, Java 1.6.0_26-32bit



 Description   

If the system property java.io.tmpdir refers to a non-existing directory, TrueZip is not able to detect or access sub archives.

The assertion from the following test excerpt fails:

System.setProperty("java.io.tmpdir", "target/temp");
TFile subArchive = new TFile("src/test/resources/archive.zip/files.zip");
assertTrue("File must act like a directory: " + subArchive, file.isDirectory());

I know that java.io.tmpdir should always exist. But the real world sometimes is different. At least TrueZip should test it and log a warning to the console. Otherwise, the reason for a failed archive detection is hard to find.



 Comments   
Comment by palich [ 06/Jun/13 ]

Attaching complete sample JUnit Test:

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import org.junit.Test;

import de.schlichtherle.truezip.file.TFile;

public class TempDirTest {

	@Test
	public void test() {
		System.setProperty("java.io.tmpdir", "target/temp");

		// ZIP file OK
		TFile archive = new TFile("src/test/resources/archive.zip");
		assertArchive(archive);

		// Sub ZIP file FAILS
		TFile subArchive = new TFile("src/test/resources/archive.zip/files.zip");
		assertArchive(subArchive);
	}

	private void assertArchive(TFile file) {
		assertTrue("File must exist: " + file, file.exists());
		assertTrue("File must be detected as archive: " + file,
				file.isArchive());
		assertTrue("File must act like a directory: " + file,
				file.isDirectory());
		// See TFile#falsePositives
		assertEquals("File length must be 0: " + file, 0l, file.length());
	}
}

Comment by Christian Schlichtherle [ 07/Jun/13 ]

I've looked into this but I am still undecided what to do about it.

I don't like the concept of logging. Most of the time logging messages is a waste of resources. And then again they still don't provide exactly the information you need to fix a problem. Overall, I don't want to litter my code with logging statements for very rare corner cases like this.

I generally prefer to throw an IOException. However, in this particular context the IOException gets resolved as if a false positive archive file had been encountered, which is why you get these responses. Note that the Kernel and the drivers use temp files for other purposes too, not just accessing nested archive files. I cannot possibly handle all special cases. Even if I handle this special case, you can only get true or false from these boolean methods, so it doesn't help a lot.

As of now, I think I should probably throw an AssertionError or an IllegalStateException in this case. The app should then simply terminate with a nice stack trace clearly pointing out the problem. There is no easy way to recover from this error condition at runtime, but It's easy to fix at compile time.

Comment by Christian Schlichtherle [ 13/Jun/13 ]

What I could do is to respond to the IOException with a check if the temp dir exists. If not, then create the temp dir and retry to create the temp file. Otherwise, or if this fails, simply throw a new AssertionError().

When applied to your case, this strategy would solve the issue by silently creating the temp dir. Only if this impossible, you would get an AssertionError. Throwing the AssertionError is a good thing because if the issue can't get resolved, then continuing the program is pretty much pointless because temp files are used at several places.

Comment by palich [ 20/Jun/13 ]

I like your latest proposal, trying to create the temp directory if it does not yet exist and throwing an AssertionError otherwise. This is actually the workaround I have implemented for the software I am working on.

When I have understood you correctly, you would check the temp directory only in case of IOExceptions. Please make sure that you repeat the current operation after creating the temp directory successfully.

Comment by Christian Schlichtherle [ 19/Jul/13 ]

As discussed, except that an IllegalArgumentException is thrown instead of an AssertionError. Both should terminate the app, though.





[TRUEZIP-320] Update dependencies Created: 31/Jan/13  Updated: 19/Jul/13  Resolved: 15/May/13

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.7.1
Fix Version/s: TrueZIP 7.7.2

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

org.tukaani:xz is now at 1.3.






[TRUEZIP-319] Files are sometimes incorrectly umounted, leaving behind tmp files. Created: 10/Jan/13  Updated: 07/Feb/15  Resolved: 30/Jan/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.3.4, TrueZIP 7.4.3, TrueZIP 7.5.5, TrueZIP 7.6.6, TrueZIP 7.7, TrueZIP 7.7.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: jvanerp Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

We're working on a product where we need to have access to zip and jar files and write to them. Unfortunately sometimes the directories wherein these live are named with archive extensions. For example:

.../dummy.war/foo.zip/realArchive.zip

But every once in so many executions the realArchive.zip is incorrectly umounted and leaved behind a realArchive.zip.XXXXXXXXX.tmp file.

I've verified this with 7.3.4, 7.4.3 and now 7.7.1



 Comments   
Comment by Christian Schlichtherle [ 10/Jan/13 ]

Have you tried to call TVFS.umount(...) or TVFS.sync(...) when you are done with the processing?

In a standalone application, TVFS.umount() gets called by an automatically managed shutdown hook, but in container environments, you may need to call one of these methods manually.

Comment by jvanerp [ 10/Jan/13 ]

Yes, I call the TFile/TFVS umount methods. Even in finally blocks, so that's not the problem. It seems like (after debugging quite some code), that after overwriting a few files using TFIleWriters (which are also closed in finally blocks), that sometimes the archive just isn't cleanly umounted, leaving behind such a tmp file.

Comment by Christian Schlichtherle [ 21/Jan/13 ]

Sorry for the late answer but I have been very busy.

I tried the following program to reproduce this issue:


package com.company.mavenproject3;

import de.schlichtherle.truezip.file.*;
import java.io.*;

public class HelloWorld {
    public static void main(final String[] args) throws IOException {
        final Writer writer = new TFileWriter(
                new TFile("dummy.war/foo.zip/realAchive.zip/HälloWörld.txt"));
        try {
            writer.write("Hello world!\n");
        } finally {
            writer.close();
        }
    }
}

Note that I've created dummy.war and foo.zip as regular directories before running the program. I then run the program many times using JDK 1.7.0.11 and JDK 1.6.0.37 on Mac OS X 10.8.2. The result was that I cannot reproduce this issue. When I debug-step through the programm I can see that the temp file gets created upon the constructor call and unmounted in the application shutdown hook (which calls TVFS.umount()). So everything looks fine.

I do get reports about file system errors occasionally. However, they are all from Linux and were not reproducible. TrueZIP has comprehensive multithreaded integration tests which pass with flying colors on Windows and Mac OS X, so I am led to believe that this may be an issue with the Linux port of the JDK, the OS, its file system implementation or a weird combination of all three.

If you can provide me with a reproducible issue, I am happy to fix it asap.

Comment by Christian Schlichtherle [ 21/Jan/13 ]

I just realized you never said you are using Linux. Let's see.





[TRUEZIP-318] KeyManagerLocator in truezip-driver-zip has runtime dependency on truezip-swing Created: 10/Jan/13  Updated: 07/Feb/15  Resolved: 10/Jan/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.7.1
Fix Version/s: None

Type: Bug Priority: Trivial
Reporter: jvanerp Assignee: Christian Schlichtherle
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I'm building a non-swing project (http://github.com/hierynomus/scannit), which I wanted to upgrade to TrueZIP 7.7.1. However I now need to add truezip-swing as an extra dependency because it won't work otherwise. The stracktrace is:

java.util.ServiceConfigurationError: de.schlichtherle.truezip.key.spi.KeyManagerService: Provider de.schlichtherle.truezip.fs.archive.zip.PromptingKeyManagerService could not be instantiated: java.lang.NoClassDefFoundError: de/schlichtherle/truezip/swing/EnhancedPanel
	at java.util.ServiceLoader.fail(ServiceLoader.java:207) ~[na:1.6.0_37]
	at java.util.ServiceLoader.access$100(ServiceLoader.java:164) ~[na:1.6.0_37]
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:360) ~[na:1.6.0_37]
	at java.util.ServiceLoader$1.next(ServiceLoader.java:428) ~[na:1.6.0_37]
	at de.schlichtherle.truezip.key.sl.KeyManagerLocator$Boot.<clinit>(KeyManagerLocator.java:60) ~[truezip-driver-zip-7.7.1.jar:na]
	at de.schlichtherle.truezip.key.sl.KeyManagerLocator.get(KeyManagerLocator.java:41) ~[truezip-driver-zip-7.7.1.jar:na]
	at de.schlichtherle.truezip.key.AbstractKeyManagerProvider.get(AbstractKeyManagerProvider.java:22) ~[truezip-driver-zip-7.7.1.jar:na]
	at de.schlichtherle.truezip.fs.archive.zip.KeyController.getKeyManager(KeyController.java:62) ~[truezip-driver-zip-7.7.1.jar:na]
	at de.schlichtherle.truezip.fs.archive.zip.KeyController.sync(KeyController.java:129) ~[truezip-driver-zip-7.7.1.jar:na]
	at de.schlichtherle.truezip.fs.FsDecoratingController.sync(FsDecoratingController.java:131) ~[truezip-kernel-7.7.1.jar:na]
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.sync(FsFalsePositiveArchiveController.java:480) ~[truezip-kernel-7.7.1.jar:na]
	at de.schlichtherle.truezip.fs.FsManager.sync(FsManager.java:105) ~[truezip-kernel-7.7.1.jar:na]
	at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:291) ~[truezip-file-7.7.1.jar:na]
	at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:236) ~[truezip-file-7.7.1.jar:na]
	at de.schlichtherle.truezip.file.TFile.sync(TFile.java:957) ~[truezip-file-7.7.1.jar:na]
	at de.schlichtherle.truezip.file.TFile.umount(TFile.java:1005) ~[truezip-file-7.7.1.jar:na]
	at nl.javadude.scannit.reader.TFiles.umountQuietly(TFiles.java:28) ~[scannit/:na]
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.closeTFile(ArchiveEntrySupplier.java:37) [scannit/:na]
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.gatherEntries(ArchiveEntrySupplier.java:72) [scannit/:na]
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.gatherEntries(ArchiveEntrySupplier.java:59) [scannit/:na]
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.list(ArchiveEntrySupplier.java:45) [scannit/:na]
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.withArchiveEntries(ArchiveEntrySupplier.java:25) [scannit/:na]
	at nl.javadude.scannit.Worker.scanFiles(Worker.java:59) [scannit/:na]
	at nl.javadude.scannit.Worker.scanURI(Worker.java:53) [scannit/:na]
	at nl.javadude.scannit.Worker.scan(Worker.java:46) [scannit/:na]
	at nl.javadude.scannit.Scannit.<init>(Scannit.java:40) [scannit/:na]
	at nl.javadude.scannit.ScannitTest.init(ScannitTest.java:41) [scannit/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_37]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_37]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_37]
	at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_37]
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) [junit-4.10.jar:na]
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit-4.10.jar:na]
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) [junit-4.10.jar:na]
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) [junit-4.10.jar:na]
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) [junit-4.10.jar:na]
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300) [junit-4.10.jar:na]
	at org.junit.runners.Suite.runChild(Suite.java:128) [junit-4.10.jar:na]
	at org.junit.runners.Suite.runChild(Suite.java:24) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) [junit-4.10.jar:na]
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300) [junit-4.10.jar:na]
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157) [junit-4.10.jar:na]
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:76) [junit-rt.jar:na]
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195) [junit-rt.jar:na]
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63) [junit-rt.jar:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_37]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_37]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_37]
	at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_37]
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) [idea_rt.jar:na]
Caused by: java.lang.NoClassDefFoundError: de/schlichtherle/truezip/swing/EnhancedPanel
	at java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.6.0_37]
	at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) ~[na:1.6.0_37]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:615) ~[na:1.6.0_37]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) ~[na:1.6.0_37]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) ~[na:1.6.0_37]
	at java.net.URLClassLoader.access$000(URLClassLoader.java:58) ~[na:1.6.0_37]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:197) ~[na:1.6.0_37]
	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_37]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190) ~[na:1.6.0_37]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306) ~[na:1.6.0_37]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) ~[na:1.6.0_37]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ~[na:1.6.0_37]
	at de.schlichtherle.truezip.fs.archive.zip.PromptingKeyManagerService.<init>(PromptingKeyManagerService.java:37) ~[truezip-driver-zip-7.7.1.jar:na]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.6.0_37]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) ~[na:1.6.0_37]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) ~[na:1.6.0_37]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513) ~[na:1.6.0_37]
	at java.lang.Class.newInstance0(Class.java:355) ~[na:1.6.0_37]
	at java.lang.Class.newInstance(Class.java:308) ~[na:1.6.0_37]
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:356) ~[na:1.6.0_37]
	... 58 common frames omitted
Caused by: java.lang.ClassNotFoundException: de.schlichtherle.truezip.swing.EnhancedPanel
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202) ~[na:1.6.0_37]
	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_37]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190) ~[na:1.6.0_37]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306) ~[na:1.6.0_37]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) ~[na:1.6.0_37]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ~[na:1.6.0_37]
	... 78 common frames omitted

After which the following stracktrace occurs which breaks things:

java.lang.NoClassDefFoundError: Could not initialize class de.schlichtherle.truezip.key.sl.KeyManagerLocator$Boot
	at de.schlichtherle.truezip.key.sl.KeyManagerLocator.get(KeyManagerLocator.java:41)
	at de.schlichtherle.truezip.key.AbstractKeyManagerProvider.get(AbstractKeyManagerProvider.java:22)
	at de.schlichtherle.truezip.fs.archive.zip.KeyController.getKeyManager(KeyController.java:62)
	at de.schlichtherle.truezip.fs.archive.zip.KeyController.sync(KeyController.java:129)
	at de.schlichtherle.truezip.fs.FsDecoratingController.sync(FsDecoratingController.java:131)
	at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.sync(FsFalsePositiveArchiveController.java:480)
	at de.schlichtherle.truezip.fs.FsManager.sync(FsManager.java:105)
	at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:291)
	at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:236)
	at de.schlichtherle.truezip.file.TFile.sync(TFile.java:957)
	at de.schlichtherle.truezip.file.TFile.umount(TFile.java:1005)
	at nl.javadude.scannit.reader.TFiles.umountQuietly(TFiles.java:28)
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.closeTFile(ArchiveEntrySupplier.java:37)
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.gatherEntries(ArchiveEntrySupplier.java:72)
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.gatherEntries(ArchiveEntrySupplier.java:59)
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.list(ArchiveEntrySupplier.java:45)
	at nl.javadude.scannit.reader.ArchiveEntrySupplier.withArchiveEntries(ArchiveEntrySupplier.java:25)
	at nl.javadude.scannit.Worker.scanFiles(Worker.java:59)
	at nl.javadude.scannit.Worker.scanURI(Worker.java:53)
	at nl.javadude.scannit.Worker.scan(Worker.java:46)
	at nl.javadude.scannit.Scannit.<init>(Scannit.java:40)
	at nl.javadude.scannit.ScannitTest.init(ScannitTest.java:41)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:24)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:76)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

I think the dependency on truezip-swing is unneeded, or its absence should not trigger failures.



 Comments   
Comment by Christian Schlichtherle [ 10/Jan/13 ]

I agree this is a bit nasty, but there is a trivial workaround - just include the TrueZIP Swing module. It won't get used until there's really a need to.

If you absolutely can't tolerate this, please consider upgrading to TrueVFS. The module structure in TrueZIP is a bit flawed. This has been fixed in TrueVFS. Unfortunately, I can't fix it in TrueZIP without breaking binary compatibility and even then the result would be mirrored in TrueVFS.

Comment by Christian Schlichtherle [ 10/Jan/13 ]

If you are afraid that users could get prompted for a password with a Swing dialog, then you could implement your own key management by following this tutorial: http://truezip.java.net/truezip-driver/truezip-driver-zip/key-management.html





[TRUEZIP-316] TFile.mv(...) should not copy-delete if the source and destination are regular files or directories Created: 21/Dec/12  Updated: 19/Jul/13  Resolved: 21/Dec/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*
Affects Version/s: TrueZIP 7.7.1
Fix Version/s: TrueZIP 7.7.2

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 21/Dec/12 ]

Changeset: 60581e47acb4
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-12-21 18:04
Message: Improved TFile.mv(...).
Issue #TRUEZIP-316 - TFile.mv(...) should not copy-delete if the source and destination are regular files or directories





[TRUEZIP-315]  de.schlichtherle.truezip.rof.AbstractReadOnlyFile.readFully throws java.io.EOFException when operating on a path containing symbolic links Created: 06/Dec/12  Updated: 07/Feb/15  Resolved: 30/Jan/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.7.1
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: rmuetze Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

SLES 11, SP1 and SP2, x86_64, Quadcore virtual machine with 16GB of RAM


Tags: EOFException

 Description   

I am an end user who runs a software which usess TrueZIP. Whenever the software tried to insert an element into a ZIP file, the application log filled with:

<pre>
2012-12-06 18:12:59,241 WARN [SchedulerWorker-ZipArchiver-11,] (ZipArchiverService) - Could not archive /xxxxxx/YYYYYYYYYYYYY/zzzz/bbb/sssssssss/optlink/files/2012/12/06/18/47f2dab5-0f67-4382-9
71b-0f961f2a6dce inside archive /xxxxxx/YYYYYYYYYYYYY/zzzz/bbb/sssssssss/optlink/files/archive/2012/12/06/5959b628-1bed-4620-a6f1-954560b9df72.zip, source file exists, archive exists
java.io.EOFException
at de.schlichtherle.truezip.rof.AbstractReadOnlyFile.readFully(AbstractReadOnlyFile.java:37)
</pre>

The outcome of the operation was a 22 Byte empty ZIP file (so the minimum size of a ZIP). However, sometimes the operation succeeded after I removed all the orphan ZIP files from previous operations. In a shell, the successfully filled ZIP would then appear as an directory in Suse SLES 11. Weird enough.

I tried almost everything to find the cause since many other machines running the same software would not show this issue. I reproduced the issue at least with

  • Apache Tomcat 7.0.22 and 7.0.32
  • Oracle Java 1.6.0_35 and openJDK 1.6.0_24
  • ext3 and ext4 file systems

I changed various other minor system parameters like file and folder ownership, umask, etc. without solving the issue.

I finally found that I initially installed the software into a folder which I used a symbolic link in "/" for to shorten the path. So all configuration options would use my symbolic link name instead of the full path name. So I configured the actual path and the issue was gone. Hooray!
As a test, I inserted a symbolic link into my configuration randomly. Instead of using "/path/opt/files" I created a symbolic link "/path/optlink/files", where optlink is a symbolic link to opt in the very same folder (and filesystem, of course. The issue reappeared.

At least in the environment described above, symbolic links seem to break one or more methods of TrueZIP.



 Comments   
Comment by Christian Schlichtherle [ 06/Dec/12 ]

Thanks for your report. Would you mind telling me what application you are using? The problem is that I don't know what this application was trying to do when the exception happened.

As a general information, TrueZIP is agnostic to symbolic links. So I suppose this issue may be caused by the Java platform or even the operating system.

Comment by rmuetze [ 13/Dec/12 ]

Excuse the delay please. The application is a homegrown message archiving system running a a web application. Since I will not have access to the project sources, I asked the developer to create a testcase with the code which is being used in the app. Right now we can reproduce the issue on different systems when running the app inside Tomcat but cannot reproduce when running a standalone Java program with the same code. I am awaiting a decision whether we will further dig into this or not. For the time being, we refrain from using symbolic links in our installation.

Comment by Christian Schlichtherle [ 13/Dec/12 ]

Maybe the application should simply call TFile.getCanonicalFile() to resolve symbolic links for the archive file path?





[TRUEZIP-314] Improve exception messages Created: 06/Dec/12  Updated: 19/Jul/13  Resolved: 06/Dec/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: TrueZIP 7.7.2

Type: Improvement Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-313] CLONE -IllegalArgumentException when creating a TPath for a rooted path name with spaces Created: 20/Nov/12  Updated: 19/Jul/13  Resolved: 20/Nov/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Path
Affects Version/s: TrueZIP 7.7
Fix Version/s: TrueZIP 7.7.1

Type: Bug Priority: Major
Reporter: Konstantin Gribov Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

linux x64
oracle/sun jdk7, openjdk7



 Description   

When I use Paths.get("/some/path/with spaces") I get usual Path object that allows me to get it contens.

But if I try to create instance of TPath new TPath("/some/path/with spaces") I'll get an IllegalArgumentException:

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in path at index 15: /some/path/with spaces
	at net.java.truevfs.access.TPath.name(TPath.java:371)
	at net.java.truevfs.access.TPath.<init>(TPath.java:138)
	...
Caused by: java.net.URISyntaxException: Illegal character in path at index 15: /some/path/with spaces
	at java.net.URI$Parser.fail(URI.java:2829)
	at java.net.URI$Parser.checkChars(URI.java:3002)
	at java.net.URI$Parser.parseHierarchical(URI.java:3086)
	at java.net.URI$Parser.parse(URI.java:3044)
	at java.net.URI.<init>(URI.java:595)
	at net.java.truevfs.access.TPath.name(TPath.java:367)


 Comments   
Comment by Christian Schlichtherle [ 20/Nov/12 ]

This happens only if the path name starts with a path name separator ('/' on Unix), i.e. if it's rooted.

Comment by Christian Schlichtherle [ 20/Nov/12 ]

Changeset: 4854ba2daa6d
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-11-20 20:50
Message: Fixed.
Issue #TRUEZIP-313 - CLONE -IllegalArgumentException when creating a TPath for a rooted path name with spaces





[TRUEZIP-312] ServiceLocator duplicate services in multithreading case. Created: 15/Nov/12  Updated: 19/Nov/12  Resolved: 15/Nov/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: None

Type: Bug Priority: Major
Reporter: darknesstm Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

win7 jdk1.6 eclipse3.7


Tags: ServiceLocator

 Description   

when I use TrueZip in my eclipse plugin project, first new a TFile in a separate thread by org.eclipse.jface.dialogs.ProgressMonitorDialog.ProgressMonitorDialog, get a warnning:

警告: Found two services with the same priority 0
First: de.schlichtherle.truezip.fs.file.TempFilePoolService[priority=0]
Second: de.schlichtherle.truezip.fs.file.TempFilePoolService[priority=0]

by debug code, find in de.schlichtherle.truezip.util.ServiceLocator.getServices(Class<S>), l1 and l2 are not same not find the same class de.schlichtherle.truezip.fs.file.TempFilePoolService



 Comments   
Comment by Christian Schlichtherle [ 15/Nov/12 ]

This can't be a multithreading issue because this code is properly isolated by the JVM when accessing the IOPoolLocator.Boot class. The effect you see is the result if your current thread's context class loader is not the same as the class loader which defined the IOPoolLocator class, yet both class loaders can locate an implementation class. You can try to set up the current thread's context class loader to be identical the class loader which defined the IOPoolLocator class. Otherwise, it's just a warning and should work anyway.

Note that in TrueVFS, the successor of TrueZIP, a more sophisticated class loading framework is used which can resort this issue if one class loader is a parent of the other.

Comment by darknesstm [ 19/Nov/12 ]

Thanks for the feedback.The problem is really not a multithreading issue.Actually the two class loaders load the same class by a same class loader.





[TRUEZIP-311] Update documentation Created: 13/Nov/12  Updated: 19/Jul/13  Resolved: 30/Jan/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*, TrueZIP Path
Affects Version/s: TrueZIP 7.7
Fix Version/s: TrueZIP 7.7.2

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-309] Issue with the RDC feature which supresses redundant reprocessing when copying archive entries Created: 12/Nov/12  Updated: 16/Nov/12  Resolved: 12/Nov/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP.RAES (TZP)
Affects Version/s: TrueZIP 7.6.5, TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Critical
Reporter: torsten.janke Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

Java HotSpot(TM) 64-Bit Server VM (23.5-b02 mixed mode linux-amd64 compressed oops)
openSuse 12.2 linux 64bit


Attachments: Zip Archive archive.zip    

 Description   

When copying an encrypted archive & then extracting it to an directory an exception is thrown.

Steps to reproduce:
1. Created archive.zip with problematic GIFs (attached)

2. Created archive.tzp:
java -cp truezip-samples-7.6.6-jar-with-dependencies.jar de.schlichtherle.truezip.sample.file.app.Encrypt archive.zip archive.tzp

3. Created copy of archive.tzp (RAES -> RAES copy). According to our analysis, problem happens in this step:
java -cp truezip-samples-7.6.6-jar-with-dependencies.jar de.schlichtherle.truezip.sample.file.app.Nzip cp archive.tzp archiveCopy.tzp

4. Unzipping archiveCopy.tzp to plain directory. Crash with ZipException. Directory "output" does not exists before this step:
java -cp truezip-samples-7.6.6-jar-with-dependencies.jar de.schlichtherle.truezip.sample.file.app.Nzip cp archiveCopy.tzp output

The attached tzp file is encrypted using a 256-bit key and password "passwordpasswordpasswordpassword".

Exception thrown:
de.schlichtherle.truezip.io.InputException: java.util.zip.ZipException: invalid block type
at de.schlichtherle.truezip.io.Streams.cat(Streams.java:258)
at de.schlichtherle.truezip.io.Streams.copy(Streams.java:71)
at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:118)
at de.schlichtherle.truezip.file.TBIO.cp0(TBIO.java:221)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:179)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:169)
at de.schlichtherle.truezip.file.TBIO.cp_r(TBIO.java:138)
at de.schlichtherle.truezip.file.TFile.cp_rp(TFile.java:3318)
at de.schlichtherle.truezip.sample.file.app.Nzip.cpOrMv(Nzip.java:406)
at de.schlichtherle.truezip.sample.file.app.Nzip.runChecked(Nzip.java:162)
at de.schlichtherle.truezip.sample.file.app.Application.work(Application.java:93)
at de.schlichtherle.truezip.file.TApplication.run(TApplication.java:59)
at de.schlichtherle.truezip.sample.file.app.Nzip.main(Nzip.java:122)
Caused by: java.util.zip.ZipException: invalid block type
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at de.schlichtherle.truezip.io.DisconnectingInputStream.read(DisconnectingInputStream.java:47)
at de.schlichtherle.truezip.io.LockInputStream.read(LockInputStream.java:63)
at de.schlichtherle.truezip.io.DecoratingInputStream.read(DecoratingInputStream.java:54)
at de.schlichtherle.truezip.io.DecoratingInputStream.read(DecoratingInputStream.java:54)
at de.schlichtherle.truezip.io.DecoratingInputStream.read(DecoratingInputStream.java:54)
at de.schlichtherle.truezip.io.DecoratingInputStream.read(DecoratingInputStream.java:54)
at de.schlichtherle.truezip.io.Streams$1ReaderTask.run(Streams.java:174)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)



 Comments   
Comment by Christian Schlichtherle [ 12/Nov/12 ]

Some findings:

The two entries in the original archive.zip are using the STORED compression method. Each contains two extra fields:

0x5455 (5 bytes): Extended Timestamp
0x7875 (11 bytes): Unknown

Comment by Christian Schlichtherle [ 12/Nov/12 ]

Got it: The ZipRaesDriver makes compression mandatory, even when copying STORED entries. However, the data is copied using RDC and so there is a mismatch between the entry contents and its meta data, which corrupts the archive integrity. Now I need to check where in these many abstraction layers I have to fix the issue without breaking other things.

Comment by Christian Schlichtherle [ 12/Nov/12 ]

Please find attached a snapshot which resolves the issue.

Comment by Christian Schlichtherle [ 12/Nov/12 ]

Changeset: 34701f5e7c7e
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-11-12 18:34
Message: Resolved.
Issue #TRUEZIP-309 - Issue with the RDC feature which supresses redundant reprocessing when copying archive entries





[TRUEZIP-307] TArchiveDetector bug when accessing inner archives without accessing outer archive before Created: 12/Nov/12  Updated: 16/Nov/12  Resolved: 13/Nov/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*
Affects Version/s: TrueZIP 7.6.5, TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Minor
Reporter: torsten.janke Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

Java HotSpot(TM) 64-Bit Server VM (23.5-b02 mixed mode linux-amd64 compressed oops)
openSuse 12.2 linux 64bit


Attachments: Java Source File InnerArchiveTest.java    

 Description   

When accessing an inner archive in a certain way, TrueZIP throws an exception. Code snippet is (testclass is attached):

final TArchiveDetector detector = new TArchiveDetector("zip");
final File testArchiveAsFile = new File("archive1.zip");

final TPath mainArchive;
try (TConfig config = TConfig.push())

{ config.setArchiveDetector(detector); mainArchive = new TPath(testArchiveAsFile.getPath()); }

final TPath innerArchive;
try (TConfig config = TConfig.push())

{ config.setArchiveDetector(detector); innerArchive = mainArchive.resolve("archive2.zip"); }

//System.out.println(Files.isDirectory(innerArchive));

// Crash with error - ZIP is not known scheme (see ServiceConfigurationError.txt)
System.out.println(Files.isRegularFile(innerArchive.toNonArchivePath()));

When line with "Files.isDirectory(innerArchive)" is used, everything works ok.

Exception:
java.util.ServiceConfigurationError: zip (Unknown file system scheme! May be the class path doesn't contain the respective driver module or it isn't set up correctly?)
at de.schlichtherle.truezip.fs.FsAbstractCompositeDriver.newController(FsAbstractCompositeDriver.java:33)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController0(FsDefaultManager.java:95)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController(FsDefaultManager.java:78)
at de.schlichtherle.truezip.nio.file.TFileSystem.<init>(TFileSystem.java:53)
at de.schlichtherle.truezip.nio.file.TFileSystemProvider.getFileSystem(TFileSystemProvider.java:259)
at de.schlichtherle.truezip.nio.file.TPath.getFileSystem0(TPath.java:574)
at de.schlichtherle.truezip.nio.file.TPath.getFileSystem(TPath.java:570)
at de.schlichtherle.truezip.nio.file.TPath.getFileSystem(TPath.java:100)
at java.nio.file.Files.provider(Files.java:65)
at java.nio.file.Files.readAttributes(Files.java:1675)
at java.nio.file.Files.isDirectory(Files.java:2124)
...



 Comments   
Comment by Christian Schlichtherle [ 12/Nov/12 ]

Thanx - I'll look after it. As discussed on the user mailing list, the workaround is to access the outer archive file before the inner archive file, so that the TrueZIP Kernel has it mounted with a TArchiveDetector which contains the driver.

Comment by Christian Schlichtherle [ 13/Nov/12 ]

Changeset: 740841a36af8
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-11-13 09:13
Message: Resolved.
Issue #TRUEZIP-307 - TArchiveDetector bug when accessing inner archives without accessing outer archive before





[TRUEZIP-306] Key file selection doesn't always work Created: 11/Nov/12  Updated: 16/Nov/12  Resolved: 12/Nov/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-305] Preserve last-modified timestamp on archives when copying them with cp_rp Created: 09/Nov/12  Updated: 07/Feb/15  Resolved: 13/Nov/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: vpartington Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When recursively copying a ZIP file using TFile.cp_rp(TFile), the last-modified timestamp of the archive itself is not preserved. Neither is the last-modified timestamp of any archive within that ZIP file.

The code I am executing looks like this:

TFile in = new TFile("/home/vinny/PetClinic-1.0.war");
TFile out = new TFile("/tmp/PetClinic-1.0.war");
in.cp_rp(out);

If I execute that code, the last modified timestamp of /tmp/PetClinic-1.0.war will not be the same as that of /home/vinny/PetClinic-1.0.war, even though the latter is a copy of the first. Also, the timestamp of any archive inside of PetClinic-1.0.war will be touched as well, even though they are also copied as-is from the source to the target ZIP. I would expect the timestamp of a new archive (top-level or within another archive) to be touched when I invoke TFile.mkdir(), not when I invoke TFile.cp_rp().

We have a use case where this behaviour causes problems. In Deployit (a deployment automation product we build), we use TrueZIP to process deployable archives like WAR and EAR files to replace placeholders with environment specific values while we deploy them. A customer of ours uses that feature to modify configuration files inside the WEB-INF directory. But the last-modified timestamps of the JARs inside of WEB-INF/lib get touched in the process, even though they are not modified by our placeholder replacer. That

(as discussed on the users mailing list on November 9th 2012)



 Comments   
Comment by Christian Schlichtherle [ 13/Nov/12 ]

Ok, because this issue is backed by a valid and interesting use case, it deserves a detailed answer why I am not going to support it. The answer will detail two summaries:

1) It would confuse users and tools.
2) It would downgrade performance.
3) It would still not be a verbatim copy.
4) There is another way to support the use case.

Before I go into the details, please note that this "feature" was once supported in older versions of TrueZIP. However, I removed it for the reasons that I am now going to explain:

1) It would confuse users and tools:

TrueZIP/TrueVFS maintains a virtual file system for archive files. Like in a real file system, the last modification time of a directory changes if and only if the list of its (direct) children changes. Let's consider the following virtual file system path:

application.war/WEB-INF/lib/library.jar/META-INF/MANIFEST

Now if a tool would update the contents of the MANIFEST file, its last modification time would change. However, if all parent path elements were regular directories, then neither the last modification time of the directory META-INF nor its ancestors would change.

Now imagine this would happen if application.war and library.jar are ZIP files rather than regular directories. Without using TrueZIP/TrueVFS, a tool or a user might not notice that the data in both ZIP files has changed. The side effect of this unnoticed change may be disastrous when using backup or synchronization tools because they might not notice that they have to process such an updated archive file - the application.war file in this case.
To avoid these issues, whenever the state of the virtual file system changes, the TrueZIP/TrueVFS Kernel would reassemble the updated archive file with an updated last modification time.

Now the very same logic applies when copying archive files because the kernel doesn't know what the user's intention is, i.e. it cannot tell that you would prefer to have an exact copy of the time stamp.

2) It would downgrade performance:

Changing the last modification time of an entity in a regular file system is simple and fast. Just call TFile.setLastModified() and be done with it.

TrueZIP/TrueVFS expands this feature to be applicable for entries within archive files, too. However, although this is still simple, it may not be fast at all: Archive file formats like the ZIP file format et al require an entry's meta data to be written BEFORE the entry's content. That means you can't really change the last modification time of an entry AFTER you have written its contents, like you would when copying files in a regular directory. If you do this, then the kernel would automatically sync the archive file and restart another update cycle to ensure that the meta data is correctly set. However, this means the whole archive file gets copied and if you do this in a loop when iterating over all archive entries, the run time complexity would increase to O( s^2 ), where s is the total size of the archive file. Now consider a large archive file and it becomes clear that this would be a disaster.

Now the access tier applies some dark art spells when talking to the kernel tier to ensure that this doesn't happen when using TFile.cp(...) and TFile.cp_rp(...). However, the spells have their limits when it comes to the virtual root directory of a nested archive file - for library.jar in the example.

3) It would still not be a verbatim copy:

Even if this could be solved (by applying even more dark art spells), then what you would get would still not be a verbatim copy. This is because all archive entry meta data needs to get processed by the driver tier and chances are that the support for a particular archive file format is incomplete. In fact, even the TrueZIP Driver ZIP supports only a fraction of the ZIP File Format Specification, so you may expect some "loss of precision" when copying archive files this way.

4) There is another way to support the use case:

To make a verbatim copy of an archive file, simply instruct the kernel to unmount its virtual file system and then perform the copy. Instructions for doing this in a safe, simple and fast way are provided in the Javadoc for the TFile class here: http://truezip.java.net/apidocs/de/schlichtherle/truezip/file/TFile.html#verbatimCopy

I hope this helps.

Comment by vpartington [ 13/Nov/12 ]

Hi Christian,

Pity you won't fix this, but thank you for the clear motivation. Especially number #2 is a compelling argument. Dynamically determining whether to copy the existing last-modified timestamp or to take the current system time would be too slow, so you have to pick one of the two at the moment of creation of the ZIP. In that case, picking the current system time seems the safest choice.

Is there a chance I could convince you to add an option to control this?

Regards, Vincent.

Comment by Christian Schlichtherle [ 13/Nov/12 ]

Well, I could sprinkle some more dark art spells over the code so you could have an FsSyncOption to control this. However, reason #1, #3 and #4 would still apply. Even if you don't care for #1 and #3, then #4 still gives you a viable alternative which is more accurate (no loss of precision in the meta data) and of slightly better performance (no mounting required).

So I wonder why you would still want it? As far as I know other users are happily applying the procedure in #4. I think this should work well for you, too.

Please let me you know experience with it.

Comment by vpartington [ 19/Nov/12 ]

Hi Christian,

Agreed. We'll try #4 first and let you know how it worked out. Thanks again for replying so quickly and thinking along with us.

Regards, Vincent.





[TRUEZIP-304] InvalidPathException: Illegal char <:> Created: 09/Nov/12  Updated: 07/Feb/15  Resolved: 09/Nov/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

One of my users encountered this crash:

program.name=DocFetcher
program.version=1.1.3
program.build=20120830-2321
program.portable=false
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.7.0_07-b11
java.version=1.7.0_07
sun.arch.data.model=32
os.arch=x86
os.name=Windows 7
os.version=6.1
user.language=de
java.nio.file.InvalidPathException: Illegal char <:>
at index 5: Liste: Was für Arbeitsbedingungen mich an einem Angebot interessieren Vorlage Version 28401253882450907068.odt
at sun.nio.fs.WindowsPathParser.normalize(Unknown Source)
at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at sun.nio.fs.WindowsPath.parse(Unknown Source)
at sun.nio.fs.WindowsFileSystem.getPath(Unknown Source)
at sun.nio.fs.AbstractPath.resolve(Unknown Source)
at de.schlichtherle.truezip.fs.nio.file.FileEntry.(FileEntry.java:60)
at de.schlichtherle.truezip.fs.nio.file.FileController.getOutputSocket(FileController.java:161)
at de.schlichtherle.truezip.file.TBIO.getOutputSocket(TBIO.java:311)
at de.schlichtherle.truezip.file.TBIO.cp0(TBIO.java:218)
at de.schlichtherle.truezip.file.TBIO.cp(TBIO.java:208)
at de.schlichtherle.truezip.file.TFile.cp(TFile.java:2932)
at net.sourceforge.docfetcher.model.parse.ParseService.doParse(ParseService.java:278)
at net.sourceforge.docfetcher.model.parse.ParseService.parse(ParseService.java:229)
at net.sourceforge.docfetcher.model.index.file.FileContext.index(FileContext.java:146)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleFile(FileIndex.java:285)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:123)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:272)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:50)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:383)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
...

According to the user, the crash occurred because:

The bug occured when docfetcher on a windows machine was parsing a .tar.bz2-Archive created on a linux machine and found a colon character (":") in the filename of the file it was about to process.
Windows does not allow a ":" in the filename, linux has no such requirement.



 Comments   
Comment by Christian Schlichtherle [ 09/Nov/12 ]

As you figured, on Windows you can't use a colon in a path name because it's supposed to separate a drive letter from a path name. This issue appears because your application tries to copy/extract the Linux-originated TAR file to a Windows directory. So maybe you can change this behavior?

Comment by qforce [ 09/Nov/12 ]

I can think of two solutions:
1) Swallow the exception and report to the user that the file couldn't be extracted.
2) Rename the file on the fly during extraction. For example, I could replace colons and other invalid characters with underscores.

Does TrueZip support the second option?

Comment by Christian Schlichtherle [ 09/Nov/12 ]

The TFile.cp methods don't support path name mapping. You could implement this feature in your application if you would replicate what this method does and vary appropriately. But honestly, I wouldn't recommend to do this because this task is more complex as it seems at first sight (have a look at the source code if you are interested in the details).

So I'm afraid you are more or less left with the first option.

Comment by qforce [ 09/Nov/12 ]

Okay, thanks for your help!





[TRUEZIP-303] Update documentation Created: 30/Oct/12  Updated: 16/Nov/12  Resolved: 30/Oct/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: TrueZIP 7.7

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-302] CLONE -Add an @ExpertFeature annotation to any method which accepts a TArchiveDetector parameter Created: 30/Oct/12  Updated: 16/Nov/12  Resolved: 30/Oct/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: None
Fix Version/s: TrueZIP 7.7

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

Directly injecting a TArchiveDetector into a TFile or any of its methods can easily lead to data corruption - see http://truezip.java.net/truezip-file/usage.html#Third_Party_Access . This can lead to a lot of aggravation on the user side and the perception that TrueVFS would be unreliable and eventually corrupt data.

However, there are some use cases where these features are strictly required, so there should be a documented @ExpertFeature annotation.



 Comments   
Comment by Christian Schlichtherle [ 30/Oct/12 ]

Changeset: 0d306f147022
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-10-30 13:09
Message: Added @ExpertFeature and updated documentation.
Issue #TRUEZIP-302 - CLONE -Add an @ExpertFeature annotation to any method which accepts a TArchiveDetector parameter





[TRUEZIP-301] Some JAR|WAR files in EAR are treated as file instead of directory when using TFile.isDirectory() function Created: 29/Oct/12  Updated: 16/Nov/12  Resolved: 29/Oct/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: None

Type: Bug Priority: Major
Reporter: krase_tian Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

JDK1.6, windowsXP



 Description   

I used follow code to apply recursion to traverse the directory tree of one archive file.

public static void main(String[] args)
	{
		String path = "D:\\AA7.2.0\\av.7.2.0\\av.biz\\deploy\\av-biz.ear";
		TConfig.get().setArchiveDetector(
			new TArchiveDetector(TArchiveDetector.NULL, new Object[][] {{"jar|ear|war",
					new JarDriver(IOPoolLocator.SINGLETON)},}));
		try
		{
			printName(new TFile(path));
		}
		catch (IOException e)
		{
			e.printStackTrace();
		}
		try
		{
			TVFS.umount();
		}
		catch (FsSyncException e)
		{
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}
	
	private static void printName(TFile file) throws IOException
	{
		if (file.isDirectory())
		{
			TFile[] children = file.listFiles();

			for (TFile child : children)
			{
				printName(child);
			}
		}
		else
		{
			print(file.getAbsolutePath());
		}

	}

	private static void print(String filepath) throws IOException
	{
		if(filepath.endsWith("jar")||filepath.endsWith("war")||filepath.endsWith("ear"))
		{
			System.out.println(filepath);
		}
		
		File file  = new File("D:/a.txt");
		if(!file.exists())
		{
			file.createNewFile();
		}
		FileWriter writer = new FileWriter(file,true);
		writer.write(filepath);
		writer.write("\n");
		writer.flush();
		writer.close();
		
	}

I found some JAR|WAR files in EAR was treated as file, isDirectory return false



 Comments   
Comment by Christian Schlichtherle [ 29/Oct/12 ]

First, you should wrap the creation of the TFile object into a TConfig.push()/pop() pair, otherwise you permanently change the configuration for the current thread.

Second, I need the archive file which wasn't recognized. Can you attach it to this issue please?

Comment by Christian Schlichtherle [ 29/Oct/12 ]

Forget my first note, I forgot that this is the main method, so you are effectively changing the global configuration and I suppose this is a wanted side effect.

Comment by krase_tian [ 29/Oct/12 ]

The file I operated is more than 70M, I delete some files in it in order to upload it.

Comment by Christian Schlichtherle [ 29/Oct/12 ]

Thanks, I've downloaded the file. I suppose it contains business confidential information, so I deleted the attachment.

Comment by Christian Schlichtherle [ 29/Oct/12 ]

I checked the EAR with a project generated from TrueZIP Archetype File* 7.6.6 and I could perfectly graph it's contents using the generated Tree class, so it works for me.

To double check, could you please try to move the EAR to the current directory? Maybe the issue is only present when accessing an absolute path, although I have no reason to assume this.

Comment by Christian Schlichtherle [ 29/Oct/12 ]

The submitter has asked me to close this issue on his behalf.





[TRUEZIP-300] Incorrect positioning and closing logic in de.schlichtherle.truezip.io.SeekableByteBufferChannel Created: 24/Oct/12  Updated: 16/Nov/12  Resolved: 24/Oct/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Critical
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Fixed.






[TRUEZIP-299] CLONE -ArrayIndexOutOfBounds when reading an Extra field Created: 20/Oct/12  Updated: 19/Jul/13  Resolved: 21/Nov/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7.1

Type: Bug Priority: Major
Reporter: jamieb22 Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

BSD


Tags: extra, field

 Description   

2012-10-19 16:25:33.127 ERROR - error occurred while archiving blob. blob will be reprocessed on server restart
java.lang.ArrayIndexOutOfBoundsException: null
at java.lang.System.arraycopy(Native Method)
at net.java.truevfs.comp.zip.DefaultExtraField.readFrom(DefaultExtraField.java:48)
at net.java.truevfs.comp.zip.ExtraFields.readFrom(ExtraFields.java:166)
at net.java.truevfs.comp.zip.ZipEntry.setExtraFields(ZipEntry.java:611)
at net.java.truevfs.comp.zip.ZipEntry.setRawExtraFields(ZipEntry.java:591)
at net.java.truevfs.comp.zip.AbstractZipFile.recoverLostEntries(AbstractZipFile.java:541)
at net.java.truevfs.comp.zipdriver.AbstractZipDriver.newInput(AbstractZipDriver.java:320)
at net.java.truevfs.comp.zipdriver.AbstractZipDriver.newInput(AbstractZipDriver.java:45)
at net.java.truevfs.kernel.spec.FsArchiveDriver.newInput(FsArchiveDriver.java:196)
at net.java.truevfs.kernel.impl.TargetArchiveController.liftedTree1$1(TargetArchiveController.scala:148)
at net.java.truevfs.kernel.impl.TargetArchiveController.mount0(TargetArchiveController.scala:147)
at net.java.truevfs.kernel.impl.TargetArchiveController.mount(TargetArchiveController.scala:107)
at net.java.truevfs.kernel.impl.FileSystemArchiveController$ResetFileSystem.autoMount(FileSystemArchiveController.scala:69)
at net.java.truevfs.kernel.impl.FileSystemArchiveController.autoMount(FileSystemArchiveController.scala:36)
at net.java.truevfs.kernel.impl.BasicArchiveController.checkAccess(BasicArchiveController.scala:55)
at net.java.truevfs.kernel.impl.DefaultManager$BackController.net$java$truevfs$kernel$impl$SyncController$$super$checkAccess(DefaultManager.scala:161)
at net.java.truevfs.kernel.impl.SyncController$$anonfun$checkAccess$1.apply$mcV$sp(SyncController.scala:38)
at net.java.truevfs.kernel.impl.SyncController$$anonfun$checkAccess$1.apply(SyncController.scala:38)
at net.java.truevfs.kernel.impl.SyncController$$anonfun$checkAccess$1.apply(SyncController.scala:38)
at net.java.truevfs.kernel.impl.SyncController$class.net$java$truevfs$kernel$impl$SyncController$$apply(SyncController.scala:119)
at net.java.truevfs.kernel.impl.SyncController$class.checkAccess(SyncController.scala:38)
at net.java.truevfs.kernel.impl.DefaultManager$BackController.net$java$truevfs$kernel$impl$LockController$$super$checkAccess(DefaultManager.scala:161)
at net.java.truevfs.kernel.impl.LockController$$anonfun$checkAccess$1.apply$mcV$sp(LockController.scala:47)
at net.java.truevfs.kernel.impl.LockController$$anonfun$checkAccess$1.apply(LockController.scala:47)
at net.java.truevfs.kernel.impl.LockController$$anonfun$checkAccess$1.apply(LockController.scala:47)
at net.java.truevfs.kernel.impl.LockingStrategy.apply(LockingStrategy.scala:84)
at net.java.truevfs.kernel.impl.LockController$class.timedReadOrWriteLocked(LockController.scala:103)
at net.java.truevfs.kernel.impl.LockController$class.checkAccess(LockController.scala:47)
at net.java.truevfs.kernel.impl.DefaultManager$BackController.checkAccess(DefaultManager.scala:161)
at net.java.truevfs.kernel.impl.ArchiveControllerAdapter.checkAccess(ArchiveControllerAdapter.scala:22)
at net.java.truevfs.kernel.spec.FsAccessOptionsController.checkAccess(FsAccessOptionsController.java:83)
at net.java.truevfs.kernel.spec.FsDecoratingController.checkAccess(FsDecoratingController.java:53)
at net.java.truevfs.kernel.impl.FalsePositiveArchiveController$$anonfun$checkAccess$1.apply(FalsePositiveArchiveController.scala:74)
at net.java.truevfs.kernel.impl.FalsePositiveArchiveController$$anonfun$checkAccess$1.apply(FalsePositiveArchiveController.scala:74)
at net.java.truevfs.kernel.impl.FalsePositiveArchiveController$TryChild$.apply(FalsePositiveArchiveController.scala:195)
at net.java.truevfs.kernel.impl.FalsePositiveArchiveController.net$java$truevfs$kernel$impl$FalsePositiveArchiveController$$apply(FalsePositiveArchiveController.scala:172)
at net.java.truevfs.kernel.impl.FalsePositiveArchiveController.checkAccess(FalsePositiveArchiveController.scala:74)
at net.java.truevfs.ext.pacemaker.AspectController$$anonfun$checkAccess$1.apply$mcV$sp(AspectController.scala:37)
at net.java.truevfs.ext.pacemaker.AspectController$$anonfun$checkAccess$1.apply(AspectController.scala:37)
at net.java.truevfs.ext.pacemaker.AspectController$$anonfun$checkAccess$1.apply(AspectController.scala:37)
at net.java.truevfs.ext.pacemaker.PaceController.apply(PaceController.scala:24)
at net.java.truevfs.ext.pacemaker.AspectController.checkAccess(AspectController.scala:37)
at net.java.truevfs.access.TFile.exists(TFile.java:1660)
at com.stimulus.archiva.store.VolumeStoreV1.insertBlob(VolumeStoreV1.java:344)
at com.stimulus.archiva.store.BlobStore.insertBlob(BlobStore.java:157)
at com.stimulus.archiva.archive.ArchiveEngine.archiveBlob(ArchiveEngine.java:286)
at com.stimulus.archiva.archive.ArchiveEngine.archiveBlob(ArchiveEngine.java:201)
at com.stimulus.archiva.archive.ArchiveEngine.archiveBlob(ArchiveEngine.java:166)
at com.stimulus.archiva.receive.ReceiveService.archive(ReceiveService.java:361)
at com.stimulus.archiva.receive.ReceiveService.route(ReceiveService.java:444)
at com.stimulus.archiva.receive.ReceiveService.processFileQueueItem(ReceiveService.java:322)
at com.stimulus.archiva.queue.FileQueue$ProcessQueueConsumer.run(FileQueue.java:208)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)



 Comments   
Comment by Christian Schlichtherle [ 20/Oct/12 ]

The cause for this exception is invalid data in the Zip entry extra field.
This should have been a ZipException instead of an IndexOutOfBoundsExcepion.

Comment by Christian Schlichtherle [ 20/Nov/12 ]

The issue is still present if an extra field holds less than four bytes.

Comment by Christian Schlichtherle [ 21/Nov/12 ]

Changeset: 6ee0a5235e66
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-11-21 12:17
Message: Fixed.
Issue #TRUEZIP-299 - CLONE -ArrayIndexOutOfBounds when reading an Extra field





[TRUEZIP-297] Bugs in integration test suite Created: 01/Oct/12  Updated: 16/Nov/12  Resolved: 01/Oct/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

T(File|Path)ITSuite.test(BusyFileInputStream|IllegalDeleteOfEntryWithOpenStream) sometimes fails when running test classes and methods in parallel.






[TRUEZIP-296] TrueZIP Extension PaceManager should not sync a file system controller if the current thread has created open streams or channels for it Created: 27/Sep/12  Updated: 16/Nov/12  Resolved: 27/Sep/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Extension PaceManager
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes

Issue Links:
Duplicate
is duplicated by TRUEZIP-293 Exception when accessing an archive f... Closed

 Comments   
Comment by Christian Schlichtherle [ 27/Sep/12 ]

Changeset: 87fdcc942554
Author: christian@schlichtherle.de
Date: 2012-09-27 22:20
Message: Done.
Issue #TRUEZIP-296 - TrueZIP Extension PaceManager should not sync a file system controller if the current thread has created open streams or channels for it

Comment by Christian Schlichtherle [ 02/Oct/12 ]

Changeset: 96a1534faede
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-10-02 19:01
Message: Fixed dead lock.
Issue #TRUEZIP-296 - TrueZIP Extension PaceManager should not sync a file system controller if the current thread has created open streams or channels for it





[TRUEZIP-295] Enhance logging messages in FsFinalizeController Created: 21/Sep/12  Updated: 27/Sep/12  Resolved: 27/Sep/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.6.6
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Invalid Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-293] Exception when accessing an archive file with TrueZIP Extension PaceManager installed Created: 21/Sep/12  Updated: 16/Nov/12  Resolved: 27/Sep/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Extension PaceManager
Affects Version/s: TrueZIP 7.6, TrueZIP 7.6.1, TrueZIP 7.6.2, TrueZIP 7.6.4, TrueZIP 7.6.5, TrueZIP 7.6.6
Fix Version/s: TrueZIP 7.7

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates TRUEZIP-296 TrueZIP Extension PaceManager should ... Closed

 Description   

The TrueZIP Extension PaceManager may report an FsSyncException with an FsResourceOpenException as its cause telling you that it couldn't sync because the current thread has open streams or channels. Here is an example:

java.io.FileNotFoundException: /xyz.zip/...
        at de.schlichtherle.truezip.file.TFileInputStream.newInputStream(TFileInputStream.java:108)
        at de.schlichtherle.truezip.file.TFileInputStream.<init>(TFileInputStream.java:95)
        ...
Caused by: de.schlichtherle.truezip.fs.FsSyncException: zip:file:/xyz.zip!/
        at de.schlichtherle.truezip.fs.FsResourceController.waitIdle(FsResourceController.java:91)
        at de.schlichtherle.truezip.fs.FsResourceController.sync(FsResourceController.java:74)
        at de.schlichtherle.truezip.fs.FsCacheController.sync(FsCacheController.java:217)
        at de.schlichtherle.truezip.fs.FsSyncController.sync(FsSyncController.java:232)
        at de.schlichtherle.truezip.fs.FsLockController$1Sync.call(FsLockController.java:235)
        at de.schlichtherle.truezip.fs.FsLockController$1Sync.call(FsLockController.java:232)
        at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
        at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
        at de.schlichtherle.truezip.fs.FsLockController.sync(FsLockController.java:240)
        at de.schlichtherle.truezip.fs.FsDecoratingController.sync(FsDecoratingController.java:131)
        at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.sync(FsFalsePositiveArchiveController.java:480)
        at de.schlichtherle.truezip.fs.FsManager.sync(FsManager.java:105)
        at de.schlichtherle.truezip.extension.pace.PaceManagerModel.retain(PaceManagerModel.java:83)
        at de.schlichtherle.truezip.extension.pace.PaceManagerController.retain(PaceManagerController.java:42)
        at de.schlichtherle.truezip.extension.pace.PaceController$1Input.newInputStream(PaceController.java:159)
        at de.schlichtherle.truezip.file.TFileInputStream.newInputStream(TFileInputStream.java:104)
        ... 13 common frames omitted
Caused by: de.schlichtherle.truezip.fs.FsResourceOpenException: Thread-local / total number of open I/O resources (streams, channels etc): 1 / 1
        at de.schlichtherle.truezip.fs.FsResourceController.waitIdle(FsResourceController.java:104)
        at de.schlichtherle.truezip.fs.FsResourceController.waitIdle(FsResourceController.java:88)
        ... 28 common frames omitted

The key point here is that only the current thread had open streams or channels and so the PaceManager should simply log and discard this exception because its only effect is that the PaceManager can't evict the archive file from its LRU cache yet; but it might be able to do it later if still required and the current thread had eventually closed all its open streams or channels.






[TRUEZIP-291] Too restrictive POSIX permissions when creating an archive file on JSE 7 Created: 19/Sep/12  Updated: 20/Sep/12  Resolved: 20/Sep/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE
Affects Version/s: TrueZIP 7.3.4, TrueZIP 7.4.3, TrueZIP 7.5.5, TrueZIP 7.6, TrueZIP 7.6.1, TrueZIP 7.6.2, TrueZIP 7.6.4, TrueZIP 7.6.5
Fix Version/s: TrueZIP 7.6.6

Type: Bug Priority: Minor
Reporter: T-Gergely Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

Linux 64bit


Issue Links:
Duplicate
is duplicated by TRUEVFS-61 CLONE - Too restrictive POSIX permiss... Closed

 Description   

When I create a zip with TFile.mkdir(), its permissions become "rw-----" instead of "-rw-rr-" when my umask is 0022.



 Comments   
Comment by Christian Schlichtherle [ 19/Sep/12 ]

Please provide java -version. There are different execution paths for JSE 6 and JSE 7 in the TrueZIP Driver FILE.

Comment by Christian Schlichtherle [ 20/Sep/12 ]

I can verify this bug to be present on Mac OS X when running on JSE 7. It is NOT present when running on JSE 6.

Comment by Christian Schlichtherle [ 20/Sep/12 ]

Changeset: 4d238c601036
Author: christian@schlichtherle.de
Date: 2012-09-20 04:16
Message: Done.
Issue #TRUEZIP-291 - Too restrictive POSIX permissions when creating an archive file on JSE 7





[TRUEZIP-290] TFile.compact() should not swallow the original exception when TFile.rm() fails Created: 13/Sep/12  Updated: 16/Sep/12  Resolved: 13/Sep/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*
Affects Version/s: TrueZIP 7.6.4
Fix Version/s: TrueZIP 7.6.5

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

When calling TFile.compact() fails with an IOException, the buffer file used for compacting gets removed. However, this may fail with another IOException, too. In this case, the original IOException gets suppressed. This is bad because it makes it hard to analyze the root cause.



 Comments   
Comment by Christian Schlichtherle [ 13/Sep/12 ]

Changeset: e36024cdc3b6
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-09-13 19:09
Message: Now using Throwable.addSuppressed if JSE7.AVAILABLE.
Otherwise just discards the second IOException.
Issue #TRUEZIP-290 - TFile.compact() should not swallow the original exception when TFile.rm() fails





[TRUEZIP-288] Windows UNC path names not always resolved correctly Created: 28/Aug/12  Updated: 31/Aug/12  Resolved: 29/Aug/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver FILE
Affects Version/s: TrueZIP 7.5.5, TrueZIP 7.6, TrueZIP 7.6.1
Fix Version/s: TrueZIP 7.6.2

Type: Bug Priority: Major
Reporter: qui.tiqe Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

Windows



 Description   

The following code will result in an java.lang.IllegalArgumentException exception when using TrueZIP in combination with Java 1.7 (the path is assumed to be valid):

TFile tfile = new TFile("\\\\hostname\\share
test.zip");
System.out.println( tfile.length() );

results in:

Exception in thread "main" java.lang.IllegalArgumentException: URI authority component has undefined host
at sun.nio.fs.WindowsUriSupport.fromUri(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.getPath(Unknown Source)
at java.nio.file.Paths.get(Unknown Source)
at de.schlichtherle.truezip.fs.nio.file.FileController.<init>(FileController.java:50)
at de.schlichtherle.truezip.fs.nio.file.FileDriver.newController(FileDriver.java:31)
at de.schlichtherle.truezip.fs.FsAbstractCompositeDriver.newController(FsAbstractCompositeDriver.java:33)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController0(FsDefaultManager.java:59)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController0(FsDefaultManager.java:63)
at de.schlichtherle.truezip.fs.FsDefaultManager.getController(FsDefaultManager.java:51)
at de.schlichtherle.truezip.file.TFile.getController(TFile.java:1495)
at de.schlichtherle.truezip.file.TFile.getController(TFile.java:1490)
at de.schlichtherle.truezip.file.TFile.length(TFile.java:2070)

More surprisingly, when using Java 1.6 this error does not occur. Instead I get the result I desire.

Executing the following code will produce the expected result on my machine both for Java 1.6 AND 1.7:

File file = new File("\\\\hostname\\share
test.zip");
System.out.println( file.length() );

So as far as I can tell there seems to be a bug in TrueZIP. I also tested the latest TrueZIP revision (5541). This revision yields the same results.



 Comments   
Comment by Christian Schlichtherle [ 28/Aug/12 ]

Which version of Windows and Java are you using?

If I run the Nzip main class (in TrueZIP Samples), I can run the ll command with a path name like \\localhost\c$ without a problem.

I am running this on Windows Vista and JDK 1.7.0_06.

Comment by Christian Schlichtherle [ 28/Aug/12 ]

The Nzip class runs normal for me with UNC path names. I can list directories, archive files, etc.

Comment by qui.tiqe [ 29/Aug/12 ]

I am using Windows 7 Ultimate i.c.w. Java 1.7.0_05. I use Eclipse Indigo to run the program.

On my machine the issue occurs in the constructor of class FileController because model.getMountPoint().toUri() returns a URI that is not considered to be correct by Paths.get().

That is, a call equivalent to the following call is made:

this.target = Paths.get( new URI("file://hostname/share/") );

while a call equivalent to the following call should have been made:

this.target = Paths.get( new URI("file:////hostname/share/") );

Comment by qui.tiqe [ 29/Aug/12 ]

The following crude patch indeed solves the issue for me:

@@ -17,6 +17,7 @@
import de.schlichtherle.truezip.util.BitField;
import java.io.FileNotFoundException;
import java.io.IOException;
+import java.net.URI;
import java.nio.file.AccessDeniedException;
import java.nio.file.Files;
import static java.nio.file.Files.*;
@@ -46,7 +47,22 @@
super(model);
if (null != model.getParent())
throw new IllegalArgumentException();

  • this.target = Paths.get(model.getMountPoint().toUri());
    +
    + URI uri = model.getMountPoint().toUri();
    +
    + if ( uri.getScheme().equals("file") && uri.getRawAuthority() != null && uri.getRawSchemeSpecificPart().startsWith("//") )
    +
    Unknown macro: {+ try+ { + uri = new URI( "file://" + uri.getRawSchemeSpecificPart() ); + }+ catch (Exception e)+ { + /* Ignore */ + }+ }

    +
    + this.target = Paths.get(uri);
    }

private BasicFileAttributeView getBasicFileAttributeView(Path file) {

Comment by Christian Schlichtherle [ 29/Aug/12 ]

Using JDK 1.7.0_06 on my Windows Vista, the following program runs fine:

package javaapplication1;

import java.net.*;
import java.nio.file.*;

public class JavaApplication1 {

    public static void main(String[] args) throws Exception {
        Path legal = Paths.get(new URI("file://localhost/c$"));
        System.out.println(Files.size(legal));
        Path wierd = Paths.get(new URI("file:////localhost/c$"));
        System.out.println(Files.size(wierd));
    }
}

Note that according to the URI specification, the path "legal" is perfectly conformant, i.e. it specifies "localhost" as its authority.

In contrast, the path "wierd" has an empty authority and an empty first path segment.

I know from my test suite that early versions of the JDK 7 had issues with URI parsing. I haven't got access to JDK 1.7.0_05 anymore to verify this one, but I suggest that you upgrade your JDK to 1.7.0_06.

Comment by qui.tiqe [ 29/Aug/12 ]

On my computer the above test application indeed yields the expected results (both with JDK 1.7.0_05 and 1.7.0_06). So I did some additional tests. As it turns out, the problem only occurs if there is an underscore in computer name. So "file://computer/share/" works, but "file://computer_name/share/" does not. However, as stated before, "file:////<path>" always works. And, this is also how Java 7 seems to do it.

Try:

File file = new File("//localhost/c$");
System.out.println( file.toURI().toString() );

The output will be:

file:////localhost/c$/

So I hope I have provided two compelling reasons to make the change. Actually, it is more like reverting a previous change as the code that solves the above issue used to be present in de.schlichtherle.truezip.fs.nio.file.FileController:

[code]
FileController(final FsModel model) {
super(model);
if (null != model.getParent())
throw new IllegalArgumentException();

URI uri = model.getMountPoint().toUri();
if ('
' == separatorChar && null != uri.getRawAuthority()) {
try

{ // Postfix: Move Windows UNC host from authority to path // component because the File class can't deal with this. // Note that the authority parameter must not be null and that // you cannot use the UriBuilder class - using either of these // would result in the authority property of the new URI object // being equal to the original value again. // Note that the use of the buggy URI constructor is authorized // for this case! uri = new URI( uri.getScheme(), "", TWO_SEPARATORS + uri.getAuthority() + uri.getPath(), uri.getQuery(), uri.getFragment()); }

catch (URISyntaxException ex)

{ throw new AssertionError(ex); }

}

this.target = Paths.get(uri);
}
[/code]

Comment by Christian Schlichtherle [ 29/Aug/12 ]

You mean de.schlichtherle.truezip.fs.file.FileController?! Thank you for reminding me where I had seen this issue before. Unfortunately the URI class has lots of bugs and I obviously worked around it for the JSE 6 driver, but somehow thought it had been fixed in JSE 7 (maybe it has only been partially fixed and that's why it used to work for me).

I'll incorporate the workaround from the JSE 6 driver to the JSE 7 driver. Sorry for this annoyance.

Comment by Christian Schlichtherle [ 29/Aug/12 ]

Changeset: 4d62a89c0ec3
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-08-29 18:34
Message: Fixed.
Issue #TRUEZIP-288 - TrueZIP i.c.w. Java 1.7 not playing nice with Windows file shares‏

Comment by Christian Schlichtherle [ 29/Aug/12 ]

Qui, since you have cloned the source code repository anyway, could you please quickly give the update a try? I have applied the workaround from the JSE 6 driver to the JSE 7 driver.

Comment by qui.tiqe [ 30/Aug/12 ]

I have tested the patch, and it solves the issue for me. Thank you.





[TRUEZIP-286] Unable to untar a TAR archive entry with a name length of more than 66 characters. Created: 20/Aug/12  Updated: 19/Jul/13  Resolved: 15/May/13

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR
Affects Version/s: TrueZIP 7.6, TrueZIP 7.6.1, TrueZIP 7.6.2, TrueZIP 7.6.3, TrueZIP 7.6.4, TrueZIP 7.6.5, TrueZIP 7.6.6, TrueZIP 7.7
Fix Version/s: TrueZIP 7.7.2

Type: Bug Priority: Major
Reporter: dantran Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows/unix


Issue Links:
Duplicate
is duplicated by TRUEVFS-15 CLONE -Unable to untar a TAR archive ... Closed

 Description   

detail discussion is at http://java.net/projects/truezip/lists/users/archive/2012-08/message/5



 Comments   
Comment by Christian Schlichtherle [ 20/Aug/12 ]

I have written a small round trip test and it appears the limit is 66 characters.

The strange thing is that I call TarArchiveOutputStream.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU), so it may be a bug in commons-compress 1.4.1 - let me validate.

Comment by Christian Schlichtherle [ 20/Aug/12 ]

First issue in TarArchiveOutputStream.putArchiveEntry, line 269: The array may be longer than the limit of the returned buffer, which is ignored. This is why the bug already appears with entry names of more than 66 (US-ASCII) characters.

Comment by Christian Schlichtherle [ 20/Aug/12 ]

Second issue in line 284 of the same file: The oversized array (101 bytes instead of 67) is written to the TarArchiveOutputStream.

Comment by Christian Schlichtherle [ 20/Aug/12 ]

Next issue: TarArchiveInputStream.getNextTarEntry() reads back in the oversized array and fails to truncate all null characters at the end - it discards only one null character in line 264.

The resulting entry name is then padded with redundant null characters at its end, which is why ultimately my round trip test fails to locate the entry with its original name.

Comment by Christian Schlichtherle [ 20/Aug/12 ]

Reported as an issue in Commons Compress 1.4.1: https://issues.apache.org/jira/browse/COMPRESS-200 .

Comment by Christian Schlichtherle [ 20/Aug/12 ]

Note to self: Consider using LONGFILE_POSIX instead of LONGFILE_GNU as a workaround: http://commons.apache.org/compress/apidocs/org/apache/commons/compress/archivers/tar/TarArchiveOutputStream.html#LONGFILE_POSIX .

Comment by Christian Schlichtherle [ 20/Aug/12 ]

Nope, using LONGFILE_POSIX doesn't change a thing.

Comment by Christian Schlichtherle [ 23/Aug/12 ]

Dan, it might help if you vote for the JIRA issue #200 at commons-compress.

Comment by bodewig [ 27/Dec/12 ]

I can see how Compress is using array() the wrong way, but I fail to create a simple testcase - the one I've committed to Compress' trunk passes). The posix testcase I've written passes as well and on first glance it shouldn't be affected by the array() mishap since nameBytes isn't used in that case.

Comment by bodewig [ 27/Dec/12 ]

Rather than going back and forth between two JIRA instances, the key was to use an encoding other than the default encoding. Should be fixed in Compress' trunk soonish. I'll comment here once a fix is available in svn.

Comment by bodewig [ 27/Dec/12 ]

Commons Compress svn trunk should be fixed now. There are a few other bug reports open that need to get addressed before we can cut a new release but it would be great if you could give the next SNAPSHOT build a try (or build trunk yourself). I guess COMPRESS-203 (not verified, yet) might hit TrueZIP users as well.

Comment by Christian Schlichtherle [ 27/Dec/12 ]

Thanks a lot!





[TRUEZIP-285] NoSuchMethodError: de.schlichtherle.truezip.file.TConfig.getFsManager()Lde/schlichtherle/truezip/fs/FsManager; Created: 16/Aug/12  Updated: 17/Aug/12  Resolved: 16/Aug/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hello,

One of my users reported the following crash:

program.name=DocFetcher
program.version=1.1.1
program.build=20120815-1733
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.6.0_33-b05
java.version=1.6.0_33
sun.arch.data.model=32
os.arch=x86
os.name=Windows XP
os.version=5.1
user.language=en
java.lang.NoSuchMethodError: de.schlichtherle.truezip.file.TConfig.getFsManager()Lde/schlichtherle/truezip/fs/FsManager;
at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:284)
at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:231)
at de.schlichtherle.truezip.file.TVFS.umount(TVFS.java:82)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.runFinally(FileIndex.java:402)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:59)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:272)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:50)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:383)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:272)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$200(FileIndex.java:50)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleDir(FileIndex.java:383)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:144)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:272)
at net.sourceforge.docfetcher.model.index.file.FileIndex.doUpdate(FileIndex.java:158)
at net.sourceforge.docfetcher.model.TreeIndex.update(TreeIndex.java:136)
at net.sourceforge.docfetcher.model.index.Task.update(Task.java:98)
at net.sourceforge.docfetcher.model.index.IndexingQueue.threadLoop(IndexingQueue.java:161)
at net.sourceforge.docfetcher.model.index.IndexingQueue.access$100(IndexingQueue.java:44)
at net.sourceforge.docfetcher.model.index.IndexingQueue$2.run(IndexingQueue.java:116)



 Comments   
Comment by qforce [ 16/Aug/12 ]

I've been thinking about this and the thought occurred to me that I might simply be missing some TrueZIP jars, since I was trying to keep the download size of my program low. The following jars were present:

truezip-driver-file-7.5.5.jar
truezip-driver-tar-7.5.5.jar
truezip-driver-zip-7.5.5.jar
truezip-file-7.5.5.jar
truezip-kernel-7.5.5.jar
truezip-swing-7.5.5.jar

Comment by Christian Schlichtherle [ 16/Aug/12 ]

Apparently the class path or the class loader environment is set up incorrectly. Note that some classes are loaded dynamically, so the current thread's context class loader should be able to see the classes.

Comment by qforce [ 17/Aug/12 ]

FYI, I found out what the problem was: Multiple incompatible versions of TrueZIP were loaded into the runtime, because jars from older program versions weren't removed during installation.

Comment by Christian Schlichtherle [ 17/Aug/12 ]

You're welcome!





[TRUEZIP-284] List of file system models in the JMX agent may not be accurate. Created: 05/Aug/12  Updated: 29/Aug/12  Resolved: 05/Aug/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Extension JMX/JUL
Affects Version/s: TrueZIP 7.6
Fix Version/s: TrueZIP 7.6.1

Type: Bug Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

If both the TrueZIP Extension JMX/JUL and the TrueZIP Extension PaceManager is used, then the list of file system models registered in the MBean server by the TrueZIP Extension JMX/JUL may be larger than the maximum number of mounted archive file systems managed by the PaceManager. This is due to some caching in the TrueZIP Extension JMX/JUL.



 Comments   
Comment by Christian Schlichtherle [ 05/Aug/12 ]

Changeset: b136216aa488
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-08-05 21:50
Message: Fixed.
Issue #TRUEZIP-284 - List of file system models in the JMX agent may not be accurate.





[TRUEZIP-282] Raise logging level for calling close() on stale streams or channels in finalizer-threads to INFO Created: 31/Jul/12  Updated: 29/Aug/12  Resolved: 31/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

A stale resource is an I/O stream or channel which has been opened by the client application but was somehow not close()d after its use.

Depending upon the nature of the entity to which they are referring to, resources allocate memory in the operating system and/or the TrueZIP Kernel, so a stale resource is a memory leak. Even worse, a stale resource may yield unwanted side effects, for example redundant buffering of output data to temporary files or even not being able to unmount a virtual archive file system. Hence, a stale resource is considered to be a programming bug in the client application!

To provide a limited level of protection against unwanted side effects, TrueZIP 7 always close()s stale resources in finalizer-Threads. However, this will only work if the stale resource gets eventually picked up by a finalizer-Thread, which is unreliable.

Until TrueZIP 7.5.5, finalizer-threads logged about a successful call to close() on a stale resource only at the level FINE. This enabled users to trace for such events, but only if java.util.logging was configured appropriately.

As of TrueZIP 7.6, the logging level is raised to INFO for a successful call to close() on a stale resource. By default, this will notify users of such an event on the console unless java.util.logging has been configured otherwise.

Note that if a call to close() on a stale resource fails, a WARNING message is logged - this hasn't changed.

To prevent annoying users with such logging messages, application developers should ensure to close() their streams and channels by applying the following idiom:

InputStream in = new TFileInputStream(...);
try {
    in...
} finally {
    in.close(); // may throw an IOException, too!
}

In this example, a TFileInputStream was used, but this idiom applies likewise to any kind of I/O stream or channel.

Note that the call to close() may fail with an IOException, too. It is an error to conceal an IOException from a call to close()!

Using Java 7, this can get shortened to:

try (InputStream in = new TFileInputStream(...)) {
    in...
}


 Comments   
Comment by Christian Schlichtherle [ 02/Aug/12 ]

Changeset: 4179db8c7cd5
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-08-02 09:34
Message: Done.
Issue #TRUEZIP-282 - Raise logging level for calling close() on stale streams or channels in finalizer-threads to INFO





[TRUEZIP-281] DateTimeConverter.ZIP fails to recognize change of daylight savings in long running applications Created: 24/Jul/12  Updated: 29/Aug/12  Resolved: 24/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 24/Jul/12 ]

Changeset: 6b3960f87306
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-24 15:55
Message: Fixed.
Issue #TRUEZIP-281 - DateTimeConverter.ZIP fails to recognize change of daylight savings in long running applications

Comment by Christian Schlichtherle [ 25/Jul/12 ]

Changeset: 8c442983d8ea
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-24 15:57
Message: Fixed.
Issue #TRUEZIP-281 - DateTimeConverter.ZIP fails to recognize change of daylight savings in long running applications





[TRUEZIP-280] Upgrade to Apache HttpClient 4.2.1 Created: 18/Jul/12  Updated: 29/Aug/12  Resolved: 18/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver HTTP(S)
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 18/Jul/12 ]

Changeset: b97a00e57a86
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-18 14:37
Message: Done
Issue #TRUEZIP-280 - Upgrade to Apache HttpClient 4.2.1

Comment by Christian Schlichtherle [ 18/Jul/12 ]

Changeset: 184cbc146b54
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-18 14:37
Message: Done
Issue #TRUEZIP-280 - Upgrade to Apache HttpClient 4.2.1





[TRUEZIP-279] ResourceController$ResourceOutputStream.close() et al should stop accounting for the resource unless a control flow exception gets thrown Created: 17/Jul/12  Updated: 29/Aug/12  Resolved: 19/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Critical
Reporter: arjohn Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

Issue is described on the mailing list: http://java.net/projects/truezip/lists/users/archive/2012-07/message/11

TFile.cp(File, File) does not always close all streams that it opens. This has been seen in a case where an attempt was made to copy a virus file to a zip. The virus scanner would block/remove the concerning file, triggering a FileNotFoundException somewhere deep inside the truezip framework:

FileNotFoundException(Throwable).<init>(String) line: 198
FileNotFoundException(Exception).<init>(String) line: not available
FileNotFoundException(IOException).<init>(String) line: not available
FileNotFoundException.<init>(String, String) line: not available
FileInputStream.open(String) line: not available [native method]
FileInputStream.<init>(File) line: not available
FileInputSocket.newInputStream() line: 43
ZipOutputShop$BufferedEntryOutputStream.storeBuffer() line: 385
ZipOutputShop$BufferedEntryOutputStream.close() line: 368
FsMultiplexedOutputShop$EntryOutputStream.close() line: 202
DisconnectingOutputShop$DisconnectingOutputStream.close() line: 250
LockOutputStream.close() line: 85
FsResourceController$ResourceOutputStream.close() line: 352
FsSyncController.close(Closeable) line: 297
FsSyncController$SyncOutputStream.close() line: 527
FsLockController$1Close.call() line: 620
FsLockController$1Close.call() line: 617
FsLockController.locked(IOOperation<T>, Lock) line: 364
FsLockController.writeLocked(IOOperation<T>) line: 303
FsLockController.close(Closeable) line: 625
FsLockController$LockOutputStream.close() line: 612
FsFinalizeController$FinalizeOutputStream.close() line: 340
Streams.copy(InputStream, OutputStream) line: 78
IOSocket<LT,PT>.copy(InputSocket<?>, OutputSocket<?>) line: 118
TBIO.cp0(boolean, File, File) line: 221
TBIO.cp(boolean, File, File) line: 208
TFile.cp(File, File) line: 2994
...

This FileNotFoundException prevents FsResourceController$ResourceOutputStream.close() from deregistering the stream with the FsResourceController.

A possible work-around



 Comments   
Comment by Christian Schlichtherle [ 17/Jul/12 ]

I possibly fixed it - please give it a try and let me know.

Comment by Christian Schlichtherle [ 17/Jul/12 ]

Changeset: 094436fc73fb
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-17 22:12
Message: Hopefully fixed.
Issue #TRUEZIP-279 - TFile.cp(File,File) doesn't always close all streams

Comment by Christian Schlichtherle [ 17/Jul/12 ]

Changeset: a7c636738600
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-17 22:15
Message: Hopefully fixed.
Issue #TRUEZIP-279 - TFile.cp(File,File) doesn't always close all streams

Comment by arjohn [ 18/Jul/12 ]

The process no longer freezes, but the produced zip file is broken. I found the following new stack trace in our logs:

Failed to commit the updates
de.schlichtherle.truezip.fs.FsSyncException: zip:file:/C:/Users/Arjohn%20Kampman/AppData/Roaming/Intella/cases/avtest/itemcontent/itemcontent.zip!/
at de.schlichtherle.truezip.fs.FsTargetArchiveController.close(FsTargetArchiveController.java:544)
at de.schlichtherle.truezip.fs.FsTargetArchiveController.sync(FsTargetArchiveController.java:433)
at de.schlichtherle.truezip.fs.FsContextController.sync(FsContextController.java:205)
at de.schlichtherle.truezip.fs.FsResourceController.sync(FsResourceController.java:77)
at de.schlichtherle.truezip.fs.FsCacheController.sync(FsCacheController.java:218)
at de.schlichtherle.truezip.fs.FsSyncController.sync(FsSyncController.java:232)
at de.schlichtherle.truezip.fs.FsLockController$1Sync.call(FsLockController.java:235)
at de.schlichtherle.truezip.fs.FsLockController$1Sync.call(FsLockController.java:232)
at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
at de.schlichtherle.truezip.fs.FsLockController.sync(FsLockController.java:240)
at de.schlichtherle.truezip.fs.archive.zip.KeyController.sync(KeyController.java:128)
at de.schlichtherle.truezip.fs.FsDecoratingController.sync(FsDecoratingController.java:131)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.sync(FsFalsePositiveArchiveController.java:480)
at de.schlichtherle.truezip.fs.FsManager.sync(FsManager.java:105)
at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:284)
at de.schlichtherle.truezip.file.TVFS.sync(TVFS.java:231)
at ...
Caused by: java.io.IOException: This multiplexed output shop is still busy with writing a stream!
at de.schlichtherle.truezip.socket.MultiplexedOutputShop.close(MultiplexedOutputShop.java:158)
at de.schlichtherle.truezip.socket.DisconnectingOutputShop.close(DisconnectingOutputShop.java:102)
at de.schlichtherle.truezip.socket.LockOutputShop.close(LockOutputShop.java:70)
at de.schlichtherle.truezip.fs.FsTargetArchiveController.close(FsTargetArchiveController.java:539)
... 22 common frames omitted

The concerning directory has been excluded from the virus scanner.

Comment by Christian Schlichtherle [ 18/Jul/12 ]

I'll check this.

Comment by Christian Schlichtherle [ 18/Jul/12 ]

Changeset: b13c4d405541
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-18 15:19
Message: Another fix.
Issue #TRUEZIP-279 - ResourceController$ResourceOutputStream.close() et al should stop accounting for the resource unless a control flow exception gets thrown

Comment by Christian Schlichtherle [ 18/Jul/12 ]

Changeset: 4e5d379c976c
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-18 15:19
Message: Another fix.
Issue #TRUEZIP-279 - ResourceController$ResourceOutputStream.close() et al should stop accounting for the resource unless a control flow exception gets thrown

Comment by Christian Schlichtherle [ 18/Jul/12 ]

Please try the new attachments and let me know your results.

Comment by arjohn [ 18/Jul/12 ]

First results look good. The problems with the small test that I used are gone. I'll try a larger test now. This will take a couple of hours to complete, so I'll report back tomorrow morning.

Comment by Christian Schlichtherle [ 18/Jul/12 ]

Great. Meanwhile I'll review the entire controller chain to check if I need to apply the pattern there, too.

Comment by arjohn [ 19/Jul/12 ]

All is looking fine here. I haven't seen any truezip issues during today's testing with the last snapshot.





[TRUEZIP-278] FsEntry.toString() throws IllegalArgumentException if getTypes() returns an empty set. Created: 14/Jul/12  Updated: 29/Aug/12  Resolved: 14/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 14/Jul/12 ]

Changeset: 42efe15106d0
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-14 14:31
Message: Fixed.
Issue #TRUEZIP-278 - FsEntry.toString() throws IllegalArgumentException if getTypes() returns an empty set.





[TRUEZIP-277] tomcat reports truezip fails to remove the inheritable thread local configuration stack Created: 13/Jul/12  Updated: 29/Aug/12  Resolved: 13/Aug/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*
Affects Version/s: TrueZIP 7.6
Fix Version/s: TrueZIP 7.6.1

Type: Bug Priority: Minor
Reporter: jamieb22 Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

Darwin e4-ce-8f-0f-c2-b0.dummy.porta.siemens.net 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64


Attachments: Java Archive File truezip-extension-jmx-jul-7.6.1-SNAPSHOT.jar     Java Archive File truezip-extension-pace-7.6.1-SNAPSHOT.jar     Java Archive File truezip-samples-7.6.1-SNAPSHOT-jar-with-dependencies.jar    
Tags: memory-leak, thread-local

 Description   

Jul 13, 2012 4:22:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/mailarchiva] created a ThreadLocal with key of type [de.schlichtherle.truezip.util.InheritableThreadLocalStack.Stacks] (value [de.schlichtherle.truezip.util.InheritableThreadLocalStack$Stacks@4c17a1bc]) and a value of type [java.util.LinkedList] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Jul 13, 2012 4:22:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks



 Comments   
Comment by Christian Schlichtherle [ 14/Jul/12 ]

http://stackoverflow.com/questions/11484104/how-to-test-if-a-threadlocal-has-been-initialized-without-actually-doing-that

Comment by Christian Schlichtherle [ 13/Aug/12 ]

This appears to be the "Custom ThreadLocal class" issue described on this page:
http://wiki.apache.org/tomcat/MemoryLeakProtection

While upgrading to TomCat >= 7.0.6 should fix the problem, I am investigating how to fix the root cause.

Comment by Christian Schlichtherle [ 13/Aug/12 ]

Changeset: 5b598eb3ae45
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-08-13 23:23
Message: Fixed.
Issue #TRUEZIP-277 - tomcat reports truezip fails to remove the inheritable thread local configuration stack

Comment by Christian Schlichtherle [ 13/Aug/12 ]

Changeset: 59171255fbe0
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-08-13 23:23
Message: Fixed.
Issue #TRUEZIP-277 - tomcat reports truezip fails to remove the inheritable thread local configuration stack

Comment by Christian Schlichtherle [ 13/Aug/12 ]

Uploaded JARs for testing.





[TRUEZIP-274] NegativeArraySizeException in Tar archive Created: 07/Jul/12  Updated: 29/Aug/12  Resolved: 08/Jul/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.4.3
Fix Version/s: TrueZIP 7.5.1

Type: Bug Priority: Major
Reporter: qforce Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on TRUEZIP-251 Update to Apache Commons Compress 1.4 Closed

 Description   

Hi there! One of my users reported the following crash:

program.name=DocFetcher
program.version=1.1-beta6
program.build=20120303-0018
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.6.0_31-b05
java.version=1.6.0_31
sun.arch.data.model=32
os.arch=x86
os.name=Windows XP
os.version=5.1
user.language=es
java.lang.NegativeArraySizeException
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.paxHeaders(TarArchiveInputStream.java:278)
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextTarEntry(TarArchiveInputStream.java:222)
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextEntry(TarArchiveInputStream.java:361)
at de.schlichtherle.truezip.fs.archive.tar.TarInputShop.<init>(TarInputShop.java:91)
at de.schlichtherle.truezip.fs.archive.tar.TarDriver.newTarInputShop(TarDriver.java:159)
at de.schlichtherle.truezip.fs.archive.tar.TarGZipDriver.newTarInputShop(TarGZipDriver.java:82)
at de.schlichtherle.truezip.fs.archive.tar.TarDriver.newInputShop(TarDriver.java:151)
at de.schlichtherle.truezip.fs.archive.tar.TarDriver.newInputShop(TarDriver.java:47)
at de.schlichtherle.truezip.fs.archive.FsDefaultArchiveController.mount(FsDefaultArchiveController.java:170)
at de.schlichtherle.truezip.fs.archive.FsFileSystemArchiveController$ResetFileSystem.autoMount(FsFileSystemArchiveController.java:98)
at de.schlichtherle.truezip.fs.archive.FsFileSystemArchiveController.autoMount(FsFileSystemArchiveController.java:47)
at de.schlichtherle.truezip.fs.archive.FsArchiveController.autoMount(FsArchiveController.java:129)
at de.schlichtherle.truezip.fs.archive.FsArchiveController.getEntry(FsArchiveController.java:160)
at de.schlichtherle.truezip.fs.archive.FsContextController.getEntry(FsContextController.java:117)
at de.schlichtherle.truezip.fs.FsDecoratingController.getEntry(FsDecoratingController.java:76)
at de.schlichtherle.truezip.fs.FsDecoratingController.getEntry(FsDecoratingController.java:76)
at de.schlichtherle.truezip.fs.FsConcurrentController.getEntry(FsConcurrentController.java:164)
at de.schlichtherle.truezip.fs.FsSyncController.getEntry(FsSyncController.java:108)
at de.schlichtherle.truezip.fs.FsFederatingController.getEntry(FsFederatingController.java:156)
at de.schlichtherle.truezip.file.TFile.isDirectory(TFile.java:2108)
at net.sourceforge.docfetcher.model.index.file.FileIndex.switchSolidToArchive(FileIndex.java:782)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitSolidArchive(FileIndex.java:550)
at net.sourceforge.docfetcher.model.index.file.FileIndex.switchDirZipToSolid(FileIndex.java:469)
at net.sourceforge.docfetcher.model.index.file.FileIndex.access$000(FileIndex.java:50)
at net.sourceforge.docfetcher.model.index.file.FileIndex$1.handleFile(FileIndex.java:268)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.runWithHtmlPairing(HtmlFileLister.java:123)
at net.sourceforge.docfetcher.model.index.file.HtmlFileLister.doRun(HtmlFileLister.java:56)
at net.sourceforge.docfetcher.util.Stoppable.run(Stoppable.java:57)
at net.sourceforge.docfetcher.model.index.file.FileIndex.visitDirOrZip(FileIndex.java:263)
at net.sourceforge.docfetcher.model.index.file.FileIndex.doUpdate(FileIndex.java:156)
at net.sourceforge.docfetcher.model.TreeIndex.update(TreeIndex.java:126)
at net.sourceforge.docfetcher.model.index.Task.update(Task.java:98)
at net.sourceforge.docfetcher.model.index.IndexingQueue.threadLoop(IndexingQueue.java:162)
at net.sourceforge.docfetcher.model.index.IndexingQueue.access$100(IndexingQueue.java:45)
at net.sourceforge.docfetcher.model.index.IndexingQueue$2.run(IndexingQueue.java:117)



 Comments   
Comment by Christian Schlichtherle [ 08/Jul/12 ]

This issue really depends on commons-compress. TrueZIP 7.5.1 has updated its dependency to commons-compress 1.4 which greatly improves its TAR implementation. Currently, TrueZIP 7.5.5 is the most recent version. This week, TrueZIP 7.6 should get released which updates it's dependency to commons-compress 1.4.1. Check with commons-compress for their issue list.

Comment by qforce [ 08/Jul/12 ]

Ah, OK. I searched the commons-compress issue tracker for "NegativeArraySizeException", with no results, so I guess I'll just update the commons-compress jar and report to the commons-compress people directly when this bug shows up again.

Comment by Christian Schlichtherle [ 08/Jul/12 ]

Right. Don't forget to update your TrueZIP dependencies. Especially the new TrueZIP Extension Pacemaker that'll appear in 7.6 should be beneficial for your use case.





[TRUEZIP-273] Welcome TrueZIP Extension PaceManager! Created: 07/Jul/12  Updated: 29/Aug/12  Resolved: 07/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Extension PaceManager
Affects Version/s: None
Fix Version/s: TrueZIP 7.6

Type: New Feature Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

This module constrains the number of mounted archive files in order to save some heap space. The savings can be significant if an application is accessing many archive files.

The module provides a JMX interface for monitoring and management. Check for the ObjectName "de.schlichtherle.truezip.extension.pace:type=PaceManager".






[TRUEZIP-272] Make archive controller chain eligible for GC after sync() if the selective entry cache is empty. Created: 07/Jul/12  Updated: 29/Aug/12  Resolved: 07/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

This would be a major improvement of memory management when accessing thousands of archive files.



 Comments   
Comment by Christian Schlichtherle [ 07/Jul/12 ]

Changeset: 6d3e4aa9bf9e
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-07 09:56
Message: RCB
Issue #TRUEZIP-272 - Make archive controller chain eligible for GC after sync() if the selective entry cache is empty.

Comment by Christian Schlichtherle [ 07/Jul/12 ]

Changeset: f5d510e38e7e
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-07 12:26
Message: Had to revert the change because it fails the integration tests.
Issue #TRUEZIP-272 - Make archive controller chain eligible for GC after sync() if the selective entry cache is empty.

Comment by Christian Schlichtherle [ 07/Jul/12 ]

Changeset: e0e1ca177538
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-07 19:07
Message: Done.
Issue #TRUEZIP-272 - Make archive controller chain eligible for GC after sync() if the selective entry cache is empty.





[TRUEZIP-271] Memory leak in TFileSystemProvider.filesystems Created: 06/Jul/12  Updated: 29/Aug/12  Resolved: 07/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Path
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

... because the filesystem values of this map refer to their mount point keys. Wrap the filesystem in a weak reference before putting it in the map to solve this issue.



 Comments   
Comment by Christian Schlichtherle [ 07/Jul/12 ]

Changeset: 4c9363d9126c
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-07 09:31
Message: Done.
Issue #TRUEZIP-271 - Memory leak in TFileSystemProvider.filesystems





[TRUEZIP-270] TrueZIP Extension JMX should sync() without CLEAR_CACHE Created: 06/Jul/12  Updated: 29/Aug/12  Resolved: 06/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Extension JMX/JUL
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on TRUEZIP-268 Using FsSyncOption.CLEAR_CACHE induce... Closed
depends on TRUEZIP-269 Using FsSyncOption.CLEAR_CACHE induce... Closed

 Description   

... in order to avoid potential dead locks and busy loops.



 Comments   
Comment by Christian Schlichtherle [ 06/Jul/12 ]

Done.





[TRUEZIP-269] Using FsSyncOption.CLEAR_CACHE induces busy loops in other threads Created: 06/Jul/12  Updated: 29/Aug/12  Resolved: 17/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks TRUEZIP-270 TrueZIP Extension JMX should sync() w... Closed
Related
is related to TRUEZIP-268 Using FsSyncOption.CLEAR_CACHE induce... Closed
is related to TRUEZIP-268 Using FsSyncOption.CLEAR_CACHE induce... Closed

 Description   

... when working with nested archive files and there are copy operations where an input stream has been successfully acquired and then acquiring the output stream would require an automatic sync() of the same target archive file from which the input stream is reading.



 Comments   
Comment by Christian Schlichtherle [ 06/Jul/12 ]

Fixing this issue requires a bigger refactoring and should get addressed in TrueZIP 8, a.k.a. TrueVFS.

For now, the workaround is not to use CLEAR_CACHE when synchronizing archive files from a different thread than those who access these archive files.





[TRUEZIP-268] Using FsSyncOption.CLEAR_CACHE induces extended dead locks in other threads Created: 06/Jul/12  Updated: 29/Aug/12  Resolved: 17/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes

Issue Links:
Dependency
blocks TRUEZIP-270 TrueZIP Extension JMX should sync() w... Closed
Related
is related to TRUEZIP-269 Using FsSyncOption.CLEAR_CACHE induce... Closed
is related to TRUEZIP-269 Using FsSyncOption.CLEAR_CACHE induce... Closed

 Description   

When running TrueZIP's integration test suite, an extended dead lock may happen as the following output from JVisualVM's ThreadInspector shows:

2012-07-06 10:39:25

"de.schlichtherle.truezip.io.Streams$ReaderThread" - Thread t@656
   java.lang.Thread.State: WAITING
	at sun.misc.Unsafe.park(Native Method)
	- waiting to lock <14f5969> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) owned by "main" t@1
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
	at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290)
	at de.schlichtherle.truezip.io.LockInputStream.read(LockInputStream.java:61)
	at de.schlichtherle.truezip.io.DecoratingInputStream.read(DecoratingInputStream.java:54)
	at de.schlichtherle.truezip.io.DecoratingInputStream.read(DecoratingInputStream.java:54)
	at de.schlichtherle.truezip.io.Streams$1ReaderTask.run(Streams.java:174)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

   Locked ownable synchronizers:
	- locked <32c746> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"main" - Thread t@1
   java.lang.Thread.State: WAITING
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for <51a79b> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at de.schlichtherle.truezip.io.Streams.cat(Streams.java:210)
	at de.schlichtherle.truezip.io.Streams.copy(Streams.java:71)
	at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:118)
	at de.schlichtherle.truezip.socket.IOCache$InputBufferPool.allocate(IOCache.java:318)
	at de.schlichtherle.truezip.socket.IOCache$Input.getDelegate(IOCache.java:274)
	at de.schlichtherle.truezip.socket.DelegatingInputSocket.getBoundSocket(DelegatingInputSocket.java:43)
	at de.schlichtherle.truezip.socket.DelegatingInputSocket.newReadOnlyFile(DelegatingInputSocket.java:53)
	at de.schlichtherle.truezip.socket.DelegatingInputSocket.newReadOnlyFile(DelegatingInputSocket.java:53)
	at de.schlichtherle.truezip.fs.FsSyncController$Input.newReadOnlyFile(FsSyncController.java:400)
	at de.schlichtherle.truezip.fs.FsLockController$Input$1NewReadOnlyFile.call(FsLockController.java:481)
	at de.schlichtherle.truezip.fs.FsLockController$Input$1NewReadOnlyFile.call(FsLockController.java:478)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:352)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:303)
	at de.schlichtherle.truezip.fs.FsLockController$Input.newReadOnlyFile(FsLockController.java:486)
	at de.schlichtherle.truezip.fs.FsFinalizeController$Input.newReadOnlyFile(FsFinalizeController.java:170)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input$NewReadOnlyFile.call(FsFalsePositiveController.java:328)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input$NewReadOnlyFile.call(FsFalsePositiveController.java:321)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$TryChild.call(FsFalsePositiveController.java:536)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController.call(FsFalsePositiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input.newReadOnlyFile(FsFalsePositiveController.java:318)
	at de.schlichtherle.truezip.fs.archive.zip.ZipDriver.newInputShop(ZipDriver.java:468)
	at de.schlichtherle.truezip.fs.archive.FsTargetArchiveController.mount0(FsTargetArchiveController.java:219)
	at de.schlichtherle.truezip.fs.archive.FsTargetArchiveController.mount(FsTargetArchiveController.java:174)
	at de.schlichtherle.truezip.fs.archive.FsFileSystemArchiveController$ResetFileSystem.autoMount(FsFileSystemArchiveController.java:86)
	at de.schlichtherle.truezip.fs.archive.FsFileSystemArchiveController.autoMount(FsFileSystemArchiveController.java:38)
	at de.schlichtherle.truezip.fs.archive.FsArchiveController$1Output.mknod(FsArchiveController.java:274)
	at de.schlichtherle.truezip.fs.archive.FsArchiveController$1Output.getLocalTarget(FsArchiveController.java:221)
	at de.schlichtherle.truezip.fs.archive.FsArchiveController$1Output.getLocalTarget(FsArchiveController.java:218)
	at de.schlichtherle.truezip.fs.archive.FsContextController$Output.getLocalTarget(FsContextController.java:327)
	at de.schlichtherle.truezip.fs.archive.FsContextController$Output.getLocalTarget(FsContextController.java:311)
	at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:47)
	at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:21)
	at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:47)
	at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:21)
	at de.schlichtherle.truezip.fs.FsSyncController$Output.getLocalTarget(FsSyncController.java:455)
	at de.schlichtherle.truezip.fs.FsSyncController$Output.getLocalTarget(FsSyncController.java:442)
	at de.schlichtherle.truezip.fs.FsLockController$Output$1GetLocalTarget.call(FsLockController.java:539)
	at de.schlichtherle.truezip.fs.FsLockController$Output$1GetLocalTarget.call(FsLockController.java:536)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:352)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:303)
	at de.schlichtherle.truezip.fs.FsLockController$Output.getLocalTarget(FsLockController.java:543)
	at de.schlichtherle.truezip.fs.FsLockController$Output.getLocalTarget(FsLockController.java:525)
	at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:47)
	at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:21)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Output$GetLocalTarget.call(FsFalsePositiveController.java:404)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Output$GetLocalTarget.call(FsFalsePositiveController.java:397)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$TryChild.call(FsFalsePositiveController.java:536)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController.call(FsFalsePositiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Output.getLocalTarget(FsFalsePositiveController.java:394)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Output.getLocalTarget(FsFalsePositiveController.java:378)
	at de.schlichtherle.truezip.socket.InputSocket.getPeerTarget(InputSocket.java:50)
	at de.schlichtherle.truezip.fs.archive.zip.ZipInputShop$1Input.newInputStream(ZipInputShop.java:130)
	at de.schlichtherle.truezip.socket.DisconnectingInputShop$Input.newInputStream(DisconnectingInputShop.java:162)
	at de.schlichtherle.truezip.socket.LockInputShop$1Input.newInputStream(LockInputShop.java:147)
	at de.schlichtherle.truezip.fs.archive.FsTargetArchiveController$1Input.newInputStream(FsTargetArchiveController.java:320)
	at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
	at de.schlichtherle.truezip.fs.archive.FsContextController$Input.newInputStream(FsContextController.java:304)
	at de.schlichtherle.truezip.fs.FsResourceController$Input.newInputStream(FsResourceController.java:248)
	at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
	at de.schlichtherle.truezip.fs.FsSyncController$Input.newInputStream(FsSyncController.java:412)
	at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:494)
	at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:491)
	at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:364)
	at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:303)
	at de.schlichtherle.truezip.fs.FsLockController$Input.newInputStream(FsLockController.java:499)
	at de.schlichtherle.truezip.fs.FsFinalizeController$Input.newInputStream(FsFinalizeController.java:176)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input$NewInputStream.call(FsFalsePositiveController.java:363)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input$NewInputStream.call(FsFalsePositiveController.java:356)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$TryChild.call(FsFalsePositiveController.java:536)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController.call(FsFalsePositiveController.java:104)
	at de.schlichtherle.truezip.fs.FsFalsePositiveController$1Input.newInputStream(FsFalsePositiveController.java:353)
	at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:100)
	at de.schlichtherle.truezip.file.TBIO.cp0(TBIO.java:221)
	at de.schlichtherle.truezip.file.TBIO.cp(TBIO.java:208)
	at de.schlichtherle.truezip.file.TFile.cp_p(TFile.java:3083)
	at de.schlichtherle.truezip.file.TFileITSuite.assertCopyDelete0(TFileITSuite.java:941)
	at de.schlichtherle.truezip.file.TFileITSuite.assertCopyDelete0(TFileITSuite.java:925)
	at de.schlichtherle.truezip.file.TFileITSuite.assertCopyDelete(TFileITSuite.java:906)
	at de.schlichtherle.truezip.file.TFileITSuite.assertCopyDelete(TFileITSuite.java:889)
	at de.schlichtherle.truezip.file.TFileITSuite.testCopyDelete(TFileITSuite.java:873)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.apache.maven.surefire.junitcore.ClassDemarcatingRunner.run(ClassDemarcatingRunner.java:58)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:24)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.apache.maven.surefire.junitcore.SynchronousRunner.schedule(SynchronousRunner.java:13)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:62)
	at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:139)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)

   Locked ownable synchronizers:
	- locked <14f5969> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

	- locked <818376> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)

	- locked <ddc385> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)

As you can see, the main thread is calling Streams.cat(InputStream, OutputStream). This method is awaiting a Condition (51a79b) which should get signaled by the ReaderThread. However, the ReaderThread needs to lock a ReentrantLock (14f5969) which is already acquired by the main thread. Voila, an extended dead lock!

I call this an "extended dead lock" because it involves awaiting a condition (in the main thread signaled by the ReaderThread) rather than locking a monitor. Unfortunately, such a condition is not detected by JConsole and YourKit. I'm not sure if this should get addressed by these tools, but that's not the topic here.



 Comments   
Comment by Christian Schlichtherle [ 06/Jul/12 ]

There are some options to consider:

  1. Integrate the lock used by the LockInputShop into the lock management applied by the LockController. Unfortunately, in TrueZIP 7 this would require a serious refactoring. In TrueZIP 8 (a.k.a. TrueVFS), it would be much simpler.
  2. Make the LockInputShop temporarily release the lock when calling getPeerTarget(). Mind you that in TrueVFS sockets are stateless and therefore there is no getPeerTarget().
  3. Give the ReaderThread somehow reentrant access to all locks acquired by the calling thread of Streams.cat(InputStream, OutputStream). This would be a nice, general solution to avoid dead locks induced by a call to this method, but I don't think this is possible at all.
  4. Change the execution path somehow so that the call to Streams.cat(InputStream, OutputStream) is not required.
Comment by Christian Schlichtherle [ 06/Jul/12 ]

Fixing this issue requires a bigger refactoring and should get addressed in TrueZIP 8, a.k.a. TrueVFS.

For now, the workaround is not to use CLEAR_CACHE when synchronizing archive files from a different thread than those who access these archive files.

Comment by Christian Schlichtherle [ 07/Jul/12 ]

Changeset: 848924649349
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-06 17:16
Message: Remove usage of CLEAR_CACHE in order to prevent possible dead locks or busy loops.
Issue #TRUEZIP-268 - Using FsSyncOption.CLEAR_CACHE induces extended dead locks in other threads





[TRUEZIP-267] Improve concurrency of the default archive manager Created: 06/Jul/12  Updated: 29/Aug/12  Resolved: 07/Jul/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

... by using a ReentrantReadWriteLock instead of synchronized methods where some methods can safely use a ReadLock.

This will help applications which do a lot of sync()ing, e.g. when using the new TrueZIP Extension Pacemaker.



 Comments   
Comment by Christian Schlichtherle [ 07/Jul/12 ]

Changeset: dd33c337bd18
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-07-06 13:43
Message: Done.
Issue #TRUEZIP-267 - Improve concurrency of the default archive manager





[TRUEZIP-266] The sample code for key management should customize the ZipDriver class, not the JarDriver class Created: 22/Jun/12  Updated: 29/Aug/12  Resolved: 22/Jun/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

..., because WinZip AES is not used for JAR files, but only for ZIP files.

See http://truezip.java.net/truezip-driver/truezip-driver-zip/key-management.html



 Comments   
Comment by Christian Schlichtherle [ 22/Jun/12 ]

Changeset: c0c698834571
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-06-22 10:32
Message: Resolved.
Issue #TRUEZIP-266 - The sample code for key management should customize the ZipDriver class, not the JarDriver class





[TRUEZIP-265] Upgrade to commons-compress 1.4.1 Created: 13/Jun/12  Updated: 29/Aug/12  Resolved: 13/Jun/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR, TrueZIP Driver ZIP, TrueZIP Driver ZIP.RAES (TZP)
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes





[TRUEZIP-264] Covariant archive entries may not be readable Created: 10/Jun/12  Updated: 29/Aug/12  Resolved: 07/Jul/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

... if the directory entry has been read from the archive file AFTER the file entry for the same archive entry name.






[TRUEZIP-263] Improve documentation Created: 05/Jun/12  Updated: 29/Aug/12  Resolved: 07/Jul/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Task Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Reminder.






[TRUEZIP-262] ZipOutputShop.EntryOutputStream and TarOutputShop.EntryOutputStream will write to the underlying container even if they have been closed Created: 04/Jun/12  Updated: 29/Aug/12  Resolved: 04/Jun/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR, TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.5
Fix Version/s: TrueZIP 7.6

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 04/Jun/12 ]

Changeset: 558a12b08975
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-06-04 20:05
Message: Fixed.
Issue #TRUEZIP-262 - ZipOutputShop.EntryOutputStream and TarOutputShop.EntryOutputStream will write to the underlying container even if they have been closed





[TRUEZIP-261] Deprecate TFile.getFile() Created: 01/Jun/12  Updated: 04/Jun/12  Resolved: 01/Jun/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP File*
Affects Version/s: TrueZIP 7.5.4
Fix Version/s: TrueZIP 7.5.5

Type: Improvement Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Using the resulting File object for file system access would compete with the TrueZIP Kernel for I/O and may easily corrupt the state of the (virtual) file system space, including loss of data!






[TRUEZIP-260] .tmp file is not cleaned up if archiving fails Created: 24/May/12  Updated: 04/Jun/12  Resolved: 25/May/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: laubrino Assignee: Christian Schlichtherle
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

windows7/64bit, JDK6



 Description   

While adding files to an archive (with method TFile.cp_p(TFile)) IOException "Not enought space od disk" is thrown (tested on a small flashdisk). Then I want to cleanup all partial files. I can delete TFile.getFile() file, however .tmp file is still there and is locked by some process (JVM?) so it cannot be deleted even manually. It stays there even after JVM exit.
When I call TVFS.umount(TFile) I get FsSyncException caused by IOException: This multiplexed output shop is still busy with writing a stream!



 Comments   
Comment by Christian Schlichtherle [ 25/May/12 ]

Sounds as if everything works as designed:

(1) When there is an error writing to an archive file, the temp file in the directory of the target archive file is intentionally left over. This is (a) to enable post mortem analysis and more importantly (b) to enable recovery of the partial written archive file with tools like plain old ZipInputStream or whatever. Imagine I had deleted the temp file, then all your changes since the last TVFS.umount() would be lost!

(2) Though shall not operate on TFile.getFile()! This method exists for technical reasons and is not intended for users. If you do, you are competing for file access with the TrueZIP Kernel and may corrupt its state. What you have witnessed is the result of this.

(3) You can't remove the java.io.File because the TrueZIP Kernel still holds an open stream. On Windows, this locks the file for concurrent access (see above). To unlock resources, make sure to close your streams and call TVFS.umount().

(4) After the JVM terminates, the file must be accessible again. A persistent lock mechanism doesn't exist.

I'll consider deprecating TFile.getFile() so that users don't get confused over this.

Comment by Christian Schlichtherle [ 25/May/12 ]

Changeset: 2001204a70b0
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-05-25 20:10
Message: Deprecated TFile.getFile().
Issue #TRUEZIP-260 - .tmp file is not cleaned up if archiving fails

Comment by laubrino [ 28/May/12 ]

Thank you for your comment.

I use TrueZip for backups. I do want to remove unfinished zip files, because they have no value for me. I don't want to keep only partial archives on a disk. Is there any way how to achieve that? Without calling TFile.getFile() and so on?

Comment by Christian Schlichtherle [ 29/May/12 ]

There are some options, but they depend on your use case. If you are doing repetitive backups to the same target ZIP file, then I would recommend a different archiving scheme by doing:

TConfig config = TConfig.get()
config.setOutputPreferences(config.getOutputPreferences.set(FsOutputOption.GROW));

Then all new entries would get appended to the end of the target ZIP file and all changes would be recorded in the Central Directory at its end.

With this scheme you need lesser space on the target file system (because it does not use a temp file) and faster processing times (because it does not need to reassemble the target ZIP file from a temp file).

If the target file system runs full, then you can still access the target ZIP file because TrueZIP automatically recovers partially written ZIP files: You will loose at most the last partial written entry, not the entire ZIP file.

Let me know if this helps.





[TRUEZIP-259] Improve Javadoc. Created: 16/May/12  Updated: 04/Jun/12  Resolved: 01/Jun/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.4
Fix Version/s: TrueZIP 7.5.5

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-258] Optimize code. Created: 16/May/12  Updated: 04/Jun/12  Resolved: 01/Jun/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.4
Fix Version/s: TrueZIP 7.5.5

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-257] Improve web site. Created: 16/May/12  Updated: 04/Jun/12  Resolved: 01/Jun/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5.4
Fix Version/s: TrueZIP 7.5.5

Type: Task Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[TRUEZIP-256] FsController.setTime/3 should fail with the boolean return value false instead of throwing an exception on a negative access time Created: 16/May/12  Updated: 04/Jun/12  Resolved: 16/May/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.4
Fix Version/s: TrueZIP 7.5.5

Type: Improvement Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 16/May/12 ]

Changeset: 38da91b949a7
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-05-16 18:20
Message: Fixed.
Issue #TRUEZIP-256 - FsController.setTime/3 should fail with the boolean return value false instead of throwing an exception on a negative access time





[TRUEZIP-255] The shutdown hook fails! Created: 04/May/12  Updated: 04/Jun/12  Resolved: 04/May/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.3
Fix Version/s: TrueZIP 7.5.4

Type: Bug Priority: Blocker
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Unfortunately, the shutdown hook in TrueZIP 7.5.3 fails, so that any uncommitted changes to archive files get lost!

This is very bad indeed - please accept my apologies for any inconveniences this may have caused.






[TRUEZIP-252] Fix rare dead lock condition Created: 02/May/12  Updated: 04/Jun/12  Resolved: 02/May/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.2
Fix Version/s: TrueZIP 7.5.3

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

Nested archive file operations may still dead lock in multithreaded environments due to incorrect local resolution of sync collisions in the selective entry cache - I said it's rare!



 Comments   
Comment by Christian Schlichtherle [ 02/May/12 ]

Changeset: 7259687e0c30
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-05-02 12:52
Message: Fixed.
Issue #TRUEZIP-252 - Fix rare dead lock condition





[TRUEZIP-251] Update to Apache Commons Compress 1.4 Created: 22/Apr/12  Updated: 08/Jul/12  Resolved: 01/May/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Improvement Priority: Major
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks TRUEZIP-219 IllegalArgumentException when Files.n... Closed
blocks TRUEZIP-274 NegativeArraySizeException in Tar arc... Closed

 Description   

Commons Compress is used for reading and writing TAR files. This update will greatly improve interoperability with third party TAR file tools.



 Comments   
Comment by Christian Schlichtherle [ 01/May/12 ]

Done





[TRUEZIP-250] Wrong use of ThreadLocal may introduce memory leaks Created: 22/Apr/12  Updated: 04/Jun/12  Resolved: 22/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP, TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

... if ThreadLocal is subclassed and ThreadLocal.remove() is not properly used.



 Comments   
Comment by Christian Schlichtherle [ 22/Apr/12 ]

Changeset: 7133469eef8d
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-22 18:37
Message: Fixed.
Issue #TRUEZIP-250 - Use of ThreadLocal may introduce memory leaks

Comment by Christian Schlichtherle [ 22/Apr/12 ]

Changeset: 10c63746f459
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-22 18:32
Message: Fixed.
Issue #TRUEZIP-250 - Use of ThreadLocal may introduce memory leaks





[TRUEZIP-249] de.schlichtherle.truezip.fs.archive.zip.ZipOutputShop.EntryOutputStream.close() may erratically close the current entry Created: 22/Apr/12  Updated: 04/Jun/12  Resolved: 22/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

... because it doesn't check if it has already closed the current entry.



 Comments   
Comment by Christian Schlichtherle [ 22/Apr/12 ]

Changeset: f7978571e2cf
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-22 15:36
Message: Fixed.
Issue #TRUEZIP-249 - de.schlichtherle.truezip.fs.archive.zip.ZipOutputShop.EntryOutputStream.close() may erratically close the current entry

Comment by Christian Schlichtherle [ 22/Apr/12 ]

Changeset: 10746a2f0b80
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-22 15:36
Message: Fixed.
Issue #TRUEZIP-249 - de.schlichtherle.truezip.fs.archive.zip.ZipOutputShop.EntryOutputStream.close() may erratically close the current entry





[TRUEZIP-248] FsResourceController.*.close() may produce racing conditions. Created: 17/Apr/12  Updated: 04/Jun/12  Resolved: 17/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

... because of wrong order of statements in close() methods.



 Comments   
Comment by Christian Schlichtherle [ 17/Apr/12 ]

Changeset: 064cf932fad4
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-17 18:36
Message: Fixed.
Issue #TRUEZIP-248 - FsResourceController.*.close() may produce racing conditions.

Comment by Christian Schlichtherle [ 17/Apr/12 ]

Changeset: 0d54fc18eb34
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-17 18:36
Message: Fixed.
Issue #TRUEZIP-248 - FsResourceController.*.close() may produce racing conditions.





[TRUEZIP-247] FsMultiplexedOutputShop.storeBuffers may erratically remove a buffer if an exception gets thrown. Created: 10/Apr/12  Updated: 04/Jun/12  Resolved: 10/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Kernel
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 10/Apr/12 ]

Changeset: 326c4391d857
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-10 21:08
Message: Fixed.
Issue #TRUEZIP-247 - FsMultiplexedOutputShop.storeBuffers may erratically remove a buffer if an exception gets thrown.





[TRUEZIP-246] Key Prompting Cancellation not treated correctly for WinZip AES encryption Created: 08/Apr/12  Updated: 04/Jun/12  Resolved: 08/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Comments   
Comment by Christian Schlichtherle [ 08/Apr/12 ]

Changeset: 07ca88300cbe
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-08 10:58
Message: Fixed.
Issue #TRUEZIP-246 - Key Prompting Cancellation not treated correctly for WinZip AES encryption

Comment by Christian Schlichtherle [ 08/Apr/12 ]

Changeset: 23646782f79b
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-08 11:07
Message: Added test coverage.
Issue #TRUEZIP-246 - Key Prompting Cancellation not treated correctly for WinZip AES encryption





[TRUEZIP-245] Wrong description in ZipEntry.toString() Created: 06/Apr/12  Updated: 04/Jun/12  Resolved: 06/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Trivial
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes
Environment:

The returned string contains "time=" instead of "gpbf=" (general purpose bit flags) and "crc=" instead of "ea=" (external attributes).



 Comments   
Comment by Christian Schlichtherle [ 06/Apr/12 ]

Changeset: 83d3ff09881e
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-06 16:15
Message: Fixed.
Issue #TRUEZIP-245 - Wrong description in ZipEntry.toString()

Comment by Christian Schlichtherle [ 06/Apr/12 ]

Changeset: 8d0224d866d0
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-06 16:15
Message: Fixed.
Issue #TRUEZIP-245 - Wrong description in ZipEntry.toString()





[TRUEZIP-244] Only half of the authentication code is checked for WinZip AES encrypted files Created: 06/Apr/12  Updated: 04/Jun/12  Resolved: 06/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver ZIP
Affects Version/s: TrueZIP 7.5.1
Fix Version/s: TrueZIP 7.5.2

Type: Bug Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

Which increases the probability of a collision, i.e. a false positive password check, from 1/2^10 to 1/2^5.



 Comments   
Comment by Christian Schlichtherle [ 06/Apr/12 ]

Changeset: b8566913d7ed
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-06 17:01
Message: Fixed.
Issue #TRUEZIP-244 - Only half of the authentication code is checked for WinZip AES encrypted files

Comment by Christian Schlichtherle [ 06/Apr/12 ]

Changeset: 887494a8e2e7
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-06 17:01
Message: Fixed.
Issue #TRUEZIP-244 - Only half of the authentication code is checked for WinZip AES encrypted files





[TRUEZIP-243] Improve web site Created: 03/Apr/12  Updated: 04/Jun/12  Resolved: 03/Apr/12

Status: Closed
Project: TrueZIP
Component/s: None
Affects Version/s: TrueZIP 7.5
Fix Version/s: TrueZIP 7.5.1

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Christian Schlichtherle [ 03/Apr/12 ]

The home page, getting-started, tutorial etc. have been improved. Changes will get visible with the next push to the web repository.





[TRUEZIP-242] The TarDriver should use the default character set instead of US-ASCII Created: 03/Apr/12  Updated: 04/Jun/12  Resolved: 03/Apr/12

Status: Closed
Project: TrueZIP
Component/s: TrueZIP Driver TAR
Affects Version/s: TrueZIP 7.5
Fix Version/s: TrueZIP 7.5.1

Type: Improvement Priority: Minor
Reporter: Christian Schlichtherle Assignee: Christian Schlichtherle
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes

Issue Links:
Dependency
blocks TRUEZIP-241 AssertionError with CharConversionExc... Closed

 Comments   
Comment by Christian Schlichtherle [ 03/Apr/12 ]

Changeset: a13227777d3b
Author: Christian Schlichtherle <christian AT schlichtherle DOT de>
Date: 2012-04-03 13:35
Message: RCB
Issue #TRUEZIP-242 - The TarDriver should use the default character set instead of US-ASCII





[TRUEZIP-241] AssertionError with CharConversionException when accessing TAR files with non-US-ASCII characters Created: 02/Apr/12  Updated: 04/Jun/12