[XADISK-149] JVM (stack guard) warning while loading xadisk native library. Created: 13/Oct/13  Updated: 27/Dec/15  Resolved: 27/Dec/15

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to George for identifying this through this thread: https://groups.google.com/forum/#!topic/xadisk/Hc4iZv8_6mk

The JVM reports the following warning while starting xadisk:

"Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /tmp/xadisk/native/unix_32_xadisk.native which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'."



 Comments   
Comment by gastaldi [ 13/Jan/14 ]

As stated in the Forum thread, the solution is to change the enum to this:

private enum NATIVE_LIB_NAMES {
        unix_64_xadisk, unix_32_xadisk,
        windows_64_xadisk,windows_32_xadisk,
        mac_64_xadisk, mac_32_xadisk,
        placeholder_xadisk
};
Comment by Nitin Verma [ 27/Dec/15 ]

Checked-in the changes to svn/trunk (revision 574). They are same as suggested by George above.





[XADISK-81] Provide an i/o interface for random read/write, by extending java.io.RandomAccessFile. Created: 31/Jul/11  Updated: 22/Jun/15

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Martin for bringing up this feature idea.

Currently, XADisk is providing implementations for java.io.InputStream and java.io.OutputStream. If feasible, provide an i/o interface which extends the standard class java.io.RandomAccessFile.

Thanks,
Nitin



 Comments   
Comment by puce [ 22/Jun/15 ]

Or SeekableByteChannel/ FileChannel when moving to the NIO.2 File API (XADISK-175)

http://docs.oracle.com/javase/tutorial/essential/io/legacy.html
http://docs.oracle.com/javase/tutorial/essential/io/rafs.html





[XADISK-175] Migrate to NIO.2 File API Created: 22/Jun/15  Updated: 22/Jun/15

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: puce Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Since Java SE 7, java.io.File has become a legacy API and replaced by the NIO.2 File API:
http://docs.oracle.com/javase/tutorial/essential/io/legacy.html

Migrate the existing code to NIO.2 File API.






[XADISK-174] Consider to migrate to GitHub Created: 22/Jun/15  Updated: 22/Jun/15

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: puce Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

For easier collaboration (fork/ pull request), consider to migrate to Git and GitHub.






[XADISK-138] XASystemNoMoreAvailableException after interruption Created: 11/Jul/13  Updated: 14/Mar/15  Resolved: 16/Aug/13

Status: Closed
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7, JDK 7


Tags: thread

 Description   

I got a XASystemNoMoreAvailableException thrown in NativeXAFileSystem.notifySystemFailure due to an Thread.interupt()-ion during completeReadOnlyTransaction() which caused a ClosedByInterruptException.
I couldn't reproduce it (it occurred only once in a running test-system) even though I tried with the below test code. That code produces some other but less problematic issue.

Note that I have an environment which relies heavily on interrupting cooperating Threads which run for too long or (partly) failed. Thus I cannot simply or easily avoid triggering interuption.

The below test code produces an issue during shutdown, but not during the transaction:

System.out.println("Booting XADisk...");
StandaloneFileSystemConfiguration configuration = new StandaloneFileSystemConfiguration(
"target/xadisk-fail/system", "id-1");
XAFileSystem xafs = XAFileSystemProxy.bootNativeXAFileSystem(configuration);
xafs.waitForBootup(-1);
System.out.println("XADisk is now available for use.");

File testFile = File.createTempFile("xa-file", ".dat");
testFile.deleteOnExit();
long waitUntil = System.currentTimeMillis() + Times.MILLIS_PER_SECOND;
while (System.currentTimeMillis() < waitUntil)

{ Session session = xafs.createSessionForLocalTransaction(); XAFileOutputStream xaos = session.createXAFileOutputStream(testFile, false); xaos.write("Hello World!".getBytes(Misc.UTF_8)); Thread.currentThread().interrupt(); xaos.write("Hello World2!".getBytes(Misc.UTF_8)); Thread.currentThread().interrupt(); xaos.close(); Thread.currentThread().interrupt(); session.commit(); }

Thread.interrupted();

xafs.shutdown();
System.out.println("XADisk is down.");



 Comments   
Comment by Nitin Verma [ 14/Jul/13 ]

A similar issue has been reported at thread: https://groups.google.com/forum/?hl=en#!topic/xadisk/sqYH82vOhDw by Ravi, where undeploying the application on jboss 7.2 triggers an interrupt. In that case, the thread which gets interrupted is the hornetq resource adapter's worker thread during delivery of a message to an mdb (the mdb performs some operations on xadisk).

Comment by Nitin Verma [ 15/Aug/13 ]

Hi Gunnar,

I ran your test-case and to confirm, I see the ClosedByInterruptException during session.commit(), and the xadisk instance becomes unavailable after that.

A detailed summary...

When an application invokes the xadisk operations (not the remote xadisk invocation case, in which the application thread just does the network transfers for remote method calls), most of the work gets done in the application thread (in addition, there are a few xadisk worker threads running, doing their job asynchronously). XADisk uses NIO FileChannels heavily, which are interruptible. When an application thread calls an xadisk api which does any such FileChannel operation, and receives an interrupt, the channel would get closed and an exception will result. This channel may be for the transaction logs or any application data file etc. At this stage, the channel operation is incomplete (eg, an incomplete unknown amount of data was written), there is nothing much xadisk can do to continue working, as per its current design.

Altering the current implementation to cater to such interrupts (see interrupt, clean-up, and let only the current transaction go away leaving xadisk in-memory and persistent data in a consistent state) looks quite complex.

This should be a generic issue with any library which makes use of the interruptible io library like NIO. I tried finding out some workaround...

1. Disabling interrupt for a thread or for a FileChannel does not seem possible as per current Java. So, the application can run such application-threads by overriding the interrupt() method of the Thread class, where it can allow delaying the effective delivery of such interrupts. But note that this approach can interfere with some features of xadisk like deadlock handling, which uses thread interrupt internally (for transaction in wait state for lock). I am not sure how prevalent is this approach; just found it here and sharing: http://cs.oswego.edu/pipermail/concurrency-interest/2008-March/005064.html

2. Using the approach in 1. would involve frequent calls for enabling/disabling interrupts. As an alternative, the application can do away with the interrupt approach altogether and simply use a application level flag (instead of Thread.interrupt()) to communicate interrupts. Again, the application code would be filled-up with checks for this flag.

3. Use xadisk's remote method invocation. This would decouple the application-thread from the actual thread of the xadisk instance doing the job, but it would lead to slower performance due to socket based communication.

Please feel free to comment.

Thanks & Regards,
Nitin

Comment by gunnar_zarncke [ 16/Aug/13 ]

What about a lightweight decoupling:

4. Decouple the interruptible threads (my client logic) from the non-interruptible XA-threads.
Use one Thread per XAFile(Input|Output)Stream (or a ThreadPool) to do the non-interruptible reading/wrinting and pass the data to/from the client (Input|Output)Stream via a java.util.concurrent.Exchanger.
Then if the clients thread gets interrupted at worst the Exchanger call gets interrupted. The XA thread will of course fail to sync with the exchanger but this can be handled by just waiting (for a later commit/rollback).
Advantages: less overhead than full remoting; fully compatible with Java interruption (provided the XA threads don't get interrupted; this can be achieved by putting them into a different ThreadGroup).
This can be implemented as an alternative to XAFile(Input|Output)StreamWrapper.

Comment by Nitin Verma [ 16/Aug/13 ]

Hoping that this issue can be worked around by developers and realizing the complexity of a fix in xadisk for this, I am closing this issue for now.

Comment by gunnar_zarncke [ 20/Aug/13 ]

An implementation (beta) of approach 4 exists and was sent to Nitin for review.

Comment by misisko [ 12/Mar/15 ]

Hi @gunnar_zarncke, we have a big problem with this issue - what's about your patch? How did you solved this problem? @Nitin, do developers work on fix for this problem? I think, this problem is verry annoying.

Comment by gunnar_zarncke [ 12/Mar/15 ]

The mentioned fix is productive since about a year and fairly stable. In the context of that application I do experience some strange thread and/or interrupt related issues (threads-pools not shutting down in time) but I'm fairly certain that it's not due to my patch and the XADisk transactions run fast and stable. So I'd call my fix (basically a thread-decoupling layer around the XADisk part stable and reliable. It can be used without patching XADisk but I didn't publish it because I expected Nitin to provide it as is in some sub packge. I can send it to you directly if you want, just mail me at gunnar dot zarncke at gmx dot de.

Comment by Nitin Verma [ 14/Mar/15 ]

Hi Misisko,

Code modification is done only by me; I have not planned any fix for this as of now.

It just checked that I had missed reviewing the fix proposed by Gunnar. You may get it from Gunnar and see if it solves the problem.

Thanks,
Nitin

Comment by misisko [ 14/Mar/15 ]

Looking forward. Thanks for information.





[XADISK-172] Missing target file for copy is logged as InsufficientPermissionOnFileException when it should be FileNotExistsException Created: 17/Feb/15  Updated: 18/Feb/15

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: pjlharvey Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux version 3.8.13-16.2.1.el6uek.x86_64 (mockbuild@ca-build44.us.oracle.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Thu Nov 7 17:01:44 PST 2013



 Description   

If i execute the following code

XAFileSystem xafs = null;
Session session = null;
String instanceId = generateUniqueId();

try {
	StandaloneFileSystemConfiguration configuration = new StandaloneFileSystemConfiguration(xadiskTransactionFolder, instanceId);
	xafs = XAFileSystemProxy.bootNativeXAFileSystem(configuration);
	xafs.waitForBootup(-1);
	session = xafs.createSessionForLocalTransaction();
	File srcFile = new File("/tmp/existingfolder", "non_existent_file");
	File destFile = new File("/tmp/targetFolder", "target_file");
	session.copyFile(srcFile, destFile);

} catch (Exception e) {
	// act accordingly
}

then the xadisk copy method throws the following exception.

org.xadisk.filesystem.exceptions.InsufficientPermissionOnFileException: Permission of type [READ_FILE] is needed over the file/directory with path [/tmp/existingfolder/non_existent_file] for the i/o operation to succeed.
        at org.xadisk.filesystem.NativeSession.checkPermission(NativeSession.java:1157) ~[xadisk-1.2.2.jar:na]
        at org.xadisk.filesystem.NativeSession.copyFile(NativeSession.java:359) ~[xadisk-1.2.2.jar:na]

This is a little misleading, as the actual problem is that file does not exist.



 Comments   
Comment by Nitin Verma [ 18/Feb/15 ]

Yes, it is a known problem. You may find similar issue logged where non-existing file/directory error is reported as InsufficientPermissionOnFileException.





[XADISK-173] XADisk api for creating file/directory while creating all parent directories required. Created: 18/Feb/15  Updated: 18/Feb/15

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Phillip Harvey for raising this feature suggestion.

It could be useful to provide an xadisk api for doing the following in a single operation:
-create file or directory,
-create all non-existent parent directories.






[XADISK-171] NativeXAFileInputStream.read() instead of returning a 0-255 int value, returns byte. Created: 03/Feb/15  Updated: 05/Feb/15  Resolved: 05/Feb/15

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Milo for reporting this issue through the discussion thread: https://groups.google.com/forum/#!topic/xadisk/X_J5JESS92w

NativeXAFileInputStream's read() method which just reads one byte, is returning the "byte" read as signed byte. In other words, it would return values in the range -128 to 127, instead of the standard 0-255. Due to this, -1 would get returned in two cases: end-of-file reached as well as 255 seen in a binary file.



 Comments   
Comment by Nitin Verma [ 03/Feb/15 ]

SVN connection at my ide seems to be slow, so not able to checkin the fix to trunk as of now. The fix is simply this:

NativeXAFileInputStream, line#128, modify:
return nextByte;
to
return nextByte & 0xff;

Thanks,
Nitin

Comment by Nitin Verma [ 05/Feb/15 ]

Checked-in the changes to trunk: revision #573.





[XADISK-170]  fileExistsAndIsDirectory return false when a folder in the path chain is not accessible Created: 11/Dec/14  Updated: 11/Dec/14

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: sberthouzoz Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows, NTFS filesystem, xa-disk 1.2.2



 Description   

suppose the following call fileExistsAndIsDirectory("\\server\share$\folder\subfolder") return false even when the folder exists.
You need the following condition to reproduce it:

  • no READ Permission on \\server\share$\
  • READ Permission set on \\server\share$\folder
  • READ and WRITE Permission set on \\server\share$\folder\subfolder

The cause of the problem is that the code is doing a (""\\server\share$\").isDirectory() which returns false (because of file Permission) in org.xadisk.filesystem.virtual.TransactionVirtualView.getVirtualViewDirectory(File) on line 304.

Here is an abstract of the call chain:

  1. session.fileExistsAndIsDirectory("\\server\share$\folder\subfolder")
  2. NativeSession.fileExistsAndIsDirectory("\\server\share$\folder\subfolder"", false)
  3. NativeSession.checkPermission(READ_DIRECTORY, "\\server\share$\folder")
  4. TransactionVirtualView.isDirectoryReadable("\\server\share$\folder")
  5. TransactionVirtualView.getVirtualViewDirectory(\\server\share$"), here the line 305 or 312 will throw the FileNotExistsException which will cause the methods to return false;

The problem is also that the method checks the parent of the parent during the chain, instead of just checking the given path






[XADISK-169] When moving directory across file-systems fails, provide the possible reason. Created: 06/Oct/14  Updated: 06/Oct/14  Resolved: 06/Oct/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Improvement Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Marco for suggesting this at discussion thread: https://groups.google.com/forum/#!topic/xadisk/Yn4cyCe8CIo

On some operating systems, the rename directory call would fail if the rename is across different file-systems. While throwing an error when rename is failing for a directory, it may help to remind the user about this possibility of failure.



 Comments   
Comment by Nitin Verma [ 06/Oct/14 ]

Checked-in the changes to trunk (revision #572).





[XADISK-168] Interleaving of GatheringDiskWriter's submitBuffer and writeBuffersToTransactionLog causing NPE. Created: 21/Sep/14  Updated: 21/Sep/14  Resolved: 21/Sep/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Ravi Soni who reported a problem of NullPointerException in GatheringDiskWriter:submitBuffer, which is possibly related to the code issue being described below. The discussion thread, where 2-3 other exceptions were also identified along with this one is here:https://groups.google.com/forum/#!topic/xadisk/W_r8ikT3cgw.

This particular NPE could not be reproduced at will, and was seen only very rarely in a highly concurrent environment. The exception reports NPE in the below line of the method submitBuffer:

int currentCumulativeSize = cumulativeBufferSize.addAndGet(logEntry.getBuffer().remaining());

As is visible, logEntry.getBuffer is found to be null.

Though this problem, being concurrency related, could not be reproduced, but doing some analysis from the code, it appears that the following instruction from method writeBuffersToTransactionLog gets a chance to execute (for the buffer being "submit"ted) after the addition of this buffer to the buffers' list in the submitBuffer, but BEFORE the above logEntry.getBuffer gets executed:

temp.makeOnDisk(temp.getOnDiskInfo());

The method Buffer.makeOnDisk sets the byteBuffer inside the Buffer object to null. Hence the NullPointerException.



 Comments   
Comment by Nitin Verma [ 21/Sep/14 ]

Checked-in the changes to trunk (revision #571). Note that though the code issue being fixed here is a problem in itself, but whether it also fixes the problem encountered by Ravi in his environment can only be realized based on test of time.





[XADISK-156] Response of XAResourceImpl in case of self-initiated rollbacks by XADisk. Created: 02/Dec/13  Updated: 06/Sep/14  Resolved: 06/Sep/14

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Alex for bringing this to my attention at thread:

https://groups.google.com/forum/#!topic/xadisk/SWxa1o03ccw

There are some cases when XADisk rolls back its transactions by itself, even in case of a distributed transaction (which is co-ordinated by the transaction-manager[TM]), eg transaction timeout, deadlock detection.

As the TM, being unaware of these rollbacks, would still call, as in general, methods like end, prepare, commit, rollback etc, the XADisk should choose the right mechanism to signal the early self-initiated rollback to the TM. There is an error code of XA_RB* (in XAException), can XADisk employ these error codes?



 Comments   
Comment by Nitin Verma [ 06/Sep/14 ]

During any kind of rollback, whether application initiated or xadisk initiated (eg timeout), the cleanup process for the rollback removes the information of the transaction/session from the xadisk in-memory data-structure. Keeping this information after rollback (and it is not clear for how long this information should be kept) is not straighforward in the current design/implementation. So, closing this bug.





[XADISK-155] XAResourceImpl does not honor the timeout value of 0 in setTransactionTimeout. Created: 02/Dec/13  Updated: 06/Sep/14  Resolved: 06/Sep/14

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As per the XAResource API specification, http://docs.oracle.com/javaee/5/api/javax/transaction/xa/XAResource.html#setTransactionTimeout%28int%29

"To reset the timeout value to the default value used by the resource manager, set the value to zero."

XADisk does not follow this.



 Comments   
Comment by Nitin Verma [ 02/Dec/13 ]

Similar semantics can be used for other APIs in xadisk, eg XADiskUserLocalTransaction.

Comment by Nitin Verma [ 06/Sep/14 ]

Checked-in the changes to trunk (revision #569).





[XADISK-167] The backup file creation needs a better mechanism with practical upper-bounds. Created: 31/Aug/14  Updated: 31/Aug/14  Resolved: 31/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Marco for raising this issue at the discussion thread: https://groups.google.com/forum/#!topic/xadisk/O0wbc44ufaU.

The current mechanism for backup file creation is going to exhaust fast on practical systems due to the long resulting path names. Once the path length limit is reached, xadisk can no longer create its backup files. Suppose, in a particular environment the existing path naming pattern can only go till 500 steps in the path, in that case, the xadisk instance would require a restart after 500*100000 files were created (in past + in use) in the backup directory after its last restart.

We either need an approach with infinite number of backup file creations, or if finite, the upper limit should be large enough for xadisk instance to remain up for a few months atleast (in general, this should apply to all in-memory data-structures, processing time, and on-disk files xadisk is keeping; the accumulations over the months should be in practical limits).



 Comments   
Comment by Nitin Verma [ 31/Aug/14 ]

Checked-in the changes to trunk (revision #568).





[XADISK-158] Problem with truncate method Created: 10/Jan/14  Updated: 23/Aug/14

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: hell_keeper Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Unix x64 (test on centos 6.5)



 Description   

This problem seems affect the Unix system (I dont' have this problem with win7 x64). For a project, I test multiple solutions to manage "a transactional filesystem". My transactional service must be able to write into a file (appending or not).
The following test scenario illustrates the bug. I have on my disk a file with a serialized map ( a JSON serialization with a map of beans):

{"meta2":"key":"key2","value":"value2","type":"USER"}

,"meta1":"key":"key1","value":"value1","type":"TECHNICAL"}}

My transactional service must update this file with a new serialized map:

{"meta3":{"key":"key3","value":"value3","type":"TECHNICAL"}}

The transactional method calls a XADisk storage connector. To avoid to append the new map in the file I use the truncate method:

session.truncateFile(file, 0);
try (OutputStream out = new XAFileOutputStreamWrapper(session.createXAFileOutputStream(file, true))) {
try (InputStream in = source.getInputStream())

{ IOUtils.copy(in, out); }

} catch (FileNotExistsException | NoTransactionAssociatedException | InterruptedException | IOException | FileUnderUseException e) {
...
}

In the transaction I throw an exception to cause a rollback. I expect to have the initial file with the following information:

{"meta2":"key":"key2","value":"value2","type":"USER"}

,"meta1":"key":"key1","value":"value1","type":"TECHNICAL"}}

But the file contains:

{"meta3":"key":"key3","value":"value3","type":"TECHNICAL"}}"meta2":"key":"key2","value":"value2","type":"USER"}

The first point is the file is inconsistent and it seems that the truncate method is equivalent to a insert a the beginning of the file.

Best regards.



 Comments   
Comment by Nitin Verma [ 11/Jan/14 ]

Hello.

Have you explicitly called rollback method on the session object?

Thanks,
Nitin

Comment by hell_keeper [ 11/Jan/14 ]

I use Spring to manage the transaction so the rollback has been called on the XaResource (by spring).

Comment by Nitin Verma [ 14/Jan/14 ]

I am not able to reproduce this issue with a standalone test-case as below. The file's contents were "this is some data. this is some data. this is some data.", and they remain as-is after the test completes:

File f = new File("C:
a.txt");
session.truncateFile(f, 0);
XAFileOutputStream xafos = session.createXAFileOutputStream(f, true);
xafos.write("new content".getBytes());
session.rollback();

Do you see something which can be different with your test?

Thanks,
Nitin

Comment by hell_keeper [ 15/Jan/14 ]

Expected the fact that I use a XAFileOutputStreamWrapper, I dont see difference...

Comment by hell_keeper [ 15/Jan/14 ]

I just created a little project (under eclipse and using maven) that illustrates my problem. Please see it attached hopping that the bug is present to you (https://drive.google.com/file/d/0B2sj7pmEYCG5WlJwNmhVeEJCQ1U/edit?usp=sharing). Please take care about my system (unix x64) because on windows I don't have this problem.

This test is based on the sample present in the example section of the xaDisk website.

Please note also that the test prints this message in my console (may be the cause of the problem).

Botting an XADisk instance...
OpenJDK 64-Bit Server VM warning: You have loaded library /home/thomas/workspace/problem-truncate/target/testXadisk1859041122059/native/unix_32_xadisk.native which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Successfully booted the XADisk instance.

Best regards.

Comment by Nitin Verma [ 15/Jan/14 ]

Hello.

I am able to reproduce the issue on my Fedora linux installation. I am sorry I ignored this from the bug description - "This problem seems affect the Unix system".

I will let you know as and when I discover the reason/fix for this bug.

Many Thanks for your effort.

Nitin

Comment by Nitin Verma [ 15/Jan/14 ]

I find that the below code does not work as expected on the linux installation:
______________________________

File s = new File("C:/a.txt");
File d = new File("C:/b.txt");
FileChannel fcs = new FileInputStream(s).getChannel();
FileChannel fcd = new FileOutputStream(d, true).getChannel();
fcd.transferFrom(fcs, 3, 10);
______________________________

From the javadoc of FileChannel.transferFrom, one would expect the bytes transfer to begin at 3rd position of the destination file. This works fine on my windows xp installation, but on linux it appends at the end of the destination file. Note that using "append=false" for opening FileOutputStream truncates the file immediately (even before starting to write into the file).

Seems like a bug to me.

Any opinions?

Comment by Jasper Siepkes [ 31/Jan/14 ]

Well the doc ( http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#transferFrom%28java.nio.channels.ReadableByteChannel,%20long,%20long%29 ) says "An attempt is made to read up to count bytes from the source channel and write them to this channel's file starting at the given position." (with position being written in monospaced font to signify its an method argument). So yeah one would expect that, JVM bug perhaps?. Which JVM version are you using on Linux Nitin?

Comment by Nitin Verma [ 31/Jan/14 ]

Hello Jasper,

I think it was 1.6. I will confirm it and update (need to reboot and switch OS etc )

Nitin

Comment by Nitin Verma [ 23/Aug/14 ]

I did some jdk source code analysis and came to know:

1. the same bug on Linux would also occur (and as tested, it does) for write(ByteBuffer, int), where the second argument for position in the file is being ignored if the fileChannel was opened in append mode. Basically, the following code too does not work as expected:
File d = new File ("b.txt");
FileChannel fcd = new FileOutputStream (d, true).getChannel ();
ByteBuffer bb = ByteBuffer.wrap("bytebuffer".getBytes());
fcd.write(bb, 3);

It does not write at the position of 3, but simply appends.

2. The code flow analysis takes us to the method pwrite0 in http://hg.openjdk.java.net/jdk6/jdk6/jdk/file/b6c45a8b14fb/src/solaris/native/sun/nio/ch/FileDispatcher.c. This further makes use of a library function pwrite64.

3. I wrote a simple c code to verify that even pwrite64 does not work as expected on the linux machine:
FILE *f = fopen("b.txt", "a");
printf("%d", pwrite64(fileno(f), "tobewritten, 8, 3));

This too appends at the end of the file.

4. Came to see that the man page here says it is a known bug with pwrite64:
http://linux.die.net/man/2/pwrite64: "However, on Linux, if a file is opened with O_APPEND, pwrite() appends data to the end of the file, regardless of the value of offset. "

Comment by Nitin Verma [ 23/Aug/14 ]

I will check if we can workaround this jdk bug in xadisk source, wherever it can affect us, including the place being discussed here.





[XADISK-160] ConcurrentModificationException in TransactionVirtualView.updateDescendantVVDsWithPrefix. Created: 10/Feb/14  Updated: 23/Aug/14  Resolved: 23/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Marco for identifying and raising this issue via the thread: https://groups.google.com/forum/#!topic/xadisk/hlj0YlO5qDg

In the method updateDescendantVVDsWithPrefix of class TransactionVirtualView, we are iterating over the keys over the hashmap (virtualViewDirs) and within the loop calling updateVVDWithPath which modifies this map. This results in ConcurrentModificationException.



 Comments   
Comment by Nitin Verma [ 23/Aug/14 ]

Checked-in the changes to trunk (revision #567).





[XADISK-161] In directory move case, all the descendants of the "src" directory should be removed from directoriesToForce list. Created: 10/Feb/14  Updated: 23/Aug/14  Resolved: 23/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In case of directory move, all the descendant directories of the "src" directory (in addition to "src" directory itself) should be removed from the tracking list of directories to force to disk, i.e. from the list named "directoriesToForce" in DurableDiskSession. If this is not done, the descendant directories in which some work was done, which would no longer exist due to the move of the ancestor directory ("src"), would give rise to failure in the force directory method.



 Comments   
Comment by Nitin Verma [ 23/Aug/14 ]

Checked-in the changes to trunk (revision #567).





[XADISK-163] XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available Created: 15/May/14  Updated: 17/Aug/14  Resolved: 17/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Blocker
Reporter: steroberti Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows 7, jdk 1.6, websphere 7



 Description   

if i send a jms message and i write a lot of number file in xatransaction on roolback i catch this stack error:

[14/05/14 12.24.23:267 CEST] 00000026 XATransaction E J2CA0027E: Si è verificata un'eccezione nel richiamare rollback in un adattatore risorse XA dal DataSource xadiskcf, nell'ID transazione

{XidImpl: formatId(57415344), gtrid_length(36), bqual_length(54), data(00000145fa42f07c0000000100000006715745520ac55af173a42fa2316d88b0ade83ce800000145fa42f07c0000000100000006715745520ac55af173a42fa2316d88b0ade83ce8000000010000000000000000000000000002)}

: javax.transaction.xa.XAException
at org.xadisk.filesystem.utilities.MiscUtils.createXAExceptionWithCause(MiscUtils.java:18)
at org.xadisk.connector.XAResourceImpl.rollback(XAResourceImpl.java:117)
at com.ibm.ejs.j2c.XATransactionWrapper.rollback(XATransactionWrapper.java:1303)
at com.ibm.tx.jta.JTAXAResourceImpl.rollback(JTAXAResourceImpl.java:363)
at com.ibm.tx.jta.RegisteredResources.deliverOutcome(RegisteredResources.java:1654)
at com.ibm.tx.jta.RegisteredResources.distributeOutcome(RegisteredResources.java:1933)
at com.ibm.tx.jta.RegisteredResources.distributeRollback(RegisteredResources.java:2586)
at com.ibm.tx.jta.TransactionImpl.internalRollback(TransactionImpl.java:1952)
at com.ibm.tx.jta.TransactionImpl.internalRollback(TransactionImpl.java:1915)
at com.ibm.tx.jta.TransactionImpl.rollback(TransactionImpl.java:1317)
.............
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:528)
at org.xadisk.filesystem.NativeSession.rollback(NativeSession.java:1068)
at org.xadisk.connector.XAResourceImpl.rollback(XAResourceImpl.java:113)
... 23 more
Caused by: java.io.EOFException
at org.xadisk.filesystem.utilities.FileIOUtility.readFromChannel(FileIOUtility.java:171)
at org.xadisk.filesystem.TransactionLogEntry.getNextTransactionLogEntry(TransactionLogEntry.java:475)
at org.xadisk.filesystem.NativeSession.rollback(NativeSession.java:1021)

for replicate use this code in MDB spring:
XADiskConnection connection = xaDisk.getConnection();
File folder = new File("d:
temp");
if(!folder.exists())
connection.createFile(folder, true);
for (int j = 0; j < 10000; j++) {
File f = new File("d:\\temp
" + j + ".txt");
connection.createFile(f, false);
XAFileOutputStream out = connection.createXAFileOutputStream(f, false);
out.write((System.currentTimeMillis() + "").getBytes());
out.close();
}



 Comments   
Comment by Nitin Verma [ 17/Aug/14 ]

Thanks for identifying and reporting the issue. I have checked-in the fix to trunk. Please refer to revision #566. To resolve the issue, you can patch your local version of the xadisk with these changes.





[XADISK-164] NullPointerException in NativeSession Created: 15/May/14  Updated: 17/Aug/14  Resolved: 17/Aug/14

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: sberthouzoz Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The method checkValidParent will never throw FileNotExistsException

private void checkValidParent(File f) throws FileNotExistsException {
        if (f.getParentFile() == null) {
            throw new FileNotExistsException(f.getParentFile().getAbsolutePath());
        }
    }

It should probably use f.getAbsolutePath() in the FileNotExistsException



 Comments   
Comment by Nitin Verma [ 17/Aug/14 ]

Checked-in the changes to trunk. Modified the path set in the new exception to <Parent directory of f>.





[XADISK-146] CrashRecovery silently fails on error during recoverHeavyWriteTransactionsForRollback Created: 30/Sep/13  Updated: 16/Aug/14  Resolved: 16/Aug/14

Status: Closed
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Improvement Priority: Major
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7, broken Eclipse Juno, JDK 6



 Description   

The after crash recovery of an XADisk istance may fail silently when files written in heavy write mode are deleted (or otherwise missing). This leads to hard to find and recover szenrios (luckily for me only in a unit test).

The problem starts when hanging transaction logs are executed in recoverHeavyWriteTransactionsForRollback.
Exceptions throw by rollback in TransactionCompleter are caught in WorkRunnable but in absence of a WorkListener silently discarded.

I recommend to add a WorkListener during CrashRecovery which logs these Exception loudly. Or alternatively to use a WorkListener passed in during startup.



 Comments   
Comment by Nitin Verma [ 16/Aug/14 ]

I have created another bug to discourage the use of work scheduling methods (in WorkManager) without using WorkListener: https://java.net/jira/browse/XADISK-166. As updated there, at present there are two such places and the above mentioned "recoverHeavyWriteTransactionsForRollback" was one of them.

I have checked-in the changes. Now, the workListener instance from NativeXAFileSystem, which is used for all kinds of work scheduling, would be used for observing the failures during "recoverHeavyWriteTransactionsForRollback" too. Though, there is no logging at present by the workListener (class: CriticalWorkListener). When an exception is noticed, "xaFileSystem.notifySystemFailure" would be triggered.

Comment by Nitin Verma [ 16/Aug/14 ]

Changes made to use a workListener, through bug https://java.net/jira/browse/XADISK-166.





[XADISK-166] Never use workManager without a workListener. Created: 16/Aug/14  Updated: 16/Aug/14  Resolved: 16/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

While using any of the methods in the WorkManager for scheduling work, always provide a workListener instance, so that the exceptions seen during the execution of the work instance can be noticed by the workListener.

At present, we only use the startWork method. There are two places when we are calling the overloaded version of this method without workListener - CrashRecoveryWorker:recoverOnePhaseTransactions and CrashRecoveryWorker:recoverHeavyWriteTransactionsForRollback. Replace these calls to use a workListener instance (can use NativeXAFileSystem.startWork() which uses its criticalWorkListener).

Also remove the implementation of startWork() without workListener from our StandaloneWorkManager implementation.



 Comments   
Comment by Nitin Verma [ 16/Aug/14 ]

Checked-in the changes to trunk.





[XADISK-159] In NativeXAFileOutputStream.write, replace the recursive call to write with a loop. Created: 11/Jan/14  Updated: 16/Aug/14  Resolved: 16/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Marco for sharing this over the discussion thread: https://groups.google.com/forum/#!topic/xadisk/n9xj3OuWW9o

In NativeXAFileOutputStream.write, the recursive call to write can lead to a high number of stack frames (eg for 1M byte[] and with default xadisk buffer size of 4k, there would be around 250 recursive calls). Replace this with a loop alternative (the changes are going to be trivial).



 Comments   
Comment by Nitin Verma [ 16/Aug/14 ]

Checked-in the changes to trunk.





[XADISK-165] PointOfContact may not get released for cluster's master node. Created: 27/Jul/14  Updated: 16/Aug/14  Resolved: 16/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Ravi for identifying this issue and suggesting a fix through the discussion thread at: https://groups.google.com/forum/#!topic/xadisk/vLpaHjn8OZs

When using the cluster mode, the master node of the cluster also initiates a server socket through PointOfContact. During shutdown for the xadisk instance, we are calling PointOfContact.release() only based on the general remote invocation flag. We should also consider looking at the flag for clustering remote invocation flag (set for the master instance).

As suggested by Ravi,

  • if(configuration.getEnableRemoteInvocations()) {
    + if(getHandleClusterRemoteInvocations() || getHandleGeneralRemoteInvocations()) { pointOfContact.release(); }


 Comments   
Comment by Nitin Verma [ 16/Aug/14 ]

Checked-in the changes to trunk. The fix was suggested by Ravi himself on the above discussion thread.





[XADISK-154] Transaction log may not get deleted even after all of its related transactions' completion. Created: 22/Nov/13  Updated: 16/Aug/14  Resolved: 16/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Daniel for identifying this problem at the thread: https://groups.google.com/forum/#!topic/xadisk/oJBK47CiyJM.

A transactions complete, there is a check to see if the logs they are using are not used by any other ongoing transaction. If so, such logs is deleted. Due to incomplete tracking of the log usage, this was not working as expected.

Also, if a completing transaction was the last remaining transaction in the current log, and when any further log request ends up creating a new log, the last log can remain undeleted. We need to take care of this too.



 Comments   
Comment by Nitin Verma [ 16/Aug/14 ]

Checked-in the changes to trunk. The changes had been uploaded on the above discussion thread and were verified by Daniel.





[XADISK-145] Add a mechanism to cleanup lock related data-structures for files/directories (eg LockTreeNode). Created: 28/Sep/13  Updated: 16/Aug/14  Resolved: 16/Aug/14

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Marco for identifying and reporting this on the discussion thread at: https://groups.google.com/forum/#!topic/xadisk/n9xj3OuWW9o

The NativeConcurrencyControl is missing a mechanism to cleanup the lock related data-structure for files/directories when they are no longer needed (when the lock is safe to be de-allocated, i.e. when no transaction is holding the lock or waiting for it or has locked an ancestor directory etc). Without such cleanup, there is a risk of intolerably huge number of such ("currently" useless) objects accumulating over time under a load with large number of files.



 Comments   
Comment by Nitin Verma [ 16/Aug/14 ]

Checked-in the changes to trunk. The changes had been uploaded in the same discussion thread as above, and were verified by Marco.





[XADISK-162] Generate OSGi bundle manifest attributes Created: 28/Feb/14  Updated: 28/Feb/14

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Jasper Siepkes Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I've added a small patch to add options to the Maven POM which adds OSGi Bundle attributes to the manifest when building. This allows XADisk to be used in an OSGi framework.



 Comments   
Comment by Jasper Siepkes [ 28/Feb/14 ]

Can't attach files to JIRA So here's the patch pasted:

 
--- a/pom.xml
+++ b/pom.xml
@@ -14,8 +14,8 @@ Contributor : Christoph Beck
     <modelVersion>4.0.0</modelVersion>
     <groupId>java.net.xadisk</groupId>
     <artifactId>xadisk</artifactId>
-    <version>SNAPSHOT</version>
-    <packaging>jar</packaging>
+    <version>1.3.0-SNAPSHOT</version>
+    <packaging>bundle</packaging>
     <name>XADisk</name>
     <description>Transactional Wrapper for File Systems</description>
 	
@@ -83,6 +83,22 @@ Contributor : Christoph Beck
                     </execution>
                 </executions>
             </plugin>
+            <plugin>
+                <groupId>org.apache.felix</groupId>
+                <artifactId>maven-bundle-plugin</artifactId>
+                <extensions>true</extensions>
+                <configuration>
+                    <instructions>
+                        <Bundle-SymbolicName>org.xadisk</Bundle-SymbolicName>
+                        <Export-Package>org.xadisk.*</Export-Package>
+                        <Import-Package>
+                            javax.naming.*;resolution:=optional,
+                            javax.resource.*;resolution:=optional,
+                            *
+                        </Import-Package>
+                    </instructions>
+                </configuration>
+            </plugin>
         </plugins>
         <sourceDirectory>src</sourceDirectory>
         <resources>
-- 
1.8.3.1




[XADISK-120] fileExists throws InsufficientPermissionOnFileException Created: 22/Jun/12  Updated: 03/Jan/14  Resolved: 10/Aug/13

Status: Closed
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: current

Type: Bug Priority: Major
Reporter: titmus Assignee: Nitin Verma
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64bit
JDK 1.6.0_29



 Description   

fileExists sometimes thows InsufficientPermissionOnFileException instead of returning false.

Suppose we have some directory structure with root directory <dataRoot> (which exiss).
Then we check if directories <dataRoot>/dir3, <dataRoot>/dir3/subdir1 and <dataRoot>/dir3/subdir1/subdir2 exist (but they don't):

File dir1 = new File(dataRoot, "dir3");
File dir2 = new File(dir1, "subdir1");
File dir3 = new File(dir2, "subdir2");

session.fileExists(dir1); // works
session.fileExists(dir2); // throws InsufficientPermissionOnFileException:
session.fileExists(dir3); // works

When fileExiss is invoked for <dataRoot>/dir3/subdir1 it throws InsufficientPermissionOnFileException

org.xadisk.filesystem.exceptions.InsufficientPermissionOnFileException: Permission of type [READ_DIRECTORY] is needed over the file/directory with path [C:\Users\titmus\AppData\Local\Temp\junit283329822371778462\data\dir3] for the i/o operation to succeed.
at org.xadisk.filesystem.NativeSession.checkPermission(NativeSession.java:1078)
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:401)
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:385)



 Comments   
Comment by titmus [ 22/Jun/12 ]

The fileExistsAndIsDirectory method behaves in the same way.

Comment by Nitin Verma [ 10/Aug/13 ]

All operations in XADiskBasicIOOperations are preceded by a permission check over the file/parent-file(directory). The fileExists operation is preceded by a read permission check over the parent directory. We call (in VirtualViewDirectory) Java File's canRead/canWrite methods without first checking for existence (this saves us an extra call to Java File's exists method), and these methods from Java File do not report any existence related exception - they simply say canRead/canWrite or not. This is the reason we cannot distinguish permission or existence issue.

Leaving it as-designed for now, assuming a small compromise would be possible with the api users for the exception handling part. But anyone please feel free to comment back.

Comment by hell_keeper [ 03/Jan/14 ]

Hi, I agree with you with the fact that a small compromise can be possible with the user of the API. However, about the semantic of the method, I think it’s a bad idea if a folder or a file does not exist to return an exception about the read/write authorization because its parent does not exist. I’m testing and using XADisk API to handle files/folders with path and it’s a little bit frustrating to don’t understand why we do not have the expected behavior when testing if a file exists because the some previous folders of the path have not be created (not because I have not managed it but because it can be possible in my workflow :/).

What do you think about the semantic and the fact to force the user to handle an Illegitimate exception just to avoid to perform a File.exists ? Does the File.exists has huge drawback on performance ?

Please let me know your point of view about that .

Best regards.





[XADISK-157] InsufficientPermissionOnFileException sometimes missing [READ_DIRECTORY] Created: 07/Dec/13  Updated: 07/Dec/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Open JDK 1.6 and 1.7, Windows 7 and Linux (RedHat and Ubuntu)



 Description   

I often get InsufficientPermissionOnFileException on files where the real permissions in the file-system do not show any missing permissions.
I ignore the error and consider the file non-readable with the following code:

try {
Session session = getSession();
File file = getFile();
return session.fileExists(file) && !session.fileExistsAndIsDirectory(file);
} catch (final Exception e) {
Warden.disregardAndReport(e);
return false;
}

I am wondering what the causes may be for this issue.

I have seen this happen in the following circumstances:

  • Checking existence of a directory the parent directory of which doesn't exist either (in code which creates neccessary parent directories as needed).
  • Checking existence of hard-linked files (here the cause is less clear).

Stacktrace Fragment:

org.xadisk.filesystem.exceptions.InsufficientPermissionOnFileException: Permission of type [READ_DIRECTORY] is needed over the file/directory with path [/var/lib/preprocessor/configversions/20131206T185427.534Z/sequences] for the i/o operation to succeed.
at org.xadisk.filesystem.NativeSession.checkPermission(NativeSession.java:1157)
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:407)
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:391)



 Comments   
Comment by Nitin Verma [ 07/Dec/13 ]

Thanks Gunnar.

The case of parent directory missing you mentioned is a known issue: https://java.net/jira/browse/XADISK-120. Please let me know if you discover any detail about the hard-link cases.





[XADISK-153] File.canWrite() returning wrong value, leading to invalid permission check by xadisk. Created: 22/Nov/13  Updated: 22/Nov/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Luc for bringing this up on the discussion thread: https://groups.google.com/forum/#!topic/xadisk/cuQAhyxMjyI

XADisk does a permission check during the io operations, in an attempt to make sure that they succeed during commit. During create file, it check the parent directory write permission. As discussed on above thread, the check returns wrong value (on the given windows environment), leading to failure in commit.

The following code, as provided by Luc, works as follows on his windows machine:

===============================

File f = new File( "C:/");
System.out.println("f canwrite " + f.canWrite());
File newFile = new File( f.getAbsolutePath(), "zap.txt");
boolean created = newFile.createNewFile();
System.out.println( "created: " + created);

The output:

f canwrite true
Exception in thread "main" java.io.IOException: A required privilege is not held by the client
at java.io.WinNTFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1006)
at zap.main(zap.java:16)

===============================






[XADISK-95] DirectorySync fails on Win7 64 bit Created: 21/Nov/11  Updated: 24/Oct/13

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: 1.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: kanebonnette Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: 2 days
Time Spent: Not Specified
Original Estimate: 2 days
Environment:

Windows 7, 64 bit, Java 1.6.0_026


Attachments: Java Source File Example.java    

 Description   

discussion thread here: http://groups.google.com/group/xadisk/browse_thread/thread/aab722ef7450ceab

Basic problem is that when StandaloneFileSystemConfiguration.setSynchronizeDirectoryChanges is set to true and absolute paths are use, XADisk crashes/hangs with the exception:
Exception in thread "main"
org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The
XADisk instance has encoutered a critial issue and is no more
available. Such a condition is very rare. If you think you have setup
everything right for XADisk to work, please consider discussing in
XADisk forums, or raising a bug with details
at
org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSy stem.java:
488)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:707)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:
1196)
at selected.Example.example0(Example.java:108)
at selected.Example.main(Example.java:213)
Caused by: java.io.IOException: Fatal Error: Directory changes could
not be forced-to-disk during transaction commit.
at
org.xadisk.filesystem.DurableDiskSession.forceToDisk(DurableDiskSession.jav a:
122)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:695)
... 3 more

When StandaloneFileSystemConfiguration.setSynchronizeDirectoryChanges is set to true and relative paths are used, XADisk prints the following error:
XADisk Error [Native Module] Directory C:\ does not exist.

in both cases, setting StandaloneFileSystemConfiguration.setSynchronizeDirectoryChanges to false does not report any errors.

The problem apparently does not occur in WinXP, 32 bit.

Estimate set at two days because... I needed a value.



 Comments   
Comment by Nitin Verma [ 17/Feb/12 ]

A workaround (if it fits, depending upon your application requirements) is to disable directory synchronization using setSynchronizeDirectoryChanges

Comment by Nitin Verma [ 21/Aug/13 ]

Just FYI. If developers using xadisk face such native library issues, they can do the following without any need to modify and re-build the xadisk java code:

-modify the native code if necessary (available in svn/trunk/native/*.c).

-compile the code according to the target platform and prepare the dynamic library.

-rename the dynamic library as "placeholder_xadisk.native" and place it inside the xadisk.jar as xadisk.jar/native/placeholder_xadisk.native.

Comment by gastaldi [ 24/Oct/13 ]

Hi,

Is there any estimate of when will this issue be solved?

Thanks

Comment by Nitin Verma [ 24/Oct/13 ]

Hi George,

Unfortunately not. Would you (or anyone else reading this bug) like to try out the compilation for windows 7 and see if the issue gets fixed. The generated lib can then be shared with everyone.

Thanks,
Nitin





[XADISK-150] Savepoint like feature in XADisk. Created: 13/Oct/13  Updated: 24/Oct/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: New Feature Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to George for his comments on https://groups.google.com/forum/#!topic/xadisk/9_Fy3Lr3P5k.

The savepoint is a database concept enabling changes to be bookmarked and reverted-to (rolled-back) at a later point. http://en.wikipedia.org/wiki/Savepoint

It is not yet confirmed how feasible it would be to implement this, and how much value will it really add to xadisk users in general.



 Comments   
Comment by Jasper Siepkes [ 24/Oct/13 ]

Would be nice if this is implemented in a pluggable way (ie. support multiple implementations). Some underlying filesystems on which XADisk runs already have capabilities for this functionality. For example if you run XADisk on a system with ZFS it would be nice if we could leverage ZFS' snapshot capabilities.





[XADISK-152] When using clustering, the sessions do not close connections to the master concurrency control. Created: 14/Oct/13  Updated: 15/Oct/13  Resolved: 15/Oct/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When using clustering, the sessions created in the slave xadisk instances depend upon the concurrency-control of the master xadisk instance. If the master xadisk instance is a remote one, a proxy named RemoteConcurrencyControl is handed over to the sessions for the same. As this proxy object encapsulate a communication channel, so a new proxy object (hence a new channel) is needed for every such session. But sessions do not disconnect from these remote proxy objects. This leads to accumulation of channels, conversation-context (and their hosted object contexts) etc. Hence memory "leaks".

Solution is to disconnect from remote-concurrency-control as a session is done (eg in cleanup).



 Comments   
Comment by Nitin Verma [ 15/Oct/13 ]

Checked-in the changes to trunk. Revision #557.





[XADISK-151] Some objects put in HostedContext remain there even if no longer needed. Created: 14/Oct/13  Updated: 14/Oct/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

HostedContext is used to keep objects for invoking methods upon them remotely. It comes into play in situations like: remote xadisk invocation for filesystem operations, endpoint activation on remote xadisk instance, clustering (xadisk slave instances making use of a single master instance's concurrency control).

ConversationalHostedContext is bound to a specific communication channel and when the communication is closed, the context would go away (garbage collected) automatically.

GlobalHostedContext remains alive for the lifetime of the instance.

Currently, the three kinds of objects kept in the global hosted context are removed at certain later time (message-endpoint-factory, message-endpoint, xaresource for message delivery).

But, for conversational hosted context, the objects are never removed. This can lead to accumulation of references to the unused objects. For example, in case of remote xadisk operations, each new session is kept in this context. So, as long as the same xafs proxy object is used to create new and new sessions, all the session object would keep accumulating in the conversational context (of the xafs proxy).

Implement the removal of unused objects from the conversational context at the appropriate times (eg commit/rollback in case of a remote session).






[XADISK-148] In case of rollback, transactionSubmittedBuffers is not freed from the transaction entry. Created: 11/Oct/13  Updated: 11/Oct/13  Resolved: 11/Oct/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: current

Type: Bug Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In case of rollback, transactionSubmittedBuffers is not freed from the transaction entry. During commit, the corresponding entry is removed from this map, but in case of rollback, the transaction's entry can remain.

Such remaining entry contains the "xid" object which refers to the "owningSession". This results in the rolledback session object not becoming free for gc.



 Comments   
Comment by Nitin Verma [ 11/Oct/13 ]

Checked-in the changes to trunk. Revision 556.





[XADISK-147] Add method to get transaction status in Session. Created: 08/Oct/13  Updated: 08/Oct/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to George for suggesting this at: https://groups.google.com/forum/#!topic/xadisk/6Lbe3CVZSBE

It could be useful for some applications to query the Session object (usually, from some other thread other than the one handling the Session object) about its transaction's status (open, committed or rolledback).



 Comments   
Comment by Nitin Verma [ 08/Oct/13 ]

Parallel to the Session interface, there are two other interfaces, XADiskConnection (JCA environments) and XASession (standalone JTA environments); all the three interfaces implement XADiskBasicIOOperations. But the last two interfaces do not hold the responsibility for transaction management as does the Session interface (which is for standalone non-JTA environments) through its commit/rollback methods. In these two cases, the same is achieved by the JTA standard interfaces. So, the new method only need be added to the Session interface.





[XADISK-144] out-of-file-handles on transaction involing large number of copied files Created: 11/Sep/13  Updated: 11/Sep/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS, Linux testhost 2.6.32-358.2.1.el6.x86_64 #1 SMP Wed Mar 13 00:26:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux, java version "1.7.0_25"
OpenJDK Runtime Environment (rhel-2.3.10.4.el6_4-x86_64)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)



 Description   

An XADisk transaction involving a very large number of files extracted from an archive fails fatally due to an OS-level out-of-file-handle situation.

The issue is critical in so far as we want to write the files extracted from the archive under transaction control and the transaction fails fatally:

XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details

We took care to close all files immediately after writing the data and we also ensured that only a limited number of files are processed concurrently.

At first it looked as if the files might be cached in NativeSession.getCachedXAFileOutputStream.
On further inspection of the XA-Disk code it became clear that closing the XAFileOutputStream doesn't actually close the underlying file handle/channel. The close happens delayed in VirtualViewFile.forceAndFreePhysicalChannel() upon commit/rollback. This is also the case for heavyWrite which we use in this case as the files are completely written in one chunk.

This means that we can do nothing about this because the number of files that are to be extracted is potentially arbitrarily large - in in particular significantly larger than any acceptable ulimit.

I am wondering why the file handles cannot be closed earlier.
I have seen that the data is flushed upon XAFileOutputStream.close().
It should be possible to close them upon close at least in the heavy write case. Alternatively it might be possible to close the file and later reopen at the same position - should that unexpetcedly be neccessary.


Additional information:

The file handle limits is at the default of 1024:

bash-4.1$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 29738
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Reported Exception:

[...]
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:489)
at org.xadisk.filesystem.virtual.NativeXAFileOutputStream.<init>(NativeXAFileOutputStream.java:56)
at org.xadisk.filesystem.NativeSession.getCachedXAFileOutputStream(NativeSession.java:1266)
at org.xadisk.filesystem.NativeSession.createXAFileOutputStream(NativeSession.java:187)
at org.xadisk.filesystem.NativeSession.createXAFileOutputStream(NativeSession.java:43)
at de.konzentrik.lib.io.xa.XaDecoupledFileSystem$XaDecoupledSession.createXAFileOutputStream(XaDecoupledFileSystem.java:216)
at de.konzentrik.lib.io.XaDiskStore.getOutputStream(XaDiskStore.java:263)
at de.konzentrik.app.preprocess.process.step.ImageCopierStep.process(ImageCopierStep.java:65)
at de.konzentrik.app.preprocess.process.step.ImageCopierStep.process(ImageCopierStep.java:25)
[...]
at de.konzentrik.app.preprocess.process.ProcessStepRunner$StepRunner.run(ProcessStepRunner.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more
Caused by: java.io.IOException: Zu viele offene Dateien [german for "too many open files"]
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:947)
at org.xadisk.filesystem.utilities.FileIOUtility.createFile(FileIOUtility.java:91)
at org.xadisk.filesystem.DurableDiskSession.createFile(DurableDiskSession.java:158)
at org.xadisk.filesystem.virtual.VirtualViewFile.safeSetupForPhysicalFileExistence(VirtualViewFile.java:142)
at org.xadisk.filesystem.virtual.VirtualViewFile.setUpForHeavyWriteOptimization(VirtualViewFile.java:154)
at org.xadisk.filesystem.virtual.NativeXAFileOutputStream.<init>(NativeXAFileOutputStream.java:54)
... 16 more






[XADISK-135] Remove class-path entry from the manifest file. Created: 23/May/13  Updated: 04/Sep/13  Resolved: 04/Sep/13

Status: Closed
Project: xadisk
Component/s: misc
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Jason for reporting this at thread: http://groups.google.com/group/xadisk/browse_thread/thread/47b7324ece6b884a?hl=en.

The class-path entries in the manifest file of xadisk.jar (both standalone and bundled inside .rar) are redundant and may raise warnings during deployment. We should remove them.



 Comments   
Comment by Nitin Verma [ 04/Sep/13 ]

The problematic class-path entries are not present in 1.2.2 binaries.





[XADISK-129] Add support to filesystem synchronization for Solaris OS on Sparc Architecture Created: 24/Jan/13  Updated: 03/Sep/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Marco Quaranta Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File solaris_32_xadisk.native     File solaris_64_sparc.native     File solaris_64_xadisk.native    
Tags: filesystem, native, solaris, synchronization

 Description   

I've successfully compiled native code (forceDirectory method, find library attached) on SunOS 64 running on a SPARC architecture. I've also tested it with TestXADiskNative.java and it seems to work.
Please add this library in the next XADisk version.
Thank you



 Comments   
Comment by Nitin Verma [ 16/Feb/13 ]

Thanks to Marco for sending the compiled code at: http://groups.google.com/group/xadisk/browse_thread/thread/97364e2a821a93a7?hl=en

Comment by Nitin Verma [ 21/Aug/13 ]

Just FYI. If developers using xadisk face any native library issues, they can do the following without any need to modify and re-build the xadisk java code:

-modify the native code if necessary (available in svn/trunk/native/*.c).

-compile the code according to the target platform and prepare the dynamic library.

-rename the dynamic library as "placeholder_xadisk.native" and place it inside the xadisk.jar as xadisk.jar/native/placeholder_xadisk.native.

Comment by Nitin Verma [ 22/Aug/13 ]

I am attaching the three files sent by Marco over the google group.

Hi Marco,

Can you please add a brief comment below about the three files - for which platform/architecture are they (I could not figure out the two files named with 64):

solaris_64_sparc.native
solaris_32_xadisk.native
solaris_64_xadisk.native

I hope these libs would be useful for users; though I am not putting them with xadisk as of now, but they can easily replace them as "placeholder_xadisk.native" inside the xadisk.jar.

Thanks,
Nitin

Comment by Marco Quaranta [ 03/Sep/13 ]

Hi Nitin,
a little comment about these three files:

solaris_64_sparc.native - This is the first attempt I made and, as opposed the name said, it is for 32bit (it was an almost failed attempt). It's compiled without -fPIC options. Please, use solaris_32_xadisk.native in place of this.

solaris_32_xadisk.native - This is the compiled version for 32bit jvm compiled with the following command:
/usr/sfw/bin/gcc -shared -fPIC -I/usr/jdk/instances/jdk1.7.0_09/include -I/usr/jdk/instances/jdk1.7.0_09/include/solaris -o ./nativelib.so ./XADiskUnix.c

solaris_64_xadisk.native - This is the compiled version for 64bit jvm compiled with the following command:
/usr/sfw/bin/gcc -shared -fPIC -m64 -I/usr/jdk/instances/jdk1.7.0_09/include -I/usr/jdk/instances/jdk1.7.0_09/include/solaris -L/usr/sfw/lib/64 -R/usr/sfw/lib/64 -o ./nativelib.so ./XADiskUnix.c

I already know about "placeholder_xadisk.native" and it's the way I have used those files in my applications.

Thank you,
Marco





[XADISK-128] NativeXAFileInputStream skip method throws an unexpected IllegalArgumentException with message "New position cannot be negative or more than file size" Created: 23/Jan/13  Updated: 27/Aug/13  Resolved: 09/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Critical
Reporter: Marco Quaranta Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: NativeXAFileInputStream, inputstream, skip

 Description   

Calling skip method on XAFileInputStream generate IllegalArgumentException due to wrongly calculated readPositionAfterSkip.
This variable is evaluated with this expression:

(NativeXAFileInputStream.java line 175)
long readPositionAfterSkip = (position - byteBuffer.remaining()) + n;

Using a position equals to 0 and skipping 200 bytes with a generic file larger than 200 bytes this expression return a negative value and than, the subsequential calling to "position(readPositionAfterSkip)" method, will throws IllegalArgumentException.

Why don't simply use this?
long readPositionAfterSkip = position + n;

Thank you
Marco



 Comments   
Comment by Nitin Verma [ 07/Feb/13 ]

Thanks Marco.

I could reproduce the issue if I try to skip some bytes just after opening the input stream, before any call to read etc. (so, a workaround is to read one byte anyway for the first time, xafis.read())

The logic is failing because byteBuffer is not yet filled. There needs to be a check on flag filledAtleastOnce.

We need to take into account the value of byteBuffer.remaining(); the "position" is not the position of the next byte in the file (to be returned to the user), it is an internal variable which is used to track from where in the file the buffer is to be filled next.

Thanks Again...
Nitin

Comment by Nitin Verma [ 09/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-118] NullPonterExcepton when listing file Created: 22/Jun/12  Updated: 27/Aug/13  Resolved: 10/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: titmus Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64bit
JDK 1.6.0_29



 Description   

Invoking method XADiskBasicIOOperations.listFiles causes NPE when passed file is actually a file and not a directory:

at java.util.Arrays$ArrayList.<init>(Arrays.java:3357)
at java.util.Arrays.asList(Arrays.java:3343)
at org.xadisk.filesystem.virtual.VirtualViewDirectory.listFilesAndDirectories(VirtualViewDirectory.java:186)
at org.xadisk.filesystem.virtual.TransactionVirtualView.listFiles(TransactionVirtualView.java:127)
at org.xadisk.filesystem.NativeSession.listFiles(NativeSession.java:483)



 Comments   
Comment by Nitin Verma [ 10/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-76] Moving a file with xADiskConnection.moveFile(...) gets stuck if java.io.File.rename() fails Created: 17/May/11  Updated: 27/Aug/13  Resolved: 21/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.2.2

Type: Bug Priority: Minor
Reporter: juliusblank Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

WinXP



 Description   

Scenario:
An EJB tries to rename a file ("testfile") with XADisk (xADiskConnection.moveFile(...)), that has already been opened by someone else (in this case it's a simple new FileReader(new File("testfile")); inside a test).

Expected:
I would expect some notification about XADisk being unable to rename the file.

Result:
Instead the test "hangs" until a CORBA-timeout (after 30 minutes) triggers, interrupting the loop (see below) that is trying to rename the file.

Suggestion:
I tracked down the issue to the class org.xadisk.filesystem.utilities.FileIOUtility, where there is the following method:

public static void renameTo(File src, File dest) throws IOException

{...}

My problem here is that if the rename fails (for the above reason) the loop in there won't end until some external interruption occurs. I would think that the method returns an IOException or similar in case the rename/delete goes wrong.

So my suggestion would be to implement a counter and run through the loop only a certain number of times (depending on what the method "makeSpaceForGC(...)" is supposed to do...). If by eg the 5th time the file could not be deleted, delete the second file (created by the renameTo()-method and throw an exception.



 Comments   
Comment by Nitin Verma [ 17/May/11 ]

Hi Julius,

Thanks for your detailed and clear explanation.

XADisk maintains an assumption that there is no other software operating over the files/directories which are being modified through its APIs.
This assumption was kept because if we are operating over a set of files in a transactional manner, we won't be having (in most scenarios) other applications operating over the same set of files in non-transactional manner.

For your specific scenario, we can do any of these:

a. write the other application also to use XADisk API (XAFileInputStream, you can wrap it with java's InputStreamReader) instead of using FileReader(). This way, when the first application goes to rename, it throws an exception (LockTimeOut) if the other one is reading it. Just make sure that the two applications invoke the same XADisk instance. If it is running on some other JVM, it can use remoting to invoke APIs on the XADisk instance.

b. in the first application, which is using XADisk, write a check to see if the file is already locked or not (depending on the OS, a file read operation may not have locked the file), and proceed for rename only if the file is not locked. This solution need not work in cases when the other application comes and reads the file "anytime", because it may happen that the check for lock passes, and then the other application suddenly starts reading the file.

c. if possible, modify the second application such that the FileReader is closed as soon as the reading job is done. This would let the rename continue and not wait for so long.

Please feel free to write to me at [nitin_verma AT java.net] if you have any questions, suggestions or want to discuss more alternatives.

Thanks,
Nitin

Comment by juliusblank [ 08/Nov/12 ]

Hi Nitin,

time to reactivate this issue
I agree with your explanation: other applications should not mess around with files that are under XADisks (transactional) management.
As with databases or other data sources it should be ensured by external means (ie access restrictions and such) that no unwanted modifications can be made to the data.

As it is still bad for our application if we run into a (CORBA) timeout in this "strange" case, I'd like to make a suggestion that should not harm XADisks design principles:
How about introducing a configurable parameter that sets the number of retries in the mentioned scenario and which has a default of, say, 0 (or whatever value is suitable for saying "off"), which would imply an unlimited number of retries, leaving the current behaviour unchanged.
That should solve our problem without infering with XADisk too much.

What do you say?

Cheers,
Julius

Comment by Nitin Verma [ 18/Aug/13 ]

Hi Julius,

I have implemented one fix for this. XADisk will come out of the commit/rollback when it will detect ioexceptions like the one reported in this bug. It will provide an interface for obtaining the identifier for such transactions and then removing them off from the xadisk's memory/txn-logs (so that xadisk can unlock those files, if still running, or come-out of the recovery, if such failure happens during recovery). It will not retry to auto-commit or rollback such transactions; the user would need to look for the transaction's changes (whatever took place) and make them consistent manually. And once done, the user will inform xadisk to now safely close the transaction; in other words, mark the transaction as complete. So that, rest of the transactions can keep going, unlike today when we bring the whole xadisk instance down.

I will update in the javadoc in more detail. Please feel free to question and comment.

Thanks & Regards,
Nitin

Comment by Nitin Verma [ 18/Aug/13 ]

Checked-in the changes to trunk (revision #538). Javadoc pending.





[XADISK-98] Safe close FileInput/OutputStreams Created: 07/Jan/12  Updated: 27/Aug/13  Resolved: 19/Aug/13

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: 1.2
Fix Version/s: 1.2.2

Type: Bug Priority: Minor
Reporter: romaerzhuk Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File Safe-close-streams.patch    

 Description   

FileIOUtility uses unsafe close operation. Any exceptions lead to leaks file descriptors, for example:

    public static void copyFile(File src, File dest, boolean append) throws IOException {
        FileChannel srcChannel = new FileInputStream(src).getChannel();
        FileChannel destChannel = new FileOutputStream(dest, append).getChannel();

        // Any exceptions lead to leaks file descriptors!
        long contentLength = srcChannel.size();
        long num = 0;
        while (num < contentLength) {
            num += srcChannel.transferTo(num, NativeXAFileSystem.maxTransferToChannel(contentLength - num), destChannel);
        }

        destChannel.force(false);
        srcChannel.close();
        destChannel.close();
    }

Correct code:

    public static void copyFile(File src, File dest, boolean append) throws IOException {
        FileInputStream srcIn = null;
        FileOutputStream destOut = null;
        try {
          srcIn = new FileInputStream(src);
          destOut = new FileOutputStream(dest, append);
          FileChannel srcChannel = srcIn.getChannel();
          FileChannel destChannel = destOut.getChannel();
          long contentLength = srcChannel.size();
          long num = 0;
          while (num < contentLength) {
              num += srcChannel.transferTo(num, NativeXAFileSystem.maxTransferToChannel(contentLength - num), destChannel);
          }

          destChannel.force(false);
        } finally {
          close(srcIn);
          close(destOut);
        }
    }
    static void close(Closeable c) {
      if (c != null) {
        try {
          c.close();
        } catch (Throwable t) {
           logger.warn("Unable to close: " + t);
        }
      }
    }


 Comments   
Comment by romaerzhuk [ 07/Jan/12 ]

This patch solves the issue

Comment by Nitin Verma [ 21/Jan/12 ]

Many Thanks for identifying this, romaerzhuk. [sorry, don't know your name, I wish I did :)]

I will fix it by the next release.

Just wanted to add an FYI here...

One basic premise when XADisk is used to interact with a set of files is that, (at a broad level) no other software is simultaneously working over the same set of files. If someone else is also accessing those files which are being worked on by XADisk, (say XADisk delete operation is going on), the application runs into the risk of inconsistency.

When XADisk finds that it is not able to do certain operations (say delete a file, because someone else is writing to it), it makes a guess that someone else is simultaneously tempering with the same file, and disables itself from doing any other operation; applications will then start receiving XASystemNoMoreAvailableException.

Now, the above copyFile code is one such place where, if XADisk receives an exception, it is very likely that someone else doing something, and with current implementation, XADisk will call it "end of the world" and will disable itself. In practice, this would require a more serious remedy by the application administrator. But I agree that doing channel.close() is always better, even in these cases.

Thanks Again...
Nitin

Comment by Nitin Verma [ 19/Aug/13 ]

Checked-in the changes to trunk.

Just as an additional information: the xadisk issue #76 is now fixed, and we have a mechanism to leave transactions in incomplete failed state without bringing the xadisk instance down. The user may be doing manual management of the files, which were probably worked upon inside the transaction prior to failure. So, it becomes more important to take care that we always close the fileChannels (specially to the application files operated over by the transaction, if not xadisk system transaction-logs).





[XADISK-131] xadisk should not allow creation of sessions after it has been shutdown Created: 16/Feb/13  Updated: 27/Aug/13  Resolved: 14/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Stephane for bringing up this issue at: http://stackoverflow.com/questions/14887643/xadisk-trying-to-save-a-byte-array-to-a-file

XADisk should not allow creation of new sessions after it has been shutdown. It can stop creating new sessions just at the beginning of the shutdown call, and can inform the existing set of sessions about the shutdown (which it does already).



 Comments   
Comment by Nitin Verma [ 14/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-133] time-out on long transaction gives no indication in ClosedStreamException Created: 25/Feb/13  Updated: 27/Aug/13  Resolved: 14/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Trivial
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64-bit, Java 1.7, Eclipse Juno, Open EJB



 Description   

When running a performance test writing 12 GB to disk with XADisk I stumbled over an unexpected IOException (full stacktrace below), which was caused by a transaction time-out.
This time-out was triggered in TransactionTimeoutDetector.doWorkOnce(). This I found out by debugging only, because the IOException gave no indication of same but was raising during normal continuous writing.

I recommend testing the rolledbackPrematurely flag when reporting the exception in NativeXAFileOutputStream.checkIfCanContinue(). Locating the out-of-bound close() is difficult (I was lucky that I had no complex code running which might have accidentally closed the stream here or there).

Now the stacktrace:

java.io.IOException
at org.xadisk.additional.Utilities.wrapWithIOException(Utilities.java:15)
at org.xadisk.additional.XAFileOutputStreamWrapper.write(XAFileOutputStreamWrapper.java:77)
at java.nio.channels.Channels$WritableByteChannelImpl.write(Channels.java:458)
[...]
at de.konzentrik.app.epg.backend.XaTest.testXa(XaTest.java:77)
at [JUnit]
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at [Eclipse.JDT]
Caused by: org.xadisk.filesystem.exceptions.ClosedStreamException
at org.xadisk.filesystem.virtual.NativeXAFileOutputStream.checkIfCanContinue(NativeXAFileOutputStream.java:171)
at org.xadisk.filesystem.virtual.NativeXAFileOutputStream.write(NativeXAFileOutputStream.java:79)
at org.xadisk.additional.XAFileOutputStreamWrapper.write(XAFileOutputStreamWrapper.java:75)
... 25 more



 Comments   
Comment by Nitin Verma [ 02/Mar/13 ]

Thanks Gunnar. Yes, it would definitely help to report the reason for stream close.

Regards,
Nitin

Comment by Nitin Verma [ 14/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-142] close() method should be called on the corresponding stream instead of the FileChannel. Created: 19/Aug/13  Updated: 27/Aug/13  Resolved: 19/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

All over the xadisk code, close() method should be called on the corresponding stream instead of the FileChannel (which has come from that stream). Though, as observed, closing the FileChannel is closing the corresponding stream (FileInputStream/FileOutputStream/RandomAccessFile) too, but this behavior is decided by the Java runtime, and is not documented.

We should instead close the stream, which will close the channel by itself as per the JavaDoc of FileInputStream/FileOutputStream/RandomAccessFile. Quoting from the Java 5 API documentation of FileInputStream's close method:

"If this stream has an associated channel then the channel is closed as well."



 Comments   
Comment by Nitin Verma [ 19/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-115] JBoss 7.1 expects #run() and #release to be explicitly declared Created: 15/May/12  Updated: 27/Aug/13  Resolved: 09/Aug/13

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Critical
Reporter: Simon Dierl Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

JBoss 7.1.1


Attachments: Text File xadisk-run-release.patch    

 Description   

The #run() and #release() methods required by javax.resource.spi.work.Work are expected by JBoss 7.1 to be overridden in the specific subclasses of Work used in an RA. Unfortunately, XADisk only declared these methods in abstract superclasses; the concrete implementations only inherit these.

While this is perfectly fine with respect to the Spec, JBoss will refuse to load XADisk unless the subclasses override the methods from the abstract Work classes (overriding with super.run() / super.release()) is okay).

While JBoss is quite certainly wrong here, XADisk will not load on JBoss 7.1; the attached patch fixes that (without changing behavior).



 Comments   
Comment by Nitin Verma [ 15/May/12 ]

Hi Simon,

Thanks for finding and reporting this.

You may also be interested in some relevant information at http://groups.google.com/group/xadisk/browse_thread/thread/47b7324ece6b884a - refer to my posts on 7th and 10th April. Quoting below...

"The second one is an issue in JBoss/iron-jacamar. One can either use
the workaround you mentioned ("Ensure every Class in
org.xadisk.filesystem.workers implements run()
and release().") and create a new xadisk binary locally or hopefully
it will be fixed soon in a JBoss release. Also, thanks for filing the
iron-jacamar bug: https://community.jboss.org/message/728756."

"The second issue, which was in JBoss/iron-jacamar, has been fixed
through bug - https://issues.jboss.org/browse/JBJCA-789. This fix is
available in iron-jacamar version 1.0.10.Final. Though, the latest
version of JBoss AS, which is 7.1.1.Final as of today, uses iron-
jacamar version 1.0.9.Final as per https://community.jboss.org/wiki/AS711FinalReleaseNotes.
So, from the JBoss side, we will still have to wait till JBoss
upgrades its iron-jacamar version further."

Thanks,
Nitin

Comment by Nitin Verma [ 09/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-140] Resource leak in XADisk NIO TCP Server Created: 08/Aug/13  Updated: 27/Aug/13  Resolved: 16/Aug/13

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: 1.2.1, current
Fix Version/s: 1.2.2

Type: Bug Priority: Critical
Reporter: Jasper Siepkes Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, OpenJDK 7


Attachments: Java Source File ConversationGateway.java    

 Description   

When stress testing XADisks TCP remote filesystem server I found a massive number of TCP connections in my OS were hanging in the CLOSE_WAIT state. The TCP NIO server which is build into XA Disk keeps references to sockets that have been closed by the client causing TCP sockets never to be released and connections to hang in CLOSE_WAIT. I have attached a new version of 'PointOfContact.java' which remedies this problem. I rewrote a large part of it because I had trouble wrapping everything neatly in try / finally blocks to ensure connections are always cleaned up.



 Comments   
Comment by Jasper Siepkes [ 08/Aug/13 ]

Ugh.. apparently you can't attach anything on java.net's Jira...

Nitin, please send me a message if you want the patch and I'll send it to you.

Comment by Nitin Verma [ 10/Aug/13 ]

I have made below addition in the ConversationGateway.java (attached) when it sees the end-of-stream by the client:
...
...
context.updateWithConversation(buffer);
} catch (IOException ioe)

{ selectionKey.cancel(); channel.socket().close();<---added this. }

...
...

If you are not able to attach your changes, please send them to me at nitin_verma [AT] java.net.

Thanks,
Nitin

Comment by Jasper Siepkes [ 11/Aug/13 ]

Hi Nitin,

Apparently only people with the "Developer" role can add attachments to Jira ( https://weblogs.java.net/blog/edburns/archive/2013/07/15/mojarra-jira-tips?force=954 ) I've E-Mailed you a version of PointOfContact.java which I modified to remedy the above described situation.

I combined it with "ConversationGateway.java" to make it easier to understand the flow and be able to easier wrap stuff in try - catch - finally blocks to ensure not leaks away. I was also able to remove the Queue in which all the SocketChannel's are stored (which I think was the cause of the resource leaks).

Let me know if it useful for XADisk.

Kind regards,

Jasper Siepkes

Comment by Nitin Verma [ 16/Aug/13 ]

Checked-in the changes to trunk.

Many Thanks to Jasper Siepkes for his efforts on this bug.

Nitin





[XADISK-121] moveFile fails with java.lang.AssertionError Created: 22/Jun/12  Updated: 27/Aug/13  Resolved: 22/Jun/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: titmus Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64bit
JDK 1.6.0_29



 Description   

I'm trying to rename an empty directory

File emptyDir = new File(dataRoot, "empty");
File newDir = new File(dataRoot, "dir3");
session.moveFile(emptyDir, newDir);

The dataRoot is a root of directory three, XADisk operates on. The empty directory already exists (and is empty) and dir3 does not exist.
The moveFile operation fails with:

java.lang.AssertionError
at org.xadisk.filesystem.NativeLock.getHolders(NativeLock.java:54)
at org.xadisk.filesystem.NativeConcurrencyControl.pinLockTreeNode(NativeConcurrencyControl.java:266)
at org.xadisk.filesystem.NativeConcurrencyControl.pinDirectoryTree(NativeConcurrencyControl.java:241)
at org.xadisk.filesystem.NativeConcurrencyControl.pinDirectoryForRename(NativeConcurrencyControl.java:236)
at org.xadisk.filesystem.NativeSession.moveFile(NativeSession.java:304)



 Comments   
Comment by Nitin Verma [ 22/Jun/12 ]

Hello.

Thanks for identifying and reporting the issues.

I wrote a simple test (inlined below) for the use-case you described, but it works fine for me; may be something is different in this test-case and the situation where you saw the reported issue:

____________________________________________________________________________________________________________

StandaloneFileSystemConfiguration configuration = new StandaloneFileSystemConfiguration("C:/xadisk", "1");
XAFileSystem xafs = XAFileSystemProxy.bootNativeXAFileSystem(configuration);
xafs.waitForBootup(-1);
System.out.println("Successfully booted xadisk.");

Session session = xafs.createSessionForLocalTransaction();
session.moveFile(new File("C:/empty"), new File("C:/dir3"));
session.moveFile(new File(new File("C:/dataRoot"), "empty"), new File(new File("C:/dataRoot"), "dir3"));
session.commit();
____________________________________________________________________________________________________________

May be you can describe your test-case in more detail.

Thanks,
Nitin

Comment by titmus [ 22/Jun/12 ]

Did you run your code with assertions enabled (-ea switch for java)?
Without it it runs fine indeed. I noticed it in my unit tests which are by default run with assertions enabled by surefire Maven plugin. When assertions are disabled the test pass and does the move. But when assertions are enabled it fails.

Comment by Nitin Verma [ 22/Jun/12 ]

Thanks! Yes, I was actually running with assertions disabled. Now I could reproduce the issue. The fix was trivial. So, I am also checking-in the changes to trunk for you to see over svn (please ignore those other corrections related to indentation).

Comment by Nitin Verma [ 22/Jun/12 ]

Checked-in the changes to trunk (http://java.net/projects/xadisk/sources/svn/revision/525).

Comment by titmus [ 06/Sep/12 ]

I tried the version from the trunk and it works for me. Great job!





[XADISK-122] NullPointerException on transaction completion when timeout has been detected Created: 16/Jul/12  Updated: 27/Aug/13  Resolved: 10/Aug/13

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: titmus Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

GlassFish 3.1.2
JDK 1.7.0_05



 Description   

When XADisk detectes transaction timeout, NPE is raised on transaction completion.

First timeout occurs and XADisk throws TransactionRolledBackException which is intercepted by application:

Caused by: org.xadisk.filesystem.exceptions.TransactionRolledbackException: The method call expected a transaction to be associated, but no such transaction exists. An associated transaction that could be expected by the client had been rolled back earlier.
at org.xadisk.filesystem.NativeSession.checkIfCanContinue(NativeSession.java:1146) ~[xadisk.jar:na]
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:396) ~[xadisk.jar:na]
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:385) ~[xadisk.jar:na]
at org.xadisk.connector.outbound.XADiskConnectionImpl.fileExists(XADiskConnectionImpl.java:94) ~[xadisk.jar:na]
at pl.ispik.iuip.infrastructure.vfs.bean.XaDiskOperations.exists(XaDiskOperations.java:43) ~[iuip-impl-vfs.jar:na]
... 173 common frames omitted
org.xadisk.filesystem.exceptions.TransactionTimeoutException: The transaction associated earlier was rolled back by XADisk because the transaction was open for more than the transaction timeout value.
at org.xadisk.filesystem.workers.TransactionTimeoutDetector.doWorkOnce(TransactionTimeoutDetector.java:38) ~[xadisk.jar:na]
at org.xadisk.filesystem.workers.TimedWorker.run(TimedWorker.java:42) ~[xadisk.jar:na]
at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:114) ~[work-management.jar:3.1.2]
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497) ~[glassfish-corba-orbgeneric.jar:na]
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540) ~[glassfish-corba-orbgeneric.jar:na]

When transaction completes XADisk throws NPE:

java.lang.NullPointerException: null
at org.xadisk.filesystem.NativeXAFileSystem.getSessionForTransaction(NativeXAFileSystem.java:290) ~[xadisk.jar:na]
at org.xadisk.filesystem.NativeXAFileSystem.getSessionForTransaction(NativeXAFileSystem.java:54) ~[xadisk.jar:na]
at org.xadisk.connector.XAResourceImpl.end(XAResourceImpl.java:68) ~[xadisk.jar:na]
at com.sun.jts.jta.TransactionState.beforeCompletion(TransactionState.java:160) ~[jts.jar:3.1.2]
at com.sun.jts.jta.SynchronizationImpl.before_completion(SynchronizationImpl.java:135) [jts.jar:3.1.2]
at com.sun.jts.CosTransactions.RegisteredSyncs.distributeBefore(RegisteredSyncs.java:158) [jts.jar:3.1.2]
at com.sun.jts.CosTransactions.TopCoordinator.beforeCompletion(TopCoordinator.java:2548) [jts.jar:3.1.2]
at com.sun.jts.CosTransactions.CoordinatorTerm.commit(CoordinatorTerm.java:279) [jts.jar:3.1.2]
at com.sun.jts.CosTransactions.TerminatorImpl.commit(TerminatorImpl.java:250) [jts.jar:3.1.2]
at com.sun.jts.CosTransactions.CurrentImpl.commit(CurrentImpl.java:633) [jts.jar:3.1.2]
at com.sun.jts.jta.TransactionManagerImpl.commit(TransactionManagerImpl.java:332) [jts.jar:3.1.2]
at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.commitDistributedTransaction(JavaEETransactionManagerJTSDelegate.java:185) [jts.jar:3.1.2]
at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.commit(JavaEETransactionManagerSimplified.java:861) [jta.jar:3.1.2]
at com.sun.ejb.containers.BaseContainer.completeNewTx(BaseContainer.java:5136) [ejb-container.jar:3.1.2]
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4901) [ejb-container.jar:3.1.2]
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2045) [ejb-container.jar:3.1.2]
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1994) [ejb-container.jar:3.1.2]
at com.sun.ejb.containers.EJBObjectInvocationHandler.invoke(EJBObjectInvocationHandler.java:213) [ejb-container.jar:3.1.2]
at com.sun.ejb.containers.EJBObjectInvocationHandlerDelegate.invoke(EJBObjectInvocationHandlerDelegate.java:79) [ejb-container.jar:3.1.2]



 Comments   
Comment by Nitin Verma [ 10/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-124] Performance in invoking methods of remote instances Created: 19/Nov/12  Updated: 27/Aug/13  Resolved: 11/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: None
Fix Version/s: 1.2.2

Type: Improvement Priority: Major
Reporter: mjsafa Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 1 day
Time Spent: Not Specified
Original Estimate: 1 day
Environment:

one instance running on the a server in the local network
and another instance is running in client mode and invoking the remote instance


Attachments: Java Source File RemoteMethodInvoker.java    

 Description   

when i call remote instance in a local network, each remote method call (even checking a file exist) takes about 40 milliseconds to perform.
the result is very low performance of read and write and checking files in batch mode.

i solved the problem.
the problem is in RemoteMethodInvoker when writing to the socket, it doesn't use BufferdInputStream and BufferedOutputStream.
i wrapped the socket.getOutputStream in a BufferedOutputStream and the socket.getInputStream in a BufferedInputStream. and added a invokationBufferSize property to the class to indicate the size of that buffer.
the result was 40x increase in speed of calling remote methods.

the edited class RemoteMethodInvoker is attached.

sorry for poor english writing.



 Comments   
Comment by Nitin Verma [ 20/Nov/12 ]

Hello. Many Thanks for identifying the problem and coming up with the solution. I have reviewed it, I will check-in the changes after performing some tests. Thanks again.

Regards,
Nitin

Comment by Nitin Verma [ 11/Aug/13 ]

Hello.

The attached changes are fine. For the output stream though, the use of buffered stream seems to reduce performance (due to overhead, and there are two write calls only?) so i have removed it. Also, the buffer size for the input stream can be 1024 bytes.

Please feel free to comment. Thanks again.

Nitin

Comment by Nitin Verma [ 11/Aug/13 ]

Checked-in the changes to trunk.

Comment by mjsafa [ 13/Aug/13 ]

hi
thanks for your review.
about buffered stream: be sure it enhances performance.
on single machine, your idea is right about overhead and decrease in performance, but in a LAN network, i tested it, if you don't use buffered streams, java will send every byte of content as a separate packet, so for big files and big batch operations the performance will be horrible. so not think a second about using buffered streams in networks.

write some code to save and load millions of 100K files from a remote instance on the network! it will take so much time.

thanks a lot
M.Safaeian

Comment by Jasper Siepkes [ 13/Aug/13 ]

Hi Nitin,

I can confirm what mjsafa says. I've bumped into this issue too, performance over a real network with a lot or "larger" (> 200KB) is really bad without his patch.

Comment by Nitin Verma [ 14/Aug/13 ]

Thanks for your useful comments.

I did some more analysis around Socket's handling of write operations. What I could figure out is that, each socket-write operation (via socket's output-stream) is resulting in the operating system's call for socket write, and each such call copies the data bytes into the kernel buffers for tcp send. From this perspective, it is better to buffer write calls (even if 2 such. Based on comments from the reporter of this bug, and more testing on my local machine, it is still better to use buffering for as less as 2 write operations). They seem to save atleast a JNI call of copying the data to tcp send buffers.

Thanks,
Nitin

[As an additional info, the actual sending of the bytes can take place at any time:
http://www.unixguide.net/network/socketfaq/2.11.shtml

As per my observation with Wireshark, the actual sending of the data bytes was taking place around 0.5 seconds after output-stream's write. But this is just a sample observation, just to quote here.]

Comment by Nitin Verma [ 14/Aug/13 ]

Checked-in the changes to trunk. Two checkins - revision #530 & #533.





[XADISK-141] In GatheringDiskWriter, method writeRemainingBuffersNow should wait for processEvent to complete. Created: 13/Aug/13  Updated: 27/Aug/13  Resolved: 13/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In GatheringDiskWriter, method writeRemainingBuffersNow should wait for processEvent to complete. Else, there is a danger (not so reported till now, but in theory) of buffer re-ordering, where the processEvent could end-up writing the de-queued buffers for the same transaction AFTER writeRemainingBuffersNow does so (for the later buffers).

Solution is to expand the scope of transactionLogLock while will ensure no interference. The locking was already there around the log-writing, but it should also enclose the de-queuing of buffers to make de-queue + log-writing atomic.



 Comments   
Comment by Nitin Verma [ 13/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-116] Using XAFileInputStream.read(byte[]...) throws ClassCastException when using remote invocation to XADisk. Created: 15/May/12  Updated: 27/Aug/13  Resolved: 15/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2, 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Kane for identifying this issue through discussion thread at http://groups.google.com/group/xadisk/browse_thread/thread/65b38f0425a54997.

When using XAFileInputStream.read(byte[] b), or XAFileInputStream.read(byte[] b, int off, int length), a ClassCastException is thrown as below in case the xadisk instance is a remote one:

________________________________________________________________________________________________________________________

java.lang.ClassCastException: org.xadisk.filesystem.virtual.NativeXAFileInputStream cannot be cast to java.lang.Integer
at org.xadisk.bridge.server.conversation.RemoteMethodInvocationHandler.processOptimizedRemoteReferences(RemoteMethodInvocationHandler.java:196)
at
org.xadisk.bridge.server.conversation.RemoteMethodInvocationHandler.handleRemoteInvocation(RemoteMethodInvocationHandler.java:142)
at
org.xadisk.bridge.server.conversation.RemoteMethodInvocationHandler.run(RemoteMethodInvocationHandler.java:65)

...
...
________________________________________________________________________________________________________________________

This should occur both on release 1.2 and 1.2.1, and not on 1.0 and 1.1.

In class RemoteMethodInvocationHandler, method processOptimizedRemoteReferences was being called with wrong parameter (instead of response, the targetObject was sent as input. This occurred as a side-effect of the method restructuring done in 1.2).

Once this issue is fixed, another issue is seen where, if end of input stream is reached, the read method will throw NegativeArraySizeException because of the value -1 being returned by the read method. This case needs to be handled.



 Comments   
Comment by Nitin Verma [ 15/May/12 ]

Checked-in the changes to trunk.





[XADISK-123] FileNotFoundException if file is stored on a NFS-Server different to the xaDiskHome (XADiskSystemDirectory) Created: 30/Aug/12  Updated: 27/Aug/13  Resolved: 11/Aug/13

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: stendres Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Websphere7,
Spring 3.1.2,
AIX,
XADisk 1.2.1 as JCA Resource Adapter



 Description   

File operations on files are failing with a FileNotFoundException during the commit, if the files are stored on a NFS-Server different to the filesystem of the xaDiskHome.
It works, if the direcory is mounted on the same NFS-Server where the xaDiskHome reside's.

Stacktrace:

Caused by: javax.transaction.SystemException: XAResource start association error:XAER_RMFAIL
        at com.ibm.tx.jta.RegisteredResources.startRes(RegisteredResources.java:1041)
        at com.ibm.ws.tx.jta.RegisteredResources.enlistResource(RegisteredResources.java:1099)
        at com.ibm.ws.tx.jta.TransactionImpl.enlistResource(TransactionImpl.java:2169)
        at com.ibm.ws.tx.jta.TranManagerSet.enlist(TranManagerSet.java:526)
        at com.ibm.ejs.j2c.XATransactionWrapper.enlist(XATransactionWrapper.java:715)
        ... 29 more
Caused by: javax.transaction.xa.XAException
        at org.xadisk.filesystem.utilities.MiscUtils.createXAExceptionWithCause(MiscUtils.java:16)
        at org.xadisk.connector.XAResourceImpl.start(XAResourceImpl.java:44)
        at com.ibm.ejs.j2c.XATransactionWrapper.start(XATransactionWrapper.java:1463)
        at com.ibm.ws.Transaction.JTA.JTAResourceBase.start(JTAResourceBase.java:148)
        at com.ibm.tx.jta.RegisteredResources.startRes(RegisteredResources.java:990)
        ... 33 more
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Su
ch a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug w
ith details
        at org.xadisk.filesystem.NativeXAFileSystem.checkIfCanContinue(NativeXAFileSystem.java:501)
        at org.xadisk.filesystem.NativeXAFileSystem.createSessionForXATransaction(NativeXAFileSystem.java:270)
        at org.xadisk.filesystem.NativeXAFileSystem.createSessionForXATransaction(NativeXAFileSystem.java:54)
        at org.xadisk.filesystem.NativeXASession.refreshSessionForNewXATransaction(NativeXASession.java:201)
        at org.xadisk.connector.XAResourceImpl.start(XAResourceImpl.java:41)
        ... 36 more
Caused by: java.io.FileNotFoundException: /usr/data/test/usrdat/MB999MTEST.79.7917179 (No such file or directory)
        at java.io.FileInputStream.<init>(FileInputStream.java:123)
        at org.xadisk.filesystem.utilities.FileIOUtility.renameTo(FileIOUtility.java:33)
        at org.xadisk.filesystem.DurableDiskSession.renameTo(DurableDiskSession.java:142)
        at org.xadisk.filesystem.NativeSession.commitMove(NativeSession.java:871)
        at org.xadisk.filesystem.NativeSession.commitFileMove(NativeSession.java:856)
        at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:714)
        at org.xadisk.connector.XAResourceImpl.commit(XAResourceImpl.java:131)
        at com.ibm.ejs.j2c.XATransactionWrapper.commit(XATransactionWrapper.java:485)
        at com.ibm.tx.jta.JTAXAResourceImpl.commit(JTAXAResourceImpl.java:296)
        at com.ibm.tx.jta.RegisteredResources.deliverOutcome(RegisteredResources.java:1574)
        at com.ibm.tx.jta.RegisteredResources.distributeOutcome(RegisteredResources.java:1926)
        at com.ibm.tx.jta.RegisteredResources.distributeCommit(RegisteredResources.java:2238)
        at com.ibm.tx.jta.TransactionImpl.internalCommit(TransactionImpl.java:1868)
        at com.ibm.tx.jta.TransactionImpl.internalCommit(TransactionImpl.java:1844)
        at com.ibm.tx.jta.TransactionImpl.coreStage2CommitProcessing(TransactionImpl.java:1091)
        at com.ibm.tx.jta.TransactionImpl.stage2CommitProcessing(TransactionImpl.java:1136)
        at com.ibm.tx.jta.TransactionImpl.processCommit(TransactionImpl.java:990)
        at com.ibm.tx.jta.TransactionImpl.commit(TransactionImpl.java:920)
        at com.ibm.ws.tx.jta.TranManagerImpl.commit(TranManagerImpl.java:436)
        at com.ibm.tx.jta.TranManagerSet.commit(TranManagerSet.java:161)
        at com.ibm.ws.tx.jta.UserTransactionImpl.commit(UserTransactionImpl.java:293)
        at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1010)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)


 Comments   
Comment by Nitin Verma [ 12/Sep/12 ]

Hello.

Can you please let me know the output of the following APIs (say, f1 is the file you want to move, and it is located on a different NFS than xadiskHome. this is the case which is failing, right?):

f1.renameTo(f2); //here f2 is some path in the file-system of xadiskHome and there should not be any file at this location.

Please let me know the return value of the above method. Then, please also check the return value of:

f1.exists();

Thanks,
Nitin

Comment by tengvig [ 03/Dec/12 ]

I got the same error in the same situation. The error seems to occur in org.xadisk.filesystem.utilities.FileIOUtility
When the source and destination files are on different file systems, the file is copied and deleted.

public static void renameTo(File src, File dest) throws IOException {
if (!src.renameTo(dest)) {
if (renamePossible(src, dest)) {
while (!src.renameTo(dest)) {

The while loop on line 27 above does not terminate causing the exception to occur on the second run when the file already is copied and the source file is deleted.
I changed it to:
while (src.exists() && !src.renameTo(dest)) {

which fixes the problem for me.

Comment by Nitin Verma [ 05/Jan/13 ]

Hello stendres (sorry I don't know your name),

Can you please confirm if the above change fixes the problem you mentioned.

Thanks,
Nitin

Comment by Nitin Verma [ 11/Aug/13 ]

Based on the comments from tengvig, I will fix and close this issue. Though, we could not yet get a confirmation from the original reporter of the issue if the proposed solution fixes the problem at his end.

Thanks tengvig for identifying the problem.

Regards,
Nitin

Comment by Nitin Verma [ 11/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-125] ConcurrentModificationException over txnBuffers in class GatheringDiskWriter Created: 26/Nov/12  Updated: 27/Aug/13  Resolved: 13/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Øyvind for identifying and bringing forth this issue at thread:

https://groups.google.com/forum/#!msg/xadisk/ydrDw_ArKYg/a2ugS8iJR0YJ

The method submitBuffer in class GatheringDiskWriter may modify txnBuffers ArrayList concurrently to the iteration over it inside processEvent method.



 Comments   
Comment by Nitin Verma [ 13/Aug/13 ]

Solution is to use ConcurrentLinkedQueue.

Comment by Nitin Verma [ 13/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-132] XADisk hangs on commit after directory delete and recreate Created: 24/Feb/13  Updated: 27/Aug/13  Resolved: 21/Aug/13

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64-bit, Java 1.7, Eclipse Juno, Open EJB



 Description   

When doing some performance tests on XADisk which delete and recreate directories I stumbled over this issue (note: the performance is completely satisactory):

XADisk totally hangs on the second commit of changes to a file in a directory which has been deleted and recreated (over XADisk).
And even worse this persists after stopping the thread and recovery (the recovery hangs). Only deleteing the XADisk work dir helps.

The issue can be easily reproduced with this JUnit test:

public class XaDirectoryTest extends TestCase {

public void testXaDirectory() throws Exception

{ IOTools.deleteAll(new File("target/xadisk-fail")); testXaDirectory(true); testXaDirectory(false); }

private void testXaDirectory(final boolean first) throws Exception {
String xadiskSystemDirectory = "target/xadisk-fail/system";
File sampleDataDir1 = new File("target/xadisk-fail/data");
XAFileSystem xafs = null;

try {
System.out.println("Booting XADisk...");
StandaloneFileSystemConfiguration configuration = new StandaloneFileSystemConfiguration(
xadiskSystemDirectory, "id-1");

xafs = XAFileSystemProxy.bootNativeXAFileSystem(configuration);
xafs.waitForBootup(-1);
System.out.println("XADisk is now available for use.");

Session session = xafs.createSessionForLocalTransaction();

try {
File testFile = new File(sampleDataDir1, "a.txt");
if (first)

{ assertFalse(session.fileExists(sampleDataDir1, false)); }

else

{ assertTrue(session.fileExists(sampleDataDir1, false)); assertTrue(session.fileExists(testFile, false)); System.out.println("deleting existing " + testFile + "."); session.deleteFile(testFile); // without this delete (and recreate below) everything works fine System.out.println("deleting existing " + sampleDataDir1 + "."); session.deleteFile(sampleDataDir1); }

System.out.println("creating " + sampleDataDir1 + ".");
session.createFile(sampleDataDir1, true);
System.out.println("creating " + testFile + ".");
session.createFile(testFile, false);

System.out.println("writing " + testFile + ".");
OutputStream xaos = new XAFileOutputStreamWrapper(session.createXAFileOutputStream(testFile, true));
xaos.write("Hellow World".getBytes());
xaos.close();

System.out.println("committing " + testFile + ".");
session.commit();

if (!first)

{ System.out.println("never gets here because it hangs."); }

} catch (XAApplicationException xaae)

{ session.rollback(); throw xaae; }

} finally {
if (xafs != null) {
try

{ xafs.shutdown(); }

catch (IOException ioe)

{ ioe.printStackTrace(); }

}
}
System.out.println("XADisk is down.");
}
}



 Comments   
Comment by Nitin Verma [ 02/Mar/13 ]

Hi Gunnar,

Many Thanks for identifying the issue and providing the test case. I could reproduce the issue and have done some diagnosis. It is a bug in xadisk.

(It does not occur for heavyWrite=false in above test-case.)

I will update here as soon as I have a fix.

Thanks,
Nitin

Comment by Nitin Verma [ 21/Aug/13 ]

Checked-in the changes to trunk.





[XADISK-117] XADisk has failed to load its native library Created: 30/May/12  Updated: 21/Aug/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Fireworks Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

System: Microsoft Windows Server 2003 R2
WebSphere Application Server V6.1.0.41



 Description   

After deploying Resource adapter and register JCA factory, during initialization in XADisk work directory was created log file with next content:

May 30, 2012 4:38:52 PM : XADisk has failed to load its native library required for directory-synchronization.
Now, it will override the configuration property "synchronizeDirectoryChanges" and set it to false; but please note that this would turn-off directory-synchronization i.e. directory modifications may not get synchronized to the disk at transaction commit.
If you have any questions or think this exception is not expected, please consider discussing in XADisk forums, or raising a bug with details.

May 30, 2012 4:38:52 PM : Successfully booted the XADisk instance.

And after trying to use connection from XADiskConnectionFactory next log was generated:

2012-05-30 17:03:32,578 ERROR [ru.acs.st.core.upload.ArchiveStreamProcessorImpl] - Ошибка при сохранении jar файла.
ru.acs.st.common.exception.STFileUploadException: Ошибка создания файла: First.jar в хранилище, отсутствует транзакция для сохранения данных.
at ru.acs.st.core.jarstorage.FileSystemJarStorageImpl.saveFileTransactional(FileSystemJarStorageImpl.java:191)
at ru.acs.st.core.jarstorage.FileSystemJarStorageImpl.saveJar(FileSystemJarStorageImpl.java:118)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.saveToDBAndFile(ArchiveStreamProcessorImpl.java:244)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.processJarByteMassive(ArchiveStreamProcessorImpl.java:191)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.processSingleFile(ArchiveStreamProcessorImpl.java:154)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.processArchiveStream(ArchiveStreamProcessorImpl.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:615)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.doInTransaction(TransactionInterceptor.java:132)
at org.springframework.transaction.jta.WebSphereUowTransactionManager$UOWActionAdapter.run(WebSphereUowTransactionManager.java:337)
at com.ibm.ws.uow.UOWManagerImpl.runUnderNewUOW(UOWManagerImpl.java:950)
at com.ibm.ws.uow.UOWManagerImpl.runUnderUOW(UOWManagerImpl.java:511)
at org.springframework.transaction.jta.WebSphereUowTransactionManager.execute(WebSphereUowTransactionManager.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:127)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy136.processArchiveStream(Unknown Source)
at ru.acs.st.core.service.UploadArchiveServlet.doPost(UploadArchiveServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1213)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:658)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:526)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:764)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1478)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:133)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:457)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:300)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:271)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:196)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:751)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:881)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1551)
Caused by:
org.xadisk.filesystem.exceptions.NoTransactionAssociatedException: The method that was called can only be called with a transaction associated, butthere is no such transaction present.
at org.xadisk.filesystem.NativeXASession.getSessionForCurrentWorkAssociation(NativeXASession.java:223)
at org.xadisk.connector.outbound.XADiskConnectionImpl.createFile(XADiskConnectionImpl.java:64)
at ru.acs.st.core.jarstorage.FileSystemJarStorageImpl.saveFileTransactional(FileSystemJarStorageImpl.java:162)
... 45 more

Exception NoTransaction, but we can see from trace, that it was call inside transaction.



 Comments   
Comment by Nitin Verma [ 04/Jun/12 ]

Hello.

Thanks for creating this bug.

The first issue you mentioned seems to be related to other known issues like this one - http://java.net/jira/browse/xadisk-100.

The second one requires further discussion and analysis before we call it a bug. I would recommend the google group at http://groups.google.com/group/xadisk for such discussions. It looks like xadisk is not being "enlisted" inside the global/xa transaction. It would be great if you can provide more details about the test-case/environment at that discussion thread.

Thanks,
Nitin

Comment by Fireworks [ 05/Jun/12 ]

Hellow, the first issue really repaired by disabling directory synchronization. It is not critical for my task.

Comment by Nitin Verma [ 21/Aug/13 ]

Just FYI. If developers using xadisk face such native library issues, they can do the following without any need to modify and re-build the xadisk java code:

-modify the native code if necessary (available in svn/trunk/native/*.c).

-compile the code according to the target platform and prepare the dynamic library.

-rename the dynamic library as "placeholder_xadisk.native" and place it inside the xadisk.jar as xadisk.jar/native/placeholder_xadisk.native.





[XADISK-143] Allow ThreadFactory parameter for StandaloneWorkManager Created: 19/Aug/13  Updated: 19/Aug/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The StandaloneWorkManager uses a ThreadPoolExecutor with flexible parameters in the constructor but without an option to set the ThreadFactory. It would be helpful if were some way to specify the ThreadFactory or at least give a parent ThreadGroup and/or thread name prefix. Currently the XADisk threads worker threads will show up as nondescript pool-x-thread-x threads at the top level.






[XADISK-136] XADisk hangs on recovery after failed directory creation with ":" Created: 01/Jul/13  Updated: 18/Aug/13  Resolved: 18/Aug/13

Status: Closed
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7, JDK 1.6



 Description   

After trying to create a directory containing a ":" and stopping the JVM the CrashRecoveryWorker hangs indefinitely in FileIOUtils.createDirectory because creating the directory fails each time (due to the ":") and makeSpaceForGC() is called in a loop causing the recovery to never return.

I do not like this code. I agree that it may fail to to hanging closes (which indeed may be healed by GC), but there can be other reasons (like an accidental ":") which make this a dangerous construction.

At least I propose to add an extra exit condition (loop counter, timing, ...).



 Comments   
Comment by gunnar_zarncke [ 01/Jul/13 ]

I cannot easily reproduce this bug yet because it resulted from stopping a VM. The hanging recovery is reproducible - but only as long as I do not delete the XA system directory of course.
I backed it up and can provide it as it if need be.

Comment by Nitin Verma [ 14/Jul/13 ]

Thanks Gunnar. I agree the infinite loop is not a good idea; I will try to design some approach to take care of such transactions which are not able to progress (eg not able to create a file/directory, delete them etc) and, due to this endless loop, keep hanging and blocks the entire xadisk instance during recovery.

When the commit process had already started during such failure loops, going back to rollback is not feasible. One way is to require user/admin's interference to let the transaction proceed (to commit). Another issue was reported by Julius at https://java.net/jira/browse/XADISK-76 which would require the same fix as this one.

Regards,
Nitin

Comment by Nitin Verma [ 18/Aug/13 ]

This is similar to and will have the same broader solution as:
https://java.net/jira/browse/XADISK-76.

So closing as duplicate.





[XADISK-139] XASystemBootFailureException when starting XADisk on a large data set Created: 23/Jul/13  Updated: 16/Aug/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux rs201377.rs.hosteurope.de 2.6.32-5-amd64 #1 SMP Mon Feb 25 00:26:11 UTC 2013 x86_64 GNU/Linux
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.3) (6b24-1.11.3-2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)



 Description   

The startup exception shown below occurred when starting XADisk on a large data set after the system has been stopped regularly (though I cannot guarantee that XADisk shut down in time. At least I can say that no crash or kill -9 occurred immediately before.

I could in principle provide the xadisk transaction directory, but it is quite large (31 GB; mostly transaction logs).

This is not yet a production system, so I can still recover by deleting the xa stores.

org.xadisk.filesystem.exceptions.XASystemBootFailureException: The XADisk instance has encoutered a critial issue and could not be booted. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details.
at org.xadisk.filesystem.NativeXAFileSystem.<init>(NativeXAFileSystem.java:187)
at org.xadisk.filesystem.NativeXAFileSystem.bootXAFileSystemStandAlone(NativeXAFileSystem.java:219)
at org.xadisk.bridge.proxies.interfaces.XAFileSystemProxy.bootNativeXAFileSystem(XAFileSystemProxy.java:47)
at de.konzentrik.lib.io.XaDisk.initLocal(XaDisk.java:43)
at de.konzentrik.app.preprocess.PreprocessorModule.doStartup(PreprocessorModule.java:79)
at de.zarncke.lib.sys.module.AbstractModule.startOrRestart(AbstractModule.java:126)
at de.zarncke.lib.sys.Headquarters.startOrRestart(Headquarters.java:450)
at de.zarncke.lib.sys.Headquarters.startOrRestart(Headquarters.java:452)
at de.zarncke.lib.sys.Headquarters.startOrRestart(Headquarters.java:446)
at de.konzentrik.app.preprocess.PreprocessorInstallation.doBootProtected(PreprocessorInstallation.java:202)
at de.konzentrik.app.preprocess.PreprocessorInstallation.boot(PreprocessorInstallation.java:158)
at de.konzentrik.app.preprocess.Start.run(Start.java:164)
at de.zarncke.lib.block.Running.execute(Running.java:5)
at de.zarncke.lib.block.Running.execute(Running.java:3)
at de.zarncke.lib.err.Warden.guard(Warden.java:313)
at de.konzentrik.app.StartBase.executeGuarded(StartBase.java:216)
at de.konzentrik.app.preprocess.Start$1.run(Start.java:190)
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:480)
at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:336)
at org.xadisk.filesystem.TransactionLogEntry.parseLogEntry(TransactionLogEntry.java:319)
at org.xadisk.filesystem.TransactionLogEntry.getNextTransactionLogEntry(TransactionLogEntry.java:521)
at org.xadisk.filesystem.workers.CrashRecoveryWorker.findInCompleteTransactions(CrashRecoveryWorker.java:139)
at org.xadisk.filesystem.workers.CrashRecoveryWorker.collectRecoveryData(CrashRecoveryWorker.java:99)
at org.xadisk.filesystem.NativeXAFileSystem.<init>(NativeXAFileSystem.java:181)



 Comments   
Comment by gunnar_zarncke [ 06/Aug/13 ]

I have had another such XASystemBootFailureException.
This time with a much smaller data set (8.7 MB in xa disk directory).
I copied that and might provide it for analysis.

Note: I get no notices when you, Nitin, comment on the issues.
I might like to get in direct contact. You can reach me at gunnar at konzentrik.de.

Comment by gunnar_zarncke [ 16/Aug/13 ]

I sent you two occurrences of crashes via mail. I am not sure if these have the same cause, but maybe you can make something out of it.





[XADISK-126] FileIOUtility's renameTo method will not exit for copy-delete workaround of rename. Created: 27/Nov/12  Updated: 16/Aug/13  Resolved: 16/Aug/13

Status: Closed
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Eugen for reporting it in the discussion thread at https://groups.google.com/forum/#!msg/xadisk/9ItZ5ddNDDg/HgqgFeWjs7kJ.

The renameTo method of class FileIOUtility uses a workaround of copy-delete when rename does not seem to work, but is missing a break statement in the end.



 Comments   
Comment by Nitin Verma [ 16/Aug/13 ]

This turned out to be a duplicate of https://java.net/jira/browse/XADISK-123, where we have resolved the issue using a break statement.





[XADISK-127] fileExists* methods can return false instead of throwing InsufficientPermissionOnFileException. Created: 28/Nov/12  Updated: 13/Aug/13  Resolved: 13/Aug/13

Status: Closed
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Steve for raising this via discussion thread at: http://groups.google.com/group/xadisk/browse_thread/thread/ca2c38602d5b14b0.

fileExists* methods can return false instead of throwing InsufficientPermissionOnFileException, to be more in line with the Java's File exists method.



 Comments   
Comment by Nitin Verma [ 13/Aug/13 ]

Duplicate of https://java.net/jira/browse/XADISK-120.





[XADISK-119] Wrong exception thrown by XADiskBasicIOOperations.getFileLength for missing file Created: 22/Jun/12  Updated: 10/Aug/13  Resolved: 10/Aug/13

Status: Closed
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: current

Type: Bug Priority: Major
Reporter: titmus Assignee: Nitin Verma
Resolution: Works as designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64bit
JDK 1.6.0_29



 Description   

Invoking method XADiskBasicIOOperations.getFileLength on non-existing file throws InsufficientPermissionOnFileException instead of FileNotExistsException exception.

Caused by: org.xadisk.filesystem.exceptions.InsufficientPermissionOnFileException: Permission of type [READ_FILE] is needed over the file/directory with path [C:\Users\titmus\AppData\Local\Temp\junit1370631598005369210\data\missing] for the i/o operation to succeed.
at org.xadisk.filesystem.NativeSession.checkPermission(NativeSession.java:1078)
at org.xadisk.filesystem.NativeSession.getFileLength(NativeSession.java:508)
at org.xadisk.filesystem.NativeSession.getFileLength(NativeSession.java:495)



 Comments   
Comment by Nitin Verma [ 10/Aug/13 ]

All operations in XADiskBasicIOOperations are preceded by a permission check over the file/parent-file(directory). The fileLength operation is preceded by a read permission check over the input file. We call (in VirtualViewDirectory) Java File's canRead/canWrite methods without first checking for existence (this saves us an extra call to Java File's exists method), and these methods from Java File do not report any existence related exception - they simply say canRead/canWrite or not. This is the reason we cannot distinguish permission or existence issue.

Leaving it as-designed for now, assuming a small compromise would be possible with the api users for the exception handling part. But anyone please feel free to comment back.





[XADISK-137] Transaction log file got corrupted Created: 10/Jul/13  Updated: 06/Aug/13

Status: Open
Project: xadisk
Component/s: connector, filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: 5ar Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Websphere Application Server v6.1.0.19, CentOS 6.0, Java 1.5.0



 Description   

During submit of attachment error happened which caused corrupt transaction log file.
After that XADisk could not recover.

Here is the stack trace:

___________________________________________________________________________________________________________________________________
[7/3/13 10:07:41:639 GST] 00002abd XATransaction E J2CA0027E: An exception occurred while invoking commit on an XA Resource Adapter from dataSource eis/XADisk/LocalConnectionFactory, within transaction ID

{XidImpl: formatId(57415344), gtrid_length(36), bqual_length(54), data(0000013fa324bd3700002)}

: javax.transaction.xa.XAException
at org.xadisk.filesystem.utilities.MiscUtils.createXAExceptionWithCause(MiscUtils.java:16)
at org.xadisk.connector.XAResourceImpl.commit(XAResourceImpl.java:135)
at com.ibm.ejs.j2c.XATransactionWrapper.commit(XATransactionWrapper.java:462)
at com.ibm.ws.Transaction.JTA.JTAXAResourceImpl.commit(JTAXAResourceImpl.java:272)
at com.ibm.ws.Transaction.JTA.RegisteredResources.deliverOutcome(RegisteredResources.java:2010)
at com.ibm.ws.Transaction.JTA.RegisteredResources.distributeOutcome(RegisteredResources.java:2518)
at com.ibm.ws.Transaction.JTA.RegisteredResources.distributeCommit(RegisteredResources.java:2839)
at com.ibm.ws.Transaction.JTA.TransactionImpl.internalCommit(TransactionImpl.java:2651)
at com.ibm.ws.Transaction.JTA.TransactionImpl.stage2CommitProcessing(TransactionImpl.java:1736)
.
.
.
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:489)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:740)
at org.xadisk.connector.XAResourceImpl.commit(XAResourceImpl.java:131)
... 130 more
Caused by: java.io.EOFException
at org.xadisk.filesystem.utilities.FileIOUtility.readFromChannel(FileIOUtility.java:173)
at org.xadisk.filesystem.TransactionLogEntry.getNextTransactionLogEntry(TransactionLogEntry.java:494)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:663)
... 131 more
___________________________________________________________________________________________________________________________________



 Comments   
Comment by Nitin Verma [ 14/Jul/13 ]

Hello. Are you able to reproduce this issue, and how frequently do you see it? As such EOFExceptions don't generally happen during commit, I will need more specific details regarding the conditions when this issue occurs.

Can you provide some details like a sample test, transaction-log files etc which can enable me diagnose/reproduce this issue locally.

Thanks,
Nitin

Comment by gunnar_zarncke [ 06/Aug/13 ]

I cannot reproduce this issue and I havn't seen this issue again since then.
But I did see a few more cases where the transaction log couldn't be recovered.
I will comment on this in XADISK-139.





[XADISK-134] spurious XADisk Error [Native Module] Directory C:\ does not exist needs better error message Created: 26/Feb/13  Updated: 26/Feb/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64-bit, Java 1.7, Eclipse Juno, Open EJB



 Description   

Sometimes I get an error message

"XADisk Error [Native Module] Directory C:\ does not exist."

Which is written to stdout by XADiskWindows.c (Line 35).

I couldn't further isolate the cause of this.

As an improvement I propose to to call GetLastError to get more details of the error and log this code too. Or even better to raise an exception with the found status code.

See also
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx






[XADISK-130] Implement createHardLink functionality. Created: 07/Feb/13  Updated: 07/Feb/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Gunnar for bringing this up at the discussion thread - http://groups.google.com/group/xadisk/browse_thread/thread/a131fe6935ce67b7#

It would be a good idea to be able to create hard links via xadisk.

(There is already a tracking bug for symbolic links: http://java.net/jira/browse/xadisk-50)






[XADISK-57] replace all System.out. with the log4j logger Created: 05/Feb/11  Updated: 14/May/12  Resolved: 01/Mar/11

Status: Resolved
Project: xadisk
Component/s: None
Affects Version/s: current
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: filippo22000 Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

please starting from now we have to use log4j for everything



 Comments   
Comment by Nitin Verma [ 01/Mar/11 ]

Issue http://java.net/jira/browse/XADISK-17 will take care of this too. Logging would be introduced and that would be via standard loggers instead of system.out.





[XADISK-79] Add logging in XAResourceImpl when errors are seen. Created: 31/Jul/11  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Stelios for bringing up this issue via the discussion thread at google group.

In XAResource implementations for JTA, the methods like "start", "end" can declare to throw an XAException (standard class from JTA) only. But, the XAException class does not provide a constructor which accepts the underlying "casue" for the exception.

Due to that, the underlying root cause gets lost and makes the debugging difficult, as realized in the discussion thread at:
http://groups.google.com/group/xadisk/browse_thread/thread/3f77f81b5afd9bee

We can workaround this problem by adding logging in our XAResource implementation whenever such XAExceptions are to be thrown; so that the underlying cause for that XAException gets recorded somewhere for diagnosis.

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 01/Oct/11 ]

Please see http://java.net/jira/browse/XADISK-87

A better workaround is to use Throwable.initCause(Throwable t).





[XADISK-108] fileExists and fileExistsAndIsDirectory acquire lock over the parent directory, instead of the file/dir itself. Created: 15/Apr/12  Updated: 14/May/12  Resolved: 15/Apr/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

After the improved isolation levels in 1.1, the existence checks for files and directories should acquire read lock over the file/dir itself, not over the parent directory.



 Comments   
Comment by Nitin Verma [ 15/Apr/12 ]

Checked-in the changes to trunk.





[XADISK-109] FileIOUtility methods can return earlier when an interrupt is detected. Created: 05/May/12  Updated: 14/May/12  Resolved: 05/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Improvement Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

FileIOUtility (package - org.xadisk.filesystem.utilities) class has a couple of IO methods like createFile, deleteFile etc. These methods may, in a very rare situation, enter into a loop (the loop was a workaround for a strange jvm issue). If an interrupt is seen during this loop, its handling is deferred until the completion of the IO method. Although in practice, it hasn't affected any real scenario and is not likely to, but it will be a better design to handle the interrupt as soon as it is detected, and come out of the method.



 Comments   
Comment by Nitin Verma [ 05/May/12 ]

Checked-in the changes to trunk.





[XADISK-110] Lock acquisition methods reset the interrupt flag before returning Created: 05/May/12  Updated: 14/May/12  Resolved: 05/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Methods acquireSharedLock/acquireExclusiveLock in NativeConcurrencyControl, which are used to acquire lock over file/directory objects, invoke method removeDependencyFromRDG before returning. This method further clears the current-thread's interrupt flag. Such clearing of interrupt flag was introduced for cases when the lock acquisition was interrupted as part of handling of deadlock or transaction timeout; the interrupt flag should not be cleared in other normal cases.



 Comments   
Comment by Nitin Verma [ 05/May/12 ]

Some relevant points:

1. Even if we could detect an interrupt and confirm the cause of the interrupt to be deadlock/timeout handling, there may still be some external system which interrupted in addition. But putting a check for deadlock/timeout cases, we can only reduce the chance of clearing an interrupt flag which was simultaneously set by someone else.

2. Not clearing the interrupt flag altogether is also not an option. When a deadlock/timeout detector interrupts the thread during lock acquisition method, the InterruptedException may or may not arrive (for example, if the thread has just completed its waiting for lock and preparing to return the lock). As InterruptedException may not always get thrown, we may still be left with the interrupt flag set. So, it would be required to clear it before returning. If an interrupt is generated inside xadisk system, it should be handled and (flag) cleared without affecting the thread's status after executing the xadisk methods.

Comment by Nitin Verma [ 05/May/12 ]

Checked-in the changes to trunk.





[XADISK-111] Deadlock/Timeout detectors may send interrupt to a thread even outside lock acquisition methods. Created: 05/May/12  Updated: 14/May/12  Resolved: 05/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Deadlock/Timeout detectors may send interrupt to a thread even if it is outside lock acquisition methods. If the lock-request method has completed, we do not need to interrupt the thread, because the 2 possible motives are anyway served:

1. deadlock is over now because the lock waiting is over (due to whatever reason),

2. for transaction timeout case, the timeout-detector rolls back the transaction; interrupt was kept just to bring the thread outside wait for lock.

The relevant code is (in class NativeConcurrencyControl), abridged:

public void interruptTransactionIfWaitingForResourceLock(TransactionInformation xid, byte cause) {
ResourceDependencyGraph.Node node;
node = //node representing xid if it is inside a lock acquisition method, null otherwise.
//time t1
if(node != null) {
synchronized (node.getInterruptFlagLock())

{ node.getThreadWaitingForLock().interrupt(); }

}
}

The problem with above is that if the transaction comes out of the lock acquisition method after time t1, we would still be sending interrupts to it.



 Comments   
Comment by Nitin Verma [ 05/May/12 ]

Checked-in the changes to trunk.





[XADISK-112] In class NativeSession, list of srcFilesCopied will not get cleared for some cases. Created: 13/May/12  Updated: 14/May/12  Resolved: 13/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.2.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

This is relevant only to the source having the fix for improvement#93 (http://java.net/jira/browse/xadisk-93).

In class NativeSession, method checkPointDuringModificationAgainstCopy is wrongly passing the value of another list srcFilesMoved instead of srcFilesCopied. Though it is not associated with incorrect behavior but might result in generation of some additional checkpoints later which are not required (as the clearing of list srcFilesCopied gets skipped).



 Comments   
Comment by Nitin Verma [ 13/May/12 ]

Checked-in the changes to trunk.





[XADISK-113] During crash recovery, redo of directory creation operation should not delete the existing directory tree. Created: 13/May/12  Updated: 14/May/12  Resolved: 13/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.2.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

This issue appears only after the fix for improvement#93 (http://java.net/jira/browse/xadisk-93).

There were some optimizations done for checkpoints as part of xadisk-93, where we submit a checkpoint only for cases when a file f involved in the operation could in some way impact the (post crash redo of) "earlier" operations dependent upon that file f. The case which got skipped is post crash redo of directory creation where we are deleting the entire directory tree, if it exists, and then recreate the directory. Because of the way the checkpoint optimization have been implemented, the directory's relation to its descendants is not taken into account; and this gets manifested as a bug when we redo directory creation after crash, resulting in "loss of information" required for redo of operations later in the transaction.



 Comments   
Comment by Nitin Verma [ 13/May/12 ]

Checked-in the changes to trunk.





[XADISK-114] The paratertype while invoking mep.onMessage remotely, for DirectoryModificationEvent, should be FileSystemStateChangeEvent. Created: 13/May/12  Updated: 14/May/12  Resolved: 13/May/12

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: current
Fix Version/s: 1.2.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As part of the improvement made in xadisk-102, a new event class DirectoryModificationEvent was introduced. For remote delivery of such events to a message endpoint, the lookup of method "onMessage" will have to use parametertype as FileSystemStateChangeEvent (super class of DirectoryModificationEvent). Java reflection depends upon the exact parameter type to be provided for method lookup, not their subclass type.



 Comments   
Comment by Nitin Verma [ 13/May/12 ]

Checked-in the changes to trunk.





[XADISK-107] It takes too long to delete a file from a folder that contains large amount of files Created: 12/Apr/12  Updated: 14/May/12  Resolved: 15/Apr/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Critical
Reporter: eugeng Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows XP, Windows 7



 Description   

It takes about 5 sec to delete a file from a folder that contains 10.000 files. From a quick static analysis of code, the VirtualViewDirectory seems
to be the reason where it tries to load the names of all files in the directory during deleteFile.



 Comments   
Comment by Nitin Verma [ 15/Apr/12 ]

Since 1.0 itself, the caching of a directory's children name was introduced to avoid going to the underlying file-system for a child's (file or dir) existence check. This optimization was good for many cases as the directories won't typically contain more than 100 files. But still there are cases when this can be lead to extreme slow performance (as Eugen encountered in this bug) if there are 1000s of other files in the directory which are not even relevant to the current transaction.

Though there is one more critical reason of why this caching of children names must be removed. Since 1.1, we have switched to more flexible isolation levels which could help more concurrent transactions to work with files under a common directory. Such caching should have been removed then and there because with the new isolation levels, this might actually create serious issues. For example, transaction T1 ends up creating a cache for directory D, transaction T2 then deletes a file f from D and commits, transaction T1 checks for existence of file f and since the cache was not invalidated, T1 will think f still exists.

Comment by Nitin Verma [ 15/Apr/12 ]

Tested and checked-in the changes to trunk.

Comment by Nitin Verma [ 15/Apr/12 ]

As updated in my comment above, this actually turned out to be a bug.





[XADISK-106] XADiskResourceAdapter violates JCA Spec Section 19.4.2 Created: 29/Mar/12  Updated: 14/May/12  Resolved: 15/Apr/12

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Critical
Reporter: Simon Dierl Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File xadisk-equals-hashcode.patch    

 Description   

From the JCA 1.6 specification:

19.4.2 Equality Based on Config Properties and Class

The candidate objects are implementations of ResourceAdapter, ManagedConnectionFactory, ConnectionRequestInfo, java.security.Principal, org.ietf.jgss.GSSCredential, GenericCredential, PasswordCredential, and Record types.
These objects must override the default equals and hashCode methods, and provide an equality behavior based on the configuration properties and class information. That is, any two objects can be equal only if their configuration properties match and they have the same class implementation.

JBoss 7.1.1 checks for fulfillment of the spec and refuses to deploy XADisk because it does not meet the spec (XADiskResourceAdapter does not implement #hashCode() and #equals(Qbject)).

I have attached a patch that adds auto-generated equals and hashCode methods to FileSystemConfiguration; that is enough to make it work on JBoss 7.1.1. An official release of the patched version would still be nice, we currently use a custom patched version.



 Comments   
Comment by Nitin Verma [ 03/Apr/12 ]

This will be fixed in the release 1.2.1.

Comment by Nitin Verma [ 15/Apr/12 ]

Hi Simon,

Thanks for identifying and bringing the issue in light.

Regarding the equals/hashCode methods - actually we don't need to compare all properties of the configuration object. As instance-id for xadisk instances in a JVM are unique, so we can simply depend upon this property.

Thanks & Regards,
Nitin

Comment by Nitin Verma [ 15/Apr/12 ]

Checked-in the changes to trunk.





[XADISK-105] Should document this - "Proxy objects for a remote xadisk instance cannot be shared by multiple threads." Created: 13/Mar/12  Updated: 14/May/12  Resolved: 15/Apr/12

Status: Resolved
Project: xadisk
Component/s: documentation
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Major
Reporter: dscheibe Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

JDK 1.6, Atomikos Transaction Manager, Windows 7 64-bit



 Description   

Occasionally when running a long on-going transaction on a remote XADisk instance the following exception is thrown:

org.xadisk.filesystem.exceptions.ConnectionException: java.io.StreamCorruptedException: invalid stream header: 05000000
at org.xadisk.bridge.proxies.facilitators.RemoteMethodInvoker.invokeRemoteMethod(RemoteMethodInvoker.java:116)
at org.xadisk.bridge.proxies.facilitators.RemoteObjectProxy.invokeRemoteMethod(RemoteObjectProxy.java:39)
at org.xadisk.bridge.proxies.impl.RemoteXAFileOutputStream.write(RemoteXAFileOutputStream.java:53)
at org.xadisk.additional.XAFileOutputStreamWrapper.write(XAFileOutputStreamWrapper.java:75)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1026)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:999)
at workflow.BeanTest.testMe(BeanTest.java:67)
at workflow.WorkflowTest$2.run(WorkflowTest.java:60)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.StreamCorruptedException: invalid stream header: 05000000
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:782)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:279)
at org.xadisk.bridge.proxies.facilitators.RemoteMethodInvoker.invokeRemoteMethod(RemoteMethodInvoker.java:102)
... 8 more
org.xadisk.filesystem.exceptions.ConnectionException: java.io.StreamCorruptedException: invalid type code: 01
at org.xadisk.bridge.proxies.facilitators.RemoteMethodInvoker.invokeRemoteMethod(RemoteMethodInvoker.java:116)
at org.xadisk.bridge.proxies.facilitators.RemoteObjectProxy.invokeRemoteMethod(RemoteObjectProxy.java:39)
at org.xadisk.bridge.proxies.impl.RemoteXAFileOutputStream.write(RemoteXAFileOutputStream.java:53)
at org.xadisk.additional.XAFileOutputStreamWrapper.write(XAFileOutputStreamWrapper.java:75)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1026)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:999)
at workflow.BeanTest.testMe(BeanTest.java:67)
at workflow.WorkflowTest$1.run(WorkflowTest.java:43)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.StreamCorruptedException: invalid type code: 01
at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2463)
at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2498)
at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2570)
at java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:2711)
at java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:883)
at org.xadisk.bridge.proxies.facilitators.RemoteMethodInvoker.invokeRemoteMethod(RemoteMethodInvoker.java:103)
... 8 more
Exception in thread "pool-1-thread-5" org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:488)
at org.xadisk.filesystem.workers.observers.CriticalWorkersListener.workCompleted(CriticalWorkersListener.java:31)
at org.xadisk.filesystem.standalone.StandaloneWorkManager$WorkRunnable.run(StandaloneWorkManager.java:81)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: javax.resource.spi.work.WorkException: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
... 4 more
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:488)
at org.xadisk.filesystem.workers.TransactionTimeoutDetector.doWorkOnce(TransactionTimeoutDetector.java:42)
at org.xadisk.filesystem.workers.TimedWorker.run(TimedWorker.java:42)
at org.xadisk.filesystem.standalone.StandaloneWorkManager$WorkRunnable.run(StandaloneWorkManager.java:75)
... 3 more
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:488)
at org.xadisk.filesystem.NativeSession.rollback(NativeSession.java:934)
at org.xadisk.filesystem.NativeSession.rollbackPrematurely(NativeSession.java:121)
at org.xadisk.filesystem.NativeSession.rollbackAsynchronously(NativeSession.java:112)
at org.xadisk.filesystem.workers.TransactionTimeoutDetector.doWorkOnce(TransactionTimeoutDetector.java:38)
... 5 more
Caused by: java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:289)
at org.xadisk.filesystem.workers.GatheringDiskWriter.ensureLogFileCapacity(GatheringDiskWriter.java:340)
at org.xadisk.filesystem.workers.GatheringDiskWriter.forceWrite(GatheringDiskWriter.java:317)
at org.xadisk.filesystem.workers.GatheringDiskWriter.transactionCompletes(GatheringDiskWriter.java:232)
at org.xadisk.filesystem.NativeSession.rollback(NativeSession.java:927)
... 8 more

The initialization of the server XADisk was done as presented below:

String XADiskSystemDirectory = "c:/temp/xadisk";
StandaloneFileSystemConfiguration configuration = new StandaloneFileSystemConfiguration(XADiskSystemDirectory, "instance-1");
configuration.setEnableRemoteInvocations(true);
configuration.setServerPort(9999);
configuration.setTransactionTimeout(720);
_xaFileSystem = XAFileSystemProxy.bootNativeXAFileSystem(configuration);
_xaFileSystem.waitForBootup(-1);
......

The initialization of the client XADisk was done as presented below:

_xaFileSystem = XAFileSystemProxy.getRemoteXAFileSystemReference(_serverAddress, _serverPort);
_xaSession = _xaFileSystem.createSessionForXATransaction();
......



 Comments   
Comment by dscheibe [ 13/Mar/12 ]

Something i forgot:

There are two threads running on a single XAFileSystem instance, but each thread has gotten it's own XASession, maybe i'm missing some threading issue i should be aware of?

Comment by Nitin Verma [ 13/Mar/12 ]

Actually, in case of remote xadisk, the XAFileSystem reference (pointing to the remote xadisk instance) and subsequently derived objects like Session cannot be used by multiple threads concurrently. Such remote XAFileSystem objects keep their own communication channel until they are disconnected.

This needs to be clarified in the documentation. Many Thanks to Daniel for identifying it at discussion thread: http://groups.google.com/group/xadisk/browse_thread/thread/8cad076a1eaab99.

Comment by Nitin Verma [ 13/Mar/12 ]

Changing the Component to "documentation".

Comment by Nitin Verma [ 15/Apr/12 ]

Checked-in the changes (relevant JavaDoc comment) to trunk.





[XADISK-103] Maintain lastModified date when performing a move operation Created: 12/Mar/12  Updated: 14/May/12  Resolved: 17/Apr/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2.1

Type: Improvement Priority: Major
Reporter: sgerogia Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows, Unix



 Description   

When doing a manual move of a file, its lastModified date is preserved.
In Windows this is shown in the File Explorer, on Unix systems this can be viewed with the "stat" command.
It would be good if XADisk followed the same semantics and preserved the lastModified timestamp in move operations.



 Comments   
Comment by Nitin Verma [ 17/Apr/12 ]

Hi Stelios,

Thanks for identifying and logging this one.

I did some testing of move operations, where I called xadisk rename operation with "src" and "dest":
1. within the same file-system
2. "src" being on a local file-system, "dest" being on a network shared file-system.

In both of these cases, the lastModified time was actually retained. I think the case where you observed the lastModified time not being retained is when the move is across different (incompatible) file-systems and these are the cases when the Java File's renameTo method fails. I had seen such examples of renameTo failing in past, so had introduced a workaround in XADisk. In such cases, XADisk does what unix "mv" does, i.e. creating the "dest" file, copying the content, and deleting "src" file, and hence the lastModified time is lost. I am planning to fix this by explicitly setting "dest" lastModified time using Java File's setter method.

Thanks,
Nitin

Comment by Nitin Verma [ 17/Apr/12 ]

Checked-in the changes to trunk.





[XADISK-102] File system events for directory modifications should also contain the name of the child object. Created: 01/Mar/12  Updated: 14/May/12  Resolved: 18/Apr/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Improvement Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Nick for bringing this up via thread: http://groups.google.com/group/xadisk/browse_thread/thread/84403c5d7773c4b1

File system events for directory modifications should contain the name of the child object which was added/deleted in the directory.

To implement this, we can do either of these:
1. use the existing field "file" of "FileSystemStateChangeEvent" to hold the child object's path when the event is a directory modification event. The path of the directory (which comes through "file" variable today) for which this event was received can still be calculated from the child's path (getParent).

2. keep the "file" usage as it is today, and add an extra field to hold "more" information about the event. E.g. in directory modification case, it can contain the name/path of the child object.

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 03/Apr/12 ]

This will be fixed in the release 1.2.1.

Comment by Nitin Verma [ 18/Apr/12 ]

Checked-in the changes to trunk.





[XADISK-99] InsufficientPermissionOnFileException is thrown even for cases when an ancestor directory does not exist. Created: 16/Feb/12  Updated: 14/May/12  Resolved: 12/May/12

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Kane and Kannan for helping identify this issue by the following discussion threads, respectively: http://groups.google.com/group/xadisk/browse_thread/thread/4814f98275b526ec
http://groups.google.com/group/xadisk/browse_thread/thread/b1bcd0194d15814

For operations which require a directory (and its ancestor directories) to exist, if any of these directories do not exist, an exception of type "InsufficientPermissionOnFileException" is thrown mentioning that a READ/WRITE permission is not available on that directory.

It will be better to use a more relevant exception type, say FileNotExistsException.



 Comments   
Comment by Nitin Verma [ 12/May/12 ]

Checked-in the changes to trunk.





[XADISK-97] DurableDiskSession ignores return value of "forceDirectoryHierarchy" during method "setupDirectorySynchronization". Created: 26/Nov/11  Updated: 14/May/12  Resolved: 26/Nov/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Improvement Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Though it does not have any significant impact, but it will be better to consider the return value of "forceDirectoryHierarchy" and propagate that as a success or failure case to the caller. The "forceDirectoryHierarchy" currently is used to force directory creations done in order to have "xaDiskHome" in place.



 Comments   
Comment by Nitin Verma [ 26/Nov/11 ]

Checked-in the changes to trunk.





[XADISK-96] UNC Paths (\\host\shared-folder\) giving InsufficientPermissionOnFileException for some cases. Created: 24/Nov/11  Updated: 14/May/12  Resolved: 26/Nov/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Kane for bringing this issue to attention, at the discussion thread: http://groups.google.com/group/xadisk/browse_thread/thread/4814f98275b526ec

File APIs show different behavior with UNC paths, e.g.

new File("\\\\host1").isDirectory() returns false, while
new File("C:
").isDirectory() returns true.

Similarly,
new File("\\\\host1").listFiles() returns null.

Though, deeper paths like "\\\\host1
folder1" are working fine with these File APIs.

As the result, some XADisk operations may fail when they encounter a path with some specific structure. Specifically, those paths which lead XADisk to inspect the ancestor path "
host". For example, this error would happen during fileExistsAndIsDirectory() call for input path as "\\host\a" or "\\host\a\b".

Such failures occur during the method TransactionVirtualView:getVirtualViewDirectory which gets misguided by the outcome of the File.isDirectory() method for the path "
host", and throws FileNotExistsException. The outer methods then generalize this exception to InsufficientPermissionOnFileException.

Fix for this would be to treat these UNC paths in a specific way where one would view each shared-folder as a root of its descendants file hierarchy (instead of going one level up to "//host").



 Comments   
Comment by Nitin Verma [ 26/Nov/11 ]

To make a note, there is one more problem with UNC and Java combination. This is in the File's canRead()/canWrite() APIs. These APIs are returning true even if the particular file/directory does not expose that permission. This results in XDisk failing later during commit time. So, the user has to make sure to assign all needed permissions beforehand, else the XADisk instance would consider it a critical situation (permission checks passing, but actual operation failing) and would make itself unavailable.

Thanks,
Nitin

Comment by Nitin Verma [ 26/Nov/11 ]

Workaround to the insufficient permission exception reported in this bug : Mount the shared folder as a drive and refer to the shared files/directories using this drive's paths.

Comment by Nitin Verma [ 26/Nov/11 ]

Checked-in the changes to trunk.





[XADISK-94] Relative path for XADisk System Directory (xaDiskHome) may generate NullPointerException Created: 19/Nov/11  Updated: 14/May/12  Resolved: 26/Nov/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Kane for identifying this.

If a relative path is provided for xaDiskHome property (XADisk System Directory) in the configuration, and if this directory and none of its ancestor directories exist, then a NullPointerException is thrown:

_______________________________________________________________________________________________________________

org.xadisk.filesystem.exceptions.XASystemBootFailureException: .........
Cause By: java.lang.NullPointerException
at org.xadisk.filesystem.utilities.FileIOUtility.createDirectoriesIfRequired(FileIOUtility.java:138)
at org.xadisk.filesystem.utilities.FileIOUtility.createDirectoriesIfRequired(FileIOUtility.java:141)
at org.xadisk.filesystem.NativeXAFileSystem.<init>(NativeXAFileSystem.java:101)
... 4 more

_______________________________________________________________________________________________________________

Workaround is to use absolute path for xaDiskHome property, which is anyway a recommended approach for such a global property.

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 26/Nov/11 ]

Checked-in the changes to trunk.





[XADISK-93] Checkpointing done during move and copy operations can be optimized. Created: 16/Oct/11  Updated: 14/May/12  Resolved: 01/Dec/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: 1.2.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Olaf for helping identify this optimization, which can be of great help for transactions doing a lot of copy/move operations over files.

During copy/move, a checkpoint is submitted to the transaction-logs. This checkpoint contains the corresponding log-position for that move/copy operations in the transaction. When there is crash in the middle of committing, the recovery process starts committing this transaction only after the last checkpoint for the transaction. Such check-pointing was required to cater for subsequent modifications to the copy/move's "src" file.

As these checkpoints are forced to the logs, they incur a significant cost in terms of time. A possible optimization (which was deferred during initial code base) is to keep deferring this check-pointing until "src" actually gets modified, if at all. This would help in either reducing or avoiding unnecessary check-pointing, saving on the time cost.



 Comments   
Comment by Nitin Verma [ 16/Oct/11 ]

Checked-in the changes to trunk.

Comment by Nitin Verma [ 01/Dec/11 ]

The "Resolution" should be "Fixed", not "Cannot Reproduce" (which was set by mistake). Reopening for a moment to correct the "Resolution".





[XADISK-80] Document the concurrency control (locking) details for concurrent transactions in XADisk. Created: 31/Jul/11  Updated: 14/May/12  Resolved: 12/May/12

Status: Resolved
Project: xadisk
Component/s: documentation
Affects Version/s: 1.1
Fix Version/s: 1.2.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Tomas for raising a related question at the google group: http://groups.google.com/group/xadisk/browse_thread/thread/db5fbcbebdeeda46.

It would be nice to document the concurrency control (locking) details for concurrent transactions in XADisk. The XADisk User Guide would be the best place.



 Comments   
Comment by Nitin Verma [ 12/May/12 ]

Checked-in the changes to trunk.





[XADISK-92] TransactionTimeoutDetector can use a more precise calculation to decide if a session has timed out. Created: 02/Oct/11  Updated: 14/May/12  Resolved: 02/Oct/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The current calculation truncates the times to seconds beforehand, leading to loss of precision upto 1 second:

_____________________________________________________________
int timeoutValue = session.getTransactionTimeout();
int birthTime = (int) (session.getTimeOfEntryToTransaction() / 1000);
int timeNow = (int) (System.currentTimeMillis() / 1000);
_____________________________________________________________

This can be made better by:

_____________________________________________________________
long timeoutValue = session.getTransactionTimeout() * 1000;
long birthTime = session.getTimeOfEntryToTransaction();
long timeNow = System.currentTimeMillis();
_____________________________________________________________



 Comments   
Comment by Nitin Verma [ 02/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-91] The optimization to use collection.toArray(collection.size()), suggested by FindBugs, can be reverted in favor of concurrency safety. Created: 01/Oct/11  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: misc
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In release 1.0, there were many places where we were tempted to use collection.toArray(collection.size()). (This optimization was suggested by FindBugs).

Though this optimization is not likely to add much value, it was more likely to cause concurrency issues when the corresponding collection is likely to be accessed concurrently. As the savings are not really worth, we can revert to collection.toArray(0).



 Comments   
Comment by Nitin Verma [ 01/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-90] Locking algorithm can be improved. Created: 01/Oct/11  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The current implementation of locking can be improved in terms of performance. Here are some potential places:

1. In addition to normal locks over file objects, there is a special locking (called "pinning") over directories when they are to be moved. Though the current algorithm works correctly, but it has to pay some cost (even for operations other than directory move) of synchronization over a common global object called "directoriesPinnedForRename". Such kind of synchronizations can be broken into smaller level synchronization over smaller level objects like a directory tree which is actually being moved.

2. There is a single map containing all file locks and another single map containing all directory pins. These locks/pins can rather be kept into tree structures according to their paths so that objects are distributed into their local spaces instead of burdening a global common structure (e.g. cost of rehashing the whole big map, concurrency even if an efficient concurrentHashMap is used).

3. The new algorithm can take advantage of the fact that directory move operations are very infrequent compared to all other operations combined. So, the algorithm for a directory pin can potentially take some work off the algorithm for a routine file lock. This would enhance the overall performance.



 Comments   
Comment by Nitin Verma [ 01/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-89] In inbound eventing of XADisk, the numbering scheme of deadletter files depends on number of existing deadletter files. Created: 24/Sep/11  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When an XADisk instance is booted, it create a new deadletter file and the numbering scheme used to name this file depends upon the number of existing deadletter files. So, new deadletter file name = "letter_" + #files + 1. It is a bug because the chosen file-name may already exist (say someone processed a few deadletters and deleted them) and there is no exception handling for this case.

The new approach for naming this deadletter (during booting) would be to find the maximum number in the existing deadletters' names, and then increment that by one.



 Comments   
Comment by Nitin Verma [ 01/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-88] In inbound eventing of XADisk, the older deadLetters with 0 sizes can be deleted on restarting an XADisk instance. Created: 24/Sep/11  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Every time an XADisk instance is booted, it creates a new empty deadletter (in <xadisk-system-dir>/deadletter directory) file to store undelivered file events during its operations. At the same time, it does not delete existing deadletter files which were created during last boot but are empty. This leads to unnecessary increase in the number of files in the deadletter directory; increment of 1 per booting.



 Comments   
Comment by Nitin Verma [ 01/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-87] XAException thrown by methods in XAResource objects of XADisk should contain the underlying cause. Created: 24/Sep/11  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Stelios for bringing this issue to light on discussion thread at http://groups.google.com/group/xadisk/browse_thread/thread/3f77f81b5afd9bee

Various methods in XAResourceImpl (in org.xadisk.connector) and LocalEventProcessingXAResource (in org.xadisk.connector.inbound) throw XAException according to the standard XAResource interface specification. But, when throwing this XAException, they do not set the underlying cause of the exception.

Although the constructors of XAException do not accept "Throwable cause" as an argument, we can use initCause(Throwable cause) method of the super class "Throwable" to attach the underlying cause.

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 01/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-86] If LockTimeOut is set to 0, the request to acquire a lock may endup in a loop until the lock is acquired. Created: 29/Aug/11  Updated: 14/May/12  Resolved: 29/Aug/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

If the configuration property "LockTimeOut" is set to 0, during an operation which requires locking a file/dir, the thread will keep rotating in a loop until the lock by the existing holder is released. It is expected that the wait should happen without consuming unnecessary cpu cycles.

The underlying cause is due to our replacing of the Object.wait(time) method with the recommended Condition.await(time) method. The former method waits indefinitely if time argument is 0, but (we missed this unfortunately) the later does not.



 Comments   
Comment by Nitin Verma [ 29/Aug/11 ]

Checked-in the changes to trunk.





[XADISK-85] Java 's FileChannel.transferTo breaks for large number of bytes to be copied, resulting in failure in XADisk's copyFile API. Created: 24/Aug/11  Updated: 14/May/12  Resolved: 28/Aug/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hello,

Many Thanks to Olaf Eschenbruch for identifying and bringing this issue in front.

XADisk's copyFile API (see XADiskBasicIOOperation) relies on Java NIO's FileChannel.transferTo method. For large files (around 1.5GB/above), the transferTo method gives error both in Java 5 and Java 6.

In Java 5, it reports:
________________________________________________________________________________________________

Exception in thread "main" java.io.IOException: Not enough storage is available to process this command
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:742)
at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:448)
at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:521)
________________________________________________________________________________________________

In Java 6, it reports:

________________________________________________________________________________________________

Exception in thread "main" java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758)
at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:447)
at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:520)
...
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:755)
... 4 more
________________________________________________________________________________________________

XADisk should implement a workaround for this bug in Java. One possible alternative is to replace:

num += srcChannel.transferTo(num, contentLength - num, destChannel);

with

num += srcChannel.transferTo(num, SOME SMALL CONSTANT (eg 10000), destChannel);

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 28/Aug/11 ]

Checked-in the changes to trunk.





[XADISK-84] For standalone XA usage of XADisk, there is no defined API to retireve lists of in-doubt transactions after crash. Created: 11/Aug/11  Updated: 14/May/12  Resolved: 28/Aug/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

For standalone XA usage of XADisk (e.g with Atomikos), an XASession can be obtained from XAFileSystem reference by this method:

_________________________________________________________________________________________________________

XAFileSystem xafs;
...
xafs.createSessionForXATransaction();
_________________________________________________________________________________________________________

But this method would return the XASession only if the XADisk instance has completed its recovery, which also includes completion of in-doubt transactions which were ongoing before the crash/shutdown.

As of 1.1, the only "documented" method to retrieve XAResource in standalone XA usage is from the XASession object. An XAResource object is an XA standard object which is used by a Transaction Manager (TM) for post crash recovery, so that the TM can complete the in-doubt transactions. Since XASession object cannot be retrieved until the recovery is complete (there is a checkIfContinue() check), it means there is no way to obtain the XAResource object before the recovery completes. But, recovery is fully dependent on completion of all in-doubt transactions. This becomes a deadlock situation.

As a fix, there should be an API to retrieve XAResource object even before recovery completes so that TM can get informed what transactions are in-doubt and can then complete them.

A workaround for 1.1 is to use the following method for the purpose of obtaining XAResource object (post crash) so that it can be passed on to the TM:

_________________________________________________________________________________________________________

XAResource xarRecovery = new org.xadisk.filesystem.NativeXASession(xafs, "instance-id").getXAResource();
_________________________________________________________________________________________________________



 Comments   
Comment by Nitin Verma [ 28/Aug/11 ]

Checked-in the changes to trunk.





[XADISK-83] In JCA environment, the ra.xml configuration property named "lockTimeOut" does not become effective. Created: 10/Aug/11  Updated: 14/May/12  Resolved: 29/Aug/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Jaroslav Holubec for identifying and bringing up this issue.

In JCA environments, the ra.xml configuration property named "lockTimeOut" is supposed to be effective for all created
XADiskConnections, unless this property is overridden by the setFileLockWaitTimeout method of XADiskConnection. But this is not happening: if the method setFileLockWaitTimeout is not used to set this property, the effective value is always 100 (which is wrongly coming from the member named "fileLockWaitTimeout" in NativeXASession class).



 Comments   
Comment by Nitin Verma [ 29/Aug/11 ]

Checked-in the changes to trunk.





[XADISK-78] NullPointerException in TransactionTimeoutDetector under high load (high number of transactions). Created: 31/Jul/11  Updated: 14/May/12  Resolved: 01/Aug/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: 1.2

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0 minutes
Time Spent: Not Specified
Original Estimate: 0 minutes


 Description   

Thanks to Stelios for bringing up this issue.

For details, please see the original discussion at the google-group: http://groups.google.com/group/xadisk/browse_thread/thread/3f77f81b5afd9bee.

Here is the stack trace:

_____________________________________________________________________________________________________________________________________________

org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:719)
at org.xadisk.filesystem.workers.TransactionTimeoutDetector.doWorkOnce(TransactionTimeoutDetector.java:40)
at org.xadisk.filesystem.workers.TimedWorker.run(TimedWorker.java:42)
at weblogic.connector.security.layer.WorkImpl.runIt(WorkImpl.java:108)
at weblogic.connector.security.layer.WorkImpl.run(WorkImpl.java:44)
Truncated. see log file for complete stacktrace

Caused By: java.lang.NullPointerException
at org.xadisk.filesystem.workers.TransactionTimeoutDetector.doWorkOnce(TransactionTimeoutDetector.java:31)
at org.xadisk.filesystem.workers.TimedWorker.run(TimedWorker.java:42)
at weblogic.connector.security.layer.WorkImpl.runIt(WorkImpl.java:108)
at weblogic.connector.security.layer.WorkImpl.run(WorkImpl.java:44)
at weblogic.connector.work.WorkRequest.run(WorkRequest.java:95)

_____________________________________________________________________________________________________________________________________________

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 31/Jul/11 ]

The underlying cause has been identified. I will check-in the fix soon, which is a single line change:

In the class org.xadisk.filesystem.NativeXAFileSystem, method
getAllSessions(), change the following code from

"return sessions.toArray(new NativeSession[sessions.size()]);"

to

"return sessions.toArray(new NativeSession[0]);"

Thanks,
Nitin

Comment by Nitin Verma [ 01/Aug/11 ]

Checked-in the changes to /trunk. Thanks to Stelio for verifying the resolution.





[XADISK-3] XADisk Cluster - Multiple xadisk instances running across machines/JVMs and working over same sets of files Created: 06/Sep/10  Updated: 14/May/12  Resolved: 01/Oct/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 0.9 (Beta)
Fix Version/s: 1.2

Type: New Feature Priority: Major
Reporter: fecici Assignee: Nitin Verma
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 3

 Description   

Many Thanks to Tomas for suggesting this feature.

I have rephrased the initial description to make it more detailed.

As per the current implementation, more than one xadisk instances (they may be running on the same machine/JVM or across multiple machines/JVMs) cannot perform operations over the same set of files. The application has to take the responsibility of not performing operations over overlapping set of files through different xadisk instances.

This becomes a limitation when the application cannot be designed to take care of the above constraint, because then all application instances end up using the same common xadisk instance (whether on the same machine/JVM or a remote one). This further leads to lack of scalability in the setup.

XADisk should support clustering where multiple xadisk instances can work over same sets of files. This would allow applications distributed over machines/JVMs to work over same sets of files without depending upon a single xadisk instance.



 Comments   
Comment by Nitin Verma [ 16/Nov/10 ]

Changing the subcomponent and version.

Comment by Nitin Verma [ 15/Feb/11 ]

Implementation:
1) My quick thoughts say that if we can ONLY manage concurrency (that XADisk does today among sessions, with in-memory lock objects) among sessions on multiple XADisk instances, we are done. I am thinking on what would be the best approach to keep the in-memory locking related "state" over cluster.
2) ...
3) ...

Notes:
1) With such a feature in place, application administrators would need to be careful that ANY of the XADisk instances over the cluster is invoked by applications only after ALL instances are up. Doing otherwise runs the risk of an alive XADisk instance X1 being invoked and grabbing the locks over files/dirs which were locked (and are still locked, virtually) by another XADisk instance X2 which is not yet alive. I would clarify this more in the then-documentation.

Comment by Nitin Verma [ 01/Oct/11 ]

The approach of centralized lock server has been chosen, where a master node in the cluster is responsible for concurrency control (maintaining locks) for all other nodes in the cluster.

Comment by Nitin Verma [ 01/Oct/11 ]

Checked-in the changes to trunk.





[XADISK-75] Add a placeholder for picking user created native lib. Created: 14/May/11  Updated: 14/May/12  Resolved: 14/May/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In case none of the shipped native libs work on a particular platform, we can allow users to compile/create their own native lib for that platform from the source available on subversion.

XADisk would log a warning if it can't use any of the shipped native libs on a platform. Upon seeing this warning, and based on the requirements, a user may decide to:
1) compile the source code (as available on subversion) to generate a native lib with name "placeholder_xadisk.native" and replace it in the XADisk.jar as "XADisk.jar/native/placeholder_xadisk.native". Or,
2) can report the issue to us, and involve us to generate the native lib.

Purpose of this JIRA issue is to implement the idea in (1) above.



 Comments   
Comment by Nitin Verma [ 14/May/11 ]

Checked-in the changes.





[XADISK-73] XADisk can do some basic validation of the two mandatory configuration properties. Created: 07/May/11  Updated: 14/May/12  Resolved: 07/May/11

Status: Resolved
Project: xadisk
Component/s: misc
Affects Version/s: current
Fix Version/s: 1.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Although, it is well documented which mandatory configuration properties must be set, but as an improvement, we can
also do basic validation of the two (as of now) configuration properties (xaDiskHome and instanceId).






[XADISK-72] Multiple classLoaders may load the xadisk native lib, resulting in error. Created: 27/Apr/11  Updated: 14/May/12  Resolved: 27/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The current implementation in DurableDiskSession to load native lib would work fine when there is a single classloader for all xadisk instances. It would fail for different classloaders because a native lib cannot be loaded by more than one classloader.

For finding the right native lib to install, we rely on exceptions received from "System.load". Due to the above reason, it may be misleading. So, we can rather use a verification method by simply invoking the native method with dummy data.






[XADISK-71] XADisk can use the XADiskHome directory to keep its native lib. Created: 27/Apr/11  Updated: 14/May/12  Resolved: 27/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In the current implementation, we use temp directory of the system to copy the native lib of XADisk. As loaded libs won't get deleted (with deleteOnExit or explicitly), we end accumulating a lot of native lib copies in the temp directory. Better would be to use the XADiskHome directory itself to store the native lib.






[XADISK-70] It may be better if Directory Synchronization disables itself automatically. Created: 27/Apr/11  Updated: 14/May/12  Resolved: 27/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In cases where directory-synchronization is configured to be enabled, but XADisk faces issues applying it,
it is raising an exception. This would require users to re-configure it (disable the directory synchronization) and boot XADisk.

It may be better if we log a warning and proceed with directory-synchronization disabled. Warning alerts the users and at the same time allows those users who are happy without directory-synchronization to continue without any reconfiguration.






[XADISK-69] DurableDiskSession should not use the static variable "synchronizeDirectoryChanges". Created: 27/Apr/11  Updated: 14/May/12  Resolved: 27/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

DurableDiskSession should not use a static variable (synchronizeDirectoryChanges) as it would result in being shared among all xadisk instances in the same jvm. Rather use an object member variable.






[XADISK-67] Allow XADisk to plug-into JTA transaction managers (eg Atomikos) even in non-JCA environments. Created: 11/Apr/11  Updated: 14/May/12  Resolved: 11/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: New Feature Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As of XADisk 1.0, applications must be in JCA environment to take advantage of the XA capabilities of XADisk.

It would be a very useful feature to allow XADisk to plug into JTA Transaction Managers (e.g. Atomikos) even without
using JCA. As of now, a safe, but unclean hack exists for the same ("unclean" because it uses JCA APIs even in non-JCA applications) :
__________________________________________________________________________________________________________________________

XADiskManagedConnection managedConnection = new XADiskManagedConnection(xaFileSystem, "instanceId-1");
XAResource xar = managedConnection .getXAResource();//this XAResource can be enlisted into an XA/JTA transaction.

__________________________________________________________________________________________________________________________



 Comments   
Comment by Nitin Verma [ 11/Apr/11 ]

Checked-in the changes.





[XADISK-65] isSameRM method in XAResourceImpl will return wrong results for some cases. Created: 05/Apr/11  Updated: 14/May/12  Resolved: 05/Apr/11

Status: Resolved
Project: xadisk
Component/s: connector
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The current implementation of isSameRM method in XAResourceImpl will return wrong
results (will return true, but expected is false) for the following two cases.
Consider xar1.isSameRM(xar2) being called :

1) If xar1 and xar2 both are native xadisk instance (in the same JVM), but yet are different ones.
2) If xar1 is a native xadisk instance, but xar2 is a remote one.



 Comments   
Comment by Nitin Verma [ 05/Apr/11 ]

Checked-in the changes.





[XADISK-64] Better if XADisk loads/installs its native library automatically without even a minute manual effort. Created: 05/Apr/11  Updated: 14/May/12  Resolved: 05/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Improvement Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

With the changes done for issue#62, XADisk now carries a native library. But this would have also required
an additional manual effort from the users to install that native library in the JRE directory.

Based on some analysis, I have found a way to make this automatic. I have implemented it and committing that in a minute.

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 05/Apr/11 ]

Checked-in the changes.





[XADISK-63] Better exception handling in stream wrappers Created: 02/Apr/11  Updated: 14/May/12  Resolved: 05/Apr/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Improvement Priority: Minor
Reporter: lukiano Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

On both XAFileInputStreamWrapper and XAFileOutputStreamWrapper, if a XAApplicationException occurs, then it throws an IOException with the original message. However, IOException of Java 6 has a constructor which accepts a Throwable, so the original exception can be wrapped and a more detailed log is achieved.
This proposal is to check if this constructor exists from the IOException class, and use it if it's available. We cannot assume the constructor exists because the project is targeted for Java 5.



 Comments   
Comment by Nitin Verma [ 03/Apr/11 ]

Thanks Luciano for bringing this in front.

In Java 5:

  • there is a Throwable constructor which accepts a cause.
  • IOException does not provide a constructor which accepts a cause. (this was done in Java 6)
  • to the rescue, IOException inherits initCause from Throwable. We will use it.

Nitin

Comment by Nitin Verma [ 05/Apr/11 ]

Checked-in the changes.





[XADISK-62] Directory operations (like createNewFile, deleteFile, renameTo) in java.io.File may not be synchronized. Created: 07/Mar/11  Updated: 14/May/12  Due: 14/Mar/11  Resolved: 09/Mar/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: Improvement Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Directory operations of java.io.File like createNewFile, createDirectory, deleteFile, renameTo which add/remove children objects (files/directories) of a directory may not be synchronized ("may" because such behaviors are platform dependent).

XADisk, being a fully transactional system, should keep honoring the Durability (the D of ACID). After a thorough research, it has been found that Java (TM) itself does not provides a mechanism to make these (above) directory operations synchronized. So, a native module (via a .so or .dll) would now be added to XADisk to achieve the same.

I am done with the initial implementation part, and would keep updating here.



 Comments   
Comment by Nitin Verma [ 09/Mar/11 ]

Checked-in the changes.

Comment by Nitin Verma [ 10/Mar/11 ]

Also added a flag to disable this feature. Checking-in...





[XADISK-61] For methods in XADiskBasicIOOperations accepting "lockExclusively" parameter, add an overloaded version for default locking policy. Created: 05/Mar/11  Updated: 14/May/12  Resolved: 30/Mar/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Some of the methods in XADiskBasicIOOperations, like createXAFileInputStream, take a parameter called "lockExclusively".
This parameter was used while obtaining a lock over the related file - shared or exclusive. But for many applications,
the default locking policy for such operations would be sufficient and they don't need to worry about this
additional parameter to these methods.

In this direction, add overloaded versions of these methods, which assume value
"false" for "lockExclusively" (note that all those methods which accept "lockExclusively" do maintain serializablity with shared lock also, see http://java.net/jira/browse/XADISK-59 for the default serializable locking policy).



 Comments   
Comment by Nitin Verma [ 30/Mar/11 ]

Checked-in the changes.





[XADISK-60] Entries in transactionsAndLogsOccupied and transactionSubmittedBuffers are not cleaned up after transaction completion. Created: 05/Mar/11  Updated: 14/May/12  Resolved: 05/Mar/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Phil Merry for identifying and bringing up this issue.

Please refer to class org.xadisk.filesystem.workers.GatheringDiskWriter. Entries in the two maps: "transactionsAndLogsOccupied" and "transactionSubmittedBuffers", are not cleaned up after transaction completion. Although the data kept in them per transaction is very little, but as number of transactions grow, the memory footprint may be significant.

We can do remove(xid) over both of these maps inside the method "cleanupTransactionInfo".



 Comments   
Comment by Nitin Verma [ 05/Mar/11 ]

Checked-in the changes.





[XADISK-59] Concurrency among transactions can be improved without affecting the current isolation level. Created: 03/Mar/11  Updated: 14/May/12  Resolved: 05/Mar/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: Improvement Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Generic



 Description   

First of all, I am grateful to John Babbidge who made me rethink on the existing locking in XADisk.

I have realized that the current isolation level can still be maintained with the following locking scheme, which allows more concurrency among transactions. Below, F may mean file or directory or both based on the context.

createFile(F) Exclusive lock on F
deleteFile(F) Exclusive lock on F
createXAFileOutputStream(F) Exclusive lock on F
createXAFileInputStream(F) Shared lock on F
fileExists(F) Shared lock on F
fileExistsAndIsDirectory(F) Shared lock on F
copyFile(F1, F2) Shared lock on F1, Exclusive lock on F2
moveFile(F1, F2) Exclusive lock on F1 and F2
getFileLength(F) Shared lock on F
truncateFile(F) Exclusive lock on F
listFiles(F) This will not incur any locking.

____________________________________________________________________________________________________________________________

Note for listFiles : The original plan regarding listFiles was different, but is not being implemented.

The original plan: a serializable concurrency would require shared lock on F/* for listFiles(F/), where * includes all children (all possible, current or future). It means with listFiles(dir), the lock on "dir/" would also block createFile for "dir/a.txt" by other transactions. The intention is to not let the result of listFiles change over a transaction. As such a locking in listFiles (though, this method is rarely used) would result in another loss of concurrency again with almost no value add in most cases, so a method parameter can be provided to relax locking for listFiles as needed.

Why Not Being implemented: First, listFiles appears to be rarely used in business application dealing with files. Even if an application is using listFiles (say, to iterate and process all files of a directory), the strict requirement that new files shouldn't get added to the directory is rare. Deletion of any file would anyway remain blocked because of a read lock obtained (in createXAFileInputStream) when the application starts processing the file. Secondly, it was realized that implementation of such a special kind of lock (dir/) would incur an overhead during ALL operations, even if the application is not actually using the listFiles method. So, overall, keeping listFiles free of this (D/) kind of lock seems a practical and efficient approach. (Of course, if more new ways come up, I am open for implementing in future)
____________________________________________________________________________________________________________________________



 Comments   
Comment by Nitin Verma [ 05/Mar/11 ]

Checked-in the changes...





[XADISK-49] moveFile operation is asking for a lock twice when the parents (directory) of source and destination are same. Created: 19/Jan/11  Updated: 14/May/12  Resolved: 08/Feb/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: 1.1

Type: Bug Priority: Major
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 49

 Description   

A transaction keeps track of all acquiredLocks in the variable "acquiredLocks" of NativeSession. I update
this collection whenever I am done acquiring all locks required for the operation (say moveFile) to
complete.

This was good as long as there are no lock-reuses within the given operation.

We will have to update the "acquiredLocks" after acquiring each lock
instead of doing so after acquiring all locks required for the operation.






[XADISK-47] Implement Read-Only optimization for transactions in XADisk. Created: 05/Jan/11  Updated: 14/May/12  Resolved: 01/Mar/11

Status: Resolved
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.0
Fix Version/s: 1.1

Type: New Feature Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 47

 Description   

Performance of read-only transactions in XADisk can be improved using
optimizations derived from the "read-only" nature of the transaction. Most
important ones are:
a) for xa transactions, return RD_ONLY from XAResource.prepare call
b) for local transactions, do not do anything (ofcourse, under the covers) with
the transaction-logs.






[XADISK-17] Add logging to XADisk. Created: 20/Nov/10  Updated: 14/May/12  Resolved: 07/May/11

Status: Resolved
Project: xadisk
Component/s: misc
Affects Version/s: 0.9 (Beta)
Fix Version/s: 1.1

Type: New Feature Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 17

 Description   

It would be quite helpful to see log messages from XADisk; atleast those when
XADisk performs critical operations.






[XADISK-74] Use slf4j for logging instead of binding it to one logger Created: 07/May/11  Updated: 18/Apr/12

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: fecici Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

it could be nice to log through the slf4j, so the user can choose which logger to use...






[XADISK-104] Limitting access to a remote file system, exposed by xadisk Created: 12/Mar/12  Updated: 12/Mar/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Daniel for identifying this improvement at discussion thread http://groups.google.com/group/xadisk/browse_thread/thread/b77ee7b33f9c9574.

It could be a useful feature if xadisk can be configured to restrict operations over specified files/folders only, specially for remote file systems exposed by xadisk.






[XADISK-101] For windows share directory, permission checks done using Java APIs are misleading. Created: 17/Feb/12  Updated: 17/Feb/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

There is a problem with Windows Share and Java combination, in the File's canRead()/canWrite() APIs.

These APIs are returning true even if the particular file/directory does not expose that permission. This results in XDisk's permission checks, done during an operation itself, passing even if actual permissions don't exist. When XADisk proceeds to commit the changes, only then the lack of permissions result in exception. The XADisk instance would fail during commit and would make itself unavailable.

Workaround is to make sure to assign all needed permissions beforehand, when using windows share folders.






[XADISK-77] Problem when trying to write to a Samba shared folder Created: 25/May/11  Updated: 17/Feb/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: None

Type: Bug Priority: Major
Reporter: sgerogia Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

XADisk 1.1
Windows XP SP3 + Weblogic 10.3.3 (XADisk JCA adapter)
RedHat Ent. Linux Server 5.5 (remote folder)


Attachments: Text File stack-trace.txt     Zip Archive XADisk.zip    

 Description   

A Linux network folder has been exposed in the intranet by Samba.
The folder requires authentication with a local Linux user to allow read/write access.
It has been mounted as a network drive on the Windows machine where the Weblogic server hosting the XADisk JCA adapter.

When a move operation is attempted, the attached stacktrace is shown (copy from the server's console, as there is a mix of logs and system outs) and the operation fails.

The local XA homedir after the operation is attached as a zip.

A workaround (if it fits, depending upon your application requirements) is to disable directory synchronization using setSynchronizeDirectoryChanges



 Comments   
Comment by Nitin Verma [ 25/May/11 ]

Hi Stelios,

Thanks for bringing up the issue.

While I am investigating it and coming up with a plan, you may consider using:

http://xadisk.java.net/javadoc/1.1/org/xadisk/filesystem/FileSystemConfiguration.html#setSynchronizeDirectoryChanges%28java.lang.Boolean%29

to disable directory synchronization.

Please let me know if you have any questions.

Thanks,
Nitin





[XADISK-100] Error "Directory changes could not be forced-to-disk" for windows file share. Created: 17/Feb/12  Updated: 17/Feb/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When creating files in a windows share directory, the following exception is seen during transaction commit:

____________________________________________________________________________________________________________________________

Exception in thread "main" org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:488)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:731)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:1246)
at org.xadisk.tests.correctness.Temp.main(Temp.java:17)
Caused by: java.io.IOException: Fatal Error: Directory changes could not be forced-to-disk during transaction commit.
at org.xadisk.filesystem.DurableDiskSession.forceToDisk(DurableDiskSession.java:122)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:723)
... 2 more

____________________________________________________________________________________________________________________________

A workaround (if it fits, depending upon your application requirements) is to disable directory synchronization using setSynchronizeDirectoryChanges






[XADISK-82] Remove ra.xml from the JAR artifact Created: 03/Aug/11  Updated: 10/Aug/11  Resolved: 06/Aug/11

Status: Resolved
Project: xadisk
Component/s: misc
Affects Version/s: 1.1
Fix Version/s: 1.1

Type: Improvement Priority: Major
Reporter: Simon Dierl Assignee: Nitin Verma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File xadisk-ra-inclusion.patch    
Tags: maven

 Description   

Currently, the XADisk JAR artifact deployed to Maven central via maven.java.net contains a META-INF/ra.xml. This causes problems when deploying the exploded RAR artifact to JBoss 5.1.0, since JBoss will try to parse the ra.xml in the JAR (contained in the RAR) and fail.

The problem can be fixed by excluding the ra.xml from the JAR, where it should not be in the first place. A patch to the pom.xml that corrects this is attached.



 Comments   
Comment by Nitin Verma [ 03/Aug/11 ]

Hello,

Thanks for identifying this issue.

Due to the recent migration to the new maven repository, I had created a separate pom.xml for uploading to the repository (as there were a few additional requirements), and missed out the ra.xml in the jar.

(One won't find this "extra" ra.xml in everything.zip at the project website though).

Thanks,
Nitin

Comment by Nitin Verma [ 03/Aug/11 ]

I have uploaded the modified artifacts to Staging Repository, but the repository doesn't seem to allow updation/deletion of released artifacts (in the Release Repository). I have written an email to Sonatype folks to seek for advices regarding updation/deletion in such cases.

Comment by Simon Dierl [ 05/Aug/11 ]

If Sonatype does not allow to change an already uploaded artifact (makes sense for build reproducibility), what about releasing a 1.1.1 with the ra.xml change only? This would only mean a minor change to the pom.xml-s of those affected.

Comment by Nitin Verma [ 05/Aug/11 ]

Hello, SDierl (I am sorry I don't know how should I call you)

Thanks for your advice.

I have received response from Sonatype, and they say its not feasible to update (even slightest) or delete the released artifacts and the only option is to create another release.

On that line, I have (I got a bit late in updating this bug) created a new release as "v1.1" (earlier it was "1.1"). It is now available at:

http://search.maven.org/#browse|-815524010

Would you like to confirm if the .rar from the new release "v1.1" deploys all fine for you.

Thanks,
Nitin

Comment by Nitin Verma [ 06/Aug/11 ]

Verified that the rar in the new release "v1.1" (http://search.maven.org/#browse|-815524010) is deploying and working fine on JBoss 5.1.0.

Comment by Simon Dierl [ 10/Aug/11 ]

Works for me, too. Thanks for the prompt response!

Comment by Nitin Verma [ 10/Aug/11 ]

Many Thanks, Simon

Cheers...
Nitin





[XADISK-56] Move all not junit test classes to junit Created: 05/Feb/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: testcases
Affects Version/s: current
Fix Version/s: current

Type: Task Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[XADISK-55] add junit dependency and write the main xadisk unit and integration tests Created: 05/Feb/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: testcases
Affects Version/s: current
Fix Version/s: current

Type: Task Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[XADISK-54] Refactor the entire projet to use the standard maven layout Created: 05/Feb/11  Updated: 31/Jul/11  Due: 07/Feb/11

Status: Open
Project: xadisk
Component/s: misc
Affects Version/s: current
Fix Version/s: current

Type: Task Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Refactor the entire projet to use the standard maven layout.



 Comments   
Comment by dcoraboe [ 17/Feb/11 ]

Does it mean that XADisk is planned to be distributed in the Maven Central repository?

Comment by Nitin Verma [ 23/Feb/11 ]

Hi,
I am hoping the XADisk binaries would be on maven2 repository (java.net's) very soon. I have sent my request
to get permissions to upload to that repository.

The current issue (54th) is different and will no be tracking this activity. I have filed
another issue now : http://java.net/jira/browse/XADISK-58

Thanks,
Nitin





[XADISK-50] Implement the createSymbolicLink function Created: 19/Jan/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 50

 Description   

Would be useful to add to the session the possibility of creating symbolic links.






[XADISK-51] implement createPropertiesFile function Created: 19/Jan/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 51

 Description   

would be useful to add to the session the option to store properties file






[XADISK-53] implement the delete recursive Created: 19/Jan/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 53

 Description   

would be useful to add the recursive option to the delete method






[XADISK-18] Lock timeout functionality for directory pinning. Created: 20/Nov/10  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 0.9 (Beta)
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 18

 Description   

As of current implementation, lock timeout is not effective for cases when the
locking is not feasible due to another transaction holding a directory-pin lock
over an ancestor directory of the file/directory obje