[XADISK-139] XASystemBootFailureException when starting XADisk on a large data set Created: 23/Jul/13  Updated: 16/Aug/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux rs201377.rs.hosteurope.de 2.6.32-5-amd64 #1 SMP Mon Feb 25 00:26:11 UTC 2013 x86_64 GNU/Linux
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.3) (6b24-1.11.3-2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)



 Description   

The startup exception shown below occurred when starting XADisk on a large data set after the system has been stopped regularly (though I cannot guarantee that XADisk shut down in time. At least I can say that no crash or kill -9 occurred immediately before.

I could in principle provide the xadisk transaction directory, but it is quite large (31 GB; mostly transaction logs).

This is not yet a production system, so I can still recover by deleting the xa stores.

org.xadisk.filesystem.exceptions.XASystemBootFailureException: The XADisk instance has encoutered a critial issue and could not be booted. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details.
at org.xadisk.filesystem.NativeXAFileSystem.<init>(NativeXAFileSystem.java:187)
at org.xadisk.filesystem.NativeXAFileSystem.bootXAFileSystemStandAlone(NativeXAFileSystem.java:219)
at org.xadisk.bridge.proxies.interfaces.XAFileSystemProxy.bootNativeXAFileSystem(XAFileSystemProxy.java:47)
at de.konzentrik.lib.io.XaDisk.initLocal(XaDisk.java:43)
at de.konzentrik.app.preprocess.PreprocessorModule.doStartup(PreprocessorModule.java:79)
at de.zarncke.lib.sys.module.AbstractModule.startOrRestart(AbstractModule.java:126)
at de.zarncke.lib.sys.Headquarters.startOrRestart(Headquarters.java:450)
at de.zarncke.lib.sys.Headquarters.startOrRestart(Headquarters.java:452)
at de.zarncke.lib.sys.Headquarters.startOrRestart(Headquarters.java:446)
at de.konzentrik.app.preprocess.PreprocessorInstallation.doBootProtected(PreprocessorInstallation.java:202)
at de.konzentrik.app.preprocess.PreprocessorInstallation.boot(PreprocessorInstallation.java:158)
at de.konzentrik.app.preprocess.Start.run(Start.java:164)
at de.zarncke.lib.block.Running.execute(Running.java:5)
at de.zarncke.lib.block.Running.execute(Running.java:3)
at de.zarncke.lib.err.Warden.guard(Warden.java:313)
at de.konzentrik.app.StartBase.executeGuarded(StartBase.java:216)
at de.konzentrik.app.preprocess.Start$1.run(Start.java:190)
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:480)
at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:336)
at org.xadisk.filesystem.TransactionLogEntry.parseLogEntry(TransactionLogEntry.java:319)
at org.xadisk.filesystem.TransactionLogEntry.getNextTransactionLogEntry(TransactionLogEntry.java:521)
at org.xadisk.filesystem.workers.CrashRecoveryWorker.findInCompleteTransactions(CrashRecoveryWorker.java:139)
at org.xadisk.filesystem.workers.CrashRecoveryWorker.collectRecoveryData(CrashRecoveryWorker.java:99)
at org.xadisk.filesystem.NativeXAFileSystem.<init>(NativeXAFileSystem.java:181)



 Comments   
Comment by gunnar_zarncke [ 06/Aug/13 ]

I have had another such XASystemBootFailureException.
This time with a much smaller data set (8.7 MB in xa disk directory).
I copied that and might provide it for analysis.

Note: I get no notices when you, Nitin, comment on the issues.
I might like to get in direct contact. You can reach me at gunnar at konzentrik.de.

Comment by gunnar_zarncke [ 16/Aug/13 ]

I sent you two occurrences of crashes via mail. I am not sure if these have the same cause, but maybe you can make something out of it.





[XADISK-95] DirectorySync fails on Win7 64 bit Created: 21/Nov/11  Updated: 24/Oct/13

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: 1.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: kanebonnette Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: 2 days
Time Spent: Not Specified
Original Estimate: 2 days
Environment:

Windows 7, 64 bit, Java 1.6.0_026


Attachments: Java Source File Example.java    

 Description   

discussion thread here: http://groups.google.com/group/xadisk/browse_thread/thread/aab722ef7450ceab

Basic problem is that when StandaloneFileSystemConfiguration.setSynchronizeDirectoryChanges is set to true and absolute paths are use, XADisk crashes/hangs with the exception:
Exception in thread "main"
org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The
XADisk instance has encoutered a critial issue and is no more
available. Such a condition is very rare. If you think you have setup
everything right for XADisk to work, please consider discussing in
XADisk forums, or raising a bug with details
at
org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSy stem.java:
488)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:707)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:
1196)
at selected.Example.example0(Example.java:108)
at selected.Example.main(Example.java:213)
Caused by: java.io.IOException: Fatal Error: Directory changes could
not be forced-to-disk during transaction commit.
at
org.xadisk.filesystem.DurableDiskSession.forceToDisk(DurableDiskSession.jav a:
122)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:695)
... 3 more

When StandaloneFileSystemConfiguration.setSynchronizeDirectoryChanges is set to true and relative paths are used, XADisk prints the following error:
XADisk Error [Native Module] Directory C:\ does not exist.

in both cases, setting StandaloneFileSystemConfiguration.setSynchronizeDirectoryChanges to false does not report any errors.

The problem apparently does not occur in WinXP, 32 bit.

Estimate set at two days because... I needed a value.



 Comments   
Comment by Nitin Verma [ 17/Feb/12 ]

A workaround (if it fits, depending upon your application requirements) is to disable directory synchronization using setSynchronizeDirectoryChanges

Comment by Nitin Verma [ 21/Aug/13 ]

Just FYI. If developers using xadisk face such native library issues, they can do the following without any need to modify and re-build the xadisk java code:

-modify the native code if necessary (available in svn/trunk/native/*.c).

-compile the code according to the target platform and prepare the dynamic library.

-rename the dynamic library as "placeholder_xadisk.native" and place it inside the xadisk.jar as xadisk.jar/native/placeholder_xadisk.native.

Comment by gastaldi [ 24/Oct/13 ]

Hi,

Is there any estimate of when will this issue be solved?

Thanks

Comment by Nitin Verma [ 24/Oct/13 ]

Hi George,

Unfortunately not. Would you (or anyone else reading this bug) like to try out the compilation for windows 7 and see if the issue gets fixed. The generated lib can then be shared with everyone.

Thanks,
Nitin





[XADISK-117] XADisk has failed to load its native library Created: 30/May/12  Updated: 21/Aug/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Fireworks Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

System: Microsoft Windows Server 2003 R2
WebSphere Application Server V6.1.0.41



 Description   

After deploying Resource adapter and register JCA factory, during initialization in XADisk work directory was created log file with next content:

May 30, 2012 4:38:52 PM : XADisk has failed to load its native library required for directory-synchronization.
Now, it will override the configuration property "synchronizeDirectoryChanges" and set it to false; but please note that this would turn-off directory-synchronization i.e. directory modifications may not get synchronized to the disk at transaction commit.
If you have any questions or think this exception is not expected, please consider discussing in XADisk forums, or raising a bug with details.

May 30, 2012 4:38:52 PM : Successfully booted the XADisk instance.

And after trying to use connection from XADiskConnectionFactory next log was generated:

2012-05-30 17:03:32,578 ERROR [ru.acs.st.core.upload.ArchiveStreamProcessorImpl] - Ошибка при сохранении jar файла.
ru.acs.st.common.exception.STFileUploadException: Ошибка создания файла: First.jar в хранилище, отсутствует транзакция для сохранения данных.
at ru.acs.st.core.jarstorage.FileSystemJarStorageImpl.saveFileTransactional(FileSystemJarStorageImpl.java:191)
at ru.acs.st.core.jarstorage.FileSystemJarStorageImpl.saveJar(FileSystemJarStorageImpl.java:118)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.saveToDBAndFile(ArchiveStreamProcessorImpl.java:244)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.processJarByteMassive(ArchiveStreamProcessorImpl.java:191)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.processSingleFile(ArchiveStreamProcessorImpl.java:154)
at ru.acs.st.core.upload.ArchiveStreamProcessorImpl.processArchiveStream(ArchiveStreamProcessorImpl.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:615)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.doInTransaction(TransactionInterceptor.java:132)
at org.springframework.transaction.jta.WebSphereUowTransactionManager$UOWActionAdapter.run(WebSphereUowTransactionManager.java:337)
at com.ibm.ws.uow.UOWManagerImpl.runUnderNewUOW(UOWManagerImpl.java:950)
at com.ibm.ws.uow.UOWManagerImpl.runUnderUOW(UOWManagerImpl.java:511)
at org.springframework.transaction.jta.WebSphereUowTransactionManager.execute(WebSphereUowTransactionManager.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:127)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy136.processArchiveStream(Unknown Source)
at ru.acs.st.core.service.UploadArchiveServlet.doPost(UploadArchiveServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1213)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:658)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:526)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:764)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1478)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:133)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:457)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:300)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:271)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:196)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:751)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:881)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1551)
Caused by:
org.xadisk.filesystem.exceptions.NoTransactionAssociatedException: The method that was called can only be called with a transaction associated, butthere is no such transaction present.
at org.xadisk.filesystem.NativeXASession.getSessionForCurrentWorkAssociation(NativeXASession.java:223)
at org.xadisk.connector.outbound.XADiskConnectionImpl.createFile(XADiskConnectionImpl.java:64)
at ru.acs.st.core.jarstorage.FileSystemJarStorageImpl.saveFileTransactional(FileSystemJarStorageImpl.java:162)
... 45 more

Exception NoTransaction, but we can see from trace, that it was call inside transaction.



 Comments   
Comment by Nitin Verma [ 04/Jun/12 ]

Hello.

Thanks for creating this bug.

The first issue you mentioned seems to be related to other known issues like this one - http://java.net/jira/browse/xadisk-100.

The second one requires further discussion and analysis before we call it a bug. I would recommend the google group at http://groups.google.com/group/xadisk for such discussions. It looks like xadisk is not being "enlisted" inside the global/xa transaction. It would be great if you can provide more details about the test-case/environment at that discussion thread.

Thanks,
Nitin

Comment by Fireworks [ 05/Jun/12 ]

Hellow, the first issue really repaired by disabling directory synchronization. It is not critical for my task.

Comment by Nitin Verma [ 21/Aug/13 ]

Just FYI. If developers using xadisk face such native library issues, they can do the following without any need to modify and re-build the xadisk java code:

-modify the native code if necessary (available in svn/trunk/native/*.c).

-compile the code according to the target platform and prepare the dynamic library.

-rename the dynamic library as "placeholder_xadisk.native" and place it inside the xadisk.jar as xadisk.jar/native/placeholder_xadisk.native.





[XADISK-77] Problem when trying to write to a Samba shared folder Created: 25/May/11  Updated: 17/Feb/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: None

Type: Bug Priority: Major
Reporter: sgerogia Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

XADisk 1.1
Windows XP SP3 + Weblogic 10.3.3 (XADisk JCA adapter)
RedHat Ent. Linux Server 5.5 (remote folder)


Attachments: Text File stack-trace.txt     Zip Archive XADisk.zip    

 Description   

A Linux network folder has been exposed in the intranet by Samba.
The folder requires authentication with a local Linux user to allow read/write access.
It has been mounted as a network drive on the Windows machine where the Weblogic server hosting the XADisk JCA adapter.

When a move operation is attempted, the attached stacktrace is shown (copy from the server's console, as there is a mix of logs and system outs) and the operation fails.

The local XA homedir after the operation is attached as a zip.

A workaround (if it fits, depending upon your application requirements) is to disable directory synchronization using setSynchronizeDirectoryChanges



 Comments   
Comment by Nitin Verma [ 25/May/11 ]

Hi Stelios,

Thanks for bringing up the issue.

While I am investigating it and coming up with a plan, you may consider using:

http://xadisk.java.net/javadoc/1.1/org/xadisk/filesystem/FileSystemConfiguration.html#setSynchronizeDirectoryChanges%28java.lang.Boolean%29

to disable directory synchronization.

Please let me know if you have any questions.

Thanks,
Nitin





[XADISK-175] Migrate to NIO.2 File API Created: 22/Jun/15  Updated: 22/Jun/15

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: puce Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Since Java SE 7, java.io.File has become a legacy API and replaced by the NIO.2 File API:
http://docs.oracle.com/javase/tutorial/essential/io/legacy.html

Migrate the existing code to NIO.2 File API.






[XADISK-174] Consider to migrate to GitHub Created: 22/Jun/15  Updated: 22/Jun/15

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: puce Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

For easier collaboration (fork/ pull request), consider to migrate to Git and GitHub.






[XADISK-170]  fileExistsAndIsDirectory return false when a folder in the path chain is not accessible Created: 11/Dec/14  Updated: 11/Dec/14

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: sberthouzoz Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows, NTFS filesystem, xa-disk 1.2.2



 Description   

suppose the following call fileExistsAndIsDirectory("\\server\share$\folder\subfolder") return false even when the folder exists.
You need the following condition to reproduce it:

  • no READ Permission on \\server\share$\
  • READ Permission set on \\server\share$\folder
  • READ and WRITE Permission set on \\server\share$\folder\subfolder

The cause of the problem is that the code is doing a (""\\server\share$\").isDirectory() which returns false (because of file Permission) in org.xadisk.filesystem.virtual.TransactionVirtualView.getVirtualViewDirectory(File) on line 304.

Here is an abstract of the call chain:

  1. session.fileExistsAndIsDirectory("\\server\share$\folder\subfolder")
  2. NativeSession.fileExistsAndIsDirectory("\\server\share$\folder\subfolder"", false)
  3. NativeSession.checkPermission(READ_DIRECTORY, "\\server\share$\folder")
  4. TransactionVirtualView.isDirectoryReadable("\\server\share$\folder")
  5. TransactionVirtualView.getVirtualViewDirectory(\\server\share$"), here the line 305 or 312 will throw the FileNotExistsException which will cause the methods to return false;

The problem is also that the method checks the parent of the parent during the chain, instead of just checking the given path






[XADISK-144] out-of-file-handles on transaction involing large number of copied files Created: 11/Sep/13  Updated: 11/Sep/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS, Linux testhost 2.6.32-358.2.1.el6.x86_64 #1 SMP Wed Mar 13 00:26:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux, java version "1.7.0_25"
OpenJDK Runtime Environment (rhel-2.3.10.4.el6_4-x86_64)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)



 Description   

An XADisk transaction involving a very large number of files extracted from an archive fails fatally due to an OS-level out-of-file-handle situation.

The issue is critical in so far as we want to write the files extracted from the archive under transaction control and the transaction fails fatally:

XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details

We took care to close all files immediately after writing the data and we also ensured that only a limited number of files are processed concurrently.

At first it looked as if the files might be cached in NativeSession.getCachedXAFileOutputStream.
On further inspection of the XA-Disk code it became clear that closing the XAFileOutputStream doesn't actually close the underlying file handle/channel. The close happens delayed in VirtualViewFile.forceAndFreePhysicalChannel() upon commit/rollback. This is also the case for heavyWrite which we use in this case as the files are completely written in one chunk.

This means that we can do nothing about this because the number of files that are to be extracted is potentially arbitrarily large - in in particular significantly larger than any acceptable ulimit.

I am wondering why the file handles cannot be closed earlier.
I have seen that the data is flushed upon XAFileOutputStream.close().
It should be possible to close them upon close at least in the heavy write case. Alternatively it might be possible to close the file and later reopen at the same position - should that unexpetcedly be neccessary.


Additional information:

The file handle limits is at the default of 1024:

bash-4.1$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 29738
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Reported Exception:

[...]
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:489)
at org.xadisk.filesystem.virtual.NativeXAFileOutputStream.<init>(NativeXAFileOutputStream.java:56)
at org.xadisk.filesystem.NativeSession.getCachedXAFileOutputStream(NativeSession.java:1266)
at org.xadisk.filesystem.NativeSession.createXAFileOutputStream(NativeSession.java:187)
at org.xadisk.filesystem.NativeSession.createXAFileOutputStream(NativeSession.java:43)
at de.konzentrik.lib.io.xa.XaDecoupledFileSystem$XaDecoupledSession.createXAFileOutputStream(XaDecoupledFileSystem.java:216)
at de.konzentrik.lib.io.XaDiskStore.getOutputStream(XaDiskStore.java:263)
at de.konzentrik.app.preprocess.process.step.ImageCopierStep.process(ImageCopierStep.java:65)
at de.konzentrik.app.preprocess.process.step.ImageCopierStep.process(ImageCopierStep.java:25)
[...]
at de.konzentrik.app.preprocess.process.ProcessStepRunner$StepRunner.run(ProcessStepRunner.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more
Caused by: java.io.IOException: Zu viele offene Dateien [german for "too many open files"]
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:947)
at org.xadisk.filesystem.utilities.FileIOUtility.createFile(FileIOUtility.java:91)
at org.xadisk.filesystem.DurableDiskSession.createFile(DurableDiskSession.java:158)
at org.xadisk.filesystem.virtual.VirtualViewFile.safeSetupForPhysicalFileExistence(VirtualViewFile.java:142)
at org.xadisk.filesystem.virtual.VirtualViewFile.setUpForHeavyWriteOptimization(VirtualViewFile.java:154)
at org.xadisk.filesystem.virtual.NativeXAFileOutputStream.<init>(NativeXAFileOutputStream.java:54)
... 16 more






[XADISK-153] File.canWrite() returning wrong value, leading to invalid permission check by xadisk. Created: 22/Nov/13  Updated: 22/Nov/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Luc for bringing this up on the discussion thread: https://groups.google.com/forum/#!topic/xadisk/cuQAhyxMjyI

XADisk does a permission check during the io operations, in an attempt to make sure that they succeed during commit. During create file, it check the parent directory write permission. As discussed on above thread, the check returns wrong value (on the given windows environment), leading to failure in commit.

The following code, as provided by Luc, works as follows on his windows machine:

===============================

File f = new File( "C:/");
System.out.println("f canwrite " + f.canWrite());
File newFile = new File( f.getAbsolutePath(), "zap.txt");
boolean created = newFile.createNewFile();
System.out.println( "created: " + created);

The output:

f canwrite true
Exception in thread "main" java.io.IOException: A required privilege is not held by the client
at java.io.WinNTFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1006)
at zap.main(zap.java:16)

===============================






[XADISK-158] Problem with truncate method Created: 10/Jan/14  Updated: 23/Aug/14

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: hell_keeper Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Unix x64 (test on centos 6.5)



 Description   

This problem seems affect the Unix system (I dont' have this problem with win7 x64). For a project, I test multiple solutions to manage "a transactional filesystem". My transactional service must be able to write into a file (appending or not).
The following test scenario illustrates the bug. I have on my disk a file with a serialized map ( a JSON serialization with a map of beans):

{"meta2":"key":"key2","value":"value2","type":"USER"}

,"meta1":"key":"key1","value":"value1","type":"TECHNICAL"}}

My transactional service must update this file with a new serialized map:

{"meta3":{"key":"key3","value":"value3","type":"TECHNICAL"}}

The transactional method calls a XADisk storage connector. To avoid to append the new map in the file I use the truncate method:

session.truncateFile(file, 0);
try (OutputStream out = new XAFileOutputStreamWrapper(session.createXAFileOutputStream(file, true))) {
try (InputStream in = source.getInputStream())

{ IOUtils.copy(in, out); }

} catch (FileNotExistsException | NoTransactionAssociatedException | InterruptedException | IOException | FileUnderUseException e) {
...
}

In the transaction I throw an exception to cause a rollback. I expect to have the initial file with the following information:

{"meta2":"key":"key2","value":"value2","type":"USER"}

,"meta1":"key":"key1","value":"value1","type":"TECHNICAL"}}

But the file contains:

{"meta3":"key":"key3","value":"value3","type":"TECHNICAL"}}"meta2":"key":"key2","value":"value2","type":"USER"}

The first point is the file is inconsistent and it seems that the truncate method is equivalent to a insert a the beginning of the file.

Best regards.



 Comments   
Comment by Nitin Verma [ 11/Jan/14 ]

Hello.

Have you explicitly called rollback method on the session object?

Thanks,
Nitin

Comment by hell_keeper [ 11/Jan/14 ]

I use Spring to manage the transaction so the rollback has been called on the XaResource (by spring).

Comment by Nitin Verma [ 14/Jan/14 ]

I am not able to reproduce this issue with a standalone test-case as below. The file's contents were "this is some data. this is some data. this is some data.", and they remain as-is after the test completes:

File f = new File("C:
a.txt");
session.truncateFile(f, 0);
XAFileOutputStream xafos = session.createXAFileOutputStream(f, true);
xafos.write("new content".getBytes());
session.rollback();

Do you see something which can be different with your test?

Thanks,
Nitin

Comment by hell_keeper [ 15/Jan/14 ]

Expected the fact that I use a XAFileOutputStreamWrapper, I dont see difference...

Comment by hell_keeper [ 15/Jan/14 ]

I just created a little project (under eclipse and using maven) that illustrates my problem. Please see it attached hopping that the bug is present to you (https://drive.google.com/file/d/0B2sj7pmEYCG5WlJwNmhVeEJCQ1U/edit?usp=sharing). Please take care about my system (unix x64) because on windows I don't have this problem.

This test is based on the sample present in the example section of the xaDisk website.

Please note also that the test prints this message in my console (may be the cause of the problem).

Botting an XADisk instance...
OpenJDK 64-Bit Server VM warning: You have loaded library /home/thomas/workspace/problem-truncate/target/testXadisk1859041122059/native/unix_32_xadisk.native which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Successfully booted the XADisk instance.

Best regards.

Comment by Nitin Verma [ 15/Jan/14 ]

Hello.

I am able to reproduce the issue on my Fedora linux installation. I am sorry I ignored this from the bug description - "This problem seems affect the Unix system".

I will let you know as and when I discover the reason/fix for this bug.

Many Thanks for your effort.

Nitin

Comment by Nitin Verma [ 15/Jan/14 ]

I find that the below code does not work as expected on the linux installation:
______________________________

File s = new File("C:/a.txt");
File d = new File("C:/b.txt");
FileChannel fcs = new FileInputStream(s).getChannel();
FileChannel fcd = new FileOutputStream(d, true).getChannel();
fcd.transferFrom(fcs, 3, 10);
______________________________

From the javadoc of FileChannel.transferFrom, one would expect the bytes transfer to begin at 3rd position of the destination file. This works fine on my windows xp installation, but on linux it appends at the end of the destination file. Note that using "append=false" for opening FileOutputStream truncates the file immediately (even before starting to write into the file).

Seems like a bug to me.

Any opinions?

Comment by Jasper Siepkes [ 31/Jan/14 ]

Well the doc ( http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#transferFrom%28java.nio.channels.ReadableByteChannel,%20long,%20long%29 ) says "An attempt is made to read up to count bytes from the source channel and write them to this channel's file starting at the given position." (with position being written in monospaced font to signify its an method argument). So yeah one would expect that, JVM bug perhaps?. Which JVM version are you using on Linux Nitin?

Comment by Nitin Verma [ 31/Jan/14 ]

Hello Jasper,

I think it was 1.6. I will confirm it and update (need to reboot and switch OS etc )

Nitin

Comment by Nitin Verma [ 23/Aug/14 ]

I did some jdk source code analysis and came to know:

1. the same bug on Linux would also occur (and as tested, it does) for write(ByteBuffer, int), where the second argument for position in the file is being ignored if the fileChannel was opened in append mode. Basically, the following code too does not work as expected:
File d = new File ("b.txt");
FileChannel fcd = new FileOutputStream (d, true).getChannel ();
ByteBuffer bb = ByteBuffer.wrap("bytebuffer".getBytes());
fcd.write(bb, 3);

It does not write at the position of 3, but simply appends.

2. The code flow analysis takes us to the method pwrite0 in http://hg.openjdk.java.net/jdk6/jdk6/jdk/file/b6c45a8b14fb/src/solaris/native/sun/nio/ch/FileDispatcher.c. This further makes use of a library function pwrite64.

3. I wrote a simple c code to verify that even pwrite64 does not work as expected on the linux machine:
FILE *f = fopen("b.txt", "a");
printf("%d", pwrite64(fileno(f), "tobewritten, 8, 3));

This too appends at the end of the file.

4. Came to see that the man page here says it is a known bug with pwrite64:
http://linux.die.net/man/2/pwrite64: "However, on Linux, if a file is opened with O_APPEND, pwrite() appends data to the end of the file, regardless of the value of offset. "

Comment by Nitin Verma [ 23/Aug/14 ]

I will check if we can workaround this jdk bug in xadisk source, wherever it can affect us, including the place being discussed here.





[XADISK-129] Add support to filesystem synchronization for Solaris OS on Sparc Architecture Created: 24/Jan/13  Updated: 03/Sep/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Marco Quaranta Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File solaris_32_xadisk.native     File solaris_64_sparc.native     File solaris_64_xadisk.native    
Tags: filesystem, native, solaris, synchronization

 Description   

I've successfully compiled native code (forceDirectory method, find library attached) on SunOS 64 running on a SPARC architecture. I've also tested it with TestXADiskNative.java and it seems to work.
Please add this library in the next XADisk version.
Thank you



 Comments   
Comment by Nitin Verma [ 16/Feb/13 ]

Thanks to Marco for sending the compiled code at: http://groups.google.com/group/xadisk/browse_thread/thread/97364e2a821a93a7?hl=en

Comment by Nitin Verma [ 21/Aug/13 ]

Just FYI. If developers using xadisk face any native library issues, they can do the following without any need to modify and re-build the xadisk java code:

-modify the native code if necessary (available in svn/trunk/native/*.c).

-compile the code according to the target platform and prepare the dynamic library.

-rename the dynamic library as "placeholder_xadisk.native" and place it inside the xadisk.jar as xadisk.jar/native/placeholder_xadisk.native.

Comment by Nitin Verma [ 22/Aug/13 ]

I am attaching the three files sent by Marco over the google group.

Hi Marco,

Can you please add a brief comment below about the three files - for which platform/architecture are they (I could not figure out the two files named with 64):

solaris_64_sparc.native
solaris_32_xadisk.native
solaris_64_xadisk.native

I hope these libs would be useful for users; though I am not putting them with xadisk as of now, but they can easily replace them as "placeholder_xadisk.native" inside the xadisk.jar.

Thanks,
Nitin

Comment by Marco Quaranta [ 03/Sep/13 ]

Hi Nitin,
a little comment about these three files:

solaris_64_sparc.native - This is the first attempt I made and, as opposed the name said, it is for 32bit (it was an almost failed attempt). It's compiled without -fPIC options. Please, use solaris_32_xadisk.native in place of this.

solaris_32_xadisk.native - This is the compiled version for 32bit jvm compiled with the following command:
/usr/sfw/bin/gcc -shared -fPIC -I/usr/jdk/instances/jdk1.7.0_09/include -I/usr/jdk/instances/jdk1.7.0_09/include/solaris -o ./nativelib.so ./XADiskUnix.c

solaris_64_xadisk.native - This is the compiled version for 64bit jvm compiled with the following command:
/usr/sfw/bin/gcc -shared -fPIC -m64 -I/usr/jdk/instances/jdk1.7.0_09/include -I/usr/jdk/instances/jdk1.7.0_09/include/solaris -L/usr/sfw/lib/64 -R/usr/sfw/lib/64 -o ./nativelib.so ./XADiskUnix.c

I already know about "placeholder_xadisk.native" and it's the way I have used those files in my applications.

Thank you,
Marco





[XADISK-137] Transaction log file got corrupted Created: 10/Jul/13  Updated: 06/Aug/13

Status: Open
Project: xadisk
Component/s: connector, filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: 5ar Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Websphere Application Server v6.1.0.19, CentOS 6.0, Java 1.5.0



 Description   

During submit of attachment error happened which caused corrupt transaction log file.
After that XADisk could not recover.

Here is the stack trace:

___________________________________________________________________________________________________________________________________
[7/3/13 10:07:41:639 GST] 00002abd XATransaction E J2CA0027E: An exception occurred while invoking commit on an XA Resource Adapter from dataSource eis/XADisk/LocalConnectionFactory, within transaction ID

{XidImpl: formatId(57415344), gtrid_length(36), bqual_length(54), data(0000013fa324bd3700002)}

: javax.transaction.xa.XAException
at org.xadisk.filesystem.utilities.MiscUtils.createXAExceptionWithCause(MiscUtils.java:16)
at org.xadisk.connector.XAResourceImpl.commit(XAResourceImpl.java:135)
at com.ibm.ejs.j2c.XATransactionWrapper.commit(XATransactionWrapper.java:462)
at com.ibm.ws.Transaction.JTA.JTAXAResourceImpl.commit(JTAXAResourceImpl.java:272)
at com.ibm.ws.Transaction.JTA.RegisteredResources.deliverOutcome(RegisteredResources.java:2010)
at com.ibm.ws.Transaction.JTA.RegisteredResources.distributeOutcome(RegisteredResources.java:2518)
at com.ibm.ws.Transaction.JTA.RegisteredResources.distributeCommit(RegisteredResources.java:2839)
at com.ibm.ws.Transaction.JTA.TransactionImpl.internalCommit(TransactionImpl.java:2651)
at com.ibm.ws.Transaction.JTA.TransactionImpl.stage2CommitProcessing(TransactionImpl.java:1736)
.
.
.
Caused by: org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:489)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:740)
at org.xadisk.connector.XAResourceImpl.commit(XAResourceImpl.java:131)
... 130 more
Caused by: java.io.EOFException
at org.xadisk.filesystem.utilities.FileIOUtility.readFromChannel(FileIOUtility.java:173)
at org.xadisk.filesystem.TransactionLogEntry.getNextTransactionLogEntry(TransactionLogEntry.java:494)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:663)
... 131 more
___________________________________________________________________________________________________________________________________



 Comments   
Comment by Nitin Verma [ 14/Jul/13 ]

Hello. Are you able to reproduce this issue, and how frequently do you see it? As such EOFExceptions don't generally happen during commit, I will need more specific details regarding the conditions when this issue occurs.

Can you provide some details like a sample test, transaction-log files etc which can enable me diagnose/reproduce this issue locally.

Thanks,
Nitin

Comment by gunnar_zarncke [ 06/Aug/13 ]

I cannot reproduce this issue and I havn't seen this issue again since then.
But I did see a few more cases where the transaction log couldn't be recovered.
I will comment on this in XADISK-139.





[XADISK-81] Provide an i/o interface for random read/write, by extending java.io.RandomAccessFile. Created: 31/Jul/11  Updated: 22/Jun/15

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.1
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Martin for bringing up this feature idea.

Currently, XADisk is providing implementations for java.io.InputStream and java.io.OutputStream. If feasible, provide an i/o interface which extends the standard class java.io.RandomAccessFile.

Thanks,
Nitin



 Comments   
Comment by puce [ 22/Jun/15 ]

Or SeekableByteChannel/ FileChannel when moving to the NIO.2 File API (XADISK-175)

http://docs.oracle.com/javase/tutorial/essential/io/legacy.html
http://docs.oracle.com/javase/tutorial/essential/io/rafs.html





[XADISK-101] For windows share directory, permission checks done using Java APIs are misleading. Created: 17/Feb/12  Updated: 17/Feb/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

There is a problem with Windows Share and Java combination, in the File's canRead()/canWrite() APIs.

These APIs are returning true even if the particular file/directory does not expose that permission. This results in XDisk's permission checks, done during an operation itself, passing even if actual permissions don't exist. When XADisk proceeds to commit the changes, only then the lack of permissions result in exception. The XADisk instance would fail during commit and would make itself unavailable.

Workaround is to make sure to assign all needed permissions beforehand, when using windows share folders.






[XADISK-100] Error "Directory changes could not be forced-to-disk" for windows file share. Created: 17/Feb/12  Updated: 17/Feb/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When creating files in a windows share directory, the following exception is seen during transaction commit:

____________________________________________________________________________________________________________________________

Exception in thread "main" org.xadisk.filesystem.exceptions.XASystemNoMoreAvailableException: The XADisk instance has encoutered a critial issue and is no more available. Such a condition is very rare. If you think you have setup everything right for XADisk to work, please consider discussing in XADisk forums, or raising a bug with details
at org.xadisk.filesystem.NativeXAFileSystem.notifySystemFailure(NativeXAFileSystem.java:488)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:731)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:1246)
at org.xadisk.tests.correctness.Temp.main(Temp.java:17)
Caused by: java.io.IOException: Fatal Error: Directory changes could not be forced-to-disk during transaction commit.
at org.xadisk.filesystem.DurableDiskSession.forceToDisk(DurableDiskSession.java:122)
at org.xadisk.filesystem.NativeSession.commit(NativeSession.java:723)
... 2 more

____________________________________________________________________________________________________________________________

A workaround (if it fits, depending upon your application requirements) is to disable directory synchronization using setSynchronizeDirectoryChanges






[XADISK-104] Limitting access to a remote file system, exposed by xadisk Created: 12/Mar/12  Updated: 12/Mar/12

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Daniel for identifying this improvement at discussion thread http://groups.google.com/group/xadisk/browse_thread/thread/b77ee7b33f9c9574.

It could be a useful feature if xadisk can be configured to restrict operations over specified files/folders only, specially for remote file systems exposed by xadisk.






[XADISK-18] Lock timeout functionality for directory pinning. Created: 20/Nov/10  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 0.9 (Beta)
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 18

 Description   

As of current implementation, lock timeout is not effective for cases when the
locking is not feasible due to another transaction holding a directory-pin lock
over an ancestor directory of the file/directory object. Or, when a directory
move operation cannot proceed due to a descendant file/directory object being
already locked by some other transaction.

***Directory move operations are rare and so this functionality was deferred
while implementation.***






[XADISK-51] implement createPropertiesFile function Created: 19/Jan/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 51

 Description   

would be useful to add to the session the option to store properties file






[XADISK-50] Implement the createSymbolicLink function Created: 19/Jan/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 50

 Description   

Would be useful to add to the session the possibility of creating symbolic links.






[XADISK-48] Write some tests for performance measurements. Created: 09/Jan/11  Updated: 31/Jul/11

Status: Reopened
Project: xadisk
Component/s: testcases
Affects Version/s: 1.0
Fix Version/s: current

Type: Task Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 48

 Description   

Thanks to Martijn van Tilburg for his query on performance metrics for XADisk.

It encouraged me to start with some basic performance related tests - I started
with a comparison between Java IO operations (which are transaction-less) and
XADisk operations for read/write/truncate.

What can be the most crucial thing for measurement? Some quick thoughts which
came to me:

a) It would be nice to see how the "average time taken" by a transaction would
vary with the number of other concurrent transactions. This is because of shared
resources like memory, cpu, and the huge bottleneck - the physical disk (with
its head).

b) Measurement of the time taken with number of concurrent transactions would
make more sense if we are measuring data both from

  • XADisk (which provides ACID semantics)
  • Java IO API (which does NOT provide ACID semantics).

It is well-known that the ACID transactional properties would always introduce
an additional cost for the same operation done "natively (without a ACID
transaction)".

So, what would make more sense is to plot the measurements with XADisk and with
Java IO API (for same operations) side-by-side.

Have checked-in the new tests a few minutes ago.

Thanks,
Nitin



 Comments   
Comment by Nitin Verma [ 09/Jan/11 ]

Version should be "1.0" for consistency.

Comment by Nitin Verma [ 09/Jan/11 ]

Have checked-in the tests to /trunk.

Closing...

Comment by Nitin Verma [ 11/Jan/11 ]

I mistakenly closed this issue earlier. Reopening...

It would be good to listen for feedback from community (and also research
myself) on how to improve such tests (the current ones are quite basic) and what
would be a proper setup to run the tests and also how things can be measured
more precisely to get a broader picture of everything.





[XADISK-53] implement the delete recursive Created: 19/Jan/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: current
Fix Version/s: current

Type: New Feature Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 53

 Description   

would be useful to add the recursive option to the delete method






[XADISK-74] Use slf4j for logging instead of binding it to one logger Created: 07/May/11  Updated: 18/Apr/12

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: fecici Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

it could be nice to log through the slf4j, so the user can choose which logger to use...






[XADISK-56] Move all not junit test classes to junit Created: 05/Feb/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: testcases
Affects Version/s: current
Fix Version/s: current

Type: Task Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[XADISK-55] add junit dependency and write the main xadisk unit and integration tests Created: 05/Feb/11  Updated: 31/Jul/11

Status: Open
Project: xadisk
Component/s: testcases
Affects Version/s: current
Fix Version/s: current

Type: Task Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[XADISK-54] Refactor the entire projet to use the standard maven layout Created: 05/Feb/11  Updated: 31/Jul/11  Due: 07/Feb/11

Status: Open
Project: xadisk
Component/s: misc
Affects Version/s: current
Fix Version/s: current

Type: Task Priority: Minor
Reporter: filippo22000 Assignee: Nitin Verma
Resolution: Unresolved Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Refactor the entire projet to use the standard maven layout.



 Comments   
Comment by dcoraboe [ 17/Feb/11 ]

Does it mean that XADisk is planned to be distributed in the Maven Central repository?

Comment by Nitin Verma [ 23/Feb/11 ]

Hi,
I am hoping the XADisk binaries would be on maven2 repository (java.net's) very soon. I have sent my request
to get permissions to upload to that repository.

The current issue (54th) is different and will no be tracking this activity. I have filed
another issue now : http://java.net/jira/browse/XADISK-58

Thanks,
Nitin





[XADISK-162] Generate OSGi bundle manifest attributes Created: 28/Feb/14  Updated: 28/Feb/14

Status: Open
Project: xadisk
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Jasper Siepkes Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I've added a small patch to add options to the Maven POM which adds OSGi Bundle attributes to the manifest when building. This allows XADisk to be used in an OSGi framework.



 Comments   
Comment by Jasper Siepkes [ 28/Feb/14 ]

Can't attach files to JIRA So here's the patch pasted:

 
--- a/pom.xml
+++ b/pom.xml
@@ -14,8 +14,8 @@ Contributor : Christoph Beck
     <modelVersion>4.0.0</modelVersion>
     <groupId>java.net.xadisk</groupId>
     <artifactId>xadisk</artifactId>
-    <version>SNAPSHOT</version>
-    <packaging>jar</packaging>
+    <version>1.3.0-SNAPSHOT</version>
+    <packaging>bundle</packaging>
     <name>XADisk</name>
     <description>Transactional Wrapper for File Systems</description>
 	
@@ -83,6 +83,22 @@ Contributor : Christoph Beck
                     </execution>
                 </executions>
             </plugin>
+            <plugin>
+                <groupId>org.apache.felix</groupId>
+                <artifactId>maven-bundle-plugin</artifactId>
+                <extensions>true</extensions>
+                <configuration>
+                    <instructions>
+                        <Bundle-SymbolicName>org.xadisk</Bundle-SymbolicName>
+                        <Export-Package>org.xadisk.*</Export-Package>
+                        <Import-Package>
+                            javax.naming.*;resolution:=optional,
+                            javax.resource.*;resolution:=optional,
+                            *
+                        </Import-Package>
+                    </instructions>
+                </configuration>
+            </plugin>
         </plugins>
         <sourceDirectory>src</sourceDirectory>
         <resources>
-- 
1.8.3.1




[XADISK-173] XADisk api for creating file/directory while creating all parent directories required. Created: 18/Feb/15  Updated: 18/Feb/15

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to Phillip Harvey for raising this feature suggestion.

It could be useful to provide an xadisk api for doing the following in a single operation:
-create file or directory,
-create all non-existent parent directories.






[XADISK-172] Missing target file for copy is logged as InsufficientPermissionOnFileException when it should be FileNotExistsException Created: 17/Feb/15  Updated: 18/Feb/15

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: pjlharvey Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux version 3.8.13-16.2.1.el6uek.x86_64 (mockbuild@ca-build44.us.oracle.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Thu Nov 7 17:01:44 PST 2013



 Description   

If i execute the following code

XAFileSystem xafs = null;
Session session = null;
String instanceId = generateUniqueId();

try {
	StandaloneFileSystemConfiguration configuration = new StandaloneFileSystemConfiguration(xadiskTransactionFolder, instanceId);
	xafs = XAFileSystemProxy.bootNativeXAFileSystem(configuration);
	xafs.waitForBootup(-1);
	session = xafs.createSessionForLocalTransaction();
	File srcFile = new File("/tmp/existingfolder", "non_existent_file");
	File destFile = new File("/tmp/targetFolder", "target_file");
	session.copyFile(srcFile, destFile);

} catch (Exception e) {
	// act accordingly
}

then the xadisk copy method throws the following exception.

org.xadisk.filesystem.exceptions.InsufficientPermissionOnFileException: Permission of type [READ_FILE] is needed over the file/directory with path [/tmp/existingfolder/non_existent_file] for the i/o operation to succeed.
        at org.xadisk.filesystem.NativeSession.checkPermission(NativeSession.java:1157) ~[xadisk-1.2.2.jar:na]
        at org.xadisk.filesystem.NativeSession.copyFile(NativeSession.java:359) ~[xadisk-1.2.2.jar:na]

This is a little misleading, as the actual problem is that file does not exist.



 Comments   
Comment by Nitin Verma [ 18/Feb/15 ]

Yes, it is a known problem. You may find similar issue logged where non-existing file/directory error is reported as InsufficientPermissionOnFileException.





[XADISK-151] Some objects put in HostedContext remain there even if no longer needed. Created: 14/Oct/13  Updated: 14/Oct/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

HostedContext is used to keep objects for invoking methods upon them remotely. It comes into play in situations like: remote xadisk invocation for filesystem operations, endpoint activation on remote xadisk instance, clustering (xadisk slave instances making use of a single master instance's concurrency control).

ConversationalHostedContext is bound to a specific communication channel and when the communication is closed, the context would go away (garbage collected) automatically.

GlobalHostedContext remains alive for the lifetime of the instance.

Currently, the three kinds of objects kept in the global hosted context are removed at certain later time (message-endpoint-factory, message-endpoint, xaresource for message delivery).

But, for conversational hosted context, the objects are never removed. This can lead to accumulation of references to the unused objects. For example, in case of remote xadisk operations, each new session is kept in this context. So, as long as the same xafs proxy object is used to create new and new sessions, all the session object would keep accumulating in the conversational context (of the xafs proxy).

Implement the removal of unused objects from the conversational context at the appropriate times (eg commit/rollback in case of a remote session).






[XADISK-149] JVM (stack guard) warning while loading xadisk native library. Created: 13/Oct/13  Updated: 13/Jan/14

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to George for identifying this through this thread: https://groups.google.com/forum/#!topic/xadisk/Hc4iZv8_6mk

The JVM reports the following warning while starting xadisk:

"Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /tmp/xadisk/native/unix_32_xadisk.native which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'."



 Comments   
Comment by gastaldi [ 13/Jan/14 ]

As stated in the Forum thread, the solution is to change the enum to this:

private enum NATIVE_LIB_NAMES {
        unix_64_xadisk, unix_32_xadisk,
        windows_64_xadisk,windows_32_xadisk,
        mac_64_xadisk, mac_32_xadisk,
        placeholder_xadisk
};




[XADISK-157] InsufficientPermissionOnFileException sometimes missing [READ_DIRECTORY] Created: 07/Dec/13  Updated: 07/Dec/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Open JDK 1.6 and 1.7, Windows 7 and Linux (RedHat and Ubuntu)



 Description   

I often get InsufficientPermissionOnFileException on files where the real permissions in the file-system do not show any missing permissions.
I ignore the error and consider the file non-readable with the following code:

try {
Session session = getSession();
File file = getFile();
return session.fileExists(file) && !session.fileExistsAndIsDirectory(file);
} catch (final Exception e) {
Warden.disregardAndReport(e);
return false;
}

I am wondering what the causes may be for this issue.

I have seen this happen in the following circumstances:

  • Checking existence of a directory the parent directory of which doesn't exist either (in code which creates neccessary parent directories as needed).
  • Checking existence of hard-linked files (here the cause is less clear).

Stacktrace Fragment:

org.xadisk.filesystem.exceptions.InsufficientPermissionOnFileException: Permission of type [READ_DIRECTORY] is needed over the file/directory with path [/var/lib/preprocessor/configversions/20131206T185427.534Z/sequences] for the i/o operation to succeed.
at org.xadisk.filesystem.NativeSession.checkPermission(NativeSession.java:1157)
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:407)
at org.xadisk.filesystem.NativeSession.fileExists(NativeSession.java:391)



 Comments   
Comment by Nitin Verma [ 07/Dec/13 ]

Thanks Gunnar.

The case of parent directory missing you mentioned is a known issue: https://java.net/jira/browse/XADISK-120. Please let me know if you discover any detail about the hard-link cases.





[XADISK-143] Allow ThreadFactory parameter for StandaloneWorkManager Created: 19/Aug/13  Updated: 19/Aug/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The StandaloneWorkManager uses a ThreadPoolExecutor with flexible parameters in the constructor but without an option to set the ThreadFactory. It would be helpful if were some way to specify the ThreadFactory or at least give a parent ThreadGroup and/or thread name prefix. Currently the XADisk threads worker threads will show up as nondescript pool-x-thread-x threads at the top level.






[XADISK-134] spurious XADisk Error [Native Module] Directory C:\ does not exist needs better error message Created: 26/Feb/13  Updated: 26/Feb/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: gunnar_zarncke Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 64-bit, Java 1.7, Eclipse Juno, Open EJB



 Description   

Sometimes I get an error message

"XADisk Error [Native Module] Directory C:\ does not exist."

Which is written to stdout by XADiskWindows.c (Line 35).

I couldn't further isolate the cause of this.

As an improvement I propose to to call GetLastError to get more details of the error and log this code too. Or even better to raise an exception with the found status code.

See also
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx






[XADISK-130] Implement createHardLink functionality. Created: 07/Feb/13  Updated: 07/Feb/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.1
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Many Thanks to Gunnar for bringing this up at the discussion thread - http://groups.google.com/group/xadisk/browse_thread/thread/a131fe6935ce67b7#

It would be a good idea to be able to create hard links via xadisk.

(There is already a tracking bug for symbolic links: http://java.net/jira/browse/xadisk-50)






[XADISK-150] Savepoint like feature in XADisk. Created: 13/Oct/13  Updated: 24/Oct/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: New Feature Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to George for his comments on https://groups.google.com/forum/#!topic/xadisk/9_Fy3Lr3P5k.

The savepoint is a database concept enabling changes to be bookmarked and reverted-to (rolled-back) at a later point. http://en.wikipedia.org/wiki/Savepoint

It is not yet confirmed how feasible it would be to implement this, and how much value will it really add to xadisk users in general.



 Comments   
Comment by Jasper Siepkes [ 24/Oct/13 ]

Would be nice if this is implemented in a pluggable way (ie. support multiple implementations). Some underlying filesystems on which XADisk runs already have capabilities for this functionality. For example if you run XADisk on a system with ZFS it would be nice if we could leverage ZFS' snapshot capabilities.





[XADISK-147] Add method to get transaction status in Session. Created: 08/Oct/13  Updated: 08/Oct/13

Status: Open
Project: xadisk
Component/s: filesystem
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: Nitin Verma Assignee: Nitin Verma
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Thanks to George for suggesting this at: https://groups.google.com/forum/#!topic/xadisk/6Lbe3CVZSBE

It could be useful for some applications to query the Session object (usually, from some other thread other than the one handling the Session object) about its transaction's status (open, committed or rolledback).



 Comments   
Comment by Nitin Verma [ 08/Oct/13 ]

Parallel to the Session interface, there are two other interfaces, XADiskConnection (JCA environments) and XASession (standalone JTA environments); all the three interfaces implement XADiskBasicIOOperations. But the last two interfaces do not hold the responsibility for transaction management as does the Session interface (which is for standalone non-JTA environments) through its commit/rollback methods. In these two cases, the same is achieved by the JTA standard interfaces. So, the new method only need be added to the Session interface.





Generated at Sun Aug 30 21:21:52 UTC 2015 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.