Skip to main content

New to Cajo, error with NotSerializableException

  3 posts   Feedicon  
Replies: 2 - Last Post: June 14, 2011 13:34
by: LizardSF
showing 1 - 3 of 3
Posted: June 08, 2011 18:24 by LizardSF
Greetings! Trying to use cajo to handle some fairly basic multi-VM communication tasks, ideally with minimal changes to the existing code. The basic setup is, or should be, fairly simple: I've got a main app with a front end, that spawns a certain number of threads which contain objects that do the actual work. I've got a different application, with no front end, that just spawns a worker-type object and sits around waiting for the main app to need it. There's minimal state being shared; the main app needs to know "How many records has each worker processed" and "is the worker done with its asigned tasks". The workers need a mountain of static data for settings, rules, etc. There's two objects that really matter: One which keeps track of which tasks have been assigned and which remain, and another which just tracks active workers.

Here's the problem. When I run the "Create Worker" app, it is fine, and as soon as the main/engine app starts working, the worker in the other VM, thanks to cajo, notices this and does its thing. However, when it tries to add the object it creates to the list of workers maintained by the main app, I get "java.io.NotSerializableException: gnu.cajo.Cajo" and the usual stack trace.

Here's the list to be maintained by the engine:
pendingWork = new HashSet<SplitBasedLoanProcessingTaskCajo>();

Here's how the engine exposes itself:

cajo = new Cajo();
System.out.println("Server running in split-based engine.");
cajo.export(this);

Here's the function that adds the new worker to the list:
public void addWorkerToQueue(SplitBasedLoanProcessingTaskCajo work)
{
pendingWork.add(work);
}

(I've done some other experiments with the code, but then I end up with an unserializable object exception on BufferedWriter, which leads me to believe I have a fundamental misunderstanding of what I need to do in order to share objects between VMs.)

For my purposes, I need to:
a)Have a worker start at any time during the process, and be fed the set of startup information that the engine has.

b)Let that worker access the list of unassigned tasks to be processed. This is the most important "state" to be shared among VMs. This list is the only non-static object a worker needs access to, so I have other solutions for getting it the other values, if need be.

c)Let the main engine know there's another worker, and when it's done.

d)Retrieve from the worker a set of results.

Posted: June 09, 2011 06:26 by John Catherino
A Writer is local to the JVM in which it exists, and is not serialisable.

If the worker is trying to arrive at the main app with a Writer, then it will need to be changed; to receive a serialisable data structure from the main app, where it can then pass it onto its worker JVM, to write it locally.
Posted: June 14, 2011 13:34 by LizardSF
Thank you very much! That helped a lot, and I was easily able to restructure my code as needed. I've got one more issue that is proving very frustrating, this time involving pass by reference/pass by value.

I have a structure which my "Engine" maintains -- basically, a list of workers (currently, it's a set).

The engine has a function, exposed via the interface and cajo, which adds a newly created worker to that list.

During processing, it checks the status of each worker, and, when it's done, removes it from the list.

It's the last bit that's not happening. The "worker" added to the list is apparently being done by value, not reference, so that the state changes of the workers are not being detected. I followed the suggestions on the documentation page and used interfaces,which should create automatic pass-by-reference, but it doesn't seem to be happening.

Here's the worker interface. These are the functions/properties I need to see externally:
public interface ICajoWorker
{
	public boolean isCompleted();
	public List<ArrayList<String>> getReport_names();
}


Here's the relevant declaration in the engine:
ConcurrentSet<ICajoWorker>				pendingWork;

...

public void addWorkerToQueue(ICajoWorker work)
	{
		pendingWork.add(work);
		System.out.println("Added worker");
	}


Here's the code in the "Start Worker" class (irrelevant material removed):
ICajoWorker work;
while (!done)
{
Object refs[] = engineCajo.lookup(IEngine.class);
if (refs.length > 0)
	{ // compatible server objects found
	splitBasedEngine = (IEngine) engineCajo.proxy(refs[0], IEngine.class);
	work = new SplitBasedLoanProcessingTaskCajo(....);
	int g = splitBasedEngine.getGlobalTaskID();
             splitBasedEngine.setGlobalTaskID(g++);
	((SplitBasedLoanProcessingTaskCajo) work).setTaskSize(splitBasedEngine.getLoanCount() / 1000);
	Thread t = new Thread((Runnable) work);
	t.start();
	splitBasedEngine.getPendingWork().add(work);
	done = true;
	continue;
              }


The worker does the required work, and it is correctly synchronizing with the "ChunkManager" that is also a cajo object, but the engine is getting a copy of "work", not a reference to the object. (I have tried both the addWorkerToQueue function and the code shown above; both have the same effect.
Replies: 2 - Last Post: June 14, 2011 13:34
by: LizardSF
 
 
Close
loading
Please Confirm
Close