Saving and Restoring IDL Objects

QUESTION: It occurs to me that saving and restoring object references can be useful. But when I call a method on the restored object IDL can't find it, since the Object__Define file has not yet been compiled. How can I get these methods restored, short of putting each saved object method in its own file?

ANSWER: JD Smith of Cornell University and I discussed this in the IDL newsgroup in late August 1998. Here is a slightly edited transcript of that conversation. JD writes first.

I am exploring a very promising use of the Save/Restore commands in conjunction with objects. Given some complex object which contains a host of different types of data (with pointers, etc.), as part of a class method, one adds:
   Save, self, FILENAME=fname 
to register on disk an accurate snapshot of the object. To restore, later, use:
   Restore, fname, RESTORED_OBJECTS=obj, /RELAXED_STRUCTURE_ASSIGNMENT
and the object is in obj, but also brought back as the local variable self. I'm not sure the relaxed structure assignment flag works for objects, but I don't see why it wouldn't. So this can be used in two ways ...

1. To allow an object to replace itself with another, perhaps older copy (or even an altogether different type of object -- but the utility of self-transmogrifying objects is not yet apparent to me). This works because the implicit self variable is passed by reference (as it has to be). This will lead to at least one unreferenced heap variable unless garbage collection steps are taken, i.e. by saying:
   oldself = self
   Restore, fname,/RELAXED_STRUCTURE
   Obj_Destroy, oldself
2. To allow a program module to load up an object on the fly, through the obj variable in the above statement (only one should be loaded if only one was saved).

This is all very convenient but leads to the strange situation of a loaded object in memory which exists there before any of the class methods, and/or the __define procedure for that object class are compiled. Therefore, the usual paradigm of putting all class methods in the __define procedure file before this procedure (suggested by RSI itself in the manual) fails. How can the method be found if the __define doesn't have to be compiled and isn't in it's own file? I would like to come up with a solution which doesn't involve a separate class__method.pro file for each method. Any ideas?

JD

To which I replied like this:

How about something like this:
   thisClass = Obj_Class(self)
   Resolve_Routine, thisClass + '_define'
I haven't tested this, but don't see any reason it wouldn't work. Resolve_Routine is the way IDL procedures and functions can be compiled from *within* other procedures and functions.

David

JD replied with this:

This will certainly work, but has the unfortunate side-effect of re-compiling every method each time an object is read from disk... I thought of modifying it slightly to the tune of:
   thisdef = Obj_Class(self)+'__DEFINE'
   IF (Where(Routine_Info() EQ thisdef))[0] EQ -1 THEN
   Resolve_Routine, thisdef
So that it would only compile if presently undefined.

JD

Unfortunately, even this doesn't work, since superclass methods are not automatically searched for and compiled. JD writes this in a follow-up article several months later.

The basic problem is that superclass methods are not automatically searched and compiled by the above procedure, since a restored object contains implicitly in its definition the class definitions of all its superclasses. Basically it's the same problem cropping up again. The solution is to recursively work your way up the inheritance tree by hand. But, one more wrinkle: if you're often updating your class definition, you need to have it defined before restoring the object, to avoid having the older definition (as saved with the object) shadow your new one. A new Resolve_Obj (see attached), and a method which addresses all of these concerns is therefore:
   Resolve_Obj,CLASS='class'
   Restore, file, RESTORED_OBJECTS=obj, /RELAXED_STRUCTURE_ASSIGNMENT
Notice that now we are burdened with knowing which class to pre-compile. However, if you are willing to live with the potential of older class definitions shadowing newer ones, or don't mind ensuring the most recent class version is defined before any object restoration occurs (though in practice that would eliminate the problem), you can still use:
   Restore, file, RESTORED_OBJECTS=obj, /RELAXED_STRUCTURE_ASSIGNMENT
   Resolve_Obj, obj[0]
without worrying about which class is being restored. The only difference here is that putting the restore first opens up the potential of class definition shadowing. Sometimes this isn't a concern, or the convenience and generality of a generic restoration scheme outweighs the precautions which must be taken when using it.

The updated routine is attached. Notice the use of Routine_Info to avoid compiling any class already compiled. Also note that the original routine will work well enough for classes which do not INHERIT... but most of the good ones do ;).

JD

Here is the new Resolve_Obj code itself.

   PRO Resolve_Obj, obj, CLASS=class, ROUTINE_INFO=ri
      if n_params() ne 0 then begin 
         if NOT obj_valid(obj) then begin
            message,'Object is not valid.'
         endif 
         class=obj_class(obj)
      endif 
      
      if n_elements(ri) eq 0 then ri=routine_info()
     
      for i=0,n_elements(class)-1 do begin 
         defpro=class[i]+'__DEFINE'
         if (where(ri eq defpro))[0] eq -1 then begin
            ;; Compile and define the class.
            call_procedure,defpro
         endif 
         supers=obj_class(class[i],/SUPERCLASS,COUNT=cnt)
         if cnt gt 0 then resolve_obj,CLASS=supers,ROUTINE_INFO=ri
      endfor 
   END

In Practice

I had the opportunity to use this little routine of JD's recently (August 2009), and I found a small complication with it. If your object inherits from a built-in IDL object (mine was inheriting IDL_Container), then the program fails on the Call_Procedure line, when it tries to compile a file named idl_container__define.pro.

To work around this problem, I have had to resort to a catch error handler that silently handles the problem and continues on its merry way. I have also slightly re-written JD's original program to more closely match the Coyote Library style guide. The revised program is named Resolve_Object.