Simple Rewriting Question

Nov 3, 2009 at 3:54 PM

Hi, have been using CCI to do some simple program analysis and now I want to use it to do some simple bytecode rewriting to support a debugging application. Basically I want to have a shadow stack that keeps track of all the writes to local variables on the current call frame. In theory this should be very easy to do but for whatever reason I am completely lost trying to find the parts of the framework that will do this for me.

Since I have control of the compilation step I can actually insert most of the shadow stack code in at the source level, so what I need to do is quite simple. Whenever I see a store in the bytecode to a user defined variable (x = y;) I want to place a call that looks like the following (ShadowStackUpdate('x', y);) immediately after it to update the shadow stack.

So I read in the assembly, initialize my ShadowMutator class (inherits from MetadataMutator), visit the assembly and write it (following more or less the PeToPe and RoundtripTests code):


ShadowMutator mutator = new ShadowMutator(host, pdbReader);
var modcopy = mutator.GetMutableCopy(assembly);
var modassembly = mutator.Visit(modcopy);

String nb = Path.ChangeExtension(asmFile, null);
String resExefile = nb + "_shadow" + ".exe";
String resPdbFile = nb + "_shadow" + ".pdb";

var pdbWriter = new PdbWriter(resPdbFile, pdbReader);
Stream peStream = File.Create(resExefile);
PeWriter.WritePeToStream(modassembly, host, peStream, pdbReader, pdbReader, pdbWriter);


where (with some simplification on the operation being added to the body, GenerateUpdateForLocalVarStore should of course insert several instructions to actually handle the shadow store)


internal class ShadowMutator : MetadataMutator
    internal ShadowMutator(IMetadataHost h, PdbReader r) 
        : base(h)
    public override List<IOperation> Visit(List<IOperation> operations)
        List<IOperation> oop = base.Visit(operations);
	List<IOperation> opl = new List<IOperation>();

	foreach(IOperation op in oop)
	        case OperationCode.Stloc:
		case OperationCode.Stloc_S:
		case OperationCode.Stloc_0:
		case OperationCode.Stloc_1:
		case OperationCode.Stloc_2:
		case OperationCode.Stloc_3:

	return opl;

    private void GenerateUpdateForLocalVarStore(List<IOperation> opl)
        Operation pushconst = new Operation() { OperationCode = OperationCode.Ldnull };
opl.Add(pushconst); } }


The resulting executable throws an InvalidProgramException (not supprisingly as I am pretty sure the offsets/any checksum in the code stream is getting mangled by my insertions). However, after wandering through things all afternoon I am feeling pretty stuck. So I would really appreciate a pointer in the right direction (as I know there must be a simple way to do this but I cannot find it).

Thanks in advance for any help,



Nov 3, 2009 at 4:55 PM

Rewriting at the IL level is possible, but somewhat tricky as you've found out the hard way. We have some code that will make that easier, but that still needs some cleaning up and packaging before we can expect people other than ourselves to use it.

Of course, in general, the easiest way to rewrite method bodies it to decompile it to a Code Model (i.e. a tree) and then to do tree rewriting and then finally compiling the tree back into IL. To do this, you have to use the CodeMutator and you'll probably want to override Visit(Assignment). I've just updated PeToPe to always do this.

If you can assign to locals in other ways, such as by passing their addresses to method calls, you'll need to look at more places.

Nov 3, 2009 at 5:17 PM

Thanks for the quick reply. I figured there was some better way to do the rewriting but I was just lost in the framework. I just did a svn update and see the changes to the PeToPe example (and it clarifies quite a bit), so I'll take a look at that and see how it goes.

Thanks for the tip on passing variables as references, we are aware of this possibility and have some plans to deal with it but it certainly is annoying.



Nov 4, 2009 at 4:03 PM

I spent some time reworking my code to use the CodeMutator class (subclass of) as is done in the PeToPe example. My first test was to just do roundtrip with no modification to the program, just read, then write.

Unfortunately the CodeMutator seems to produce a new executable and pdbfile that is out of sync with the source code and thus cannot be used on the resulting program (even though the bytecode/debugging info was not changed at all). This is a problem for me as the goal is to do minimal rewriting so that the program can still be debugged/items reported back into visual studio.

Below is the code that does the mutation (just as in the PeToPe example) in my simple case the use of normalization does not seem to make a difference. I also checked and if I use a the simple MetaDataMutator(host) as the mutator the debug information is preserved correctly. I can give you the source that causes the problem, but it is really simple so the issue probably comes up regularly.

Additionally, I really don't need anything as high level as the CodeMutator. All that is needed is to insert 2-3 bytecodes (no control flow) and never accross debug points. I was looking at the ILGenerator class (i.e. is it possible just copy most operations into it plus the new bytecodes and then have it regenerate the method body) but there does not seem to be an easy way to just copy operations into it (all the Emit methods take an OperationCode + additional information). Would using something along these lines be possible.



//Read in set up stuff here (same as PeToPe) .....

MetadataMutator mutator = new CodeMutator(host, ilToSourceProvider, sourceToILProvider, pdbReader); IAssembly assembly = module as IAssembly; if(assembly != null) module = mutator.Visit(mutator.GetMutableCopy(assembly)); else module = mutator.Visit(mutator.GetMutableCopy(module)); //CodeModelNormalizer cmn = new CodeModelNormalizer(host, ilToSourceProvider, sourceToILProvider, pdbReader, null); //assembly = module as IAssembly; //if(assembly != null) // module = cmn.Visit(cmn.GetMutableCopy(assembly)); //else // module = cmn.Visit(cmn.GetMutableCopy(module)); //Write out stuff here (same as PeToPe) .....




Nov 6, 2009 at 5:14 PM

I just sent you an email to talk more about this. Let me know if you don't get it.