jni4net version 0.5 released

Second public release of jni4net is out. New features are:

• Simplified proxygen to wrap all classes in DLL or JAR file from command-line, without time spent on config file. It as well provides very simplistic build.cmd. I will write another article about proxygen soon, till then look at new samples.
• Ported java.nio.ByteBuffer to .NET, so you could share same byte array in both VMs. You are able to use BinaryWriter in .NET as well, but same byte order and interface is more convenient.
• There are new Adapt.Enumeration and Adapt.Disposable methods on CLR side. They are convenience adapters from corresponding Java interfaces.

  foreach (var prop in Adapt.Enumeration(javaSystemProperties.keys()))
  {
      cnt++;
  }
  

  using (var fis = Adapt.Disposable(new FileInputStream(classPath)))
  {
      using (var zis = Adapt.Disposable(new ZipInputStream(fis.Real)))
      {
          ZipEntry entry = zis.Real.getNextEntry();
          while (entry!=null)
          {
              string name = entry.getName();
              entry = zis.Real.getNextEntry();
          }
      }
  }
  

• Improved proxygen code generator
• Few minor bugfixes
• New troubleshooter page

Download there

How calling from Java to .NET works in jni4net

Tonight I would like to explain how calling from Java to .NET works with jni4net. The other way around is described in previous article, it would be good if you read that first if you didn't yet.

So, in JNI there is method which allow you to register native implementation of Java method. Such method had "native" keyword in it's signature. I believe that it's actually how low level methods in Java runtime are implemented.

We use proxygen tool, which is part of jni4net, to reflect any public members of .NET type and then to generate Java proxies. Those proxy methods are marked native. The generator must do few translations to follow Java naming rules. Namespaces are always lower-cased to become packages. Properties are converted to bean-like get and set methods.


For exceptions the situation is similar. system.Exception proxy is inherited from Java java.lang.RuntimeException to be throw-able. Therefore it isn't inherited from system.Object proxy. But there is system.IObject interface on both of them for your convenience.

The proxy implements Java finalizer, so when JVM GC cleans up the proxy instance, the real CLR instance is released as well. The reference to CLR is implemented as GCHandle converted to long and kept on JVM side inside the proxy.

system.Object proxy as well overrides toString() method and forwards it to CLR. It seems that in version 0.4 I forgot to override hashCode() and equals(), sorry about that ;-)

The implementation is is asymmetric to CLR->JVM proxies, because for both it's CLR which does the translation work. The wrapper have signature which is expected by JNI RegisterNatives. I apply Marshal.GetFunctionPointerForDelegate to wrapper methods to get endpoint callable by JVM. Side note is that JNI RegisterNatives does not make any difference between static and instance methods. They are identified by name and signature only, strange limitation.

Last trick I mention is about initial CLR bootstrap. During initial design I realized that there is way how to export native method from managed DLL. So I currently use that trick to create DLL with signature friendly to JVM. It seems to me now, that I will need to change my approach soon in order to support multiple versions of CLR and Mono on Linux. Probably I would need to use C++/COM. For now, we have 32bit and 64bit CLR 2.0 on Windows.

Next time maybe about interfaces and type casting.

How calling from .NET to Java works in jni4net

In this article I would like to explain how calling from .NET to Java works with jni4net.

First of all you should know that Java Virtual Machine (JVM) is exposing Java Native Interface (JNI). It is native/binary interface which allows other programs to control JVM: load classes, create instances and run methods. It as well allows you to control object lifetime by holding handle to instance. Usual way how to consume JNI interface is to use %JDK%\include\jni.h in your C++ program and then load jvm.DLL. I wanted solution for .NET, so I use [DllImport] attribute and I translated the header file from C to C#. I also converted all pointers and handles to IntPtr. So now we have JNI interface accessible to any .NET code.

Now it's quite easy to use JNI methods to call Java objects, but it's far from convenient. So the next step was to use Java reflection and get signatures of core objects and generate proxies. The proxies look like Java classes, they have same name, namespace, same methods with same signatures. But it don't have the implementation in .NET but rather they call the real implementation in JVM using JNI. The reflection and proxy code generator is reusable idea, so I created 'proxygen', it's tool which is part of jni4net package. You could use it to wrap your own classes.


Now you should know few tricks. JNI calls are done on handle of the method, which could be retrieved by method name and signature. See javap. To make it faster, I pre-bind the proxy-class to real-JVM-class during start-up.

To make the proxy good citizen in .NET world, java.lang.Object overrides Equals(), GetHashCode(), ToString() .NET methods and forward the calls to appropriate Java equivalent. You should be aware that there could be multiple proxies for same Java instance. java.lang.String proxy implements implicit conversion to CLR String.

Because we need garbage collection work, proxy implements finalizer. When CLR garbage collector finds lonely proxy, we release JNI handle of the real Java instance, so it could be collected on JVM side as well. Warning, it is possible to create cycle between objects across both heaps, which would prevent GC from collection. I have no good solution for that now, just be careful.

Another feature to mention is exceptions. To make Java exceptions useful in CLR, they should be inherited from System.Exception. Simply to be able to throw and catch them in .NET. But it means that they could now be inherited from java.lang.Object proxy. This is compromise, but it's worth of it, because now you could catch exceptions thrown by Java method in .NET code because jni4net does transparent translation for you. You will receive proxy of exception. To overcome the problem with common base class, I introduced java_.lang.IObject interface with all expected methods.

Last note would be about call marshalling. It's done with PInvoke on CLR side and with JNI on JVM side. The primitive types are very well compatible. Unsigned types from CLR are transmitted binary. Strings are translated as Unicode, it's bit slow. Any other non-primitive parameter is Java object. Which means you need to pass another proxy/Java object as parameter (for brevity we ignore interfaces now). When returning from the call, we could have return value. It is the way how to send Java instances back to .NET. jni4net/proxy wraps the JNI handle for you. It finds best-fit proxy-class and returns its instance. The proxy contains JNI handle to the real Java instance.

Next time about calling back from JVM to .NET and proxies of CLR objects in Java.

jni4net - bridge JVM and CLR

I'm proud to present first public release of jni4net - Object oriented, fast, intraprocess bridge between JVM and CLR.

Hello World!

From C# to Java

using net.sf.jni4net;
public class Program
{
    private static void Main()
    {
        Bridge.CreateJVM(new BridgeSetup());
        java.lang.System.@out.println("Greetings from C# to Java world!");
    }
}

This is very basic demonstration of the principle, full version of this sample and another 3 sample applications could be found in binary distribution. The others are same Hello World but from Java to .NET, usage of Apache FOP from C# for xsl:fo and last is creation of WinForms dialog from Java.

Please download here and talk back.

Next time I will write about why I created it, how it works, what are the features and what are the next steps. Watch this space.

Export native 64bit method from .NET assembly

It is well known fact that .NET assembly could be tweaked to export native method, similar way how normal DLLs do it. There is good description and tool from Selvin for 32bit native function.
My problem was how to do it for 64bits. Here you go.

1) you IlDAsm your assembly into il code.
2) Edit the IL and change header to look like this
for 32bit

.corflags 0x00000002
.vtfixup [1] int32 fromunmanaged at VT_01
.data VT_01 = int32[1]

for 64bit

.corflags 0x00000008 
.vtfixup [1] int64 fromunmanaged at VT_01
.data VT_01 = int64[1]

3) Header of your exported method would look similar to this. This is same for 32bit version.

.vtentry 1 : 1
.export [1] as Java_net_sf_jni4net_Bridge_initDotNet

4) You IlAsm the file back into DLL. For x64 you use /x64 flag.

Hack ReSharper for x64 unit tests

If you need to develop 64bit .NET application and run unit tests with R#, you will find that it's not possible, because Visual Studio is 32bit process and R# task runner is loaded as 32bit as well. There is quick and dirty solution to that.

1) Stop Visual Studio
2) Find JetBrains.ReSharper.TaskRunner.exe
3) Drop read-only flag on the file
4) Run

CorFlags.exe JetBrains.ReSharper.TaskRunner.exe /32BIT- /Force

The assembly have now broken signature and VS/R# complains about that. But now you could debug your 64bit unit tests. More info would be probably here

Robocode modules as in version 1.7

In version 1.7 we managed to split Robocode into several modules, introduce dependency injection using PicoContainer. We as well started using Maven 2 for builds. Now I will try to explain it a bit.

Modules Description

• robocode.api - This is really robocode.jar as you know it.
  • But now it contains only Robot API, everything what's visible to the robot in package robocode.
  • It also contains Control API, which you can use to automate the game, for example RoboRumble does exactly that.
  • Main entry point of the application. It finds robocode.core and calls it very soon after start.
  • There are few interfaces and base classes to allow robocode.core and other modules implement them.
  • It is very important to not add any compile time dependency of robocode.api on any other .jar, because that would cause all robot authors would need to have that dependency as well.
• robocode.core - The Hub.
  • It loads all robocode modules. Module is any .jar file in libs directory with name starting with "robocode". It finds class Module from package of the same name as the .jar file. For example net.sf.robocode.repository.Module. The class should initialize itself in static constructor. That's module entry-point. Good way how to do it is to register components (classes and instances) into PicoContainer. Or as a singleton or as a factory (transient). There is only single container in whole application. Components could be also registered under name, which is interesting for extensions, example is JavaHost.
  • Because core is referenced by every other module, it serves as place where all interfaces are defined. But most of the implementations are outside of core. This helps us to prevent spaghetti of dependencies.
  • In the future, we may have more extension points, they will be defined by interface which any extension should implement. The interface will be added into core if they are not specific to one of existing modules.
  • Currently we do NOT consider any interfaces public API, so if you want to create extension to Robocode, you should talk to us on our discussion email group first.
• robocode.battle - The rules and main battle loop.
  • It runs the battle thread, which is driving battle life-cycle. Battle has rounds, turns, and each turn have several steps. See Battle.runTurn()
  • The battle thread also creates robots, by using robocode.host, pauses them between turns, reads their commands and produces events for robots, like ScannedRobotEvent or HitByBulletEvent. It transfers the event from GUI to interactive robots.
  • It also produces all events for IBattleListener, they are produces each turn and we call them snapshots, about position of robots, bullets, etc. They are (99%) immutable structures. They also could contain serialized form of drawing commands from robots and text written by robots.
  • This module would expose extension points when we will implement custom extensible battle rules and battle fields. Watch this space!
• robocode.host - The security sandbox for robot.
  • Loads robot into separate ClassLoader, so he can't access anything but robocode.api. There is bit of magic to load from nested .jar files for team robots.
  • Java SecurityManager is enabled, so we prevent it from all cheating or attacks to user computer.
  • Disk operation are limited to certain directory and volume of data. Files are extracted from robot's .jar file on demand.
  • There are *RobotProxy classes, which transform robot's commands like setFire() into message, which is send each turn, on execute() call. It also validates them.
• robocode.repository - Is the dabase of robots installed on the system.
  • Robot classes are loaded and registered when the game spots them in robots directory. Robot property file is also extracted. The metadata are then stored in database, which is serialized into one file. The trick there is that the database is faster to read than to load all the classes and examine them. We do reload the classes or properties, when we found that file timestamp is later than last timestamp stored in database.
  • There is quite lot of logic to figure out the naming and versioning conflicts of robots.
• robocode.tests - Our unit tests and integration tests.
  • There are different kinds of robots made just for the purpose of testing some feature of Robocode. Some of them try to break rules, attack the security system. Others behave interesting way on battlefield and we measure them, to learn if the game physics is OK.
  • The tests are build with junit, there is base class RobotTestBed which prepares and runs the battle. It implements IBattleListener and we could read all the events produced by game and watch the game by code. During the game we make notes and in tear down phase we evaluate them with asserts.
  • There are some regular unit tests as well, but just few of them.
• robocode.ui - Windows and BattleView.
  • Windows are mostly boring stuff, but there are several problems. One of them is UI thread, which keeps window responsible during the game. But we need to make sure that the UI thread is not used to call battle components or even robot's code (as in pre 1.6). To do that BattleManager is really queue of commands for battle thread. They are executed on battle thread in each turn. Well it's not that simple, and we still have some troubles when the game is paused.
  • Another problem is how to paint all the events produced by battle. They are regullary dispatched on battle thread to anyone registered in BattleManager.addListener(), but for UI it;s different. Imagine that we could run the battle at 9K TPS (turn per second), you could easily see that it doesn't make any sense to render it with same frame rate (FPS). So we introduced AwtBattleAdaptor, which receives them all and dispatches only selected ones to UI thread. It's a bit complicated, because we could skip lots of onTurnEnded, but we need to preserve the text output from robots. So here it comes, we cache the texts per robot and inject it into those snapshots which are really dispatched. (That's only place where we broke immutability of snapshots, and we do that transparently to the recipients). Also bear in mind that we need to deliver all the events of other types, and to do that in correct order in relation to onTurnEnded. We do not dispatch onTurnStarted as current optimization. Anyone on GUI thread could register in AwtBattleAdaptor.addListener() and should properly unregister during disposal.
• robocode.ui.editor - Robot editor for real beginners. Personally I think we need to rather create Robocode as plugin to Eclipse, any volunteer ? There is also problem with Jikes, it's ages old and have bugs. But not all people have JDK installed.
• robocode.roborumble - Download robots, run the battles, upload results. Robocode@Home distributed computing, to get scores. We had issues several releases now, so we were unable to contribute to the grid with recent versions. Hopefully it's over now.
• robocode.sound - Just sound. Easy stuff.
• robocode.installer - Our hand-made installer.
• robocode.content - All other files to be deployed.
• robocode.samples - Our sample robots.

How to create new module

• Read the Developers Guide
• Unzip robocode.dummy.zip and rename all dummy files and all dummy things inside files.
• Implement Module class as described above.
• Add your module to main pom.xml
• Try if you are smart and brave enough to make at least small progress alone. And then definitely TALK to us about your idea.
• Then we decide where your extension belongs. If it's to be installed with main installer, keep it in root directory. If it should have separate installer, move it to plugins directory and create installer for it. There are several unfinished examples already. Next time about that.

Over and out.

Unity IoC - context aware

Unity will inject it self into constructor or property with IUnityContainer interface. No need to implement any "ApplicationContextAware"

public class Something
{
    //use this
    public Something(IUnityContainer container)
    {
        Container = container;
    }

    //or this
    public IUnityContainer Container { get; set; }
}

Found here

How to get non inherited interfaces

Is documented fact that GetInterfaces() method returns all interfaces implemented by given type including inherited ones. I wanted to get only interfaces defined by the class, and was searching for the solution. No luck. But it's really simple.
Just check if base class is implementing the interface, if yes filter it out.

Type type = typeof(Type);
Type[] nonInheritedInterfaces = type.GetInterfaces().Where((ifc)
     => !ifc.IsAssignableFrom(type.BaseType))
  .ToArray();

*Yes, if you are still on 2.0 you could rewrite it to loop.

URL access to nested .jar files

Tonight I implemented loading of classes from nested jar files. It seems to me that people might be interested in such trick, so here you are.

I created new protocol handler jarjar: which just redirects inner part of URL to .jar handler again. Please note that I use '^/' as jar-in-jar separator to avoid conflict with regular jar handler.

Let's have Outer.jar which contains Inner.jar which contains tested/robots/Ahead.class
Now you can do this

// initialize, just once per JVM please
JarJarURLConnection.register();

new URL(
"jar:jarjar:file:/C:/mylibs/Outer.jar^/Inner.jar!/tested/robots/Ahead.class"
).openStream();

Or create and configure ClassLoader this way

// initialize, just once per JVM please
JarJarURLConnection.register();

new URLClassLoader(
  new URL[]{new URL("jar:jarjar:file:/C:/mylibs/Outer.jar^/Inner.jar!/")}
).loadClass("tested.robots.Ahead");

If you need something more sophisticated or user friendly One-JAR is the solution I think.

English peg solitaire

My colleague at work mentioned that there is some puzzle where we have 33 holes in shape of cross and 32 bullets in all the holes but central. To move you should skip one bullet with it's neighbor into empty hole and remove the skipped one. The goal is to clean all but last bullet. He mentioned that it would be interesting to find solution with computer.
And we didn't knew the name of the game. Now it took me almost an hour to find it out for purpose of this article. So now when you know the name of the puzzle, you can easily find the solution on Internet. Actually quite interesting reading.

But I enjoyed brute force search for solution. Below is solution I found after few hours of coding and 6 minutes of intensive computing.

  XXX     XXX     XXX     XXX     XXX     XXX     XX0     XX0   
  XXX     XXX     0XX     0XX     0XX     0XX     0X0     0XX   
XXXXXXX XXXXXXX XX0XXXX 00XXXXX 0X00XXX 0X0X00X 0X0XX0X 0X0X00X 
XXX0XXX X00XXXX X0XXXXX X0XXXXX X0XXXXX X0XXXXX X0XXXXX X0XX0XX 
XXXXXXX XXXXXXX XXXXXXX XXXXXXX XXXXXXX XXXXXXX XXXXXXX XXXXXXX 
  XXX     XXX     XXX     XXX     XXX     XXX     XXX     XXX   
  XXX     XXX     XXX     XXX     XXX     XXX     XXX     XXX   
  
  XX0     XX0     XX0     XX0     XX0     XX0     XX0     XX0  
  0XX     0XX     0XX     0XX     0XX     0XX     0XX     0XX  
0X0X00X 0X0X00X 0X0X00X 0X0X000 0X0X000 0X0X000 XX0X000 00XX000
X0XXXXX X0XXXXX X0XXXXX X0XXXX0 X0XXXX0 X0XX0X0 00XX0X0 00XX0X0
XXXX0XX XXXXX00 XXX00X0 XXX00XX XXX0X00 XXX0000 0XX0000 0XX0000
  XX0     XX0     XX0     XX0     XX0     XXX     XXX     XXX  
  XXX     XXX     XXX     XXX     XXX     XXX     XXX     XXX  

  XX0     XX0     00X     00X     000     000     000     000  
  0XX     0XX     0XX     0XX     0X0     XX0     XX0     XX0  
00XX000 00XX000 00XX000 00XX000 00XXX00 000XX00 00X0000 00X0000
00XX0X0 00XX0X0 00XX0X0 00XX0X0 00XX0X0 000X0X0 000X0X0 000X0X0
0XX0X00 0XXXX00 0XXXX00 0XX00X0 0XX00X0 0XX00X0 0XX00X0 000X0X0
  XX0     X00     X00     X00     X00     X00     X00     X00  
  XX0     X00     X00     X00     X00     X00     X00     X00  

  000     000     000     000     000     000     000     000  
  XX0     0X0     0X0     0X0     0X0     0X0     0X0     000  
00XX000 000X000 000X000 00XX000 00XX0X0 0000XX0 000X000 0000000
00000X0 00X00X0 00X00X0 00000X0 0000000 0000000 0000000 000X000
00000X0 00000X0 00X00X0 00000X0 0000000 0000000 0000000 0000000
  X00     X00     000     000     000     000     000     000  
  X00     X00     000     000     000     000     000     000  

And there is the code. I renamed it to peg solitaire afterwards. I used C5 library for HashDictionary, but I'm not sure whether it was really necessary. And yes it takes some memory to compute the results as do breadth-first search. I don't even try any clever heuristics or cutting the tree.