WPF 4 Multitouch – getting started

You may have noticed I’m into multitouch (MT) technology. I don’t have any ‘proper’ MT kit – I’ve just built my own using the NUIGroup’s core software and an XNA example from CodePlex. With Win7 RC and the VS2010 and .Net 4 beta 1 I now have the opportunity to start building some real MS technology MT stuff.

I still don’t have a ‘proper’ MT device but by coupling up WIn7 with the CodePlex Multitouch Vista project (which now has a Win7 driver), it is easy to use the ‘two mice’ driver to simulate MT on your standard PC. There is nothing stopping you from having some MT fun now!

Things you need to follow along:

Win 7 RC
Multitouch Vista CodePlex project – just the binaries no need to compile, its a simple copy to HDD and run Multitouch.Service.Console and then Multitouch.Driver.Console from two CMD windows with two mice plugged in, and then you are in the land of MT! <Cough> Ok there are a couple more things to do. See the setup video.
VS2010 and .Net 4 beta 1

First thing to note is that we already know there are some changes to the manipulation API in .Net 4 coming in Beta 2. Please don’t start building that mission critical enterprise application with the code examples I’m showing here!!

In Beta 1, we have the basic manipulation events which we can code to. There are no touch aware control goodies – yet. The cool thing is that the manipulation events are all implemented on UIElement, so stuff that should have does have these events.

The manipulation events are:

ManipulationStarted
ManipulationDelta
ManipulationStarting
ManipulationCompleted

Of these ManipulationDelta seems to be the most useful in my playing to date.

You have to switch on these events per UIElement. You do this by setting the ManipulationMode property. You can be selective on the events you want or just go for All of them :-)

This all seems pretty straightforward, but without any MT aware controls switching on all the above doesn’t do alot for you. To make use of the detected events we need to add some code.

Lets look at a basic MT example. The classic MT application has to already be photo or image manipulation – being able to rotate, stretch and shrink images as you like.

In VS2010, create a new WPF application and add a Canvas and then an image control to the XAML:

</Canvas>
</Window>

Next, we need to wire up the Window1 with an event handler for ManipulationDelta. Basically, whenever a manipulation happens we going to get the change event and write the code to affect the changes onto our image. Your Window1 should look like this:

To affect our Image control we will use the RenderTransform method. We can initialise our Image control’s RenderTransform matrix using the static resource InitialMatrixTransform, and set its ManipulationMode to ALL.

Now when ever we fiddle with the Image control with using MT events will be fired and the ManipulationDelta will be captured by the Window. Now we need to make use of the manipulation data provided to us to affect the Image rendering. We’re going to use a custom matrix to implement the RenderTranform. This allows us to pre-calculate all the possible modifications to the Image that may have happened in an MT action, and then apply them in one go to the Image.

First up in our Window1 OnManipulationDelta event we want to get hold of the manipulation data and the Image control. Then we want to get the matrix defining the current RenderTransform state of our Image.

void OnManipulationDelta(object sender, ManipulationDeltaEventArgs e)
{
// Get current manipulation data
var mDelta = e.GetDeltaManipulation(this);

// Link up with source object
var i = e.OriginalSource as Image;

// Get the current matrix defining our object
var matrix = ((MatrixTransform)i.RenderTransform).Matrix;

}

Now we have all the bits we need, next we have to start applying the changes to the matrix to create a new matrix value which we will apply to the Image control via its RenderTransform method.

First thing we need to do is calculate the current centre of our Image. In normal WPF I’d use the RenderTransformOrigin method to determine the centre of the Image control ‘relative to the bounds of the object’ – ie independent of the viewport coordinates system. However, this didn’t deliver the expected results when applying the transform – for example the pivot point is way off when doing rotations. I haven’t got to the bottom of this – I’m a relative novice at WPF - so lets just calculate the actual centre point of the Image by using its ActualWidth and ActualHeight.

// Calculate the actual centre of the object, note not using RenderTransformOrigin
var orgCenter = new Point( i.ActualWidth/2, i.ActualHeight/2 );

Now we want apply the changes in location caused by the manipulation. These are held in the mDelta structure as Translation.X and Translation.Y values – basically the Vector defining the motion of our Image in the MT event. We can add these to our matrix OffsetX and OffsetY values using the matrix.Translate method:

// Append the changes in X and Y from the manipulation data to the coordinates of our object definition
matrix.Translate(mDelta.Translation.X, mDelta.Translation.Y);

Having potentially moved our Image control, we need to re-calculate its centre coordinates so that we have a valid centre to apply further transforms. We’ll re-calculate after each additional transform is applied.

    // Apply the new matrix results to get a new centre for our object
    // ie the centre after movement on on the object has been applied
    var center = matrix.Transform(orgCenter);

Lets now apply any rotation change held in the mDelta:

// Now apply any manipulation rotation to the new matrix based on the new centre position
matrix.RotateAt(mDelta.Rotation, center.X, center.Y);

Re-calculate centre again:

// Calculate the new center point after rotation has been applied
center = matrix.Transform(orgCenter);

Our last transform is to scale the Image control:

// Apply any changes in scale from the manipulation data based on recalculated centre
matrix.ScaleAt(mDelta.Scale, mDelta.Scale, center.X, center.Y);

Now we have a matrix which has had movement, rotation and scale changes applied to it. This matrix now represents all the changes in the current MT manipulation that have happened to our Image control. Let’s now apply these changes in one step, updating the Image controls rendering by setting our new matrix values as the new matrix transform.

    // Now we have our custom matrix transform calculated we can apply it to the real object
    // and see the results on screen
    i.RenderTransform = new MatrixTransform(matrix);

Finally, we tell WPF we’ve handled the event.

e.Handled = true;
}

If you have followed along, or looking to just Cut & Paste now we’ve finished :-). Here is the complete C# code: