I solved the problem:
Rotation of objects didn't fail because of code (posted here) for matrix transformation. It was a problem of the layers.
Using the following code works with IEvolution 4.1.2:
//Adjust position of annotations according to rotation angle of image
private void btnRotateOK_Click(object sender, EventArgs e)
{
    IELayer layer;
 
    //Original code:
    //ieViewerLeft.Image.LayersRemove(0);
    //layer = ieViewerLeft.Image.GetLayer(0);
 
    //New code:
    layer = ieViewerLeft.Image.GetLayer(1);
 
    int center_of_rotation_x = (int) (layer.RotateCenterX * ieViewerLeft.Image.Width);
    int center_of_rotation_y = (int) (layer.RotateCenterY * ieViewerLeft.Image.Height);
    double rotation_angle = layer.Rotate;
 
    //Calling a method, where the annotations are rotated via matrix-transformation
    rotateObjects(ieViewerLeft.Image.Annotations, center_of_rotation_x, center_of_rotation_y, rotation_angle);
 
    ieViewerLeft.EnableRotateLayers = false;
           
    //New code:
    ieViewerLeft.Image.LayersRemove(0);
 
    //some more code…
}
Best regards
Claus Gerlach