It turned out I was making a fundamental mistake when rotating the GWorld. When rotating, unlike when you're flipping, the dimensions of the GWorld change (unless it's a square, which is why I didn't catch this earlier, because I was rotating the whole icon). After I did this the rotation of the selections improved, but the shape of the selection refused to rotate. The weird thing was that when I commented out the line that rotated the contents, then the shape would rotate. In the end it turned out that this was because I had an if statement, if (rotatePix == selectionPix), which determined if the selection shape should be rotated too. However, when I was rotating the contents of the selection I was resizing the PixMap, and so it's location in memory could have changed.
Since I was done with the transformations I decided to work on drawing in selections next. Right now all the drawing tools only work in the current icon, and ignore the fact that there is a selection (this is because most of them were implemented before I did selections).
I've discovered that there are three possible states:
- no selection, the target gworld is the current gworld
- an unfloated selection, the target gworld is the current gworld but drawing is clipping to the selection region
- a floated selection, the target gworld is the selection gworld with the drawing clipped to the selection region
For the fill tool it's not only a matter of clipping, but clicks outside the selection must be rejected as well.