I've thought a bit about the irregular selection problem, and I think I can come up with a generic method of getting the shape. When calling DrawPicture, the mask is always taken into account, and the pixels that are outside the mask are left alone. So I could fill them with a special pattern, draw the picture, and look at which pixels were left untouched. However the special pattern can't be just a color, since I'm working in 32 bits thus all colors can be theoretically be used in an image. Instead I think I'll use the fact that the image contains a dummy byte (since each pixel is 32 bits, but each of the three components only wants 8 bits). When QuickDraw sets a pixel, it sets the dummy bit to zero, so if I were to set it to something else I could detect which pixels were left untouched.
(a bit later)
My implementation worked, and stuff copied from the Finder and Icon Machine is pasting in properly, with the correct selection shape. However it doesn't work for selections from Photoshop. This is because when the image from Photoshop is drawn, even for pixels which are supposed to be left untouched the dummy byte is reset to zero. I guess I'll have to use a color, since this is the only way to make it work from all source. There's the risk that the image will contain that color, but if I choose a very rare one (such as 0xFFFFFE) chances are very small that the user will set it on purpose, and it'll only show up if the image was processed.
I've also added copying and cutting of the current selection. This wasn't very hard, since I had dealt with exporting to the clipboard earlier (clip2icns has an export icon to clipboard option). The only tricky part was to get the image data from the selection if it was floated, or from the image if it wasn't. Finally I did a bunch of miscellaneous stuff, like adding select all/none/inverse commands, and deleting the current selection when the user selected clear or pressed the delete key. I also added proper menu item enabling/disabling for all the clipboard and selection commands.
The plan for tomorrow is to add support for additive/substractive selections (again, like Photoshop's) and a lasso selection tool, if I have the time (perhaps initially I'll just make it draw an irregular selection, and not shrink down just yet).