Today I worked on the lasso. The plan I described yesterday worked pretty well, except for CalcCMask. The function does the opposite of what I wanted it to, instead of returning the regions which do not contain a color (in my case white) it returns the ones that do. However the good people at Apple foresaw this, and they provided a parameter where you can specify your own color searching function (as pointer to it, of course). The function receives an RGBColor record, and returns an long and bool (the bool is actually returned, the long is part of the parameter list). The bool specifies if the color was handled or not (if it wasn't it's passed on to the default system function). The long tells the parent function if the respective color is supposed to be transparent (return 0) or is supposed to be part of the final bitmap (return 1). This worked, and now I have a tightening lasso.
CalcCMask has a sister function, SeedCFill, which I'm using for the fill and magic wand tools. In Photoshop, both of these tools have a tolerance setting, which specify if areas should be filled/selected even if they're not exactly the same color as the starting point. Using a custom color search function, I could also implement this. I think I definitely want to do this for the final version, but I'm not sure if I have enough time for the one that's going to part of my dossier. The issue isn't one of knowing how to do it (after all I just did a color search function), but one of the user interface. Since the tollerance can be chosen by the user, this would mean that I have to figure out how to let the user do this. Photoshop has a floating palette (one of many) called tool settings, whose contents change depending on the current tool. This would require a lot of work, and I don't think the IB will appreciate all the work that went into it.